openfeature-sdk-testing git:20260724.d4db756A

Immutable. This exact content never changes and is served at /api/v1/blob/e4dc5182b2c5ff17.

---
name: openfeature-sdk-testing
description: "Wraps OpenFeature (CNCF vendor-neutral SDK abstraction) testing patterns: the InMemoryProvider for hermetic tests without network calls, provider registration via OpenFeature.setProvider, the getBooleanValue/getBooleanDetails evaluation API with EvaluationDetails (value, variant, reason, errorCode), hooks for evaluation side-effects, and evaluation context for targeting-rule tests. Covers TypeScript, Java, and Python SDKs. Use when writing tests for code that resolves feature flags through the OpenFeature SDK regardless of the underlying flag management platform."
metadata:
  keywords: "openfeature, feature-flags, cncf, hermetic-tests, in-memory-provider"
---

# openfeature-sdk-testing

## Overview

OpenFeature is a CNCF project that standardises how applications
resolve feature flags. Per
[openfeature.dev/docs/reference/concepts/provider](https://openfeature.dev/docs/reference/concepts/provider),
providers are the component responsible for executing flag evaluations,
and "an application integrator can register one provider at a time."
The SDK ships an `InMemoryProvider` in every language that substitutes
real flag-management infrastructure with in-process flag state, letting
unit and integration tests run without any network call. The production
evaluation path (targeting logic, type coercion, defaults) is exercised
in full; only the data source is swapped.

**Differentiation from sibling skills:**
`launchdarkly-testing`, `unleash-testing`, `flagsmith-testing`, and
`growthbook-testing` wrap each vendor's own `TestData`/bootstrap API.
This skill covers the vendor-neutral OpenFeature layer teams adopt when
they want to keep application code decoupled from a specific provider.

## When to use

- Tests for code that calls `client.getBooleanValue()` / `get_boolean_value()`
  through the OpenFeature SDK, regardless of which provider runs in
  production.
- Tests asserting on evaluation details: variant name, reason code,
  or error code (e.g., `FLAG_NOT_FOUND`, `TYPE_MISMATCH`).
- Tests that inject hook logic (telemetry, validation) exercised during
  flag evaluation.
- Teams migrating between flag providers who want tests that survive the
  swap.
- Assignment-integrity tests per `ab-test-validity-checklist`.

## How to use

### TypeScript / Node.js

**Install** (per
[github.com/open-feature/js-sdk README](https://github.com/open-feature/js-sdk/blob/main/packages/server/README.md)):

```bash
npm install --save @openfeature/server-sdk
```

**Configure the InMemoryProvider** with flag variants and a default
variant:

```typescript
import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

const flags = {
  'show-new-ui': {
    variants: { on: true, off: false },
    disabled: false,
    defaultVariant: 'on',
  },
  'checkout-v2': {
    variants: { enabled: true, disabled: false },
    disabled: false,
    defaultVariant: 'disabled',
  },
} as const;

await OpenFeature.setProvider(new InMemoryProvider(flags));
const client = OpenFeature.getClient();
```

**Evaluate flags** using the typed evaluation API (per
[openfeature.dev/docs/reference/concepts/evaluation-api](https://openfeature.dev/docs/reference/concepts/evaluation-api)):

```typescript
// Returns the resolved value; falls back to default on error
const enabled = await client.getBooleanValue('show-new-ui', false);

// Returns full EvaluationDetails
const details = await client.getBooleanDetails('show-new-ui', false);
// details.value    - the resolved boolean
// details.variant  - e.g. "on"
// details.reason   - e.g. "STATIC", "DEFAULT", "TARGETING_MATCH"
// details.errorCode- e.g. "FLAG_NOT_FOUND" when the flag is absent
```

**Typical test pattern:**

```typescript
import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

describe('checkout feature', () => {
  let client: Client;

  beforeAll(async () => {
    await OpenFeature.setProvider(new InMemoryProvider({
      'checkout-v2': {
        variants: { enabled: true, disabled: false },
        disabled: false,
        defaultVariant: 'disabled',
      },
    }));
    client = OpenFeature.getClient();
  });

  afterAll(() => OpenFeature.close());

  it('returns false when flag defaults to disabled', async () => {
    const value = await client.getBooleanValue('checkout-v2', false);
    expect(value).toBe(false);
  });

  it('returns details with reason STATIC for static flags', async () => {
    const details = await client.getBooleanDetails('checkout-v2', false);
    expect(details.reason).toBe('STATIC');
  });

  it('returns FLAG_NOT_FOUND error code for unknown flag', async () => {
    const details = await client.getBooleanDetails('unknown-flag', false);
    expect(details.errorCode).toBe('FLAG_NOT_FOUND');
  });
});
```

### Java

**Install** (Maven, per
[github.com/open-feature/java-sdk README](https://github.com/open-feature/java-sdk/blob/main/README.md)):

```xml
<dependency>
  <groupId>dev.openfeature</groupId>
  <artifactId>sdk</artifactId>
  <version>1.20.2</version>
</dependency>
```

**Configure the InMemoryProvider:**

```java
import dev.openfeature.sdk.OpenFeatureAPI;
import dev.openfeature.sdk.Client;
import dev.openfeature.sdk.providers.memory.Flag;
import dev.openfeature.sdk.providers.memory.InMemoryProvider;

Map<String, Flag<?>> flags = new HashMap<>();
flags.put("show-new-ui", Flag.builder()
    .variant("on", true)
    .variant("off", false)
    .defaultVariant("on")
    .build());

OpenFeatureAPI api = OpenFeatureAPI.getInstance();
api.setProviderAndWait(new InMemoryProvider(flags));
Client client = api.getClient();
```

**Evaluate flags:**

```java
boolean enabled = client.getBooleanValue("show-new-ui", false);

FlagEvaluationDetails<Boolean> details =
    client.getBooleanDetails("show-new-ui", false);
// details.getValue()    - resolved value
// details.getVariant()  - "on" or "off"
// details.getReason()   - "STATIC", "DEFAULT", etc.
// details.getErrorCode()- ErrorCode.FLAG_NOT_FOUND, TYPE_MISMATCH, etc.
```

### Python

**Install** (per
[github.com/open-feature/python-sdk README](https://github.com/open-feature/python-sdk/blob/main/README.md)):

```bash
pip install openfeature-sdk==0.10.0
```

**Configure the InMemoryProvider:**

```python
from openfeature import api
from openfeature.provider.in_memory_provider import InMemoryFlag, InMemoryProvider

flags = {
    "show-new-ui": InMemoryFlag(
        default_variant="on",
        variants={"on": True, "off": False}
    ),
}

api.set_provider_and_wait(InMemoryProvider(flags))
client = api.get_client()
```

**Evaluate flags:**

```python
enabled = client.get_boolean_value("show-new-ui", False)

details = client.get_boolean_details("show-new-ui", False)
# details.value       - resolved value
# details.variant     - "on" / "off"
# details.reason      - "STATIC", "DEFAULT", etc.
# details.error_code  - "FLAG_NOT_FOUND", "TYPE_MISMATCH", etc.
```

## EvaluationDetails and reason codes

Per the OpenFeature specification
([openfeature.dev/specification/sections/flag-evaluation](https://openfeature.dev/specification/sections/flag-evaluation),
Requirements 1.4.3-1.4.15), `EvaluationDetails` carries:

| Field | Type | Meaning |
|---|---|---|
| `value` | T | Resolved flag value (spec req. 1.4.3) |
| `flagKey` | string | The requested flag identifier (1.4.5) |
| `variant` | string | Provider-supplied variant name (1.4.6) |
| `reason` | string | Resolution rationale (1.4.7) |
| `errorCode` | enum | Failure classification (1.4.8) |
| `errorMessage` | string | Optional context for the error (1.4.13) |
| `flagMetadata` | map | Immutable provider-supplied data (1.4.14) |

Canonical reason values (per
[openfeature.dev/specification/sections/providers](https://openfeature.dev/specification/sections/providers),
Requirement 2.2.5):
`STATIC`, `DEFAULT`, `TARGETING_MATCH`, `SPLIT`, `CACHED`,
`DISABLED`, `UNKNOWN`, `STALE`, `ERROR`.

Canonical error codes include `FLAG_NOT_FOUND`, `TYPE_MISMATCH`,
`PARSE_ERROR`, `TARGETING_KEY_MISSING`, `INVALID_CONTEXT`, `GENERAL`,
`PROVIDER_NOT_READY`, `PROVIDER_FATAL`.

## Evaluation context for targeting-rule tests

Evaluation context is "a container for arbitrary contextual data that
can be used as a basis for dynamic evaluation" (per
[openfeature.dev/docs/reference/concepts/evaluation-context](https://openfeature.dev/docs/reference/concepts/evaluation-context)).
The targeting key is a unique identifier (user ID, session ID) that
providers use for deterministic bucketing. Custom attributes carry
additional data (email, plan, region).

Context can be set at three levels: global (via the API object),
client, and per invocation. Lower levels override duplicate keys from
higher levels.

```typescript
// TypeScript - per-invocation context for targeting-rule tests
const ctx = { targetingKey: 'user-42', email: 'user@example.com' };
const details = await client.getBooleanDetails('beta-access', false, ctx);
expect(details.reason).toBe('TARGETING_MATCH');
```

```python
# Python - per-invocation context
from openfeature.evaluation_context import EvaluationContext
ctx = EvaluationContext(targeting_key="user-42",
                        attributes={"email": "user@example.com"})
details = client.get_boolean_details("beta-access", False, ctx)
```

```java
// Java - per-invocation context
Map<String, Value> attrs = new HashMap<>();
attrs.put("email", new Value("user@example.com"));
EvaluationContext ctx = new ImmutableContext("user-42", attrs);
boolean value = client.getBooleanValue("beta-access", false, ctx);
```

## Hooks for test-time side-effects

Hooks intercept the flag evaluation lifecycle. Per
[openfeature.dev/docs/reference/concepts/hooks](https://openfeature.dev/docs/reference/concepts/hooks)
and the specification
([openfeature.dev/specification/sections/hooks](https://openfeature.dev/specification/sections/hooks),
Requirement 4.3.1-4.3.8), the four stages are:

| Stage | Runs when |
|---|---|
| `before` | Before flag resolution; can modify evaluation context |
| `after` | After successful resolution; can validate the returned value |
| `error` | On resolution failure or unhandled before-hook error |
| `finally` | Unconditionally after all other stages |

Execution order for `before`: API - Client - Invocation. For `after`,
`error`, `finally`: reverse order (Invocation - Client - API), per
specification Requirement 4.4.2.

Register hooks at global, client, or invocation level:

```typescript
// Global hook - runs for every flag evaluation
OpenFeature.addHooks({
  before(ctx) {
    // ctx carries flagKey, flagValueType, defaultValue, evaluationContext
    console.log(`Evaluating ${ctx.flagKey}`);
  },
  after(ctx, details) {
    // details is the EvaluationDetails for this evaluation
    expect(details.errorCode).toBeUndefined();
  },
  error(ctx, err) {
    console.error(`Flag ${ctx.flagKey} failed: ${err.message}`);
  },
});
```

Test use case: attach an `after` hook to assert that no evaluation
returns an error code, surfacing `FLAG_NOT_FOUND` regressions across
the entire test run without asserting each flag individually.

## Anti-patterns

| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Registering a real (networked) provider in unit tests | Network calls; non-deterministic; slow | `InMemoryProvider` for unit tests |
| Evaluating without `setProviderAndWait` / `await setProvider` | Returns default with `PROVIDER_NOT_READY` error code before init | Always await provider readiness |
| Sharing a single `InMemoryProvider` instance across test files | Cross-test state pollution | Create a fresh provider per describe block |
| Asserting only `value`; ignoring `reason` and `errorCode` | Hides fallback-to-default failures (flag absent, type mismatch) | Assert `details.reason` and `details.errorCode` explicitly |
| Testing provider internals (variant weighting, targeting logic) | That is the provider's responsibility, not the application's | Test what the application does with the evaluated value |
| Omitting `OpenFeature.close()` in teardown | Leaks provider state and background threads | Always call `close()` / `shutdown()` in `afterAll` |

## Limitations

- `InMemoryProvider` does not replicate production targeting rules.
  To test that a real provider evaluates a segment correctly, write an
  integration test against that provider's native test harness. Consult
  the sibling skills (`launchdarkly-testing`, `unleash-testing`, etc.)
  for those patterns.
- Client-side OpenFeature SDKs (`@openfeature/web-sdk`) use a
  different `setContext` pattern; this skill targets server-side SDKs.
- Hook `before` stages can modify the evaluation context in some SDK
  versions but the specification marks this as optional behavior - verify
  against your SDK version.
- `InMemoryProvider` does not fire PROVIDER_CHANGED events on `putIfAbsent`
  in all SDK versions; check release notes when relying on event-driven
  updates in tests.

## References

- Evaluation API concept:
  [openfeature.dev/docs/reference/concepts/evaluation-api](https://openfeature.dev/docs/reference/concepts/evaluation-api)
- Provider concept and `setProvider`:
  [openfeature.dev/docs/reference/concepts/provider](https://openfeature.dev/docs/reference/concepts/provider)
- Evaluation context:
  [openfeature.dev/docs/reference/concepts/evaluation-context](https://openfeature.dev/docs/reference/concepts/evaluation-context)
- Hooks concept:
  [openfeature.dev/docs/reference/concepts/hooks](https://openfeature.dev/docs/reference/concepts/hooks)
- OpenFeature specification (flag evaluation):
  [openfeature.dev/specification/sections/flag-evaluation](https://openfeature.dev/specification/sections/flag-evaluation)
- OpenFeature specification (providers):
  [openfeature.dev/specification/sections/providers](https://openfeature.dev/specification/sections/providers)
- OpenFeature specification (hooks):
  [openfeature.dev/specification/sections/hooks](https://openfeature.dev/specification/sections/hooks)
- TypeScript/Node SDK:
  [github.com/open-feature/js-sdk](https://github.com/open-feature/js-sdk/blob/main/packages/server/README.md)
- Java SDK:
  [github.com/open-feature/java-sdk](https://github.com/open-feature/java-sdk/blob/main/README.md)
- Python SDK:
  [github.com/open-feature/python-sdk](https://github.com/open-feature/python-sdk/blob/main/README.md)
- Companion catalogs:
  `feature-flag-test-matrix-reference`,
  `flag-state-coverage-builder`
- Vendor-specific skills:
  `launchdarkly-testing`,
  `unleash-testing`,
  `flagsmith-testing`,
  `growthbook-testing`
- Cross-plugin:
  `feature-flag-test-harness`,
  `ab-test-validity-checklist`