Releasing Features Behind Feature Flags: Automate with Playwright and LaunchDarkly

In today’s fast-paced agile environments, selectively rolling out new features is more than a convenience—it’s a necessity. Feature flags provide development teams with the ability to decouple deployment from release, enabling practices like gradual rollouts, experimentation, and instant rollback. However, this flexibility introduces new complexities in testing.

In this Feature Flag Testing Ultimate Guide post, we’ll explore the benefits of using feature flags, dive into the challenges they present during testing, and demonstrate how to automate this process using Playwright for E2E tests and LaunchDarkly for feature flag management.

Why Use Feature Flags?

Feature flags (also known as feature toggles) offer a strategic advantage in modern software delivery by allowing teams to:

✅ Incrementally Roll Out Features

Release functionality to a limited subset of users before a full rollout. This staged deployment reduces risk and allows you to gather real-world feedback early.

✅ Minimize Deployment Risk

If an issue is detected, the feature can be turned off instantly—without redeploying the application. This provides a safety net for production environments.

✅ Run A/B Tests and Experiments

Feature flags allow you to test multiple variations of a feature on different user segments. Use this to validate product hypotheses and optimize performance.

✅ Accelerate Development

Teams can merge incomplete features behind flags, allowing for continuous integration without blocking the main release pipeline.

✅ Enable Business-Controlled Releases

Non-technical stakeholders can enable or disable features dynamically, without needing engineering involvement or triggering a deployment.

Testing Challenges with Feature Flags

While feature flags unlock flexibility, they introduce several testing complexities:

🔁 Multiple States and Variations

Each flag can be enabled, disabled, or have multiple variations—leading to a combinatorial explosion of test cases.

🌐 Configuration Drift

Flag settings may differ across environments (dev, staging, prod), resulting in inconsistent behavior if not managed carefully.

⏱️ Asynchronous Updates

Changes to flags may take time to propagate, leading to flaky or inconsistent tests if this is not handled correctly.

🔌 Third-Party Integration Complexity

When using services like LaunchDarkly, you add an extra layer of integration. You must validate flag behavior, rules, and targeting logic across different contexts.

Automating Feature Flag Testing with Playwright & LaunchDarkly

Automation is key to managing the complexity of feature flag testing. By combining Playwright for E2E testing and LaunchDarkly's API for dynamic flag configuration, you can build a reliable, repeatable workflow that ensures test consistency and reduces risk.

Assumptions

Feature flags are configured per user to maintain isolation.
We use launchdarkly-api-typescript and Playwright’s API to control flags programmatically.

E2E Testing Rules for Feature Flags

Test one toggle at a time - Keep tests isolated to a single flag to ensure clarity and proper coverage.
Write tests alongside flag implementation - Don’t wait—create automated tests as soon as a new flag is introduced.
Test flagged features with a user where the flag is ON - Validate new functionality with specific test accounts that have the flag enabled.
Test existing functionality with the flag OFF - Ensure that the feature being tested doesn't affect the default experience for users without the flag.
Remove flags and related tests when deprecated - Clean up retired feature flags and their associated test logic to avoid noise and confusion in your test suite.

Code Walkthrough: Automating Feature Flag Setup

Here’s a breakdown of how you can dynamically manage feature flags and validate their behavior in tests:

Step 1: Initialize the API Context

We begin by initializing the API context with Playwright’s request library, setting up headers and authentication for LaunchDarkly:

async function initializeApiContext(): Promise<APIRequestContext> {
return await request.newContext({
baseURL: 'https://app.launchdarkly.com/api/v2',
extraHTTPHeaders: {
Authorization: `Bearer ${process.env.LAUNCHDARKLY_API_KEY}`,
'Content-Type': 'application/json',
},
});
} — Don't forget about storing your LaunchDarkly API key

🔐 Handling the LaunchDarkly API Key

To interact with LaunchDarkly’s API, you'll need a personal or service account access token with appropriate permissions (usually Writer or Admin).

Here are some best practices:

Store it securely: Never hard-code the API key in your test scripts or commit it to version control. Use environment variables instead:
Use restricted tokens: Create a token with only the necessary access level and scope. For testing environments, limit its access to specific projects or environments if possible.
Access in code: Reference it securely in your Playwright setup
$Authorization: `Bearer ${process.env.LAUNCHDARKLY_API_KEY}`$
Rotate regularly: Periodically regenerate and rotate your API tokens to reduce the risk of credential leaks.

⚠️ If you're using CI/CD pipelines (e.g., GitHub Actions, GitLab CI), store your API key as a secret in the platform's secret manager.

Step 2: Validate and Ensure the Correct Flag Configuration

Before your test begins, validate that the flag is configured correctly. If not, update it:

export async function validateFeatureFlag(flagKey: string, userId: string, requiredVariation: boolean): Promise<void> {
await initializeApiContext();
const userEmail = `${TEST_EMAIL_ACCOUNT}${userId}-`;
const E2E_RULE_DESCRIPTION = `${E2E_RULE_DESCRIPTION_PREFIX} ${userEmail}`;

const url = `https://app.launchdarkly.com/api/v2/flags/${projectKey}/${flagKey}?env=${environmentKey}&expand=migrationSettings`;
const response = await apiContext.get(url, {
headers: {
Authorization: `${apiToken}`,
'Content-Type': 'application/json',
},
});

const featureFlag: FeatureFlag = await response.json();

const variationName = requiredVariation ? true : false;
const variation = featureFlag.variations.find((v: Variation) => v.value === variationName);
if (!variation) {
console.log('Variation not found.');
return;
}

// Validate the presence of required attributes
const environment = featureFlag.environments?.[environmentKey];
if (!environment || !environment.rules || environment.rules.length === 0) {
console.warn('No rules defined for this feature flag.');
await addFeatureFlagRule(flagKey, userEmail, variation._id!);
return;
}

const rule = environment.rules.find((r: Rule) => r.description === E2E_RULE_DESCRIPTION);
if (!rule) {
console.warn('E2E test setup rule not found.');
await addFeatureFlagRule(flagKey, userEmail, variation._id!);
return;
}

const clause = rule.clauses.find((c: Clause) => c.attribute === 'email' && c.op === 'startsWith');
if (!clause || !clause.values.includes(userEmail) || clause.contextKind !== 'user' || clause.negate !== false) {
console.error('Required clause value not found in feature flag rules.');
await addFeatureFlagRule(flagKey, userEmail, variation._id!);
return;
}
}

Step 3: Add a Rule to a Feature Flag

The following function adds a targeting rule for a specific feature flag. This example targets users whose email starts with a certain prefix—ideal for isolating test accounts:

const addFeatureFlagRule = async (flagKey: string, userEmail: string, variationId: string): Promise<void> => {
const E2E_RULE_DESCRIPTION = `${E2E_RULE_DESCRIPTION_PREFIX} ${userEmail}`;
const url = `https://app.launchdarkly.com/api/v2/flags/${projectKey}/${flagKey}?ignoreConflicts=true`;
const payload = {
environmentKey: environmentKey,
instructions: [
{
kind: 'addRule',
variationId: variationId,
description: E2E_RULE_DESCRIPTION,
clauses: [
{
contextKind: 'user',
attribute: 'email',
op: 'startsWith',
negate: false,
values: [userEmail],
},
],
},
],
};

const response = await apiContext.patch(url, {
headers: {
'Content-Type': 'application/json; domain-model=launchdarkly.semanticpatch',
Authorization: `${apiToken}`,
},
data: payload,
});

if (response.status() === 200) {
console.log('Successfully added rule to feature flag.');
} else {
console.error(`Failed to add rule to feature flag: ${response.status()}`);
}
};

Step 4. Integration with E2E Tests

Make sure you run the following before running the test dependent on the feature flag:

const apiContext = await initializeApiContext();

if (user.featureFlags && user.featureFlags.length > 0) {
for (const flag of user.featureFlags) {
const { flagKey, requiredVariation } = flag;

try {
await validateFeatureFlag(apiContext, flagKey, requiredVariation, user.id);
} catch (error) {
console.error(`Error validating feature flag ${flagKey}:`, error);
}
}
} else {
console.warn('No feature flags to validate.');
}

This way, any issues related to feature flag misconfigurations are caught early in the testing cycle, reducing the risk of bad deployments.

Conclusion

Feature flags are a powerful tool in modern continuous delivery processes, offering significant flexibility but also introducing intricate testing challenges. By automating the setup and validation of feature flags with Playwright and LaunchDarkly, teams can achieve a smoother, more reliable testing process. The provided code demonstrates one approach to dynamically managing feature flags in your testing suite, ensuring your applications behave correctly under different configurations.

Embracing such automated strategies can help bridge the gap between robust feature delivery and quality assurance, ultimately leading to faster, safer, and more agile deployments.

Releasing Features Behind Feature Flags: Automate with Playwright and LaunchDarkly - Feature Flag Testing Ultimate Guide