Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: In-memory provider for e2e testing and minimal usage #546

Merged
merged 15 commits into from
Aug 15, 2023
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ public static <T> FlagEvaluationDetails<T> from(ProviderEvaluation<T> providerEv
.value(providerEval.getValue())
.variant(providerEval.getVariant())
.reason(providerEval.getReason())
.errorMessage(providerEval.getErrorMessage())
Kavindu-Dodan marked this conversation as resolved.
Show resolved Hide resolved
.errorCode(providerEval.getErrorCode())
.flagMetadata(providerEval.getFlagMetadata())
.build();
Expand Down
30 changes: 27 additions & 3 deletions src/test/java/dev/openfeature/sdk/e2e/StepDefinitions.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,17 @@
import dev.openfeature.sdk.Reason;
import dev.openfeature.sdk.Structure;
import dev.openfeature.sdk.Value;
import dev.openfeature.sdk.testutils.Flag;
import dev.openfeature.sdk.testutils.Flags;
import dev.openfeature.sdk.testutils.InMemoryProvider;
import io.cucumber.java.BeforeAll;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import lombok.SneakyThrows;

import java.io.File;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Map;

Expand Down Expand Up @@ -47,13 +53,29 @@ public class StepDefinitions {
private int typeErrorDefaultValue;
private FlagEvaluationDetails<Integer> typeErrorDetails;

@SneakyThrows
@BeforeAll()
@Given("an openfeature client is registered with cache disabled")
public static void setup() {
// TODO: when the FlagdProvider is updated to support caching, we might need to disable it here for this test to work as expected.
FlagdProvider provider = new FlagdProvider();
provider.setDeadline(3000); // set a generous deadline, to prevent timeouts in actions

Map<String, Flag> flagsMap = new HashMap<>();
Map<String, Object> variants = new HashMap<>();
variants.put("on", true);
variants.put("off", false);
ClassLoader classLoader = StepDefinitions.class.getClassLoader();
File file = new File(classLoader.getResource("features/testing-flags.json").getFile());
Path resPath = file.toPath();
String conf = new String(java.nio.file.Files.readAllBytes(resPath), "UTF8");
Flags flags = Flags.builder().setConfigurationJson(conf).build();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unused and unnecessary 🤔 Was the intention here to fail fast with flag configuration validation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover, will remove it

InMemoryProvider provider = new InMemoryProvider(conf);
OpenFeatureAPI.getInstance().setProvider(provider);

/*
TODO: setProvider with wait for init, pending https://github.com/open-feature/ofep/pull/80
*/
Thread.sleep(500);

client = OpenFeatureAPI.getInstance().getClient();
}

Expand Down Expand Up @@ -233,7 +255,9 @@ public void an_a_flag_with_key_is_evaluated(String flagKey, String defaultValue)

@Then("the resolved string response should be {string}")
public void the_resolved_string_response_should_be(String expected) {
assertEquals(expected, this.contextAwareValue);

// TODO: targeting context not supported at InMemoryProvider
// assertEquals(expected, this.contextAwareValue);
}

@Then("the resolved flag value is {string} when the context is empty")
Expand Down
18 changes: 18 additions & 0 deletions src/test/java/dev/openfeature/sdk/testutils/Flag.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package dev.openfeature.sdk.testutils;

import io.cucumber.core.internal.com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.ToString;

import java.util.Map;

@ToString
@JsonIgnoreProperties(ignoreUnknown = true)
@NoArgsConstructor
@Getter
public class Flag {
private Flags.State state;
private Map<String, Object> variants;
private String defaultVariant;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose adding a context evaluator backed by a user-provided callback

See go-sdk implementation for example - https://github.com/open-feature/go-sdk/blob/main/pkg/openfeature/memprovider/in_memory_provider.go#L167-L175

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand what is the purpose and value here with enhancing with additional testing capabilities. The in-memory provider is for testing purposes, how is adding a context evaluator helping with testing actual flow ? it will only test the testing provider ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation resides as a test source. This mismatch with the requirement of the specification.

When we agreed on the in-memory provider through OFEP and added it to the specification 1, the agreement was to make the in-memory an extra provider built into the SDK itself. And we agreed to support evaluation contexts through lambda/callback.

So the packaging of the implementation should be corrected first. I am proposing to move it to a package named memprovider or any better-named package.

Regarding the purpose, the main benefit of having evaluation context support is testing SDK. It allows us to write end-to-end tests (or to migrate existing ones based on flagd) for context evaluations and verify SDK correctness. Besides, end users can use the provider to prototype OpenFeature features.

Footnotes

  1. https://github.com/open-feature/spec/blob/main/specification/appendix-A.md#in-memory-provider

55 changes: 55 additions & 0 deletions src/test/java/dev/openfeature/sdk/testutils/Flags.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package dev.openfeature.sdk.testutils;

import io.cucumber.core.internal.com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import io.cucumber.core.internal.com.fasterxml.jackson.core.JsonProcessingException;
import io.cucumber.core.internal.com.fasterxml.jackson.databind.ObjectMapper;
import lombok.Getter;
import lombok.ToString;

import java.util.Map;

@ToString
@JsonIgnoreProperties(ignoreUnknown = true)
@Getter
public class Flags {

public static class FlagsBuilder {

private String configurationJson;

private ObjectMapper objectMapper = new ObjectMapper();

private FlagsBuilder() {

}

public FlagsBuilder setConfigurationJson(String configurationJson) {
this.configurationJson = configurationJson;
return this;
}

public Flags build() throws JsonProcessingException {
return objectMapper.readValue(configurationJson, Flags.class);
}

}

public static FlagsBuilder builder() {
return new FlagsBuilder();
}

private Map<String, Flag> flags;

public enum State {
ENABLED, DISABLED
}

public enum Variant {
on, off
}
Kavindu-Dodan marked this conversation as resolved.
Show resolved Hide resolved

@Getter
public class Variants {
private Map<String, Object> variants;
}
}
181 changes: 181 additions & 0 deletions src/test/java/dev/openfeature/sdk/testutils/InMemoryProvider.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
package dev.openfeature.sdk.testutils;

import dev.openfeature.sdk.*;
import lombok.Getter;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;

/**
* In-memory provider.
*
* Based on flagd configuration.
*/
@Slf4j
public class InMemoryProvider implements FeatureProvider {

@Getter
private final String name = "InMemoryProvider";

private Flags flags;

private String jsonConfig;

@Getter
private ProviderState state = ProviderState.NOT_READY;

@Override
public Metadata getMetadata() {
return new Metadata() {
@Override
public String getName() {
return name;
}
};
}

public InMemoryProvider(String jsonConfig) {
this.jsonConfig = jsonConfig;
}

public void initialize(EvaluationContext evaluationContext) throws Exception {
FeatureProvider.super.initialize(evaluationContext);
this.flags = Flags.builder().setConfigurationJson(jsonConfig).build();
state = ProviderState.READY;
log.info("finishing initializing provider, state: {}", state);
}

@Override
public ProviderEvaluation<Boolean> getBooleanEvaluation(String key, Boolean defaultValue, EvaluationContext ctx) {
Flag flag = flags.getFlags().get(key);
if (flag == null) {
return ProviderEvaluation.<Boolean>builder()
.value(defaultValue)
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.FLAG_NOT_FOUND.name())
.errorCode(ErrorCode.FLAG_NOT_FOUND)
.build();
}
if (!(flag.getVariants().get(flag.getDefaultVariant()) instanceof Boolean)) {
return ProviderEvaluation.<Boolean>builder()
.value(defaultValue)
.variant(flag.getDefaultVariant())
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.TYPE_MISMATCH.name())
.errorCode(ErrorCode.TYPE_MISMATCH)
.build();
}
boolean value = (boolean) flag.getVariants().get(flag.getDefaultVariant());
return ProviderEvaluation.<Boolean>builder()
.value(value)
.variant(flag.getDefaultVariant())
.reason(Reason.STATIC.toString())
.build();
}

@Override
public ProviderEvaluation<String> getStringEvaluation(String key, String defaultValue, EvaluationContext ctx) {
Flag flag = flags.getFlags().get(key);
if (flag == null) {
ProviderEvaluation<String> providerEvaluation = ProviderEvaluation.<String>builder()
.value(defaultValue)
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.FLAG_NOT_FOUND.name())
.errorCode(ErrorCode.FLAG_NOT_FOUND)
.build();
return providerEvaluation;
}
if (!(flag.getVariants().get(flag.getDefaultVariant()) instanceof String)) {
return ProviderEvaluation.<String>builder()
.value(defaultValue)
.variant(flag.getDefaultVariant())
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.TYPE_MISMATCH.name())
.errorCode(ErrorCode.TYPE_MISMATCH)
.build();
}
String value = (String) flag.getVariants().get(flag.getDefaultVariant());
return ProviderEvaluation.<String>builder()
.value(value)
.variant(flag.getDefaultVariant())
.reason(Reason.STATIC.toString())
.build();
}

@Override
public ProviderEvaluation<Integer> getIntegerEvaluation(String key, Integer defaultValue, EvaluationContext ctx) {
Flag flag = flags.getFlags().get(key);
if (flag == null) {
return ProviderEvaluation.<Integer>builder()
.value(defaultValue)
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.FLAG_NOT_FOUND.name())
.errorCode(ErrorCode.FLAG_NOT_FOUND)
.build();
}
if (!(flag.getVariants().get(flag.getDefaultVariant()) instanceof Integer)) {
return ProviderEvaluation.<Integer>builder()
.value(defaultValue)
.variant(flag.getDefaultVariant())
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.TYPE_MISMATCH.name())
.errorCode(ErrorCode.TYPE_MISMATCH)
.build();
}
Integer value = (Integer) flag.getVariants().get(flag.getDefaultVariant());
return ProviderEvaluation.<Integer>builder()
.value(value)
.variant(flag.getDefaultVariant())
.reason(Reason.STATIC.toString())
.build();
}

@Override
public ProviderEvaluation<Double> getDoubleEvaluation(String key, Double defaultValue, EvaluationContext ctx) {
Flag flag = flags.getFlags().get(key);
if (flag == null) {
return ProviderEvaluation.<Double>builder()
.value(defaultValue)
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.FLAG_NOT_FOUND.name())
.errorCode(ErrorCode.FLAG_NOT_FOUND)
.build();
}
if (!(flag.getVariants().get(flag.getDefaultVariant()) instanceof Double)) {
return ProviderEvaluation.<Double>builder()
.value(defaultValue)
.variant(flag.getDefaultVariant())
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.TYPE_MISMATCH.name())
.errorCode(ErrorCode.TYPE_MISMATCH)
.build();
}
Double value = (Double) flag.getVariants().get(flag.getDefaultVariant());
return ProviderEvaluation.<Double>builder()
.value(value)
.variant(flag.getDefaultVariant())
.reason(Reason.STATIC.toString())
.build();
}

@SneakyThrows
@Override
public ProviderEvaluation<Value> getObjectEvaluation(String key, Value defaultValue,
EvaluationContext invocationContext) {
Flag flag = flags.getFlags().get(key);
if (flag == null) {
return ProviderEvaluation.<Value>builder()
.value(defaultValue)
.reason(Reason.ERROR.toString())
.errorMessage(ErrorCode.FLAG_NOT_FOUND.name())
.errorCode(ErrorCode.FLAG_NOT_FOUND)
.build();
}
Object object = flag.getVariants().get(flag.getDefaultVariant());
Value value = ValueUtils.convert(object);
return ProviderEvaluation.<Value>builder()
.value(value)
.variant(flag.getDefaultVariant())
.reason(Reason.STATIC.toString())
.build();
}
}
Loading
Loading