How could JQF handle environment variables & system properties from Maven Surefire?

[shuaiwang516]

Some tests will run with environment variables and system properties that are set in Maven Surefire from the Maven pom.xml file.

However, those environment variables and system properties are not considered by JQF.

A very simple example:

  public void testA(String s) {
    String s1 = System.getProperty("test.prefix") + s;  
    assertEquals("test-" + s, s1);
  }

In pom.xml file:

<build>
  <plugins>
      ...
      <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <configuration>
              <systemPropertyVariables>
                  <test.prefix>test-</test.prefix>       <!-- test.prefix = "test-" -->
              </systemPropertyVariables>
          </configuration>
      </plugin>
      ...
  </plugins>
</build>

This simple example can pass with mvn test but will fail with JQF because JQF does not set the system property.
We can fix this by simply adding -Dtest.prefix=test-, but would it be better to make a configurable feature so that JQF can also set the env. vars. and sys. properties from those are set in Maven Surefire?

[rohanpadhye]

That's an interesting observation! Thanks for the example.

I agree that the JQF plugin will not pick up these properties. The reason is simple: mvn jqf:fuzz runs a different plugin than Surefire. While the JQF test framework depends on JUnit and picks up all JUnit configurations (such as the @Before/@After annotations and others), the JQF Maven Plugin is not related to the Surefire plugin. I don't even know if Maven plugins can have inheritance relationships. So, any Surefire-specific configurations will not carry over to JQF.

There are many ways to get around this issue. Simply specify the properties in a different area in the pom.xml; that is, not specific to Surefire. For example, you can directly put a <properties> tag under <project>. If you want the properties to only be set during testing, you can use a <profile> that is set only when running tests. I think there is also a Maven Properties Plugin that can be configured to bind to a particular Maven phase, where you also run JQF.

If you know of a way to have the JQF plugin inherit configuration from Surefire, do let me know.