1.0 release thread

[ptrthomas]

this issue is to provide updates and collect feedback. also will hold some images needed for the wiki

this wiki page will hold details: https://github.com/intuit/karate/wiki/1.0-upgrade-guide

[ptrthomas]

• ES6 arrow functions
• string interpolation within back-ticks
• JS array behavior e.g. Array.map(), note that karate.map() is still an option

[ptrthomas]

• each level of the payload is reported as well-formed JSON (to make cutting and pasting easier)
• JSON keys will be sorted so that the payloads "line up" as far as possible - which makes it easier to eyeball
• the Json-Path (or XPath) of each "level" appears on the left, with a clear description of the mis-match

[hujunhaorobert]

@ptrthomas I had rebuilt the rewrite branch locally and generated karate-2.0.0.jar, and below is test finding:

1. the fixing solution for issue bodyPath('/Not/Existing/Path') throws java.lang.NullPointerException with stack trace: com.intuit.karate.core.ScriptBridge.xmlPath(ScriptBridge.java:421) #1365 I reported is not integrated to rewrite branch yet and the issue is still there.
   java.lang.NullPointerException stack trace: com.intuit.karate.core.ScriptBridge.xmlPath(ScriptBridge.java:426)
2. When building Standalone JAR and ZIP, under karate-netty folder, launch "mvn install -P fatjar", noticed there are some client testing.feature failures, Can you check and fix it?
   e.g.
   18:05:59.883 [main] DEBUG com.intuit.karate - abort at classpath:com/intuit/karate/client.feature:73
   18:05:59.883 [main] INFO c.intuit.karate.netty.FeatureServer - stop: shutting down
   18:05:59.884 [main] INFO c.intuit.karate.netty.FeatureServer - stop: shutdown complete
   18:05:59.884 [main] INFO com.intuit.karate.netty.ProxyServer - stop: shutting down
   18:05:59.885 [main] INFO com.intuit.karate.netty.ProxyServer - stop: shutdown complete
   [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.762 s <<< FAILURE! - in com.intuit.karate.ProxyServerSslTest
   [ERROR] testProxy(com.intuit.karate.ProxyServerSslTest) Time elapsed: 0.747 s <<< ERROR!
   com.intuit.karate.exception.KarateException:
   client.feature:11 - io/netty/handler/codec/http/HttpUtil
   client.feature:60 - io/netty/handler/codec/http/HttpUtil
   client.feature:66 - io/netty/handler/codec/http/HttpUtil
   at com.intuit.karate.ProxyServerSslTest.testProxy(ProxyServerSslTest.java:61)

The same issue also found when I use the karate-2.0.0.jar to run local karate API testing against karate mock server.

[ptrthomas]

@hujunhaorobert thanks for trying out the rewrite branch. I have made changes for bodyPath() to work in the new code: 6ce22ce

I also suspect you may not have switched cleanly to the rewrite branch, there should not be anything under karate-netty anymore, it is all moved to karate-core - anyway please see if the mocks are ok

if you still have trouble, I'll try to get a better way of trying RC releases, but I don't want to release JAR files to maven just yet

[ptrthomas]

passing JS functions top Java code that will be executed on another thread has to be wrapped using karate.toJava()

this may still have issues, in that case - last resort is to move the whole flow to Java

refer oracle/graaljs#59

[hujunhaorobert]

@ptrthomas Yes, I met some trouble to get standalone karate-{version}.jar for my mock testing. In order to regenerate that JAR, I was following the Developer Guide of Build Standalone JAR and ZIP from https://github.com/intuit/karate/wiki/Developer-Guide. I see there is nothing except README.md under karate-netty now, can you specify how to generate Karate standalone JAR, or how can I get a RC release? Thanks!

[ptrthomas]

@hujunhaorobert since all the code is in karate-core it is a little simpler.

• navigate to the karate-core directory in a terminal / console
• mvn clean install -P fatjar
• if needed add -DskipTests to the above
• the JAR and ZIP will be available in karate-core/target

[ptrthomas]

breaking change - but this was the best way to solve the js threading problem

• karate.listen() is gone
• instead there is a new DSL keyword listen that just takes a number (expressions also allowed)
• to get the result (what was passed from karate.signal()) use the "magic variable" called listenResult
  above is the example of the before--after for the message queue test, which now is green in CI

[hujunhaorobert]

@ptrthomas By following your instruction above, I re-generated the karate-2.0.0.jar successfully, which is good.

I copy this jar file to my mockTest project, and put my customised karate-config.js file under the mockTest project folder, same as main-mock-server.feature. When I launching my mock server by CLI:
java -cp karate-2.0.0.jar com.intuit.karate.Main -m main-mock-server.feature -p {{port-id}}

The program throw an exception when running below line(11) in main-mock-server.feature

• call read('classpath:karate-config.js')

Console log is as below:

12:08:41.630 [main]  INFO  com.intuit.karate.Main - Karate version: 2.0.0
12:08:43.299 [main]  ERROR com.intuit.karate - mock-server background failed - main-mock-server.feature:11
com.intuit.karate.KarateException: mock-server background failed - main-mock-server.feature:11
        at com.intuit.karate.core.MockHandler.<init>(MockHandler.java:103)
        at com.intuit.karate.core.MockServer$Builder.build(MockServer.java:112)
        at com.intuit.karate.Main.call(Main.java:221)
        at com.intuit.karate.Main.call(Main.java:47)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
        at picocli.CommandLine.access$1200(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
        at picocli.CommandLine.execute(CommandLine.java:2058)
        at com.intuit.karate.Main.main(Main.java:145)
Caused by: com.intuit.karate.KarateException: >>>> js failed:
01: read('classpath:karate-config.js')
<<<<
org.graalvm.polyglot.PolyglotException: not found: karate-config.js
- com.intuit.karate.resource.ResourceUtils.getResource(ResourceUtils.java:111)
- com.intuit.karate.core.ScenarioFileReader.toResource(ScenarioFileReader.java:122)
- com.intuit.karate.core.ScenarioFileReader.readFileAsStream(ScenarioFileReader.java:97)
- com.intuit.karate.core.ScenarioFileReader.readFileAsString(ScenarioFileReader.java:93)
- com.intuit.karate.core.ScenarioFileReader.readFile(ScenarioFileReader.java:56)
- com.intuit.karate.core.ScenarioEngine.lambda$new$0(ScenarioEngine.java:134)
- <js>.:program(Unnamed:1)

main-mock-server.feature:11

I had a quick debugging, and noticed that there was a bug when parsing the path in getResource(path), here the path=classpath:karate-config.js.
After calling path = removePrefix(path); the path will be a file name(karate-config.js), rather than a real path, which causes throwing the exception. I will leave the bug with you.

[ptrthomas]

@hujunhaorobert it is likely the previous behavior was a bug. in mocks, there is not really a classpath: concept unless you know what classpath is set when you start the JVM. in this case use a relative path or set the directory that karate will use for finding the karate-config.js as per the documentation. if you still feel this is an issue please provide a sample project

[sumargoraymond]

Returning object literal with karate.call('some_js_file.js') does not work on the RC.

Example js file:

//SampleJS.js
function fn(){
var A = "2";
var B = "3";
return {
varA:A,
varB:B
}

}

Calling the js file's method using karate.call with below code returns undefined:

var variables = karate.callSingle('SampleJS.js");
console.log(variables.varA);

[sumargoraymond]

Calling Java function using obtained variable raises an error.

The following is the sample feature file:

Scenario: Get All Queries
        * def query = karate.call('classpath:schema/GetAllQueries.js')
        * request { query: '#(query)'}
        * method POST
        * def queriesName = $response.data.__schema.queryType.fields[*].name

        * def query = karate.call('classpath:schema/GetAllMutations.js')
        * request { query: '#(query)'}
        * method POST
        * def mutationsName = $response.data.__schema.mutationType.fields[*].name

        * def allQueriesName = karate.append(mutationsName, queriesName)

       # allQueriesName = ['[QueryA, QueryB, QueryC, QueryD, MutationA, MutationB, MutationC, MutationD...']

        * def Utilities = Java.type('features._GQLCoverageTools.FilterQueries')
        * Utilities.printToFile(allQueriesName)

Java file is as below:

package features._GQLCoverageTools;
import com.intuit.karate.graal.JsList;

public class FilterQueries {

    public static void printToFile(JsList array) throws Exception{
        System.out.println(array);
    }
}

The error raised is:

org.graalvm.polyglot.PolyglotException: TypeError: invokeMember (printToFile) on features._GQLCoverageTools.FilterQueries failed due to: Cannot convert '[QueryA, QueryB, QueryC, QueryD, MutationA, MutationB, MutationC, MutationD....'(language: Java, type: com.intuit.karate.graal.JsList) to Java type 'com.intuit.karate.graal.JsList': Unsupported target type.
- <js>.:program(Unnamed:1)

Quite weird considering the data type used is an appropriate one.

[ptrthomas]

@sumargoraymond for your first comment, I cannot replicate: 263ce53

and regarding the second, JsList is not intended for end-users, just use normal Java Map / List / primitives like before, please

[sumargoraymond]

EDITED:

It works well using List

[sumargoraymond]

Hi @ptrthomas , can you try outputting 263ce53 with result.varA instead?

Thanks

[ptrthomas]

@sumargoraymond you are absolutely right ! thanks :) should be fixed now

[ptrthomas]

some improvements to the Runner.builder() that are subtle but you are going to enjoy:

• no more relying on System.setProperty()
• set karate.env
• set even the location of the karate-config.js
• and even set system properties per test-suite
• the goal here is to allow you to run 2 instances of karate.env in parallel

[sumargoraymond]

-Dkarate.options passed from the terminal does not seem to be honored when using hook on test runner.

E.g. running mvn clean test -Dkarate.options="--tags @test" -Dtest=TestRunner -Dkarate.env=production should run test with the annotation: @test

// TestRunner.java
 Results result = Runner.path("classpath:features").hook(new KarateExecutionHook()).tags("~@ignore")
              .parallel(10);

class KarateExecutionHook implements RuntimeHook {
// Here we override some of the before and after hooks
}

The test runner will run tests with the default annotation (~@ignore) instead of the @test from the terminal input.

[ptrthomas]

@sumargoraymond this is a tough one. I'm planning to change the way this works. what I propose is:

• if the user has not specified anything in code, the environment will be used, e.g. command line
• else the code has "higher priority"

in fact the code example in the post just above is an example why this is needed. here, karate.env=docker and server.port will be set by Java code and cannot be changed by setting karate.options

that said - if you want to have this flexibility at run time, you have 2 options:

• create another test runner class (recommended)
• in your runner code, check the environment manually and then call tags("~@ignore")` only if needed

comments from anyone welcome

[ptrthomas]

@sumargoraymond please ignore what I said above, I remembered that frameworks e.g spring / boot give highest priority to command-line, then system properties - and this is the right way to be able to change things at deployment / ci

so I made it work now like it used to, can you please take a look again and confirm

and thanks for trying out the early versions like this - it makes things so much easier 👍

[sumargoraymond]

Sure thing, will do. Thanks @ptrthomas

[ptrthomas]

@sumargoraymond can you provide the values of the request and response headers, specifically the Cookie and Set-Cookie ones

[sumargoraymond]

@ptrthomas Edited: I just tried hitting the same request again and it successfully loaded an xml. Yet on the RC the error is still the same:

io.netty.handler.codec.http.cookie.DefaultCookie.sameSite()Lio/netty/handler/codec/http/cookie/CookieHeaderNames$SameSite;

Cannot seem to find any Cookie nor Set-Cookie on both the request and responses though.

[ptrthomas]

@sumargoraymond unfortunately I can't do anything unless I get some way to replicate, so I'm going to move on to other things. it is possible that your server has a bug and it doesn't support some cookies: https://web.dev/samesite-cookies-explained/

I'm sure you can find a way to replicate. for e.g.

* url 'https://httpbin.org/cookies'
* cookie foo = { value: 'bar', samesite: 'Strict' }
* method get
* status 200
* match response == { cookies : { foo: 'bar', SameSite: 'Strict' } }

* url 'https://widget-pixels.outbrain.com/widget/detect/px.gif?ch=1&rn=1.3815335980445032'
* method get
* status 200
* match responseCookies.akacd_widgets_routing.samesite == 'None'

[sumargoraymond]

My bad @ptrthomas , the server does indeed require a certain key value pair for the cookie. After implementing the necessary cookie on the header, it works well on RC.

[ptrthomas]

@ivangsa does this answer your question ? https://twitter.com/KarateDSL/status/1305144489523048448

[ivangsa]

@ptrthomas yess, we knew this but didn't make the association yet.. thank you!

[ptrthomas]

@ivangsa awesome. also the same trick works for the call read('some.feature') fun where fun can be a function, so this is not restricted to scenario outlines !

to everyone listening - there are SO many changes that I'm not sure if I can add all of those to the readme :(

[Lorkenpeist]

I'm seeing an undesirable warning when my feature calls another feature which happens to use the same variable name. I have 2 feature files:

myFeature.feature

Feature:
  Scenario:
    * def myVar = "some value"
    * match myVar == "some value"

    * def otherVar = call read("getOtherVar.feature")
    * match otherVar.value == "some other value"

    # should not be overwritten by the value defined in getOtherVar
    * match myVar == "some value"

getOtherVar.feature

Feature:
  Scenario:
    * def myVar = "some temporary value"
    * match myVar == "some temporary value"

    * def value = "some other value"

Functionally, this works as expected. When getOtherVar is called, it defines its own value for myVar in its own context, so the value in myFeature's context is not overwritten. However, I am seeing this warning which does not appear in 0.9.6:

15:23:14.855 [pool-1-thread-2] WARN  com.intuit.karate - over-writing existing variable 'myVar' with new value: "some temporary value"

This warning was troubling at first as it made me suspect that a variable could be accidentally overwritten if a called feature happens to use the same variable name. Fortunately it turns out that is not the case, but I do not believe I should be seeing this warning.

[ptrthomas]

@Lorkenpeist I'm going to keep it as it is - but you are welcome to submit a PR. any variable in a calling feature "exists" in called features no matter if it is "shared" or "isolated".

[lhimo]

Has this error emerged yet?

>>>> js failed: 01: new java.math.BigDecimal(database.readValue('select field from table where condition')) <<<< org.graalvm.polyglot.PolyglotException: TypeError: instantiate on java.math.BigDecimal failed due to: no applicable overload found (overloads: [Method[public java.math.BigDecimal(java.math.BigInteger,int)],

[ptrthomas]

@lhimo no and it seems to be something you are doing wrong. follow this process please: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

[ricardgo403]

Hi, I was just trying to test my suites with the 0.9.9.RC4 (2021-02-09) Standalone JAR. I built the jar file following the Developer-Guide and everything was looking good, but when running the scenarios there are errors with the karate-config.js file, especially when reading files. These errors are not present using the standalone jar from v0.9.6.

karate-config.js
>>>> js failed:
01: function fn() {
02:   const env = karate.env
03:   karate.log("karate.env system property was:", env)
04:   if (!env) {
05:     env = "dev" // env can be anything: dev, qa, staging, etc.
06:   }
07:   const settings = read("configs/" + env + ".json")
08: 
09:   var config = {
10:     env: env,
11:     poseidonHostUrl: settings.poseidonHost,
12:     apolloHostUrl: settings.apolloHost,
13:     oneWayTemplateId: settings.oneWayTemplateId,
14:     scratchTemplateId: settings.scratchTemplateId,
15:   }
16: 
17:   if (env == "stg" || env == "dev" || env == "prd") {
18:     config.token = read("resources/token.txt")
19:     config.subOffToken = read("resources/subOffToken.txt")
20:   }
21: 
22:   karate.configure("connectTimeout", 5000)
23:   karate.configure("readTimeout", 5000)
24: 
25:   return config
26: }
<<<<
org.graalvm.polyglot.PolyglotException: java.io.FileNotFoundException: null/configs/dev.json (No such file or directory)
- com.intuit.karate.resource.FileResource.getStream(FileResource.java:94)
- com.intuit.karate.core.ScenarioFileReader.readFileAsStream(ScenarioFileReader.java:101)
- com.intuit.karate.core.ScenarioFileReader.readFileAsString(ScenarioFileReader.java:97)
- com.intuit.karate.core.ScenarioFileReader.readFile(ScenarioFileReader.java:53)
- com.intuit.karate.core.ScenarioEngine.lambda$new$0(ScenarioEngine.java:115)
- <js>.fn(Unnamed:7)

[ptrthomas]

@ricardgo403 I think using the file: prefix would work file:configs/dev.json - but this sounds like a miss in the refactoring, would you be able to follow this process ? https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

[ivangsa]

Hi, --watch option is not supported anymore when starting mocks?

$ java -cp 'target/dependency/*' com.intuit.karate.Main -m mock.feature  -p 8080 --watch
13:36:11.831 [main]  INFO  com.intuit.karate - Karate version: 0.9.9.RC4
Unknown option: '--watch'

I've tried with -w -W and --watch to no avail

[ivangsa]

also in RC4, I'm not sure if it worked before, karate expressions are replaced ok in inlined json and json files but they are not replaced in yml files
I think I have tried with all combinations of quotes and no-quotes

[ptrthomas]

@ivangsa yes, watch is gone. and I don't think we ever supported yaml embedded expressions

[ivangsa]

ah ok, I though that yml and json where treated mostly the same..
is there any plan to support yaml embedded expressions? I know it can be done in javascript

[ptrthomas]

@ivangsa nope, we designed for json / xml only

[JoshSchreuder]

Has the callSingle / path handling changed? Just testing out the RC, and my code to load some generic authentication code is no longer working. Couldn't see anything in the breaking changes.

Code:

var signIn = karate.callSingle(karate.properties['karate.config.dir'] + '/features/authentication/authentication.feature', config);

Is now throwing

org.graalvm.polyglot.PolyglotException: java.io.FileNotFoundException: c:\karate\features\authentication\C:\karate\features\authentication\authentication.feature (The filename, directory name, or volume label syntax is incorrect)
- com.intuit.karate.resource.FileResource.getStream(FileResource.java:94)
- com.intuit.karate.core.FeatureParser.parse(FeatureParser.java:73)
- com.intuit.karate.core.Feature.read(Feature.java:66)
- com.intuit.karate.core.ScenarioFileReader.readFile(ScenarioFileReader.java:64)
- com.intuit.karate.core.ScenarioBridge.read(ScenarioBridge.java:594)
- com.intuit.karate.core.ScenarioBridge.callSingle(ScenarioBridge.java:208)
- <js>.fn(Unnamed:116)

If I comment this out and log what's passed in it seems fine

karate.log("Config dir = ", karate.properties['karate.config.dir'] + '/features/authentication/authentication.feature');

Returns

16:52:17.583 [pool-1-thread-1]  INFO  com.intuit.karate - Config dir =  C:\karate/features/authentication/authentication.feature 

Which is where my feature file is.

I'm running with the JAR file (Windows) as follows

java "-Dkarate.config.dir=C:\karate" -jar "karate_rc.jar"

And it works on 0.9.6

[ptrthomas]

@JoshSchreuder certainly can be a bug. the BEST thing you can do is follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

[JoshSchreuder]

@ptrthomas 👍 I'll try and put together a minimal project that reproduces this for you

EDIT: might be similar to #1480 perhaps.

[hujunhaorobert]

@ptrthomas , I created a repo https://github.com/hujunhaorobert/demoMockFailure to replicate the failure mentioned in #1480 (java.lang.NullPointerException). Could you please have a look?

[hujunhaorobert]

@ptrthomas I re-generate karate-2.0.0.jar based on the latest develop branch and re-run the mock testing, seems the reported error is gone. Thanks for your effort on the fix solution!

[sumargoraymond]

Not really sure if this is related with current RC, however even if report verbosity is configured to show log , HTTP requests and responses are not embedded on the report.

karate.configure('report', { showLog: true, showAllSteps: true });

Or perhaps this is new intended behavior (And the documentation will need to be updated)?

Thanks!

Logback-test.xml config:

<logger name="com.intuit.karate" level="INFO"/>
  
   <root level="INFO">
       <appender-ref ref="STDOUT" />
       <appender-ref ref="FILE" />
   </root>

[ptrthomas]

@sumargoraymond if you or anyone can fix or at least replicate that will be great, I'm going to treat this as low priority. I also don't remember if the logger needs to be DEBUG for this to work

[sumargoraymond]

Hi @ptrthomas . It works perfectly on DEBUG. Thanks a lot!

[ptrthomas]

image (to use in wiki) showing rare need to use karate.toJava() when migrating to 1.0

[joelpramos]

@ptrthomas on that image - I'd recommended adding the .toJson() as well if you haven't already. Although not the best use case, how I've seen folks on my end printing map objects with the print does seem like useful and something that a lot of people are used to do (even for triaging). The map objects now need to be wrapped with the toJson().

[ptrthomas]

@joelpramos there's already a section on the wiki: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#java-in-js

and if within JS folks should use karate.log() and it should print properly - otherwise I don't think maps need to be always wrapped - unless you are making the mistake of string concatenation