Handle each line in payload exactly once and sequentially

[MoeLa]

My target is used less frequently, so a "normal" performance test with a couple of dozens of requests per seconds is not reasonable. It usually only serves one request user at any given time, but that request may take a couple of milli seconds or several minutes.

I defined a scenario, where the "payload" was defined via an environment variable. Artillery was then executed for each value., like:

export var='foo'
artillery run myscenario.yaml
export var='bar'
artillery run myscenario.yaml

I'd like to store my variables in a payload.csv (with the rows 'foo' and 'bar'), so the pseudo code from above can be reduced to artillery run myscenario.yaml. The scenario for 'foo' and 'bar shall be executed sequentially and exactly once. The duration for 'foo' and 'bar' may differ extremly.

Is there a way to set config.phases and config.http.pool (and problably further parameters) to achieve my desired behaviour?

[hassy]

hi @MoeLa👋 have you tried using the order: sequence option for payload definitions?

https://artillery.io/docs/guides/guides/test-script-reference.html#Options

One possible snag is that once all rows have been used, the next virtual user would wrap around and start from the beginning again. That wouldn't be a problem though if you have exactly 2 virtual users and 2 rows, which may be the case?

lmk if that would work, there may be other solutions if not

[MoeLa]

Hey @hassy

thanks for your quick response.

I am aware of the order: sequence option. The issue is not about iterating through the payload in sequence. The issue is about having the second user starting, not before the first user has finished his scenario.

[hassy]

I see. There's no way to coordinate between VUs in that way, they're completely independent of each other (like users / clients would be in the real world). What could possibly work is running the scenario twice (using a loop) but with different values for some request parameters between iterations?

[MoeLa]

The loop approach sounds interesting.

I wrote a small test.yaml with an outer and an inner loop. Is there a way to access the outer $loopElement inside the inner loop?

config:
  target: "https://www.google.com"
  plugins:
    metrics-by-endpoint: {}
  phases:
    - duration: 1
      arrivalRate: 1
      maxVusers: 1
scenarios:
  - name: "Test"
    flow:
      - loop:
        - log: "Outer loop: {{ $loopElement[0] }} und {{ $loopElement[1] }}" # $loopElement[1] is 'foo2' and 'bar2' here
        - get:
            url: "/search?q={{ $loopElement[0] }}"
        - loop:
          - log: "Inner loop: {{ $loopCount }}"
          - get:
              url: "/search?q={{ $loopElement[1] }}" # $loopElement[1] is undefined here
          count: 2
        over:
          - ["foo1", "foo2"]
          - ["bar1", "bar2"]

[MoeLa]

To answer my own question:

config:
  phases:
    - duration: 1
      arrivalCount: 1
      maxVusers: 1

starts one virtual user, that finishes as soon as his scenario has been completed. Instead of using config.payload,

scenarios:
  - flow:
      - loop:
          - post: ...
        over:
          - "foo"
          - "bar"

is used to iterate. "foo" and "bar" may also be an array like ["foo", "foo2"] and ["bar", "bar2"] and then be accessed by $loopElement[0].

If you need more looping inside that loop, you need to use a function step in a flow, like

scenarios:
  - flow:
      - loop:
        - function: "prepareInnerLoop" # call my-processor.js > prepareInnerLoop
        - loop:
          - post: ...
          count: "{{ repeat }}"
        over:

to write the $loopElement some where into context.vars. The inner loop messes around with $loopCount and $loopElement, that you can't use it inside or after the inner loop anymore.