Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared state/memory between VUs #532

Closed
robingustafsson opened this issue Mar 14, 2018 · 3 comments
Closed

Shared state/memory between VUs #532

robingustafsson opened this issue Mar 14, 2018 · 3 comments

Comments

@robingustafsson
Copy link
Member

We should evaluate possible ways to share larger blobs of data between VUs. The use case being that one might want to declare (or load from file on disk) a large data structure in the init context that is than used to parameterize data in the test.

Some ideas:

  1. Share parts of the init context between VUs (not certain if this is at all doable in a clean way)
  2. Create an API that is writable in the init context but only readable when running the test (executing the export default function())

An API for 2) could perhaps be something like:

import share from "k6/share";

// Parse data in to format specifc JS objects (CSV, JSON etc.)
let csvShare = share.load("./data.csv");
let jsonShare = share.load("./data.json");

export default function() {
    // Read data from the shared data blobs
    let row = csvShare.row();
    let randomRow = csvShare.random();
    let itemTitle = jsonShare.q("0.properties.title");
}
@Globegitter
Copy link

Is it possible to have any shared state at the moment? We are interested in sharing a counter between the requests, to know the how many number of requests it took until a certain condition a has been met.

@na--
Copy link
Member

na-- commented Sep 10, 2018

At the moment you can use a custom Counter metric (docs here and here) to track that at the end of the test, but there's no shared counter that you can use and check inside of the script, sorry. Depending on your use case, thresholds may be able to do the job, but if not, follow this issue for updates on any new shared state features.

@na-- na-- mentioned this issue May 16, 2019
@na-- na-- added the high prio label Dec 12, 2019
mstoykov added a commit that referenced this issue Nov 25, 2020
This is part of #532

The reason behind this not redefing ... how to open a file or load data
from one is that, as in the majority of the cases after that there is
some amount of ... transformation of the data, so that won't help much.

The original idea was to have a shareable JSON ... but the more concrete
array implementation is faster (in some cases by a lot) and also seems
like a better fix for the first iteration of this, as that will be the
majority of the data type. So it is very likely that the non array parts
will be dropped in favor of doing them later and this getting merged
faster.

There are still more things to be done and depending on whether or not
we want to support iterating (with for-of) it can probably be done
without some of the js code.
mstoykov added a commit that referenced this issue Nov 25, 2020
This is part of #532

The reason behind this not redefing ... how to open a file or load data
from one is that, as in the majority of the cases after that there is
some amount of ... transformation of the data, so that won't help much.

The original idea was to have a shareable JSON ... but the more concrete
array implementation is faster (in some cases by a lot) and also seems
like a better fix for the first iteration of this, as that will be the
majority of the data type. So it is very likely that the non array parts
will be dropped in favor of doing them later and this getting merged
faster.

There are still more things to be done and depending on whether or not
we want to support iterating (with for-of) it can probably be done
without some of the js code.
mstoykov added a commit that referenced this issue Jan 11, 2021
This is part of #532

This only implements a shareable array that is generated through a callback.
This way any additional processing can be done by any js code once and then the result will be shared between VUs in a readonly fashion.
@na-- na-- added this to the v0.30.0 milestone Jan 21, 2021
@na--
Copy link
Member

na-- commented Jan 21, 2021

This was resolved by #1739. As described in the release notes (we don't quite have the docs updated yet), k6 now has a way for scripts to use a huge read-only array without the need to have a copy in every VU. We could do something similar for a map in the future, if needs be, but that would be a separate issue. And data segmentation (i.e. VUs independently iterating over non-overlapping elements of the array) falls under #1539. So, I think this can be closed 🎉

@na-- na-- closed this as completed Jan 21, 2021
salem84 pushed a commit to salem84/k6 that referenced this issue Feb 3, 2021
This is part of grafana#532

This only implements a shareable array that is generated through a callback.
This way any additional processing can be done by any js code once and then the result will be shared between VUs in a readonly fashion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants