Issue Accessing context.vars in Playwright Scenarios with Redis Integration

[patarky]

Hi artillery.io community,

I greatly appreciate the active and helpful discussions on this board. I am currently facing an issue where I can't access context.vars from a Playwright testfunction, and I am hoping for some guidance.

Problem Description

I am attempting to build a parameterized smoke test for our internal CMS system, which is behind a login. Each virtual user must be unique as they alter server-side data. I followed the Redis/Upstash example, which was beneficial for user creation and uniqueness.

However, I am stuck on reusing the context.vars values generated from the getUser (in processor.ts) function. Although I am passing the context object along, the values for context.vars.username and context.vars.password turn up as undefined when accessed in my Playwright scenario.

"Expected" Behavior

Before each Playwright scenario, I expect a beforeScenario hook to retrieve an individual user from Redis and store the username and password in context.vars for access in subsequent scenarios. My understanding is that for all scenarios executet by a vU the beforeScenario is always executet bevorhand.

Current Behavior

• The separate test function "playwrightLoginTest" works when I do not use beforeScenario and instead use hardcoded username and password in commands.ts.
• "beforeScenario: getUser" successfully logs the correct user from Redis into the console.
• When both "beforeScenario: getUser" and "testFunction: "playplaywrightLoginTest" are run together, the all console.log statements within the playwright parts output undefined for both the username and password. Additionally, the browser window often does not open.

My gut feeling from reading the docs and other discussions is, that somehow or by design of artillery the context.vars.username and context.vars.password can only be accesed within beforeScenario. I think its not a artilley issue and more on my side - because I'm not that experienced. But I'm somehow confused because it seems so simple - to get access to the stored values. I hope someone here can give me a hint or point me in the right direction. Thank you all for taking a look.

artillery.yml

config:
  target: "https://our-cms.com"
  phases:
    - duration: 1
      arrivalRate: 1
      name: "Phase 1"
  processor: "./processor.ts"
  engines:
    playwright:
      launchOptions:
        headless: false
scenarios:
  - beforeScenario: getUser
    flow:
    - log: "Username: {{ username }} | Password: {{ password }}"
  - name: Login
    engine: playwright
    testFunction: "playwrightLoginTest"

processor.ts

import { loginCms } from '../e2e/helpers/commands';
import { Redis } from '@upstash/redis';
import { config } from 'dotenv';
config();

const redis = new Redis({
    url: process.env.UPSTASH_REDIS_URL,
    token: process.env.UPSTASH_REDIS_TOKEN
});
console.log('Redis connected successfully, .env vars are read');

interface Context {
    vars: {
        username: string;
        password: string;
    };
}

interface User {
    username: string;
    password: string;
}
export async function getUser(context, events: any, done: () => void): Promise<void> {
    const initialTime = Date.now();

    const res: User | null = await redis.lpop('users');
    console.log('Response from Redis:', res);

    if (res === null) {
        console.error('No users found in Redis');
        throw new Error('err_no_users_found_in_redis');
    }

    const user: User = res;

    context.vars.username = user.username;
    context.vars.password = user.password;
    const finalTime = Date.now();

    if (process.env.SHOW_TIMING) {
        console.log(`Time taken: ${finalTime - initialTime}ms`);
    }
    
    console.log('getUser context.vars.username', context.vars.username);
    console.log('getUser context.vars.password', context.vars.password);
    
    done();
}

export async function playwrightLoginTest(page, context, events, test) {
    const { step } = test;

    console.log('processor.ts context.vars.username', context.vars.username);
    console.log('processor.ts context.vars.password', context.vars.password);
    
    await loginCms(page, step, context);
  }

commands.ts

import { Page, expect } from '@playwright/test';
interface Context {
  vars: {
    username: string;
    password: string;
  };
}

export const loginCms = async (page: Page, step, context: Context) => {
  await step('go_login_page', async () => {
    await page.goto('/login');
  });

  await step('enter_credentials', async () => {
     
    const username = context.vars.username;
    const password = context.vars.password;

    console.log('Username from vars' , username);
    console.log('Password from vars:', password);

    await page.locator('#loginUser').fill(username);
    await page.locator('#loginPassword').fill(password);
    
    await page.getByRole('button', { name: 'Login' }).click();
    
  });

  await step('check_success', async () => {
    await expect(page.getByRole('link', { name: 'Welcome' })).toBeVisible();
  });
};

console output from a run with above yml:

performance % artillery run artillery.yml
(node:51514) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Test run id: tmkkf_qtey6j3cj86kpckgzxtjj5j93tjad_7ffe
Bundled Typescript file into JS. New processor path: /Users/user/Documents/Testing/artillery-load-cli/performance/dist/processor.js
⠹ (node:51514) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
⠴ Redis connected successfully, .env vars are read
Phase started: Phase 1 (index: 0, duration: 1s) 15:08:07(+0200)

Phase completed: Phase 1 (index: 0, duration: 1s) 15:08:08(+0200)

(node:51514) [DEP0174] DeprecationWarning: Calling promisify on a function that returns a Promise is likely a mistake.
⠙ Response from Redis: { username: 'load-testing-name-5', password: 'load-testing-pw-5' }
Time taken: 160ms
getUser at the end for context.vars.username load-testing-name-5
getUser at the end for context.vars.password load-testing-pw-5
Username: load-testing-name-5 | Password: load-testing-pw-5
--------------------------------------
Metrics for period to: 15:08:10(+0200) (width: 0.162s)
--------------------------------------

vusers.completed: .............................................................. 1
vusers.created: ................................................................ 1
vusers.created_by_name.0: ...................................................... 1
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 162.3
  max: ......................................................................... 162.3
  mean: ........................................................................ 162.3
  median: ...................................................................... 162.4
  p95: ......................................................................... 162.4
  p99: ......................................................................... 162.4


All VUs finished. Total time: 2 seconds

--------------------------------
Summary report @ 15:08:10(+0200)
--------------------------------

vusers.completed: .............................................................. 1
vusers.created: ................................................................ 1
vusers.created_by_name.0: ...................................................... 1
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 162.3
  max: ......................................................................... 162.3
  mean: ........................................................................ 162.3
  median: ...................................................................... 162.4
  p95: ......................................................................... 162.4
  p99: ......................................................................... 162.4

Infobits:
beforeScenario is currently not compatible with playwright but I try to use it not in the same scenario (which could be the issue?)
#2279
I thought about beforeScenario because of this comment:
#2770
Other solutions to this case by using .env vars:
https://medium.com/@tolandominic/front-end-load-testing-with-playwright-and-artillery-4fa1ac615fda

[bernardobridge]

Hey @patarky 👋

Thanks for the thorough explanation. As you mentioned, beforeScenario is not compatible with Playwright (yet). You can't use it in another scenario either, because each Virtual User (VU) will select which scenario to run (randomly).

My suggestion is that you integrate the getUser call as part of your playwrightLoginTest function. Something like:

export async function getUser(): Promise<void> {
    const initialTime = Date.now();

    const res: User | null = await redis.lpop('users');
    console.log('Response from Redis:', res);

    if (res === null) {
        console.error('No users found in Redis');
        throw new Error('err_no_users_found_in_redis');
    }

    const user: User = res;

    const finalTime = Date.now();

    if (process.env.SHOW_TIMING) {
        console.log(`Time taken: ${finalTime - initialTime}ms`);
    }
    
    console.log('getUser username', user.username);
    console.log('getUser password', user.password);
    
    return user
}

export async function playwrightLoginTest(page, context, events, test) {
    const { step } = test;

    const user = await getUser();
    context.vars.username = user.username;
    context.vars.password = user.password;
    
    await loginCms(page, step, context);
  }

This way it is basically the same as doing it with beforeScenario. Let me know if that works for you!

[patarky]

Hey @bernardobridge , thank you very much for looking at my case. It worked! I really appreciate it. Kudos to you and the whole artillery project.