Adjust k6/execution JS API and make it more predictable from JS perspective

[olegbespalov]

What?

Currently, if you're trying to use the k6/execution JS API and try to access some of the properties it's possible to get an error.

// script.js
import { scenario } from 'k6/execution';

console.log(scenario.name);

export default () => { }

 k6 run script

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

ERRO[0000] getting scenario information in the init context is not supported
        at file:///home/user/script.js:3:12(20)  hint="script exception"

From the JS idiomatic site, it's not expected and probably should result as the undefined, not an error since the scenario.name is a property, not the method like scenario.getName()

Ideally, the behavior should be like this:

import exec from 'k6/execution';

console.log(JSON.stringify(exec, null, 4));

export default function () {
    console.log(JSON.stringify(exec, null, 4));
}

it should just print the execution properties that were available in each context, something like this:

// Init context
{
    "instance": {
        "iterationsCompleted": 0,
        "iterationsInterrupted": 0,
        "vusActive": 0,
        "vusInitialized": 0,
        "currentTestRunDuration": 0
    },
    "test": {
        "options": { /* ... */ }
        // abort() is not printed by JSON.stringify(), but it's there
    },
    "vu": {
        "idInInstance": 0,
        "idInTest": 0
    }
} 

// VU context
{
    "instance": {
        "iterationsCompleted": 0,
        "iterationsInterrupted": 0,
        "vusActive": 1,
        "vusInitialized": 1,
        "currentTestRunDuration": 0.385693
    },
    "scenario": {
        "name": "default",
        "executor": "per-vu-iterations",
        "startTime": 1655191933532,
        "progress": 0,
        "iterationInInstance": 0,
        "iterationInTest": 0
    },
    "test": {
        "options": { /* ... */ }
        // abort is not printed by JSON.stringify(), but it's there
    },
    "vu": {
        "idInInstance": 1,
        "idInTest": 1,
        "iterationInInstance": 0,
        "iterationInScenario": 0,
        "tags": {
            "group": "",
            "scenario": "default"
        }
    }
} 

However, the error here is a nice signal to a user that using scenario.name in the init context is not the right way. So it's a nice UX and ideally, we need to keep it.

Think to consider:

• @codebien suggested that maybe we can use the --throw approach, but keep in mind init context
• Maybe instead of an error we can log a warning/error to the console, that still signals to a user that something is wrong
• Consider switching to get*() methods
• Original discussion started fix: proper error exec.scenario.* in not init context #2567

Why?

Consistent and predictable APIs are useful for any user,

[na--]

FWIW, I quite liked this suggestion from @codebien, though with some minor modifications:

1. only the properties that make sense in a certain context are available in k6/execution and its children, e.g. execution.scenario or execution.vu.iterationInScenario are not available in the init context
2. that will allow things like JSON.stringify(execution) to work without any problems
3. if the user tries to access a missing or invalid property of k6/execution or its child objects, what happens will depend on the value of --throw:
   • if --throw is enabled, an exception is thrown (current behavior)
   • if throw is not enabled, then undefined is returned (as you'd expect if you access a missing property in a normal JS object), but we also print a warning message to the console

@olegbespalov mentioned that in the first init context we haven't processed all of the option values yet. So if the user specified throw in the exported script options, we might have a situation where we get a warning from the first execution of the init context and an exception from initializing one of the script VUs later. I still think this is a mostly acceptable behavior and a better UX than the current one.

WDYT?

[vanshaj]

Hi

As this is not being assigned to anyone, starting to work on this issue.

[na--]

Hmm @vanshaj, feel free to go ahead, just be prepared that it might not be a simple issue to implement. I didn't notice that it was tagged as a good first issue, it likely isn't going to be, so I'll remove that label 😞

[vanshaj]

Hello @na--
I tried to debug the issue, but unable to resolve it. Can someone provide me some guidance.
In js/modules/k6/execution.go at line 152 we have tried to get the execution state of mi.vu.Context which basically checks whether the context has key iota set. When we are in init state that key has not been set and it returns nil and the script throws error.
Can someone give me some idea to resolve this issue?

cc: @codebien

[na--]

@vanshaj, you'd need to understand both k6 executors and goja in order to implement this task.

If the --throw option is not enabled, you'd need to adjust what the properties (and sub-properties) of k6/execution are, based on whether k6 is currently executing the init or VU context (which can be determined by whether we have the VU State object available). You might need to be creative with goja's Object.DefineAccessorProperty and DynamicObject...

As I said, it's complicated and definitely not a beginner-friendly issue. We also won't be able to help you very well, since implementing this will probably involve quite a lot of experimentation and trial-and-error even from us - we wont be able to just chime in and offer good suggestions when you are stuck, sorry... If you can't work this out on your own, I suggest you pick something else that's easier, for now.