Allow environment variables as values for some properties

[dansamsara]

The pipeline upload command supports environment variable substitution using the syntax $VAR and ${VAR}. While the schema validation works fine for properties that are strings, it will fail for properties such as parallelism where the type is defined as an integer.

What do you think about expanding some properties like parallelism to allow for a string, in addition to an integer, as long as the string resembles an environment variable?

"oneOf": [
  { "type": "integer" },
  { "type": "string", "pattern": "^\\$[A-Z_]+$|^\\$\\{[A-Z_]+\\}$" }
],

[angusfretwell]

Hi @dansamsara 👋

Thanks for raising this issue.

We don't support variable interpolation for the parallelism property (there's some discussion about it on the Buildkite forum), so in this case the existing validation is correct.

Are there any other properties you've noticed where this would apply? We'd be happy to accept a PR if so!

[dansamsara]

We don't support variable interpolation for the parallelism property (there's some discussion about it on the Buildkite forum), so in this case the existing validation is correct.

This is a dynamic pipeline. As described in the forum post you linked.

only option here is to use dynamic pipeline where you can use environment variable to define the parallelism for the pipeline definition which will be uploaded using the dynamic pipeline process.

Although it's a dynamically uploaded yaml file, we'd still like to validate it using the Buildkite pipeline schema.

For further clarification, here's how we have the pipeline is set up. The environment is configured in the pipeline like so:

env:
  TEST_PARALLELISM: 8

Then we use a command step to upload additional steps:

steps:
  - label: ":pipeline:"
    command: buildkite-agent pipeline upload .buildkite/pipeline.mobile-test.yml

In pipeline.mobile-test.yml we have a step like this:

steps:
  - label: ":horse: Parallelized mobile tests"
    command: bash run-tests.sh
    parallelism: $TEST_PARALLELISM

The generated job environment looks like this:

BUILDKITE_PARALLEL_JOB="0"
BUILDKITE_PARALLEL_JOB_COUNT="8"
TEST_PARALLELISM="8"

I see 8 jobs running in parallel for that step. The only part of this configuration that fails schema validation is parallelism: $TEST_PARALLELISM which is valid when using buildkite-agent pipeline upload.

The pipeline upload command supports environment variable substitution using the syntax $VAR and ${VAR}.

https://buildkite.com/docs/agent/v3/cli-pipeline#environment-variable-substitution

[angusfretwell]

Ah gotcha, thanks for the clarification @dansamsara.

In that case I think this would be a sensible change. I'll add this to our internal backlog, however we'd be happy to accept a PR if you'd like to get this change in sooner.