Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Atmos YAML functions #810

Open
wants to merge 78 commits into
base: main
Choose a base branch
from
Open

Introduce Atmos YAML functions #810

wants to merge 78 commits into from

Conversation

aknysh
Copy link
Member

@aknysh aknysh commented Nov 30, 2024

what

why

Atmos YAML Functions are a crucial part of Atmos stack manifests. They allow you to manipulate data and perform operations on the data to customize the stack configurations.

Atmos YAML functions are based on YAML Explicit typing and user-defined Explicit Tags (local data types). Explicit tags are denoted by the exclamation point ("!") symbol. Atmos detects the tags in the stack manifests and executes the corresponding functions.

NOTE: YAML data types can be divided into three categories: core, defined, and user-defined. Core are ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Many more advanced data types, such as binary data, are defined in the YAML specification but not supported in all implementations. Finally, YAML defines a way to extend the data type definitions locally to accommodate user-defined classes, structures, primitives, and functions.

Atmos YAML functions

NOTE; You can use Atmos Stack Manifest Templating and Atmos YAML functions in the same stack configurations at the same time. Atmos processes the templates first, and then executes the YAML functions, allowing you to provide the parameters to the YAML functions dynamically.

Examples

components:
  terraform:
    component2:
      vars:
        # Handle the output of type list from the `atmos.Component` template function
        test_1: !template '{{ toJson (atmos.Component "component1" "plat-ue2-dev").outputs.test_list }}'

        # Handle the output of type map from the `atmos.Component` template function
        test_2: !template '{{ toJson (atmos.Component "component1" .stack).outputs.test_map }}'

        # Execute the shell script and assign the result to the `test_3` variable
        test_3: !exec echo 42

        # Execute the shell script to get the `test_label_id` output from the `component1` component in the stack `plat-ue2-dev`
        test_4: !exec atmos terraform output component1 -s plat-ue2-dev --skip-init -- -json test_label_id

        # Execute the shell script to get the `test_map` output from the `component1` component in the current stack
        test_5: !exec atmos terraform output component1 -s {{ .stack }} --skip-init -- -json test_map

        # Execute the shell script to get the `test_list` output from the `component1` component in the current stack
        test_6: !exec atmos terraform output component1 -s {{ .stack }} --skip-init -- -json test_list

        # Get the `test_label_id` output of type string from the `component1` component in the stack `plat-ue2-dev`
        test_7: !terraform.output component1 plat-ue2-dev test_label_id

        # Get the `test_label_id` output of type string from the `component1` component in the current stack
        test_8: !terraform.output component1 {{ .stack }} test_label_id

        # Get the `test_list` output of type list from the `component1` component in the current stack
        test_9: !terraform.output component1 {{ .stack }} test_list

        # Get the `test_map` output of type map from the `component1` component in the current stack
        test_10: !terraform.output component1 {{ .stack }} test_map

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced new YAML functions: !exec, !template, and !terraform.output for enhanced stack manifest capabilities.
    • Added support for custom YAML tags processing in Atmos configurations.
    • Enhanced configuration options for Atlantis integration, allowing for more flexible setups.
  • Documentation Updates

    • Enhanced documentation for using remote state in Terraform components.
    • Updated guides for the atmos.Component function and the new YAML functions.
    • Clarified Atlantis integration setup options and workflows.
    • Improved explanations on handling outputs and using the new YAML functions.
    • Added documentation for new functions and updated existing guides for clarity.
  • Dependency Updates

    • Upgraded various dependencies to their latest versions for improved performance and security.

# Conflicts:
#	website/docs/integrations/github-actions/setup-atmos.mdx
# Conflicts:
#	examples/quick-start-advanced/Dockerfile
#	go.mod
#	go.sum
#	internal/exec/help.go
#	website/docs/integrations/atlantis.mdx
# Conflicts:
#	examples/quick-start-advanced/Dockerfile
#	website/docs/integrations/atlantis.mdx
@aknysh aknysh added the minor New features that do not break anything label Nov 30, 2024
@aknysh aknysh requested a review from osterman November 30, 2024 21:49
@aknysh aknysh self-assigned this Nov 30, 2024
@aknysh aknysh requested review from a team as code owners November 30, 2024 21:49
Copy link
Contributor

coderabbitai bot commented Nov 30, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request primarily updates the atmos.yaml configuration file to modify the base_path for components, stacks, and workflows, reflecting a new directory structure. Additionally, the Dockerfile for the quick-start example updates the versions of Atmos and Terraform. Several YAML configuration files for Terraform components introduce new settings and variables, while various Go files enhance error handling and processing for YAML tags. Documentation updates clarify the usage of new YAML functions and improve integration instructions for Atlantis.

Changes

File Change Summary
atmos.yaml Updated base_path for components, stacks, and workflows; logs section specifies output and log level.
examples/quick-start-advanced/Dockerfile Updated Atmos version from 1.109.0 to 1.111.0 and Terraform version from 1.9.8 to 1.5.7.
examples/tests/stacks/catalog/terraform/template-functions-test/defaults.yaml Added workspace_enabled: false under settings.spacelift.
examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml Added settings.spacelift.workspace_enabled: false, replaced test_label_id and test_label_id_2 with test_1 and test_2, and introduced multiple new variables for enhanced functionality.
go.mod Updated dependency versions for several packages, including github.com/charmbracelet/bubbletea, github.com/hashicorp/terraform-config-inspect, and others.
internal/exec/describe_stacks.go Enhanced ExecuteDescribeStacks to process custom YAML tags after converting to YAML, with added error handling.
internal/exec/terraform_generate_backends.go Modified ExecuteTerraformGenerateBackends to include processing of custom YAML tags with error handling.
internal/exec/terraform_generate_varfiles.go Updated ExecuteTerraformGenerateVarfiles to process custom YAML tags, introducing error handling.
internal/exec/utils.go Added ProcessCustomYamlTags function for YAML tag processing; updated argument handling in command functions.
internal/exec/validate_component.go Removed "cue" schema type from validation checks in validateComponentInternal.
internal/exec/validate_stacks.go Enhanced error handling in ValidateStacks, including schema file checks and downloading from URL if necessary.
internal/exec/yaml_func_exec.go Introduced processTagExec for executing shell commands within YAML functions.
internal/exec/yaml_func_template.go Added processTagTemplate for handling Atmos YAML function inputs.
internal/exec/yaml_func_terraform_output.go Introduced processTagTerraformOutput for managing Terraform outputs within YAML functions.
internal/exec/yaml_func_utils.go Added ProcessCustomYamlTags for processing YAML tags recursively.
pkg/config/const.go Added constants for Atmos YAML functions: AtmosYamlFuncTemplate, AtmosYamlFuncExec, and AtmosYamlFuncTerraformOutput.
pkg/utils/map_utils.go Introduced MapOfInterfacesToMapOfStrings for type conversion in maps.
pkg/utils/yaml_utils.go Added AtmosYamlTags variable and processCustomTags function for handling YAML tags during unmarshalling.
pkg/validate/validate_stacks_test.go Updated import statements to ensure schema package is correctly imported.
website/docs/core-concepts/components/terraform/remote-state.mdx Enhanced documentation on using remote state in Terraform components, updated syntax examples, and clarified data sharing methods.
website/docs/core-concepts/stacks/templates/functions/atmos.Component.mdx Improved clarity and detail regarding the atmos.Component function, including examples and caching behavior.
website/docs/core-concepts/stacks/templates/templates.mdx Minor textual adjustments for grammatical accuracy.
website/docs/core-concepts/stacks/yaml-functions/exec.mdx Introduced documentation for the !exec YAML function, detailing execution of shell scripts within manifests.
website/docs/core-concepts/stacks/yaml-functions/template.mdx Introduced documentation for the !template YAML function, explaining its use for converting JSON-encoded strings.
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx Introduced the !terraform.output YAML function for reading outputs from Atmos components.
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx Added documentation for Atmos YAML Functions, detailing their role and providing examples.
website/docs/integrations/atlantis.mdx Enhanced configuration options for Atlantis integration with Atmos, detailing two methods of configuration and clarifying command-line flag usage.
website/package.json Updated several dependency versions for Docusaurus and FontAwesome packages.

Possibly related PRs

Suggested reviewers

  • osterman
  • johncblandii
  • nitrocode

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary or @auto-summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @auto-title anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (22)
internal/exec/yaml_func_template.go (1)

21-21: Simplify error creation with fmt.Errorf

Use fmt.Errorf instead of errors.New(fmt.Sprintf(...)).

Apply this diff:

-		err := errors.New(fmt.Sprintf("invalid Atmos YAML function: %s", input))
+		err := fmt.Errorf("invalid Atmos YAML function: %s", input)
🧰 Tools
🪛 golangci-lint (1.62.2)

21-21: S1028: should use fmt.Errorf(...) instead of errors.New(fmt.Sprintf(...))

(gosimple)

internal/exec/yaml_func_exec.go (1)

21-21: Simplify error creation with fmt.Errorf

Use fmt.Errorf instead of errors.New(fmt.Sprintf(...)).

Apply this diff:

-		err := errors.New(fmt.Sprintf("invalid Atmos YAML function: %s", input))
+		err := fmt.Errorf("invalid Atmos YAML function: %s", input)
🧰 Tools
🪛 golangci-lint (1.62.2)

21-21: S1028: should use fmt.Errorf(...) instead of errors.New(fmt.Sprintf(...))

(gosimple)

internal/exec/yaml_func_utils.go (1)

10-12: Consider implementing error handling in ProcessCustomYamlTags

Currently, ProcessCustomYamlTags returns a nil error. Capturing and returning errors during processing could enhance robustness.

examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml (5)

16-17: Duplicate Assignments for test_1 and test_2

Both test_1 and test_2 are assigned the same value. Consider removing one if duplicate variables are unnecessary.


18-20: Redundant Assignments for test_3 and test_5

test_3 and test_5 have identical assignments. To maintain clarity, consider eliminating the redundancy.


21-24: Inconsistent Usage of Quotes with Custom YAML Tags

There's inconsistency in how custom YAML tags like !terraform.output are used—with and without quotes. Standardizing the usage enhances readability and prevents parsing issues.


25-27: Standardize Quoting for !template Tags

Similar to the previous point, ensure consistent quoting for !template tags to improve consistency.


28-32: Consistent Use of Quotes with !exec Tags

The !exec tags are used both with and without quotes. Aligning the quoting style enhances clarity.

website/docs/core-concepts/stacks/yaml-functions/exec.mdx (2)

42-42: Adjust grammatical number for correctness

The phrase "!exec atmos terraform output is just as examples" should be revised for proper grammar. Consider changing it to "!exec atmos terraform output is just an example" or "!exec atmos terraform output serves as an example".

🧰 Tools
🪛 LanguageTool

[uncategorized] ~42-~42: The grammatical number of this noun doesn’t look right. Consider replacing it.
Context: ...exec atmos terraform output` is just as examples to show that you can get the outputs of...

(AI_EN_LECTOR_REPLACEMENT_NOUN_NUMBER)


54-54: Include a serial comma for clarity

In the phrase "a complex type (list, map or object)," consider adding a comma after "map" for improved readability: "list, map, or object."

🧰 Tools
🪛 LanguageTool

[style] ~54-~54: The serial comma (Oxford comma, Harvard comma) is missing.
Context: ... If the result is a complex type (list, map or object), the script must return it as a...

(SERIAL_COMMA_ON)

website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (3)

23-23: Improve symbol formatting for clarity

Consider using backticks for the exclamation point symbol to enhance readability: "Explicit tags are denoted by the exclamation point (!) symbol."

🧰 Tools
🪛 LanguageTool

[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...

(EN_QUOTES)


[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...

(EN_QUOTES)


28-28: Add 'and' before the last item in the list

For better readability, consider adding 'and' before the last item: "(e.g., floats, ints, strings, lists, and maps)."

🧰 Tools
🪛 LanguageTool

[style] ~28-~28: A comma is missing here.
Context: ...e ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Ma...

(EG_NO_COMMA)


39-39: Add punctuation at the end of the sentence

Please add a period at the end of the sentence to complete it properly.

🧰 Tools
🪛 LanguageTool

[style] ~39-~39: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ing-maps-or-lists) returned from the atmos.Component template function - The [!exec...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[grammar] ~39-~39: Please add a punctuation mark at the end of paragraph.
Context: ...tes/functions/atmos.Component) template function - The [!exec](/core-concepts/st...

(PUNCTUATION_PARAGRAPH_END)

website/docs/core-concepts/stacks/yaml-functions/template.mdx (2)

66-66: Consider varying the phrase 'in many cases'

The phrase 'in many cases' is used frequently. You might replace it with an alternative to enhance the writing.


136-136: Limit the use of exclamation marks

Using many exclamation marks might seem excessive. Consider reducing them for a more formal tone.

🧰 Tools
🪛 LanguageTool

[style] ~136-~136: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2804 characters long)
Context: ...nctions atmos.Component, toJson and !template, use the [!terraform.output](/core-concepts/stacks...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)

129-129: Correct the typographic opening quote

Consider using the correct typographic opening quote here for consistency.

🧰 Tools
🪛 LanguageTool

[typographical] ~129-~129: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...

(EN_QUOTES)


160-160: Limit the use of exclamation marks

Using many exclamation marks might seem excessive. Consider reducing them for a more formal tone.

🧰 Tools
🪛 LanguageTool

[style] ~160-~160: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2496 characters long)
Context: ... from the same stack. The YAML function !terraform.output is executed three time...

(EN_EXCESSIVE_EXCLAMATION)

internal/exec/yaml_func_terraform_output.go (1)

60-60: Remove unnecessary initialization of outputProcessed

The variable outputProcessed is initially assigned but immediately overwritten later without being used. Removing the initial assignment cleans up the code.

Suggested change:

-	outputProcessed := map[string]any{}
🧰 Tools
🪛 golangci-lint (1.62.2)

60-60: ineffectual assignment to outputProcessed

(ineffassign)

website/docs/core-concepts/components/terraform/remote-state.mdx (2)

52-57: Consider rephrasing to enhance readability

The sentences starting with "The" can be varied to avoid repetition and improve flow.

🧰 Tools
🪛 LanguageTool

[style] ~55-~55: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...on atmos.Component function, refer to atmos.Component. The !terraform.output Atmos YAML f...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


361-361: Add missing punctuation

Please add a period at the end of the paragraph for proper grammar.

🧰 Tools
🪛 LanguageTool

[grammar] ~361-~361: Please add a punctuation mark at the end of paragraph.
Context: ...n how to use 'atmos.Component' template function <PrimaryCTA to="/core-con...

(PUNCTUATION_PARAGRAPH_END)

internal/exec/describe_stacks.go (1)

365-370: Refactor to eliminate code duplication

The code processing custom YAML tags is duplicated for Terraform and Helmfile components. Consolidating it into a shared function would enhance maintainability.

Also applies to: 550-555

internal/exec/utils.go (1)

647-653: Consider refactoring to eliminate duplicate logic

The check for single command-line arguments in processArgsAndFlags appears duplicated. Consolidating this logic would enhance maintainability.

Apply this change to remove the duplicate code:

-    // For commands like `atmos terraform clean` and `atmos terraform plan`, show the command help
-    if len(inputArgsAndFlags) == 1 {
-        info.SubCommand = inputArgsAndFlags[0]
-        info.NeedHelp = true
-        return info, nil
-    }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 3f1cad7 and 4f68de7.

⛔ Files ignored due to path filters (2)
  • go.sum is excluded by !**/*.sum
  • website/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (28)
  • atmos.yaml (1 hunks)
  • examples/quick-start-advanced/Dockerfile (1 hunks)
  • examples/tests/stacks/catalog/terraform/template-functions-test/defaults.yaml (1 hunks)
  • examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml (1 hunks)
  • go.mod (3 hunks)
  • internal/exec/describe_stacks.go (2 hunks)
  • internal/exec/terraform_generate_backends.go (1 hunks)
  • internal/exec/terraform_generate_varfiles.go (1 hunks)
  • internal/exec/utils.go (2 hunks)
  • internal/exec/validate_component.go (1 hunks)
  • internal/exec/validate_stacks.go (1 hunks)
  • internal/exec/yaml_func_exec.go (1 hunks)
  • internal/exec/yaml_func_template.go (1 hunks)
  • internal/exec/yaml_func_terraform_output.go (1 hunks)
  • internal/exec/yaml_func_utils.go (1 hunks)
  • pkg/config/const.go (1 hunks)
  • pkg/utils/map_utils.go (1 hunks)
  • pkg/utils/yaml_utils.go (2 hunks)
  • pkg/validate/validate_stacks_test.go (1 hunks)
  • website/docs/core-concepts/components/terraform/remote-state.mdx (2 hunks)
  • website/docs/core-concepts/stacks/templates/functions/atmos.Component.mdx (3 hunks)
  • website/docs/core-concepts/stacks/templates/templates.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/exec.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/template.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (1 hunks)
  • website/docs/integrations/atlantis.mdx (1 hunks)
  • website/package.json (3 hunks)
✅ Files skipped from review due to trivial changes (3)
  • pkg/validate/validate_stacks_test.go
  • website/docs/core-concepts/stacks/templates/templates.mdx
  • go.mod
🧰 Additional context used
📓 Learnings (2)
examples/quick-start-advanced/Dockerfile (2)
Learnt from: aknysh
PR: cloudposse/atmos#775
File: examples/quick-start-advanced/Dockerfile:9-9
Timestamp: 2024-11-12T05:52:05.088Z
Learning: It is acceptable to set `ARG ATMOS_VERSION` to a future version like `1.105.0` in `examples/quick-start-advanced/Dockerfile` if that will be the next release.
Learnt from: osterman
PR: cloudposse/atmos#801
File: examples/quick-start-advanced/Dockerfile:9-9
Timestamp: 2024-11-23T00:13:22.004Z
Learning: When updating the `ATMOS_VERSION` in Dockerfiles, the team prefers to pin to the next future version when the PR merges, even if the version is not yet released.
internal/exec/validate_stacks.go (1)
Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:0-0
Timestamp: 2024-11-12T05:52:05.088Z
Learning: In `internal/exec/validate_stacks.go`, when downloading the Atmos JSON Schema file to the temp directory, the temporary file is overwritten each time, so explicit removal is not necessary.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/exec.mdx

[style] ~39-~39: A comma is missing here.
Context: ...s, or even Atmos or Terraform commands (e.g. !exec atmos terraform output). :::wa...

(EG_NO_COMMA)


[uncategorized] ~42-~42: The grammatical number of this noun doesn’t look right. Consider replacing it.
Context: ...exec atmos terraform output` is just as examples to show that you can get the outputs of...

(AI_EN_LECTOR_REPLACEMENT_NOUN_NUMBER)


[style] ~54-~54: The serial comma (Oxford comma, Harvard comma) is missing.
Context: ... If the result is a complex type (list, map or object), the script must return it as a...

(SERIAL_COMMA_ON)


[style] ~67-~67: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 1895 characters long)
Context: ... templates first, and then executes the !exec function, allowing you to provide ...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx

[typographical] ~129-~129: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...

(EN_QUOTES)


[style] ~160-~160: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2496 characters long)
Context: ... from the same stack. The YAML function !terraform.output is executed three time...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/template.mdx

[style] ~65-~65: The phrase ‘in many cases’ is used quite frequently. Consider using a less frequent alternative to set your writing apart.
Context: ...ecode) function in your Terraform code. In many cases, this is not an acceptable solution bec...

(IN_MANY_STYLE_CASES)


[style] ~136-~136: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2804 characters long)
Context: ...nctions atmos.Component, toJson and !template, use the [!terraform.output](/core-concepts/stacks...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/components/terraform/remote-state.mdx

[style] ~55-~55: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...on atmos.Component function, refer to atmos.Component. The !terraform.output Atmos YAML f...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[grammar] ~361-~361: Please add a punctuation mark at the end of paragraph.
Context: ...n how to use 'atmos.Component' template function <PrimaryCTA to="/core-con...

(PUNCTUATION_PARAGRAPH_END)

website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx

[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...

(EN_QUOTES)


[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...

(EN_QUOTES)


[style] ~28-~28: A comma is missing here.
Context: ...e ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Ma...

(EG_NO_COMMA)


[style] ~39-~39: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ing-maps-or-lists) returned from the atmos.Component template function - The [!exec...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[grammar] ~39-~39: Please add a punctuation mark at the end of paragraph.
Context: ...tes/functions/atmos.Component) template function - The [!exec](/core-concepts/st...

(PUNCTUATION_PARAGRAPH_END)


[style] ~44-~44: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2141 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...

(EN_EXCESSIVE_EXCLAMATION)

🪛 golangci-lint (1.62.2)
internal/exec/yaml_func_template.go

21-21: S1028: should use fmt.Errorf(...) instead of errors.New(fmt.Sprintf(...))

(gosimple)

internal/exec/yaml_func_exec.go

21-21: S1028: should use fmt.Errorf(...) instead of errors.New(fmt.Sprintf(...))

(gosimple)

internal/exec/yaml_func_terraform_output.go

60-60: ineffectual assignment to outputProcessed

(ineffassign)

🔇 Additional comments (16)
examples/tests/stacks/catalog/terraform/template-functions-test/defaults.yaml (1)

10-12: Configuration addition looks good

pkg/utils/map_utils.go (1)

37-46: Function implementation looks solid

The MapOfInterfacesToMapOfStrings function correctly converts map keys to strings, handling non-string keys by skipping them.

examples/quick-start-advanced/Dockerfile (2)

9-9: Updating ATMOS_VERSION to future release is acceptable

Setting ATMOS_VERSION to a future version aligns with team practices highlighted in previous learnings.


12-12: Verify compatibility due to Terraform version downgrade

Downgrading Terraform from version 1.9.8 to 1.5.7 may affect module compatibility. Please ensure all configurations are compatible with the older version.

internal/exec/yaml_func_utils.go (1)

49-58: Custom YAML tag processing is well-implemented

The functions for processing custom YAML tags correctly handle the specified tags.

pkg/utils/yaml_utils.go (1)

61-78: Verify Recursive Processing of YAML Nodes

Ensure that processCustomTags correctly handles all node types, including mappings, sequences, and scalars, to prevent unexpected behavior during YAML parsing.

website/package.json (1)

54-55: Confirm Node.js Version Requirement

The engines field specifies "node": ">=20.10". Verify that requiring Node.js version 20.10 or higher is intentional, as it may affect users on older versions.

pkg/config/const.go (1)

80-82: Constants declaration looks good

The addition of the new constants for Atmos YAML functions is consistent with the existing naming conventions and enhances clarity in the code.

internal/exec/terraform_generate_varfiles.go (1)

273-277: Custom YAML tag processing integrated correctly

The addition of ProcessCustomYamlTags enhances functionality and is implemented appropriately.

internal/exec/terraform_generate_backends.go (1)

257-261: Custom YAML tag processing added successfully

The incorporation of ProcessCustomYamlTags improves the handling of custom YAML tags without introducing issues.

website/docs/core-concepts/stacks/templates/functions/atmos.Component.mdx (1)

Line range hint 123-183: Clear explanation of handling complex outputs

The added sections provide a helpful guide on using complex outputs with atmos.Component, including the use of toRawJson, toJson, and the !template function.

internal/exec/validate_stacks.go (1)

93-93: Robust handling of schema file paths

The updated logic correctly checks for the schema file's existence and handles downloads when necessary, enhancing reliability.

atmos.yaml (1)

18-18: Appropriate update of base_path

Updating the base_path to "./examples/quick-start-advanced" aligns with the new directory structure.

website/docs/core-concepts/components/terraform/remote-state.mdx (1)

355-363: Excellent addition of alternative methods

Introducing atmos.Component and !terraform.output functions provides valuable options for users.

🧰 Tools
🪛 LanguageTool

[grammar] ~361-~361: Please add a punctuation mark at the end of paragraph.
Context: ...n how to use 'atmos.Component' template function <PrimaryCTA to="/core-con...

(PUNCTUATION_PARAGRAPH_END)

internal/exec/utils.go (1)

512-515: Custom YAML tags processing integrated smoothly

The addition of ProcessCustomYamlTags and its error handling is well-implemented.

Also applies to: 517-518

website/docs/integrations/atlantis.mdx (1)

676-676: ATMOS_VERSION update is appropriate

The version update to 1.111.0 on line 676 aligns with the latest release.

internal/exec/validate_component.go Show resolved Hide resolved
internal/exec/yaml_func_terraform_output.go Show resolved Hide resolved
coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 30, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (9)
website/docs/core-concepts/stacks/yaml-functions/template.mdx (2)

17-98: Consider adding error handling examples.

The examples effectively demonstrate successful cases. Consider adding examples of error scenarios and how to handle them, such as:

  • What happens if the JSON parsing fails?
  • How to handle null or undefined values?
🧰 Tools
🪛 LanguageTool

[style] ~65-~65: The phrase ‘in many cases’ is used quite frequently. Consider using a less frequent alternative to set your writing apart.
Context: ...ecode) function in your Terraform code. In many cases, this is not an acceptable solution bec...

(IN_MANY_STYLE_CASES)


133-139: Consider elevating this tip to a callout box.

This is a crucial recommendation that could save users time and reduce complexity. Consider making it more prominent, perhaps as a callout box at the beginning of the document.

🧰 Tools
🪛 LanguageTool

[style] ~136-~136: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2804 characters long)
Context: ...nctions atmos.Component, toJson and !template, use the [!terraform.output](/core-concepts/stacks...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1)

136-164: Document cache invalidation behavior.

While the caching mechanism is well explained, consider adding:

  • Cache lifetime details
  • Cache invalidation triggers
  • Memory usage considerations
🧰 Tools
🪛 LanguageTool

[style] ~160-~160: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2496 characters long)
Context: ... from the same stack. The YAML function !terraform.output is executed three time...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/templates/functions/atmos.Component.mdx (2)

Line range hint 123-182: Add troubleshooting section for common issues.

The documentation effectively explains the concepts but would benefit from a troubleshooting section covering:

  • Common parsing errors and their solutions
  • Debugging tips for template processing
  • Best practices for complex data structures

177-196: Consider adding performance benchmarks.

The caching explanation is clear, but would be more compelling with:

  • Specific performance metrics
  • Memory usage patterns
  • Recommendations for optimal usage
examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml (1)

16-20: Consider adding error handling for component outputs

While the template function usage is correct, consider adding error handling for cases where outputs might not exist or have unexpected types.

-        test_1: '{{ (atmos.Component "template-functions-test" .stack).outputs.test_label_id }}'
+        test_1: '{{ with $output := (atmos.Component "template-functions-test" .stack).outputs }}{{ if hasKey $output "test_label_id" }}{{ .test_label_id }}{{ else }}null{{ end }}{{ end }}'
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (3)

26-33: Enhance YAML type documentation with specific examples

The explanation of YAML types is good, but would benefit from concrete examples of each type.

Consider adding:

# Core types example
string_value: "hello"
int_value: 42
list_value: [1, 2, 3]

# User-defined type example
!custom_type
  key: value
🧰 Tools
🪛 LanguageTool

[style] ~28-~28: A comma is missing here.
Context: ...e ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Ma...

(EG_NO_COMMA)


47-52: Add cautionary notes about template processing order

While the tip about template processing order is helpful, consider adding warnings about potential pitfalls when combining templates with YAML functions.

Add a warning section:

:::caution
When combining templates with YAML functions, be aware that:
- Template errors will prevent YAML function execution
- Template variables must be properly escaped when used within YAML functions
:::

56-91: Enhance examples with expected outputs

The examples would be more helpful with comments showing expected outputs and potential error cases.

Add output examples:

# Example output:
test_1: ["item1", "item2"]  # Output from test_list
test_2: {"key": "value"}    # Output from test_map
test_3: "42"                # Direct string output
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between d0749f6 and 56f194f.

📒 Files selected for processing (6)
  • atmos.yaml (1 hunks)
  • examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml (1 hunks)
  • website/docs/core-concepts/stacks/templates/functions/atmos.Component.mdx (3 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/template.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1 hunks)
  • website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • atmos.yaml
🧰 Additional context used
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx

[typographical] ~129-~129: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...

(EN_QUOTES)


[style] ~160-~160: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2496 characters long)
Context: ... from the same stack. The YAML function !terraform.output is executed three time...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/template.mdx

[style] ~65-~65: The phrase ‘in many cases’ is used quite frequently. Consider using a less frequent alternative to set your writing apart.
Context: ...ecode) function in your Terraform code. In many cases, this is not an acceptable solution bec...

(IN_MANY_STYLE_CASES)


[style] ~136-~136: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 2804 characters long)
Context: ...nctions atmos.Component, toJson and !template, use the [!terraform.output](/core-concepts/stacks...

(EN_EXCESSIVE_EXCLAMATION)

website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx

[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...

(EN_QUOTES)


[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...

(EN_QUOTES)


[style] ~28-~28: A comma is missing here.
Context: ...e ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Ma...

(EG_NO_COMMA)


[style] ~39-~39: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ing-maps-or-lists) returned from the atmos.Component template function - The [!exec...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[grammar] ~39-~39: Please add a punctuation mark at the end of paragraph.
Context: ...tes/functions/atmos.Component) template function - The [!exec](/core-concepts/st...

(PUNCTUATION_PARAGRAPH_END)


[style] ~44-~44: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2141 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...

(EN_EXCESSIVE_EXCLAMATION)

🔇 Additional comments (2)
website/docs/core-concepts/stacks/yaml-functions/template.mdx (1)

1-15: LGTM! Clear and well-structured introduction.

The introduction effectively explains the purpose of the !template function and its relationship with atmos.Component.

examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml (1)

9-11: Verify Spacelift workspace configuration impact

The disabled workspace setting might affect CI/CD pipelines. Please ensure this is the intended configuration for this environment.

✅ Verification successful

Spacelift workspace configuration is consistent with test environment patterns

The disabled workspace setting in this test configuration aligns with the observed patterns in the codebase:

  • Most test and example components have workspace_enabled: false
  • Production/actual components typically have workspace_enabled: true
  • This file is in the examples/tests/ directory, so disabling the workspace is the expected behavior
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for other Spacelift workspace configurations to ensure consistency
rg -l 'workspace_enabled:' | xargs rg 'workspace_enabled:'

Length of output: 5858

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (4)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (4)

17-19: Consider adding output type information to the usage section.

The usage section would benefit from explicitly mentioning the supported output types (string, list, map) that can be retrieved using this function. This would help users understand what to expect when using the function.

  !terraform.output <component> <stack> <output>
+ # Supports string, list, and map output types

129-129: Fix quote formatting in the tip section.

The printf example uses straight quotes instead of typographic quotes, which is inconsistent with the documentation style.

-    By using the `printf "%s-%s-%s"` function, you are constructing stack names using the stack context variables/identifiers.
+    By using the `printf "%s-%s-%s"` function, you are constructing stack names using the stack context variables/identifiers.
🧰 Tools
🪛 LanguageTool

[typographical] ~129-~129: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...

(EN_QUOTES)


138-142: Add information about cache lifecycle.

The caching explanation would benefit from clarifying when the cache is invalidated (e.g., between Atmos commands, after certain time periods, etc.). This helps users understand the caching behavior completely.


195-196: Add a code example for the sensitive attribute.

Consider adding a Terraform code example demonstrating how to use the sensitive = true attribute:

output "sensitive_value" {
  value     = aws_secretsmanager_secret.example.arn
  sensitive = true
}
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 56f194f and f00b647.

📒 Files selected for processing (1)
  • website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx

[typographical] ~129-~129: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...

(EN_QUOTES)


[style] ~168-~168: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 5180 characters long)
Context: ... Control Considerations When using the !terraform.output function (and `terrafo...

(EN_EXCESSIVE_EXCLAMATION)


[grammar] ~176-~176: With the plural noun ‘logs’, the verb inflection ‘commands’ is not correct.
Context: ...**: Maintain detailed logs of Terraform commands executed, including terraform output,...

(NNS_IN_NNP_VBZ)

🔇 Additional comments (1)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1)

166-191: Excellent security considerations coverage!

The security considerations section comprehensively addresses crucial aspects of using the !terraform.output function securely. The coverage of RBAC, audit logging, encryption, and state file handling provides valuable guidance for users.

🧰 Tools
🪛 LanguageTool

[style] ~168-~168: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 5180 characters long)
Context: ... Control Considerations When using the !terraform.output function (and `terrafo...

(EN_EXCESSIVE_EXCLAMATION)


[grammar] ~176-~176: With the plural noun ‘logs’, the verb inflection ‘commands’ is not correct.
Context: ...**: Maintain detailed logs of Terraform commands executed, including terraform output,...

(NNS_IN_NNP_VBZ)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor New features that do not break anything
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant