-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support full reconstruction of HCL from parse tree #169
Merged
kkozik-amplify
merged 14 commits into
amplify-education:main
from
weaversam8:feature/hcl2-reconstructor
Oct 7, 2024
Merged
Support full reconstruction of HCL from parse tree #169
kkozik-amplify
merged 14 commits into
amplify-education:main
from
weaversam8:feature/hcl2-reconstructor
Oct 7, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… can retain caching benefits
kkozik-amplify
approved these changes
Oct 7, 2024
@weaversam8 Hi, thank you for contributing! The PR looks good - we will release this today with |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi there, this closes #23. Many folks were asking for the ability to convert the parsed files back into HCL. This PR does a big part of the heavy lifting required to enable that functionality.
Currently the ultimate output of this library is a dictionary transformed from the underlying AST generated by Lark. I did not attempt to reverse the transformation process that this library takes while traversing the AST. (Maybe a later PR.) I did, however, use Lark's experimental "reconstruction" functionality to provide a way to transform the AST back into valid HCL. As such, if developers are willing to manipulate the AST instead of the final output dictionary, reversing the parse back into HCL is now possible with this PR.
A high level overview of the changes in this PR:
parse(file)
andparses(string)
return anAST
(Lark.Tree
) after parsing an HCL2 filewrites(ast)
writes an AST back to a valid HCL2 stringtransform(ast)
allows a user to convert the AST to a dictionary using the existingDictTransformer
class, like theload
andloads
methods already do under the hood.=
tokens, which doesn't change how those rules are ultimately parsedDictTransformer
inhcl2.transformer
to ensure it worked with the tweaks I made to the grammarhcl2.reconstructor
, which rebuilds the parser without a cache, implements the Reconstructor, and contains logic to add whitespace to the correct tokens when reassembling the filecache
option does not work withReconstructor
lark-parser/lark#1472)test/unit/test_reconstruct.py
to test my new functionality.test_write_terraform
ensures that every HCL2 file produces an identical dict when parsed, even after reconstructiontest_write_terraform_exact
ensures that the reconstructed HCL2 exactly matches the input HCL2, down to the individual character and whitespace level.test_write_terraform
test, which has no such exceptions.I have verified that all existing tests are passing, and overall test coverage is improved by this PR. Please let me know if you have any questions, or suggested changes.