Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stepfunctions: support dynamic bucket in Map ItemReader #31807

Open
1 of 2 tasks
gael-ft opened this issue Oct 18, 2024 · 1 comment
Open
1 of 2 tasks

stepfunctions: support dynamic bucket in Map ItemReader #31807

gael-ft opened this issue Oct 18, 2024 · 1 comment
Assignees
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions feature-request A feature should be added or improved. p2 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.

Comments

@gael-ft
Copy link

gael-ft commented Oct 18, 2024

Describe the feature

Goal is to allow using JsonPath for IItemReader properties when defining a Map.

For now, interface IItemReader defines bucket field as type IBucket.

But from a CFN perspective we only need a bucket name.
Using IBucket blocks users from using JsonPath as dynamic bucket name.

Example:

const distributedMap = new sfn.DistributedMap(this, 'Distributed Map State', {
  itemReader: new sfn.S3JsonItemReader({
    bucket: JsonPath.stringAt('$.myBucket'),
    key: JsonPath.stringAt('$.myKey'),
  }),
  // ...
});
"ItemReader": {
    "Resource": "arn:aws:states:::s3:getObject",
    "ReaderConfig": {
        "InputType": "JSON"
    },
    "Parameters": {
        "Bucket.$": "$.myBucket",
        "Key.$": "$.myKey"
    }
}

Current CDK implementation

Use Case

Using the provided construct instead of a CustomState for DistributedMap.

Proposed Solution

Simply using string instead of IBucket (possibly accepting both if necessary ?)

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.162.1

Environment details (OS name and version, etc.)

macOS 13.5

@gael-ft gael-ft added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Oct 18, 2024
@github-actions github-actions bot added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Oct 18, 2024
@khushail khushail added investigating This issue is being investigated and/or work is in progress to resolve the issue. p2 and removed needs-triage This issue or PR still needs to be triaged. labels Oct 18, 2024
@khushail khushail self-assigned this Oct 18, 2024
@khushail
Copy link
Contributor

Hi @gael-ft , thanks for reaching out.

I found a previous similar issue reg the support of Dynamic bucket in Map ItemReader - #29409 and PR seems to be merged 4 days ago -#31619

Please let me know if that works as per your usecase.

@khushail khushail added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions feature-request A feature should be added or improved. p2 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.
Projects
None yet
Development

No branches or pull requests

2 participants