Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

export backup to S3 #33

Open
mg98 opened this issue Oct 25, 2020 · 4 comments
Open

export backup to S3 #33

mg98 opened this issue Oct 25, 2020 · 4 comments

Comments

@mg98
Copy link

mg98 commented Oct 25, 2020

I don't know about the others, but I would like to use this backup solution with a scheduled lambda function. It would be sweet to have the possibility to export to a S3 bucket. And then it would also make sense to allow restoring from an export in a S3 bucket.

@wietlol
Copy link

wietlol commented Nov 16, 2020

An even better option would be to export to memory (string should be sufficient, looking at the code).
The user of the library can then decide where to send the backup to (file, S3, database, etc).

For @mg98 each AWS Lambda container has a temp folder which can be used to write the data to and read the data again to then send it to S3.

Each Lambda function receives 500MB of non-persistent disk space in its own /tmp directory.
https://aws.amazon.com/lambda/faqs/

@mg98
Copy link
Author

mg98 commented Nov 16, 2020

An even better option would be to export to memory

Yeah, I think that would be useful. Although, when looking at scalability, a good implementation of a S3 export could allow exports of data sets that, because of their size, exceed memory or disk limitations of the execution environment.

@russelldavis
Copy link

This would be nice. It's also pretty easy to chain the two together using the aws cli:

set -e
backup_name=$(date "+%Y-%m-%d")
cognito-backup-restore backup --pool all --use-env-vars --directory "$backup_name"
aws s3 cp "$backup_name" "s3://YOUR-BUCKET/$backup_name" --recursive

@jfstephe
Copy link

jfstephe commented Mar 25, 2022

I may not work due to #39 . Something to check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants