Skip to content

How to scale up the staging database

Laura Beaufort edited this page Sep 2, 2020 · 3 revisions

Feel free to copy this checklist into your issue:

  • Let the team know and set up a maintenance window in Pingdom, this will likely cause downtime

In the console:

  • Prod: Select the production reader, on the "Configuration" tab in the lower left-hand corner, make note of the"parameter group" because we'll want to update the stage "DB parameter group" to match this.
  • Stage: start with the writer. Click on it and hit "Modify"
  • Update DB size (currently 4.8xlarge in prod).
  • Update DB parameter group to match production (this adds more logging to be consistent with production for more consistent experiment. DON'T change the DB cluster parameter group Note: ok to leave at "Disable enhanced monitoring"
  • Choose "apply immediately"
  • Wait for the changes to take effect (status will go from "monitoring" to "available")
  • Prod: Under "database options" Look at the "DB parameter group" in production reader and "enhanced monitoring" for the reader, make note of it.
  • Repeat for reader instance (may be failed over and now is the writer, but this is fine)
  • Wait for the changes to take effect (status will go from "monitoring" to "available")
  • Add one instance (production currently at 3 instances). Select the cluster, Actions -> Add reader. Under "DB instance identifier" name it stage-aurora-inst-2
  • After you create it, double check security groups are filled out (reader should inherit from cluster settings)
  • (Optional) Set up autoscaling - we created a new policy that mirrored the policy for production docs
  • (Optional) Run "warmup script" in stage for new reader - David Z or anyone on the database team can run this. It takes about 3 hours and improves performance, but it's not required to run it or wait until it's finished, the database will just be slower until it's finished

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.DBInstance.Modifying.html