|

Learn how Meta scaled MySQL automation and fleet management using Python and how you can reuse similar design and techniques to scale your DBMS.

As Production Engineers at Meta, we strive to build and maintain our production services reliable, scalable, performant, and secure.

How Python has been leveraged to automate tasks, such as replication lag monitoring, replica promotion, and backup and restore, to name a few that in the past would have been missing or manually executed by DBAs.

The workshop will have follow-along examples to give you insights and make you learn skills you could reuse at your company.

Prerequisites 🚚

1. Install Docker [MacOS] [Linux]
2. Make sure you're able to run the followings (version I'm using):
   1. docker --version (Docker version 20.10.13, build a224086)
   2. docker-compose --version (Docker Compose version v2.3.3)
3. Join the Discord channel for communication between us by following this link: https://fb.me/pycon22-mysql-discord
   1. Troubleshooting channel: here

Get started 🚀

1. Clone this repo (git clone [email protected]:MarcoMontaltoMonella/pycon2022-mysql.git)
   1. The initial state of the system can be seen in here
2. From within the repo, run docker-compose up --build
   1. You should be seeing a front-end page at http://0.0.0.0:8000
3. Follow along the workshop, where we will cover the following (#Discord channel):
   1. Automating periodic backups (#ex1-backup)
   2. Automating healthcheck monitoring (#ex2-healthcheck)

Automating periodic backups 🗳

Discord channel: (#ex1-backup)

In this first example we will setup a job for creating backups of our MySQL databases. The high level idea is to have an external agent that will periodically run the following steps:

• Run mysqldump against our databases
• Compress the generated backup
  • useful lib: zstd
• Upload the backup to a storage location

Automating healthcheck monitoring 🚨

Discord channel: (#ex2-healthcheck)

This second example will see us creating a job for periodically check for any sign of database corruption, and notify us in case that happens. The high level idea for this code is the follow:

• Run mysqlcheck against our databases
• Analyze the output
• Alert us if database is not healthy
  • useful lib: notifiers

Thank you for attending 🙏