Updating Python

Overview

Python is the programming language that the bulk of our server-side application code is written in so it is imperative that we keep it up to date. This affords us a few nice things:

• Helps stave off technical debt and accounts for any security issues
• Gives us access to new language features and modules
• Helps with keeping our dependencies up to date as well as support for older versions of the language drop off
• Helps with keeping our cloud.gov environments up to date and functioning properly as Cloud Foundry only supports the most recent stable releases

News and updates of upcoming Python releases (either maintenance releases or new major versions) can be found on the Python Insider blog, which you can also subscribe to (side note: It might be worth setting up a subscription to the RSS feed and have it go into Slack for announcements!) directly. Two or three times a year, a new major version is released and there are usually several maintenance releases made in between so it is important to stay on top of the news.

Considerations before updating

There are several things to keep in mind before going through the update process:

• What dependencies will also need to update to account for the new version (note: these should all be kept routinely up to date independently as it is!)?
• Does CircleCI support the new version?
• Does cloud.gov (Cloud Foundry) support the new version?
• Is it available through pyenv for installation?

Be sure to do a bit of research before updating Python to make sure that it will run properly in all environments!

NOTE: The reason why we use pyenv is because most machines come with Python pre-installed, but it is an older version and coupled with other system-level dependencies. We don't want to use it but we also don't want to render our machines unusable, so pyenv allows us to install and manage multiple versions of Python independently while retaining the system-level setup.

If all of these things check out and it is safe to update, then proceed on to go through the process.

Updating Python on the project level

The update process is relatively light-weight; the bulk of the work is in the testing. Here are all of the steps to update the project at a high level. To update your local environment, see the local changes section below.

• Install the new version locally via pyenv (be sure to update pyenv first)
• Create a new virtual environment against the new version installed
• Create a new branch off of develop to account for the modifications to the code that have to happen
• Run the application to see if it starts up and things generally work
• Run the full test suite
• Fix any issues and install dependency updates as necessary
• Modify README.md to account for the new version being used by the project
• Modify runtime.txt in the project root contain the new version, using .x at the end to automatically use the latest minor release in cloud.gov (e.g., python-3.7.x)
• Modify circleci/config.yml to load the new Docker image for the testing environment
• Commit all changes, push the branch to GitHub, and create a PR for review
• Check that CircleCI passes all tests
• Optional: you can do a manual deploy of the branch to the dev space to make sure the application deploys properly
• Merge the PR into the develop branch

Let's dive into these steps a bit more by focusing on the local changes and code modifications, the CircleCI changes, and lastly the cloud.gov changes.

Local changes

NOTE: Before starting, make sure your develop branch in Git is up to date and make a new branch for working on the Python update.

First, make sure you are running the latest version of pyenv; if you already have it installed, be sure to upgrade it. On a Mac, it's easiest to do all of this with homebrew. To install it if you don't already have it:

brew update
brew install pyenv

And to update it if you do:

brew update
brew upgrade pyenv

Be sure to follow any instructions output by the installation, namely the steps found here (starting at step 2).

With pyenv installed/updated you can now install Python itself on your system. To see which versions are available, you can run this command:

pyenv install -l

A rather large list will display, but the versions that we care about are the ones listed purely by version number at the beginning of the output. To install a new version, run this command, where <version> is the desired version number (e.g., 3.7.5):

pyenv install <version>

When this finishes, you are ready to start using the new version of Python installed. Just be sure the new version of Python you installed is the active Python first! You can check this with pyenv by running this command:

pyenv version

Note: You can also see all versions currently installed by running pyenv versions.

And then switch to the desired version for your entire system (the recommended step) by running this command:

pyenv global <version>

The next thing you'll want to do is create a new virtual environment with it. I recommend keeping your existing one until you're sure the new version is working fine with everything. How you want to create the new virtual environment is up to you; pyenv can manage them for you or you can create one manually.

Once the new virtual environment is created be sure to activate that as well, then reinstall all of the Python dependencies with this command:

pip install -r requirements.txt -r requirements-dev.txt -r requirements-locust.txt

When you reinstall the dependencies into the new virtual environment you may run into an installation issue. If you do it is likely because a dependency needs to be updated to support the new version of Python. Work through the error, updating the requirements file(s) as necessary with the new dependency versions (you can check dependency versions at PyPI), and see if that resolves it.

If no new version of the dependency works with the new version of Python, then it may not be a good time to update Python just yet. To get yourself back to the previous version you had working, deactivate the virtual environment, switch the older version of Python you have installed with pyenv, and re-activate the older virtual environment to resume previous work.

If things do install fine, then it's time to start testing that the application still works fine. Run the application locally and poke around, seeing if anything crashes unexpectedly. At this point you should also run the full test suite locally with py.test. If anything breaks during this local testing, work through the error(s) and see if they can be fixed, making any modifications necessary.

Once everything is working and all changes are accounted for, be sure to commit all work done so far in the new branch. Do not forget to update the README.md file with references to the new version!

CircleCI changes

Only one change is needed to update CircleCI to use the new Python version: the Docker image. In the .circleci/config.yml there is a section toward the top that lists a couple of Docker images, one of which is Python. The easiest way to find out what to update it to is to check the Docker Hub and search for circleci, which will bring up all of CircleCI's Docker images.

The CircleCI Python image is the one that you want, and there is a tab toward the top called Tags that will list all versions available. Unfortunately this is a bit confusing as the versions contain some extra information with them that isn't always clear. Some of the images contain headless browsers with them, others contain Node.js with them, and still others combine both.

At the time of this writing, the best option seems to be one of the ones with the jessie name in the tag, without node because of our current Node.js setup with the CircleCI builds. For example, the 3.7.4-jessie-browsers image seems to be a reasonable choice here. In the future we may update the Node.js configuration in our CircleCI setup and could take advantage of one of the builds with Node.js in it, but that's a separate discussion.

Once you find the right image to use, update the .circleci/config.yml file by changing the tag on the Python image line, where <new tag> is the tag of the image you've chosen (e.g., 3.7.4-jessie-browsers):

- image: circleci/python:<new tag>

Commit the change and if you haven't already, push your branch up to GitHub and create a PR. This will kick off a CircleCI build and use the new image for the new version of Python. Monitor the test output and see if the build succeeds. If it does, success!

cloud.gov changes

The final change is for cloud.gov and is change in one file: runtime.txt. This file is what is used by the Cloud Foundry Python Buildpack. To see which versions of Python are currently supported by the latest Python buildpack, take a look at the Python buildpack releases, which each contain a table of all supported versions.

NOTE: cloud.gov may not support the latest buildpack right away, and usually doesn't! It can take a couple of weeks before cloud.gov makes the latest version available. Usually you'll know because deployments will start showing messages saying there's a new version of the buildpack available and the current version of Python will no longer be supported in the next release.

NOTE: If you follow these steps, you'll only need to modify this file when there's a major version update. By following the paradigm with the .x at the end instead of a specific number, e.g., python-3.7.x, you wouldn't need to modify this file until you're ready to update to Python 3.8. Be sure to keep this in mind for the future!

If the newest buildpack is available on cloud.gov and supports the version of Python you are upgrading to, then it is okay to modify the runtime.txt file. At the time of this writing the file has a single line in it that says python-3.7.4. This is actually sub-optimal, what we should have is python-3.7.x; this ensures that the buildpack always pulls in the latest minor release available, which helps account for security updates.

Going forward, change the file to contain the new version of Python that you are updating to and be sure it always has a .x at the end: e.g., python-3.7.x.

Optional

If you are really concerned or want to be extra careful (not a bad idea!), you can also do a manual deployment to the dev space in cloud.gov once you update runtime.txt. If the build succeeds, you are in good shape!

Finally, commit the change and push it up to GitHub and the PR.

Wrapping up

With all of these steps done and local testing complete, CircleCI tests and builds passing, and the runtime.txt file updated for cloud.gov (and again, don't forget about the documentation in README.md!), the application is set to go with the new version of Python. The final step is to merge the branch back into develop after review.

These steps should continue to work going forward for several more releases, but as with anything things can change. Should that be the case, please do not forget to update these instructions as well! 😃

Happy updating!