Skip to content

Migrations

Migrating an Open edX Instance can be a complex affair. This article isn't intended to be a comprehensive how-to, but to give some guidance on how migrations should be approached.

Upgrade Paths

Most migrations from outside companies involve upgrading the platform. This is because most organizations are not able to keep up with current releases. Even some specialized teams within the Open edX community have difficulty keeping current.

Notionally, it should be possible to install the version of tutor which corresponds to the version the client is starting from, and then perform upgrades one release at a time until arriving at the currently supported version. However, there are some issues with this:

  1. More than a few versions back, setting up Tutor becomes less and less reliable. Old resources disappear, dependencies rot, and other problems creep up.
  2. Some migrations deal with versions that predate Tutor, making Tutor's smoother upgrade path unavailable.
  3. When starting from a highly customized version of the platform, all bets are off.

When possible, the Tutor route is the most straightforward. Other options exist, however. One involves using a tool like Atlas to map out changes to the database schema between large version jumps and perform them at once. OpenCraft members can see an example of one such jump from Juniper to Sumac here1.

Resource Allocation

We offer both independent installations (customer-owned clusters on AWS or other providers) and shared hosting solutions using our internal cluster. When provisioning an independent cluster, it is easy to remember to provisioned enough disk space on the hosted database providers for MySQL and MongoDB, as you will be setting these up as part of the initial installation.

However, it's important to remember that resources on the shared cluster are limited. You must check the hosted Database instances to verify they have the disk space to take on a new tenant, and upgrade them ahead of time if not. Failure to do so can make the databases go read-only or become otherwise unavailable, which would impact all users on the cluster.

Take the total size of the database dumps you'll be importing, and add it to the current disk usage. Add a 25% buffer and then make sure that the cluster size is at least that big. If it is not, size it upward to accommodate the impending import and additional overhead room.

Additionally, for especially large migrations, it is important to have sufficient disk space in your work area so that you can perform any processing/staging ahead of the final import. Preferably, you'd want at least 3x as much space as your dumps require.

Hand-offs and Redundancy

Warning

The following notes have strong budget implications. The hours to ensure that there is redundancy during a migration MUST be accounted for during discovery.

Migrations should always have at least two concurrent team members, and if a migration has any chance of taking longer than their shifts, another pair of team members should be assigned to pick up where they left off.

Issues with weather, power, unexpected hardware issues, or medical/family emergencies can all interrupt migrations. The only way to guard against this is to have another person on standby who can take over if needed. This person need not be actively assisting the entire time, but they should at least be on-duty and working on something that can be shelved at a moment's notice.

In addition to these precautions, it is strongly recommended that as much of the migration work as possible takes place on a vm within the target cluster/datacenter, and that you use utilities such as screen or tmux (both of which should be available through most package managers) to allow another team member to take over at any time.

A written playbook should be established outlining the actions that need to be performed to complete the migration. Every team member who will work on the migration must be given time to practice it so that they are all familiar with the plan.

  1. If you do this, please budget the time to make these scripts more generalizable and publicly available, working off the research already performed. Include updating this documentation to point to your revised tooling. ↩