Thursday, April 18, 2019

Building a lite version of your site on Pantheon

We have some pretty heavy sites on Pantheon, with big tables to manage our circulation, etc, with sizes over 1Gb and close to 2 million records. When we clone the environment for CI or dev purposes, it can take a while (15-30 minutes). So we have created a reference environment called master-lite, which mimics the content, but the huge tables are truncated. This exponentially reduces the backup size and time to clone.

We've set up some CI to test new branches we push up (hattip to Steve Persch, who helped us better understand CI and DevOps) and it will create a CI environment based on the master-lite, which doesn't take as long. Then one of the things it will do is perform a visual regression test between the CI environment (based on my dev branch) and the master-lite environment (which is based on the master branch).

All that sounds great, but unless I keep master-lite up-to-date, it can get stale pretty quickly, so I have created this Bash script to run several terminus commands to build the master-lite environment from our production environment:

Along with this script, I have separate .sql files named after the sites which have site-specific SQL commands to trim away the fat, so to speak. This script also has an if branch to exclude "site3" since that site hasn't grown to the point that I need to do any trimming.

Then I set this up as a cronjob to run every day at 3am from a separate server and now my master-lite environment is kept up-to-date automatically and it's much faster/easier to clone for CI/Dev purposes.

2 comments:

  1. Jason, what tool do you use for visual regression testing?

    ReplyDelete
    Replies
    1. We use Backstop for visual regression testing. It's working really great because it saves "artifacts" in CircleCI and we can look at the test results. I don't have it tuned correctly because I get a lot of false positives, but I'm getting there!

      Delete