Upgrading Drupal from 8.3 to 8.5

Background

I am the sysadmin and developer for Art & Object, a Drupal 8 website built with the Drupal Composer project. The version pin in composer for Drupal was 8, which in hindsight was too broad for our usage. Meaning Drupal point releases (8.3 to 8.4) require study to ensure you understand all the implications, which wasn't something I did. I just blindly did a composer update, thinking everything would be handled automatically.

This really bit me when 8.4 came out because my server was running Debian Jessie, which runs PHP 5.6 and my composer didn't have a platform PHP configuration, so a lot of the underlying Symfony code updated to PHP 7. So I ended up doing a backgrade until I figured it out.

Then there were the critical security Drupal updates (SA-CORE-2018-002 and SA-CORE-2018-004) earlier this year that would not be released for 8.3, so I had to upgrade (or at least, at the time, I felt I had to, though I see now they have a patch for older 8.x releases). By that time, 8.5 was released, so I updated the composer to 8.5 and ran update and after some basic testing, moved on.

Then a few months later, I noticed the status error messages about running the contrib media module alongside the core and I knew I missed something and there was a problem.

I then started down a wicked rabbit hole of getting a local copy running and following the upgrade instructions, running into problem and going back to getting a local copy running fresh again and trying again. Lots of trail and error (mostly errors) and head-banging-on-the-desk. I looked for help on the #media IRC channel, but the best advice came from posting on Stack Overflow, where @sonfd pointed out that the media module needs to be uninstalled first. I thought I had tried that and ran into an error message that mentioned you can't uninstall the media module with media items already created.

The Fix

So after lots and lots (and lots) of local refreshes and trials and errors, here's the list I finally followed when it came time to upgrade production:

1. First, put the site in maintenance mode. Then take a database backup and make a tarball of your project directory. Don't skip over this.
2. drush pmu media crop_media_entity: pmu = pm-uninstall. Remove the media module (and crop_media_entity, if you have that, too). This was the tip from @sonfd that opened the rest of this process for me.
3. composer remove drupal/media: Remove the contrib media module from the filesystem. I should add that I prefixed all my composer commands with /usr/bin/php -d memory_limit=-1 /usr/local/bin/ because I often ran into memory limits when running composer.
4. composer require drupal/inline_entity_form drupal/crop:1.x-dev drupal/media_entity_instagram:2.x-dev drupal/media_entity:2.x-dev drupal/media_entity_slideshow:2.x-dev drupal/media_entity_twitter:2.x-dev drupal/slick_media:2.x-dev drupal/media_entity_actions: These modules are temporary to help upgrade the database records.
5. composer remove drupal/video_embed_field: For some reason, I couldn't require video_embed_field:2.x-dev, so I removed it and then...
6. composer update: When I ran this, it updated video_embed_field to 2.x-dev.
7. composer require drupal/media_entity_image drupal/media_entity_document drupal/image_widget_crop: More temporary modules to help the upgrade process.
8. drush cr: Clear cache to make sure Drupal picks up new modules and paths.
9. drush updb: Run the database updates.
10. drush pmu entity media_entity: Uninstall these modules (these were the old contrib modules)
11. composer remove drupal/media_entity drupal/media_entity_image drupal/media_entity_document drupal/crop drupal/image_widget_crop
    /usr/bin/php -d memory_limit=-1 /usr/local/bin/composer require drupal/crop:2.x-dev drupal/image_widget_crop drupal/empty_page:2: Clean out the temporary modules from the filesystem.
12. drush cr: Clear caches
13. drush updb: Run database updates
14. drush cex: Export the configuration (so you can commit it later).
15. The blazy module had an error with the core media and hasn't been updated (as of this writing), but there is a patch to fix that. So I learned how to add patches to a composer file - turned out pretty simple. Add this to composer.json in the extra section:
            "patches": {
                "drupal/blazy": {
                    "Gets Blazy to work with Drupal Core Media": "https://www.drupal.org/files/issues/2881849-8.patch"
                }
            }
16. composer update: This was odd, but I had to do an update, which picked up the patch, but didn't really install it. I can't remember exactly now, but I believe this actually deleted the blazy folder.
17. composer remove drupal/blazy: So removing this actually installed it. Who knew? Whatever ... it's still in my composer.json and now the filesystem has the module and the patch.
18. drush cr: Clear caches!
19. For some reason, this upgrade created a new field called field_media_image_1 and assigned that as the source for the image media type, which broke some of the images on the site. So I edited media.type.image.yml file to revert source_field back to my original field_image.
20. drush cim: Import my hack to get my media image type to work.
21. I had a custom field formatter that I had to edit to change the namespace from media_entity to media.
22. drush cr: Final cache clear!
23. Test and make sure all is well. If so, take the site out of maintenance mode and commit your repo changes.
    

Advice / Conclusion

A lot of this pain could be negated by studying the release notes better. I own that and this counts as one of my many scars of lessons learned. I hope others can learn from my lesson, too. Someone may end up writing a meta post about this post to point out the high cost of maintaining a Drupal site and I don't think they'd be wrong about that, but that's the price you pay for running servers that are publicly accessible.