Senior Support Lead
Allan brings technological know-how and grounds it with some simple country living. His interests include DevOps, animal husbandry (raising rabbits and chickens), hiking, and automated testing.
January 1, 1970
At the time of this blog, we have done two major version upgrades of Drupal and have refined the process along the way. There has been a lot of work in the community, through the efforts of people like Matt Glaman to make this process easier.
As a Support Engineer, I see a lot of approaches for achieving the same results in many areas of my work. Here, I’d like to share with you three different ways to achieve an upgrade of a module or theme that isn’t ready for the next major Drupal version, each with pros and cons, but all absolutely acceptable.
Why do we have this problem?
All new Drupal developers have a hard time with the layers of code changes that happen in the Drupal community. We have custom package types, custom install locations, patches, and scaffolding. To make the challenges worse, we have two ways to identify a module’s dependencies — that being a .info.yml file and for some, a composer.json. This is because some Drupal modules may want to build upon an existing PHP library or project, in addition to other Drupal modules. To ease the pain of having to define some dependencies twice, both in the .info.yml file and composer.json file, Drupal.org built their packagist, a repository of Composer packages, to read the .info.yml files from the root of the project and create Composer version constraints from that. For example, if the .info file contained the following:
name: My Module type: module core_version_requirement: ^8.8 || ^9 dependencies: - ctools:ctoolsThen Drupal.org’s packagist would create the following for the release that contained that .info.yml file, saving the contributed developer a lot of trouble.
{ "type": "drupal-module", "name": "drupal/my_module", "require": { "drupal/core": "^8.8 || ^9", "drupal/ctools": "*" } }I hit on something there, though. It will create that for the release the .info.yml was in. When most code changes come in the form of patches, this poses a challenge. You apply your patch to the .info.yml after you download the release from Drupal.org’s packagist. Additionally, Drupal.org doesn’t create a new release entry for every patch file in the issue queue. So you are left with the question, “How do I install a module on Drupal 10 that requires Drupal 9 so that I can patch it to make it compatible for Drupal 10?”
Drupal Lenient
One of the easiest methods for those who don’t understand the ins and outs of Composer is to use the Drupal Lenient plugin. It takes a lot of the manual work out of defining new packages and works with any drupal-* typed library. Types are introduced to us through the use of the Composer Installer plugin and manipulated further with something like Composer Installers Extender. Composer plugins can be quite powerful, but they ultimately add a layer of complexity to any project over using core composer tactics.
Drupal Lenient works by taking any defined package pulled in by any means via Composer, and replaces the version constraints for drupal/core currently, at the time of this writing, with “^8 || ^9 || ^10“. So where the requirements might look like the example earlier “drupal/core“: “^8.8 || ^9“, they are replaced, making it now possible to install alongside Drupal 10, even though it might not be compatible yet. This allows you to patch, test, or use the module as is, much like if you would have downloaded the zip and thrown it into your custom modules directory.
An example may look like this:
{ "name": "vendor/project", "repositories": [ { "type": "composer", "url": "https://packages.drupal.org/8" } ], "require": { "drupal/core": "^10.0.0", "drupal/my_module": "1.x-dev", "cweagans/composer-patches": "^1.7.3", "mglaman/composer-drupal-lenient": "^1.0.3" }" extra": { "composer-exit-on-patch-failure": true, "drupal-lenient": { "allowed-list": [ "drupal/my_module" ] }, "patches": { "drupal/my_module": { "3289029: Automated Drupal 10 compatibility fixes": "https://www.drupal.org/files/issues/2022-06-16/my_module.1.x-dev.rector.patch" } }, "patchLevel": { "drupal/core": "-p2" }, } }Note the Drupal-Lenient allow list. Also note that you will need to make sure and install the plugin before trying to install the module that doesn’t support Drupal 10 in this case. If you want an excellent step-by-step, Matt put one together in the Readme.
The pros:
- Easy-peasy to install
- Feeds off the original packagist packages, so if there is an upgrade, you don’t have to do anything special to transition
The cons:
- Lenient has the control and may cause inexplicable errors when updating due to unsupported core versions
- PHP devs not familiar with Drupal Lenient won’t know to look for it
- Flaky experiences when switching in and out of branches that include this plugin. If you context switch a lot, be prepared to handle some errors due to Composer’s challenges maintaining state between branches.
- Patches to other dependencies inside composer.json still require you to run through some hoops
Custom package
If you want more control over what the module can and cannot do, while keeping the core of Composer functionality without adding yet another plugin, check out this method. What we will do here is find out what version the patch or merge request is being applied against. It should be stated in the issue queue and by best practices is a dev version.
If you are a perfectionist, you can use composer install -vvv to find the url or cache file that the module came from for packages.drupal.org. It is usually one of https://packages.drupal.org/files/packages/8/p2/drupal/my_module.json or https://packages.drupal.org/files/packages/8/p2/drupal/my_module~dev.json. You will note that the Composer cache system follows a very similar structure, swapping out certain characters with dashes.
With this information, you can grab the exact package as it’s defined in the Drupal packagist. Find the version you want, and then get it into your project’s composer.json.
Let’s use Context Active Trail as an example, because at the time of this writing, there is no Drupal 10 release available.
Looking through the issue queue, we see Automated Drupal 10 compatibility fixes, which has a patch on it at. I grab the Composer package info and paste the 2.0-dev info into my composer.json under the “repositories” section as a type “package.”
Which should make your project look something like this:
{ "name": "vendor/project", "repositories": [ { "type": "package", "package": { "keywords": [ "Drupal", "Context", "Active trail", "Breadcrumbs" ], "homepage": "https://www.drupal.org/project/context_active_trail", "version": "dev-2.x", "version_normalized": "dev-2.x", "license": "GPL-2.0+", "authors": [ { "name": "Jigar Mehta (jigarius)", "homepage": "https://jigarius.com/", "role": "Maintainer" }, { "name": "jigarius", "homepage": "https://www.drupal.org/user/2492730" }, { "name": "vasi", "homepage": "https://www.drupal.org/user/390545" } ], "support": { "source": "https://git.drupalcode.org/project/context_active_trail", "issues": "https://www.drupal.org/project/issues/context_active_trail" }, "source": { "type": "git", "url": "https://git.drupalcode.org/project/context_active_trail.git", "reference": "8dc46a4cf28e0569b187e88627a30161ee93384e" }, "type": "drupal-module", "uid": "context_active_trail-3192784", "name": "drupal/context_active_trail", "extra": { "branch-alias": { "dev-2.x": "2.x-dev" }, "drupal": { "version": "8.x-2.0-rc2+1-dev", "datestamp": "1630867980", "security-coverage": { "status": "not-covered", "message": "Project has not opted into security advisory coverage!" } } }, "description": "Set the active trail based on context.", "require": { "drupal/context": "^4.1", "drupal/core": "^8.8 || ^9" } } }, { "type": "composer", "url": "https://packages.drupal.org/8" } ], "require": { "drupal/core": "^10.0.0", "drupal/context_active_trail": "2.x-dev", "cweagans/composer-patches": "^1.7.3", "mglaman/composer-drupal-lenient": "^1.0.3" }" extra": { "composer-exit-on-patch-failure": true, }, "patches": { }, "patchLevel": { "drupal/core": "-p2" }, } }Now let’s change our version criteria:
… "description": "Set the active trail based on context.", "require": { "drupal/context": "^4.1", "drupal/core": "^8.8 || ^9 || ^10" } …And then add our patch:
… extra": { "composer-exit-on-patch-failure": true, }, "patches": { "drupal/context_active_trail": { "Automated Drupal 10 compatibility fixes": "https://www.drupal.org/files/issues/2023-09-29/context_d10comp_3286756.patch" } }, "patchLevel": { "drupal/core": "-p2" }, } …Here, you will need to look to see if the patch is patching composer.json. If it is, you will need to modify your package information accordingly. For example, in this one, the fixer changes drupal/context from ^4.1 to ^5.0.0-rc1. That change looks like this:
Lastly, sometimes you run into some complications with the order packages are picked up by Composer. You may need to add an exclude element to the Drupal packagist.
Our final composer.json for our project could look something like this with all the edits:
{ "name": "vendor/project", "repositories": [ { "type": "package", "package": { "keywords": [ "Drupal", "Context", "Active trail", "Breadcrumbs" ], "homepage": "https://www.drupal.org/project/context_active_trail", "version": "dev-2.x", "version_normalized": "dev-2.x", "license": "GPL-2.0+", "authors": [ { "name": "Jigar Mehta (jigarius)", "homepage": "https://jigarius.com/", "role": "Maintainer" }, { "name": "jigarius", "homepage": "https://www.drupal.org/user/2492730" }, { "name": "vasi", "homepage": "https://www.drupal.org/user/390545" } ], "support": { "source": "https://git.drupalcode.org/project/context_active_trail", "issues": "https://www.drupal.org/project/issues/context_active_trail" }, "source": { "type": "git", "url": "https://git.drupalcode.org/project/context_active_trail.git", "reference": "8dc46a4cf28e0569b187e88627a30161ee93384e" }, "type": "drupal-module", "uid": "context_active_trail-3192784", "name": "drupal/context_active_trail", "extra": { "branch-alias": { "dev-2.x": "2.x-dev" }, "drupal": { "version": "8.x-2.0-rc2+1-dev", "datestamp": "1630867980", "security-coverage": { "status": "not-covered", "message": "Project has not opted into security advisory coverage!" } } }, "description": "Set the active trail based on context.", "require": { "drupal/context": "^5.0.0-rc1", "drupal/core": "^8.8 || ^9 || ^10" } } }, { "type": "composer", "url": "https://packages.drupal.org/8", "exclude": [ "drupal/context_active_trail" ] } ], "require": { "drupal/core": "^10.0.0", "drupal/context_active_trail": "2.x-dev", "cweagans/composer-patches": "^1.7.3", "mglaman/composer-drupal-lenient": "^1.0.3" }" extra": { "composer-exit-on-patch-failure": true, }, "patches": { "drupal/context_active_trail": { "Automated Drupal 10 compatibility fixes": "https://www.drupal.org/files/issues/2023-09-29/context_d10comp_3286756.patch" } }, "patchLevel": { "drupal/core": "-p2" }, } }The pros:
- Uses more core Composer functionality
- A PHP developer will better understand what’s going on here
- You are in complete control of how this module package and version are defined
- All the work is in one file
The cons:
- Requires some understanding of how composer.json, packagists, and the magic of Drupal’s packagist all work
- That’s a messy composer.json for the project
- If you have to use
exclude, you have to leave it up to outside forces to let you know when that module does finally put out and actual D10-ready version, and then undo all of this work
Fork it
Standard PHP composer best practice says that if you make modifications to a package, fork it, maintain your modifications, and provide a pull request if it’s functionality you wish to contribute back. You can use this same approach with Drupal modules as well. Some may even say that’s what issue forks are for! That said, issue forks come with the downside that sometimes they go away, or are overridden with changes you don’t want. They are a moving dot.
For the sake of this example, let’s assume that we have forked the module on GitHub to https://github.com/fourkitchens/context_active_trail.git. If you don’t know how to make a fork, simply do the following:
- Clone the module to your local computer using the git instructions for the module in question
- Check out the branch you want to base your changes on
- Create a new repository on GitHub
- Add it as a remote
git remote add github git@github.com:fourkitchens/context_active_trail.git - Push it!
git push github 8.x-2.x
You can do this with a version of the module that is in a merge request in Drupal.org’s issue queue, too. That way you won’t have to reapply all the changes. However, if your changes are in a patch file, consider adding them to the module at this time using your favorite patching method. Push all your changes to the github remote.
If the patch files don’t have changes to composer.json, or if the module doesn’t have one, you will likely want to provide at least a bare-bones one that contains something like the following and commit it:
{ "name": "drupal/context_active_trail", "type": "drupal-module", "require": { "drupal/context": "^5.0.0-rc1", "drupal/core": "^8.8 || ^9 || ^10" } }This will tell Composer what it needs to know inside the project about dependencies. This project already had a composer.json, so I needed to add the changes from the patch to it.
Inside our Drupal project we are working on, we need to add a new entry to the repositories section. It will look something like this:
{ "type": "vcs", "url": "https://github.com/fourkitchens/context_active_trail.git" },The VCS type repository entry tells Composer to look at the repository and poll for all its branches and tags. These will be your new version numbers.
Much like in the “Custom Package” example, you may need to add an exclude property to the Drupal packagist entry.
Now, since Drupal packagist isn’t here to give Composer some version aliases, we have to use the old notation dev-BRANCHNAME for our version. Our require entry will look something like this:
Since we already added our patches as a commit to the module, this is all you need. Your final composer.json for your project would look like this:
{ "name": "vendor/project", "repositories": [ { "type": "vcs", "url": "https://github.com/fourkitchens/context_active_trail.git" }, { "type": "composer", "url": "https://packages.drupal.org/8", "exclude": [ "drupal/context_active_trail" ] } ], "require": { "drupal/core": "^10.0.0", "drupal/context_active_trail": "dev-8.x-2.x", } }It makes for a much cleaner project json, but now you’ve split the work into two locations, requiring some synchronization. However, if multiple sites of yours use this same module and need the same fixes, this absolutely has the least resistance and ability to get those changes out more quickly.
The pros:
- Reusability
- Two smaller, simpler chunks of work
- Any PHP developer should be able to debug this setup as it uses Composer best practices. This method will be used in any project with any framework in the PHP ecosystem.
The cons:
- Changes are in two separate places
- Which patches are applied isn’t obvious in the composer.json and require looking through the commit history on the forked repository
- Requires maintenance and synchronization when upgrades happen
Final thoughts
As with almost everything out there, there are multiple ways to achieve the same goal. I hope this brings awareness, and helps provide the flexibility you need when upgrading Drupal to a new major version. Obviously, each solution has strengths, and you may need to mix it up to get the results you want.
The post Using Composer for modules not ready for the next major version of Drupal appeared first on Four Kitchens.