Five days to go until Drupal 8 EOL: automated code fixes

With five days to go until Drupal 8's end of life (on November 2, 2021), now is a good time to take stock of your Drupal 8 sites' modules. Use Upgrade Status to check for environment and module compatibility with Drupal 9.

If you took my advice a couple days ago and took on co-maintainership of a project, you may be wondering about the fastest way to get drupal.org projects Drupal 9 compatible. Last year, we introduced the Project Update Bot that submits compatibility issues with patches to drupal.org projects. If it detects all compatibility issues identified are fixed, it will also suggest updating the core compatibility information for the project within the same patch. Committing such patches will potentially entirely fix extension compatibility. Hundreds of those issues have been reviewed and marked as tested by folks in the community and a lot more are waiting for review or are under discussion.

For your own custom code, I highly advise that you use the tools behind the update bot, specifically the Drupal Rector tool funded by Palantir.net. It will attempt to fix as many deprecated API uses as possible. According to our analysis of all contributed projects, almost half of all identified Drupal 9 incompatibility issues on drupal.org are covered by Drupal Rector automated fixes. So running Drupal Rector on your own code first will save you a lot of manual work and time. Upgrade Status will help identify the rest of the problems too and link to documentation to help fix them.

You can and should make all of these changes on your Drupal 8 site, given that the fixed code will keep running fine on Drupal 8. Once you updated contributed projects used on your site, fixed your custom code and ensured a compatible environment, you can upgrade to Drupal 9.

Six days to go until Drupal 8 EOL: use incompatible extensions

With six days to go until Drupal 8's end of life (on November 2, 2021), now is a good time to take stock of your Drupal 8 sites' modules. Use Upgrade Status to check for environment and module compatibility with Drupal 9.

While a lot of Drupal extensions are now compatible with Drupal 9, it can happen that you encounter an incompatible extension. In case adopting the extension fits into your plans, that is the best way forward. However, there may be some delay until you can adopt the project and it may not fit into your plans to adopt in the first place. There are a few solutions to use drupal.org projects that are distributed as incompatible on your Drupal 9 sites.

Obtaining the extension

I do not advise to download the extension as a zip or tar.gz file and add it to your project. As I posted yesterday, it is best to use Composer to ensure all Drupal-specific and third party dependencies are resolved. Manually downloading does not ensure this. But composer will not let you download incompatible extensions!

This is where drupal.org's recently added lenient Composer endpoint comes in. Essentially this is a package repository that advertises only incompatible projects, but it exposes them as compatible instead. Using Composer 2, you can add this additional repository on top of the existing drupal.org endpoint with a higher priority. This way you can transparently add packages that are yet incompatible to your projects. As projects get compatible with Drupal 9, they will disappear from the lenient endpoint and their entry on the standard drupal.org endpoint will transparently take over on your setup.

Making code compatible

Now that you have the codebase on your site, you will still be unable to use the project, as it is indeed still incompatible. If the only patch the extension needs is to its info.yml file where it declares compatibility with Drupal 9, the simplest solution may be to install the Backward Compatibility module. If additional patching is required, or you prefer not to use the Backward Compatibility module, you can install and use the cweagans/composer-patches package for Composer and use it to apply the compatibility fix.

Other methods

The above method is not the only solution to use incompatible projects. Prior to the availability of the lenient Composer endpoint, Damien McKenna wrote a detailed blog post about using the git repository of merge requests to add to your site and Benji Fisher wrote a practical guide about that too. Even before those, James Williams wrote about using the base git repository of projects and applying fixes with composer-patches. These methods require a lot more special case entries in your composer file and would require you to manually track if the project became compatible.

The benefit of the lenient Composer endpoint is that it provides one unified source of projects. For projects you use with Backwards Compatibility, they will seamlessly convert to compatible projects on your site as their drupal.org releases become compatible. For projects you use with composer-patches, Composer will report the inability to apply patches when updates become available, where you have an explicit notification that either the project became compatible or conflicting changes were added.

Seven days to go until Drupal 8 EOL: start using Composer!

With seven days to go until Drupal 8's end of life (on November 2, 2021), now is a good time to take stock of your Drupal 8 sites' modules. Use Upgrade Status to check for environment and module compatibility with Drupal 9.

Various people in the Upgrade Status issue queue are concerned about the module requiring Composer though. While Drupal 8 or 9 do not require Composer to work, getting any extension installed on your site that depends on third party components is a sizeable challenge without Composer. With Drupal 8 core dependent on various third party components, it was inevitable that a dependency manager will be needed to build Drupal sites. Upgrade Status itself does require various third party components to check for deprecated API uses and even to locate your Drupal site root. So you cannot avoid using Composer to even check for Drupal 9 compatibility.

If you are not using Composer yet, the best case scenario is you use this opportunity to convert your Drupal site to a Composer based solution. If you started your Drupal 8 site from 8.8.0 or later, the file structure is already ready for Composer. For Drupal sites started on prior versions, the move is more work, as explained in the guide.

In case you are not ready to convert your site to Composer yet, but still want to get a Drupal 9 upgrade readiness check, you need to set up a separate, parallel site with Composer just for this purpose. Your existing site content or configuration does not matter for Upgrade Status checks, you only need to replicate the enabled projects on the site you use.

Install Composer first. It is best to use Composer 2 because it will be much faster and will use significantly less memory, but you should have no problem installing Upgrade Status with Composer 1 either. After that, run these commands to create a new Drupal site from Composer, add core developer dependencies and add this module as a requirement.


$ composer create-project 'drupal/recommended-project:~8.9' d9readiness
$ cd d9readiness
$ composer show drupal/core | grep versions
$ composer require --dev drupal/core-dev:[copy version above]
$ composer require drupal/upgrade_status


Then, copy all of your custom and contributed projects to the web/modules, web/themes, and web/profiles folders. (You could use Composer to add at least the contributed ones, but since you are not planning to use this build for anything else, this will be the quickest way). Finally, visit /admin/reports/upgrade-status, and run the report.

This should help you get a summary of where your site is at. I would suggest you plan for converting to a composer based setup and use composer for dependency management going forward. You will only need composer on your development environment and can still use your usual workflow to push your fully built codebase from there.

Eight days to go until Drupal 8 EOL: adopt a module!

With eight days to go until Drupal 8's end of life (on November 2, 2021), now is a good time to take stock of your Drupal 8 sites' modules. Use Upgrade Status to check for environment and module compatibility with Drupal 9. While more than 7 thousand modules are now Drupal 9 compatible, there are still various that are not yet compatible. However, most only need a one line change and a release or even if they need more, an automated fix is posted to their issue queue and is tested by the community.

Many of these incompatible modules have been posted by people who contributed their code but don't have the capacity to maintain it. I don't think that is a problem. That they contributed in the first place, and that their contribution is so useful that it is used on live Drupal sites itself proves the value. So I think we should appreciate and celebrate these folks. On the other hand, we need to acknowledge the lack of maintenance and find new maintainers who can bring these projects forward.

There is an existing process for taking over abandoned projects on drupal.org, however the Drupal Association decided to speed up the first steps of this by automatically opening issues in the affected project's queues two weeks before DrupalCon Europe. You can find the over 3 thousand projects that are under this process at https://bit.ly/drupaladopt (open issues tagged with "Project available for adoption"). If an issue has already been closed, that means the maintainer has declined new maintainer help, so please look at the open issues only.

The steps to request co-maintainership are:

  1. Comment on the issue explaining why you would like to maintain the module.

  2. If the project is opted in to security coverage, confirm that you have previously received security coverage opt-in permission.

  3. If an existing maintainer has not commented, move the issue to the Drupal.org Project Ownership queue by editing the 'Project' field on this issue.

  4. From there, a Drupal.org Site Moderator will review the issue and grant maintainership if the requirements are met.

Once you gain co-maintainership, make sure to commit the Drupal 9 compatibility fixes and make a release of the project, so people can rely on a fixed codebase for their sites.

You rock! Thanks for keeping a part of Drupal's ecosystem up to date!

Drupal 10 readiness will be hugely automated, 95% of deprecated Drupal APIs already covered

With the Drupal 8 end of life in a little over two months and Drupal 10's release next year, this is the time of transitions again at Drupal. However, while Drupal 7 to 8 (or 9) was a big move, the transitions from 8 to 9 and 9 to 10 are much smaller and mostly automated.

Drupal 10 is planned to be released in June, August or December 2022 and the tools are getting ready to support that. The two key tools will be the same as the previous upgrade: Upgrade Status and Drupal Rector.

Matt Glaman has been doing amazing work recently in the underlying components of both tools. Thanks to his work on updating phpstan-drupal for Drupal 10 support, Upgrade Status checks deprecated API uses on Drupal 9 too. Since my last update on that, I added reporting of deprecated modules and new system requirements as well.

Ryan Aslett at the Drupal Association built the project analysis job on top of Upgrade Status and that is run on Drupal 9 projects now as well. So we have an idea of the extent of work that will need to be done for Drupal 10 compatibility. I've updated the dev.acquia.com deprecation dashboard to show Drupal 10 results. While projects should not be expected to be Drupal 10 ready at this time, the dashboard helps us prioritise work on certain parts of the tooling to help the ecosystem upgrade.

To support that, Palantir.net has been sponsoring Matt Glaman to also bring Drupal Rector up to date for Drupal 10 readiness. The results are already outstanding! Today, of the 22204 Drupal deprecated API uses identified in Drupal 9 compatible projects. These are green and purple on the chart below. Drupal Rector has automations to fix 95% of them (green). There are a further 5391 non-Drupal deprecated API uses (yellow) including Symfony and PHPUnit deprecated APIs. Those themselves have third party rectors, so the coverage will further improve by including those. That is in the works.

Image showing the current state of Drupal 10 readiness of Drupal 9 compatible contributed projects

The drupal.org Project Update Bot resolves rectorable deprecated API uses (green) and info/composer issues (blue) when posting patches, so this means that it will be able to resolve most deprecated APIs in its suggested fixes already and we expect it will improve more with third party rectors added.

Drupal 10 itself is a moving target, the branch will be open around October/November so the above does not mean that the tools are complete, but we are significantly further ahead this time compared to the Drupal 8 to 9 transition, making the upgrade to Drupal 10 smoother for everyone.

With the Drupal 8 release, we decided upgrades must be easier going forward and thanks to the fantastic work of contributors and sponsors, we are doing it again.