Drupal 10

Last week was DrupalCon Portland, our first in-person big DrupalCon back together with 1300+ attendees. I was fortunately one of the attendees along with leaders of in-the works Drupal admin theme Claro and frontend theme Olivero. Both Claro and Olivero have been in the works for years, and both were quite close to get done, yet we did not even dream of getting both be the new defaults in Drupal within the week. But we did it!

Claro replaces Seven as the default administration theme and Olivero relaces Bartik as the default frontend theme in both Drupal 9.4.0 and Drupal 10.0.0. Both prior themes have been the Drupal defaults for 11 years, ever since Drupal 7 was released. Both themes received a lot of key contributions throughout the years, so many to even attempt to recount here. This is a guest post by Lauri Eskola (with additions by myself) on the story of how the final week went. Read on to peek into how we worked at DrupalCon to makes these historic changes happen.

Claro became stable and the default administration theme

#Drupal Admin UI / Claro BoF at @drupalcon #DrupalCon pic.twitter.com/49lMTqCj4c

— Mike Herchel (@mikeherchel) April 25, 2022

First Cristina Chumillas (ckrina) organized a BoF discussion about Claro on Monday morning. We promoted the BoF on social media and on the contribution page on the DrupalCon site.

The focus of the BoF was to discuss the remaining scope for getting Claro stable and default in the Standard profile and if there is anything in the scope that could be reduced. We were able to descope a few issues from the roadmap, which got a sign-off from Gábor Hojtsy later that day as a product manager onsite. We also created a battle plan for the remaining issues to make sure that everyone involved knew what needs to happen and who should take action on each of the issues.

After the BoF, we met Matthew Grasmick during lunch, and he half jokingly suggested that we should try to finish the roadmap by Dries’ keynote on Wednesday so that he could announce the good news there. This inspired the team to do the unthinkable; to try to finish the remaining part of the roadmap by Wednesday morning.

On Monday Lauri Eskola worked on the roadmap with Cristina, Emilie Nouveau, Mike Herchel and Kat Shaw. We also received support from Angie Byron and Gábor on making sure that we are on the right track with the solution for the last stable blocking issue that required code changes.

Hace you heard? Just one more a11y META issue to go and we can mark Claro as stable!! #DrupalCon @laurii1 go go! pic.twitter.com/gRIEYT8PBL

— Cristina (@chumillas) April 27, 2022

Gábor also did the vast majority of the actual code patches required for marking Claro stable and the default theme.

We were able to land the last stable blocking issue on Tuesday morning. Now all that was left was finishing up on the accessibility assessments. Some of the assessments were done and could be closed, but forced colors mode and focus visibility still needed some more assessment.

We received help from Ben Mullins who was offsite, and in the meanwhile Lauri was able to collaborate with Cristina, Emilie, Mike, Kat, Adam Bergstein, Baddý Breidert, and Ted Bowman onsite at the event.

All of the assessments were done by the evening. This was after we had already left the venue. Angie, Cristina, Lauri and Gábor headed for dinner and Lauri was still trying to file follow-up issues for the problems discovered as part of the assessment from Angie’s car, but he found that Ben already submitted them.

Once we got to the restaurant, all of the issues had been closed and all that was left was for someone to commit the patch that marked Claro stable.

Angie was one of the people who were involved in starting the whole project, so she was the perfect person to commit the issue.

Wohoooooo congratulations to the team working to get Claro stable and default! @chumillas @laurii1 @webchick Thank you for all your hard work!! That picture is sooo cute #DrupalCon #DrupalConPortland pic.twitter.com/cuBISEyh9m

— Fatima (@sugaroverflow) April 27, 2022

We requeued the issue to mark Claro the default theme in Standard for testing with Claro being stable and went on with our dinner. It was great vegan food by the way, we suggest you check out Blossoming Lotus if you are in Portland.

Afterwards we moved to Gábor’s hotel lobby and Angie committed the Standard profile patch too.

In the meantime, Benji Fischer, who was at the conference but not with us in person noticed that Claro became stable and restarted work on making it the administration theme of the Umami demo profile as well. Since we were at it already, Cristina reviewed and tested this patch and she committed that from the hotel lobby as well.

Twelve hours later, Dries announced the huge milestone at the Driesnote with a photo of Angie's commit from the hotel. Years of work suddenly becoming real in a couple days!

Olivero became the default frontend theme

We are getting closer to make #Olivero default in #Drupal. @gaborhojtsy @laurii1 @mikeherchel are preparing it now at #DrupalConPortland #DrupalCon pic.twitter.com/aX5V6iTOmw

— Baddy Sonja Breidert (@baddysonja) April 28, 2022

Inspired by the success of Claro, people started working on the remaining blockers to make Olivero the default theme after the Driesnote. Olivero was already stable, and all that was left was to do quality assurance checks, which was done by Mike Herchel, and to get tests to pass with Olivero being the default theme.

Getting all of the tests to pass with Olivero being the default theme in Standard profile was much more complex than we had anticipated. This had been worked on for months before the event through various spin-off issues by several contributors, but numerous details remained to solve.

The work was finished during the event by Joe Shindelar onsite with the help of Spokje, Lee Rowlands and Amber Himes Matz offsite.

We got sign-offs by Thursday morning from Nathaniel Catchpole (release manager) offsite and Gábor (product manager) onsite that Olivero could also be made the default theme of Standard.

We got all of the work done by Thursday afternoon, which raised an idea that we could commit the patch live at the beginning of trivia night. We were anxiously working on this up until the last minute, because DrupalCI was having a very rough day with most test runs across various issues running into random failures.

We were literally watching multiple DrupalCI runs live at the event so that we could start researching any test failures as soon as they appeared.

The final push to make Olivero the new default Drupal 9/10 theme landed in a live commit at @DrupalConNA trivia night! Joined on zoom were @shadow4611 @jponch @putrabon @andy__blum, while the crowd cheered on! Thanks @lullabot and all the contributors. #ContributeAtDrupalCon pic.twitter.com/Be3ACCeSnP

— Gábor Hojtsy (@gaborhojtsy) April 29, 2022

Final fixes were put in place by Joe Shindelar in the afternoon. Luckily we managed to get green test runs and had a successful live commit at the beginning of trivia to 9.4.x.

Mike got on stage to explain what is going on and offsite Olivero team members Jen Witkowski, Putra Bonaccorsi, Jared Ponchot and Andy Blum all called in via a Zoom meeting. The 10.x clean test result came back a bit later, but we merged to 10.x as well before trivia started.

This was extremely rewarding and a perfect way to end this very successful event! It felt like a full circle as Olivero got started as an idea at the last in-person DrupalCon and got to land at this in-person DrupalCon.

While getting these two themes be the new Drupal defaults was a highlight for us, this DrupalCon was notable for many other reasons. It was great to see our friends again after several years only on video calls. Want to be in the next epic story moving the drop? DrupalCon Europe is coming back to Prague in September, ticket sales are about to start. We plan to be there as well working on the next major things for Drupal!

This weekend (Friday to Sunday) is Drupal Global Contribution Weekend 2022! While there are some in person events such as in Ukraine, there are various online events that allow anyone to join. I'll highlight three opportunities to work on your Drupal 9 or Drupal 10 readiness.

1. Wim Leers just announced an opportunity to try CKEditor 5 with step by step instructions. He will be available for some office hours on all three days to consult with. CKEditor 5 is going to be the content editor available in Drupal 10, replacing CKEditor 4. So getting ready for it sooner than later is a good idea. A beta experimental version of the integration already exists in Drupal 9.3 to take for a test ride.
2. Joel Pittet with the UBC Computer Science department is hosting a Zoom event to work on Drupal 9 and 10 readiness from 9am to 5pm on all three days Vancouver time (UTC-8). Folks working on other tasks are also welcome.
3. I'll be available on Friday 9am to 5pm as well CET (UTC+1) in both the #d9readiness and #d10readiness channels on Drupal slack (https://drupal.org/slack).

Some common tasks that I think are good to take now:

• If your contributed project is not Drupal 9 compatible yet, it is a good time to do that now. You can use Upgrade Status on Drupal 8 and Drupal Rector to find problems and automate fixes.
• While your project cannot be surely Drupal 10 compatible yet, most deprecated APIs for Drupal 10 were marked in Drupal 8. You can use Upgrade Status on Drupal 9 to identify problems and use Drupal Rector again to fix most of them.
• Drupal 10 will require PHP 8 at least, so setting up testing for your project on PHP 8 and/or manually testing on PHP 8 are good ways to make sure this part of compatibility is taken care of. All Drupal 9 core releases support PHP 8, so this does not pose a risk for your existing users on supported Drupal versions.

With my and Joel's events, there will be 2*8 hours of Drupal 9/10 readiness contribution on Friday with an hour of break inbetween. That is followed by 8 hours each on the weekend days as well. Join either to get help on your journey and contribute to Drupal!

As you may know, we are planning to release Drupal 10 in 2022 (as early as June), because Drupal 9's Symfony 4 and CKEditor 4 are both end of life the year after, around the end of 2023. So we plan go give enough time for people to update to Drupal 10 before Drupal 9 goes end of life. A similar situation happened with Drupal 8 to 9 driven by Symfony 3 to 4. However, moving Drupal 10 from Symfony 4 to 5 would again only give us a couple years of time to move on to Symfony 6 next, so the current plan is to move to Symfony 6 straight.

How is it possible to move Drupal 10 from Symfony 4 to 6? Symfony 6.0 was just released a few hours ago and is stable, so by the time Drupal 10 would be released, it will already be at least on Symfony 6.1.

However, the exact same thing happened with Drupal 8, and we did not do a double Symfony version jump update from Drupal 8 to 9 for two reasons: (a) even though Drupal core minor versions are supported for 12 months, Symfony minor versions are only supported for 8 months other than the last LTS minor version and (b) jumping two versions would mean that scanning for and acting on minor version to minor version all deprecations is challenging. Those two did not resolve themselves so to speak, however we have plans to mitigate them.

Symfony security extension for Drupal-used packages

First of all, Symfony normally supports minor releases for 8 months starting with Symfony 5. On the other hand Drupal minor releases are supported for 12 months, so if there are security fixes to be made, this would not be possible on a non-LTS version of Symfony. However we made arrangements with the Symfony team to have two of the Drupal core committers be part of the Symfony security process and be able to backport security fixes as needed for components used by Drupal even if the given Symfony minor version would be normally out of support. The two Drupal core committers involved are Alex Pott and Lee Rowlands! I don't believe there was an official announcement of this agreement yet, however it was in place since September 2019. Special thanks for the open minded collaboration of the Symfony leads as well!

Bridging the deprecations jump with Symfony 5.4

Second is bridging API deprecations. Nathaniel Catchpole outlined the plan for this recently. The problem is that Symfony 4 deprecated some APIs for removal/changes in Symfony 5 and them Symfony 5 deprecated some APIs for removal/changes in Symfony 6. Jumping through two versions gives Drupal 10 a potentially longer lifetime but with a need to solve identifying all deprecated API uses for both. Now that both Symfony 5.4 and 6.0 are out, the plan is to open the Drupal 10.x-dev branch for development very soon and update to Symfony 5.4 as well as other big dependency updates. Then release a Drupal 10.0.0-alpha1 that is based off of Symfony 5.4, which would allow contributed projects and custom code to check against deprecated APIs towards Symfony 6. And only later update to Symfony 6. This way there is a middle-point that allows to make the necessary updates and check deprecated APIs against.

As jumping two major Symfony versions is not something we did before, let us know in the Drupal 10 issue if you can poke holes at the grand plan. While it looks like this will be entirely possible, it would be best to find any potential pain points ahead of Drupal implementers hitting them.

A proposal with overlapping LTS release versions of Drupal core

Lee Rowlands went even further and presented the potential benefits of this plan down the line with Drupal 11 and Drupal 12. Releasing Drupal 10 on Symfony 6 would allow us to support it longer, up to November 2027. However it does not mean that we cannot release Drupal 11 earlier than 2026 let's say. Releasing earlier would allow us to jump faster to Symfony 7 while it would still give Drupal 10 users a more comfortable, longer support timeline. Then we could take advantage of the full Symfony 7 support timeline.

That would result in Drupal users not needing to update every 12 months to keep security support but needing to update every 2.5 years instead, which would lower cost of ownership of Drupal considerably.

Check out this video where Lee explains the details and discuss in the issue titled Adopt a 2 year major release cadence and a 6 month LTS-to-LTS overlap period for Drupal 10 and beyond. (This is not yet a firm policy, it is under discussion).

With the Drupal 8 end of life in a little over two months and Drupal 10's release next year, this is the time of transitions again at Drupal. However, while Drupal 7 to 8 (or 9) was a big move, the transitions from 8 to 9 and 9 to 10 are much smaller and mostly automated.

Drupal 10 is planned to be released in June, August or December 2022 and the tools are getting ready to support that. The two key tools will be the same as the previous upgrade: Upgrade Status and Drupal Rector.

Matt Glaman has been doing amazing work recently in the underlying components of both tools. Thanks to his work on updating phpstan-drupal for Drupal 10 support, Upgrade Status checks deprecated API uses on Drupal 9 too. Since my last update on that, I added reporting of deprecated modules and new system requirements as well.

Ryan Aslett at the Drupal Association built the project analysis job on top of Upgrade Status and that is run on Drupal 9 projects now as well. So we have an idea of the extent of work that will need to be done for Drupal 10 compatibility. I've updated the dev.acquia.com deprecation dashboard to show Drupal 10 results. While projects should not be expected to be Drupal 10 ready at this time, the dashboard helps us prioritise work on certain parts of the tooling to help the ecosystem upgrade.

To support that, Palantir.net has been sponsoring Matt Glaman to also bring Drupal Rector up to date for Drupal 10 readiness. The results are already outstanding! Today, of the 22204 Drupal deprecated API uses identified in Drupal 9 compatible projects. These are green and purple on the chart below. Drupal Rector has automations to fix 95% of them (green). There are a further 5391 non-Drupal deprecated API uses (yellow) including Symfony and PHPUnit deprecated APIs. Those themselves have third party rectors, so the coverage will further improve by including those. That is in the works.

The drupal.org Project Update Bot resolves rectorable deprecated API uses (green) and info/composer issues (blue) when posting patches, so this means that it will be able to resolve most deprecated APIs in its suggested fixes already and we expect it will improve more with third party rectors added.

Drupal 10 itself is a moving target, the branch will be open around October/November so the above does not mean that the tools are complete, but we are significantly further ahead this time compared to the Drupal 8 to 9 transition, making the upgrade to Drupal 10 smoother for everyone.

With the Drupal 8 release, we decided upgrades must be easier going forward and thanks to the fantastic work of contributors and sponsors, we are doing it again.

I presented on the overall status of the Drupal 10 initiative in December at DrupalCon Europe. Then posted an update about the initiative one week ago on the Drupal Core blog.

More recently I worked on making Upgrade Status work meaningfully on Drupal 9 with Andrey Postnikov. Released Upgrade Status 8.x-3.5-alpha1 today to let you test this out. (You may need to use composer require --dev phpspec/prophecy-phpunit to make your phpunit setup complete).

How is this different from prior releases of Upgrade Status? It should run very similar on Drupal 8 as it did before. However prior releases of Upgrade Status explicitly forbid running it on Drupal 9 as the UI was very focused on the transition from 8 to 9. Now the UI elements are adapted and in some cases more general to support running either on Drupal 8 or 9.

Environment requirements are not yet defined for Drupal 10, so that part of the report is intentionally brief. However, we know Drupal 10 will require PHP 8 and the underlying phpstan-drupal library does not work on PHP 8 yet. As a super-important infrastructure piece, I encourage you to sponsor Matt Glaman to make that happen. Even included this link on the module UI.

Should Drupal 9 modules be compatible with Drupal 10 now? Absolutely not! We don't even know the full extent of Drupal 10's API or the final versions of its dependencies. However we do know a lot of deprecated APIs already, some from as far as back as Drupal 8.2.0, and projects can gradually update following those to close the gap with Drupal 10 now.

So running Upgrade Status on Drupal 9 will tell you about .info.yml file incompatibility errors and composer.json incompatibility errors, but I would ignore those for now. The more interesting bits are the already fixable deprecated API uses found that were marked deprecated in Drupal 8 and will be removed in Drupal 10. I installed the top 6 most used modules on a test site, to check the results out and there are plenty you can already help move forward in these modules. Most issues found are related to the legacy deprecated assert methods. It is no problem in Drupal 9 to use them, but those will not be present in Drupal 10 next year. Most have 1-1 replacements, so it would be best to build rector rules for them into drupal-rector and support this transition that way. (Also drupal-rector itself needs to support Drupal 9).

I started to work with the Drupal Association to set up a job to figure out the status of Drupal 10 readiness at this time of the 5000+ Drupal 9 compatible projects. (Similar to the deprecation status data we have about Drupal 8 modules tested for Drupal 9 compatibility). It may sound too early as more Drupal 9 APIs will get deprecated for Drupal 10, the reality is we already know a lot of deprecated APIs including those from Symfony and other dependencies. We can use this early data to help prioritise which deprecated APIs to cover with rectors which would allow us to start posting Project Update Bot issues for those that are Drupal 8 to 10 deprecations and are in unsupported Drupal 8 branches. This would allow us to get a headstart on fixing them, automated as much as possible.

At DrupalCon North America 2021 (less than 6 weeks away!) there will be a dedicated Drupal 10 readiness day where you will be able to learn about the state of the new version and will have a chance to work with the lead maintainers for each area to contribute to Drupal core, the tools and readiness of contributed projects.