Last time I posted an update on Upgrade Status was four months ago. It is fair to say the Drupal contributed project landscape has changed quite a bit in the meantime and Upgrade Status should evolve too. Why? The primary role of Upgrade Status is to help get your site components updated for Drupal 9 compatibility. Most of your site components are contributed modules. In many cases, either your local copies or the remote available updates will already be compatible. 38% of Drupal 8 compatible modules are now Drupal 9 compatible (3535 out of 9258) and most others have a patch waiting to land to either improve or complete compatibility.

The changing role of Upgrade Status

Therefore the role of Upgrade Status for contributed projects is more to make you realize that you are one of (a) already compatible (b) will be compatible with an update or (c) should work with the maintainer on existing issues. As opposed to scanning for problems locally and trying to fix it for yourself. But the current 2.x version of Upgrade Status does not do the job of indicating most of these. Without running the (lengthy) scans on your projects, it does not know which local projects are already compatible. And although it has access to information about whether remote updates are Drupal 9 compatible or not, it will not display them. Look at this screenshot:

We only know the Consumers project update is going to solve your Drupal 9 compatibility issues because the maintainer kindly explained that in their Drupal 9 plan. This requires manual work and is not going to happen universally (eg. as shown below neither Chaos Tools nor Markdown specify a Drupal 9 plan). For projects like Consumers, there is not even a good reason to run our lengthy compatibility checking locally because we know the remote update is available and is going to be compatible. Plus we already know from the local info file that the local version is not yet compatible and requires that update. So if we would provide all this "new" data, the user could understand the situation and decide to not even run a compatibility scan but update the project. So the first step (hopefully) improving the Upgrade Status experience was to add more raw data about local and remote Drupal 9 compatibility. Then it looks like this:

The new "Local 9-ready" column shows if the project is already ready locally (without scanning the project, based on info files in the project). The new "Drupal.org 9-ready" column shows if the remote update is Drupal 9 compatible or not (based on information already provided to core's update module). So what we can see from this example is that Chaos Tools will be entirely resolved with an update, no need to run a problem scan locally. Markdown will not be entirely resolved but is worth updating either way because likely there will be less issues. Not all projects will specify a Drupal 9 plan to go by as discussed above. But if there was no plan specified, we could still link to the issue queue. I don't think applying issue tags consistently is possible to expect from all contributors, so we cannot link to an issue tag search. The link goes to a drupal.org issue search for "Drupal 9" issues in the project. This is the fastest way to get to a place to collaborate with the maintainers of Markdown, given no Drupal 9 plan posted.

Deciding on the next step for each project

So more raw data is great, right? Well, it probably makes the display a lot more confusing. Based on what you read out about the data you would do different things to different projects. For the 2.x version Joe at Drupalize.me published a great post about how to read the data. But what if the module would make suggestions for next steps instead of you needing to carefully examine each part of the data. We could still give users the source data, but save a lot of time to compare data points as to whether a project should be updated or needs more work, etc. Enter the next step suggestion engine I built following extensive discussions with Ofer Shaal. The possible next steps for projects are:

1. Remove: if you have uninstalled projects on your site, there is likely no point in investing effort keeping them up to date.
2. Update: if your project has a remote update, that is the best course of action; even if Drupal 9 compatibility is not entirely done there, it is likely improved.
3. Collaborate with maintainer: contributed projects that are up to date could still have issues; instead of fixing locally, there is high probability that there are issues (including but not limited to from Project Update Bot) in the queue, so better start collaborating instead of fixing locally.
4. Scan: unless your custom project's info files specify Drupal 9 compatibility already, the suggested next step is to scan them
5. Fix with rector: for custom projects there is a 40+% likeliness that there is at least one problem we find that is rector-fixable, we suggest you use rector as your next step to avoid needless manual work (again) in this case
6. Fix manually: if you scanned and there were no rectorable fixes, as a last resort we suggest you fix issues manually

Instead of grouping by custom/contributed and under that installed/uninstalled, the 3.x branch now groups results based on the next suggested step. The raw data is still shown and users can still do whatever they decide if they do not agree with the suggested next step.

This is the state of the module's user interface right after installation. We can already deduct a lot of information from local Drupal 9 readiness, update information and status of projects. For the single custom project in this test setup, we can only move forward with scanning it for problems. The numeric counters of problems found overall were removed, because in the current state of contributed projects rapidly getting ready, counting their individual issues together does not really inform about anything useful.

Overview of status of your site

Usage of Upgrade Status has always been a three-step process. First you need to help the module gather data (run available update checks, run scanning of projects). Second, you would attempt to fix the problems found (update modules, run rector, etc.). Then you scan again and verify compatibility improved. So instead of putting this into documentation, I designed a summary view for the top of the page, that helps you go through the data gathering, the compatibility fixing and the celebratory steps. Here is how that looks like for my test site:

1. The data gathering steps ensure the module works with the best data possible. Having a way to accurately identify project versions is important, so projects are suggested for that (unless already installed). Having up to date available updates information is important, so it is possible to run that directly from here. There are projects where scanning on this UI is the next step to gather the required data (to move to "Fix with rector" or "Fix manually"). On this site I have one of those test modules. And finally, the UI indicates you can scan either project as you see fit.
2. Based on the data available, the second columns shows all the next step suggestions for fixing compatibility issues. Projects to remove because they are uninstalled anyway, projects to update, collaborate on, fix with rector, and so on. If your environment is incompatible that also shows up here with a link to details.
3. Finally there is a column to review what's ready to go. If your environment looks all good, that would show up here, as well as all the projects that are already compatible. A little circle chart shows how far along you are.

Comparing side by side

Check out the current user interface on the left and the new work in progress on the right. A summary of differences:

1. The current UI is almost a blank slate and is not aware of project status locally or update's compatibility remotely.
2. On the current UI to gather data you need to run the full scanner on everything. Then you need to decide on next steps based on your reading of the data.
3. On the current UI the available updates refresh and the suggestion to improve version identification with the deploy modules is sort of hidden under "help text blindess", rather than being an active part of the UI.
4. The new UI is actionable immediately with categories of projects by next step.
5. The count summaries of problems found is gone, the focus is more on the projects rather than the individual problems. (Even though that data is entirely accessible).

What do you think?

I believe that more than two months after Drupal 9's release, the role of Upgrade Status is increasingly not to tell you to do fixes locally but to allow you to obtain the fixes already made and collaborate on fixes being developed for contributed projects. And for custom projects, it should suggest you the most automated way forward (rector preferred) that will result in least work to execute. These were the guiding principles of the new user interface. I built it all in the 3.x branch of Upgrade Status, and you can try it out in 8.x-3.0-alpha1. There are definitely spots for improvement and there are probably things I still did not consider. Feedback would be great either as comments here or support requests on the issue queue. Thanks for your input, it is going to help make this better!

Drupal 9 was just released last month, and in less than two weeks we get together to celebrate it (again), learn, grow and plan together for the future at DrupalCon Global.

I presented my "State of Drupal 9" talk at various events for over a year now, and while the original direction of questions were about how the transition would work, lately it is more about what else can we expect from Drupal 9 and then Drupal 10. This is a testament and proof to the continuous upgrade path we introduced all the way back in 2017. Now that Drupal 9.0 is out, we can continue to fill the gaps and add new exciting capabilities to Drupal core.

DrupalCon Global will have various exciting events and opportunities to learn about and help shape the future of Drupal 9 and even Drupal 10. Tickets are $249 and get you access to all session content, summits and BoF discussions. As usual, contributions do not require a ticket and will happen all week as well, including a dedicated contribution day on Friday. Here is a sampling of all content elements discussing, planning on and even building the future of Drupal.

Sessions about the future of Drupal

Photo by Austin Distel on Unsplash

First there is the Driesnote of course. Dries will share the result of the Drupal 2020 Product Survey and discuss plans for Drupal 10. There is a followup Q&A session to discuss the keynote and other topics with Dries live.

The Drupal Initiatives Plenary coordinated by yours truly is going to feature various important leaders in our community working on diversity and inclusion, accessibility, events, mentoring, promotion as well as core components like the Claro admin theme and the Olivero frontend theme. This is the best way to get an overview of how Drupal's teams work, what are their plans and challenges. Even better, the plenary session is followed by a BoF where we can continue the discussion in a more interactive form.

In Drupal Core markup in continuous upgrade path Lauri Eskola will dive into why the deprecation process used for PHP and JavaScript code is not workable for HTML and CSS. This informs the direction of where markup is going in Drupal 9 and 10 onwards.

In the Drupal.org Panel the Drupal Association team discusses how key initiatives are supported on Drupal.org including Composer, Automatic Updates and even Merge Requests for Drupal contribution and plans for the future.

Mike Baynton and David Strauss will discuss Automatic updates in action and in depth showing what is possible now and what are the future plans.

There is not one but two sessions about the new proposed frontend theme. In The Olivero theme: Turning a wild idea into a core initiative Mike Herchel and Putra Bonaccorsi discusses the whole history and future plans while in Designing for chaos: The design process behind Olivero will cover the design specifically.

Moshe Weitzman leads a core conversation to take stock of the current command line tools for Drupal and discuss what a more complete core solution would look like in A robust command line tool for all Drupal sites.

In Let’s Make Drupal Core Less Complicated Ted Bowman will propose ways to simplify Drupal core for existing uses and to achieve an easier learning curve.

Finally Drupal 9: New Initiatives for Drupal offers a chance to discuss new initiatives proposed by Dries in the Driesnote. If you are interested to join in either or discuss the plans, this is your opportunity!

Birds of a Feather discussions about the future of Drupal

Attendees with tickets for DrupalCon Global will be able to participate in live discussions about key topics. BoF submission is open, so this list will possibly grow as time goes.

Ofer Shaal leads a discussion titled Standardize Rector rules as part of Drupal core deprecations to make sure the transition from Drupal 9 to 10 will be even easier than Drupal 8 to 9 is.

Submit your Birds of a Feather discussion now.

Contribute to the future of Drupal

Photo by WOCinTech Chat on Flickr

Just like in-person DrupalCons, DrupalCon Global contribution will be free to attend and does not require a ticket. The contribution spaces are especially good to go to if you are interested in the future of Drupal and making a difference.

If you've been to a DrupalCon or a DrupalCamp before, a contribution event usually involves one or more rooms with tables that have signage on them for what they are working on. This is not exactly possible online, however, we devised a system to replicate tables as groups at https://contrib2020.getopensocial.net/all-groups which allows you to see what topics will be covered and who the leads are. (Huge props to Rachel Lawson at the Drupal Association for building this out!)

If your topic is not yet there, you should create a group now. Groups indicate what they are working on and what skills they need from contributors. You should join groups you are interested to help and read their information for guidance. Teams will post group events to let you know when certain activities (introduction, review sessions, co-working on specific problems or meetings to discuss issues) will happen. Events will also be used to signify when you are most likely to find people working on the topics. The OpenSocial site is a directory of topics and events, contribution itself will happen on drupal.org with discussion on Drupal Slack for most groups.

There are already groups for Configuration Management 2.0, the Olivero theme, the Bug Smash initiative and Media. Stay tuned for more appearing as the event comes closer.

Last week I had the pleasure to present my open source State of Drupal 9 talk at Drupal India Conclave. It was great with very timely questions. While I keep the slides up to date, there is always some new development. Since the recording of this video, Drupal 7's community security support was extended with a year to November 2022. The session should be fully up to date otherwise as of today. Feel free to use the slides to present at your own meetup or in-company training. Thanks Drupal India Association for having me on the program!

I organized Drupal 9 Porting Day for April 28 as part of my #DrupalCares funding sub-campaign to help the Drupal Association bounce back from their financial losses due to the ongoing pandemic. It was a lot of fun with Lee Rowlands, Vladimir Roudakov, Adam Bergstein and Mike Lutz helping lead the contribution before and after my time of availability. 126 issues were worked on and 43 newly Drupal 9 compatible releases were made then.

Given how fun it was, with Drupal 9 coming out next week it was logical to do another event. Last Friday would have been a great opportunity in person at DrupalCon Minneapolis if not for the pandemic (again). So I decided to schedule the event for that weekend. Surabhi Gokte and Gabriele Maira helped a lot in getting the event off the ground and we announced Drupal 9 Porting Weekend for May 22-23 to accommodate people available on the workday as well as the weekend.

With more time to prepare, a lot more interested folks signed up to help lead the event in their respective timezones. 14 leads signed up and helped contributors for 52 hours, while the event lasted. Thanks Vladimir Roudakov (VladimirAus), Janna Malikova (JannaKha), Vaibhav Jain (vaibhavjain), Tsegaselassie Tadesse (tsega), Gabriele Maira (gambry), João Ventura (jcnventura), Oleh Vehera (voleger), Matthew Radcliffe (mradcliffe), Michael Lutz (mikelutz), Adam Bergstein (nerdstein), Kristen Pol, Qiangjun Ran (jungle), Jaideep Singh Kandari (JayKandari) and Digant Jagtap (digantdj), you were fantastic!

Kristen Pol and Tsegaselassie Tadesse were also very active in the planning stage, Kristen published a very detailed guide to the weekend, Tsega wrote up and posted developer tips. The Bassam Ismail posted this video based on those guides of an actual Drupal 9 project update running with Upgrade Status and Rector, ending in submitting the patch:

So with everything well prepared, Vladimir and Janna started the weekend and the leads were handing off responsibilities to each other throughout the whole event. This is how time coverage looked like for the whole 52 hours. There was not a single time when someone was not there to help:

We had a lot of fun and learned a ton from each other. While numbers will not explain the event, that is all we have after the fact to look at, so here they are:

• 119 people participated on the #d9readiness Drupal slack channel throughout the event
• 521 issues were worked on altogether
• 188 of those issues are Fixed or Closed (fixed) at the time of this writing. That is more than the number of issues even worked on on the previous event!
• An additional 111 issues are Reviewed & tested by the community at the time of this writing, so hopefully many of those will also land soon

When looking at project releases, the weekend also supported a major increase in daily newly Drupal 9 compatible releases also with several days of after-effects (I am counting these with my own script):

New releases at the weekend and shortly after included fun modules like Pirate but also seriously cool modules like the Tome static site generator, Quicklink and top 200 most used modules like Views Accordion and Schema.org Metatag.

As luck would have it Drupal 9.0.0 RC1 was also released on this weekend, which meant that people testing their updated projects also gave the Drupal 9 release candidate a test drive right away.

For me this event was amazing to organize. The results in new Drupal 9 compatible projects before the stable core release and the additional testing of the release candidate are all good material outcomes. The raised awareness around the porting process and tools as well as the know-how shared will last even longer as people use what they learned and teach others as well. Also the concentrated increased use of the tools resulted in more improvement suggestions, so we can make them even better for the next wave of porters to come.

Thanks all for your involvement, you made a lasting difference. Keep spreading your know-how and all the good things about Drupal 9!

Ps. Next up is celebrating the release on June 3rd, 2020! Post your artwork, selfies, videos and events at https://celebratedrupal.org/ and let's have some fun together.

When I announced the Drupal 9 module porting challenge two weeks ago, I did not fully understand what was gonna come. I offered to donate €900 to the Drupal Association #DrupalCares campaign for 100 projects newly ported to Drupal 9. Then more funders started to appear. Ron Northcutt offered another €900, Ofer Shaal put in another €450. QED42 offered to match Ron's €900. It certainly grew much bigger than I anticipated so it was time to step up the game.

So last week I announced and started organizing Drupal 9 porting day for April 28, 2020 to not let our funders keep their money. While my funds were almost gone in the first week, there was still the rest of the funds to get donated. The idea of the porting day sounded good because we raise funds for the Drupal Association, we get people together to do their first Drupal 9 releases, we help others' projects out, drive the tools to their boundaries, do Drupal 9 core quality assurance and grow the ready module pool before Drupal 9's launch all at the same time. Some people would learn how to get ready for Drupal 9 for the first time, so we would spread some know-how and confidence in the release as well. That is like a win-win-win-win-win-win.

Nonetheless I was still blown away by the interest to participate. Lee Rowlands and Vladimir Roudakov signed up to start leading porting day in Australia / New Zealand while I was still well asleep. By the time I woke up there were already various new releases and issues opened. I started providing feedback there and then worked my way through the top 50 used projects that needed info file changes and releases. I made sure to do the deepest research and support maintainers to do the next steps. I also started getting patches for my own projects and even though I did not think it would be even feasible, thanks to contributors, we made one of my projects, Upgrade Rector Drupal 9 compatible as well. I also helped fix a critical core bug in Drupal 9 that Christian López Espínola found while porting the Lingotek module suite. At least two companies, QED42 and Srijan had groups of people internally gathering to rally and contribute. In my afternoon, Adam Bergstein and Mike Lutz came in from the United States to continue leading the day onwards.

Closing for the day. We had an awesome contribution effort today. It is incredible to see so many contributors participating and making this day wonderful. ❤️ Thank you… #DrupalCares - Jaideep Singh Kandari

At the time of this writing, altogether 126 issues were worked on. According to my scripts identifying newly Drupal 9 compatible releases of projects, 43 newly Drupal 9 compatible releases were made, including top 50 projects like honeypot and adminimal_admin_toolbar and such developer modules as twig_xdebug and queue_ui. When I put this together with all the numbers in the challenge to date, it turns out these 43 projects exactly rounded out the second 100 projects. Yes I went back to double-check!

This means Ron will now donate his €900 (which will be matched by Dries and Vanessa Buytaert and Drupal businesses to €2700) and QED42 will also donate their €900, totalling to an impact of €3600 funding for the Drupal Association from this second milestone of the Drupal 9 porting challenge. (Including the first milestone's €900, the directly donated funds are altogether €2700, for a total matched impact of €6300 in the #DrupalCares campaign).

If you did not get to do a first Drupal 9 release on porting day, no problem! We made a ton of progress on projects other than the ones that got releases and that will result in more releases. Some of them could be very soon. In fact, this challenge is not over, as there are still two more days, and we just entered the final round for Ofer Shaal's fund of 50 newly Drupal 9 compatible releases (max €450) for #DrupalCares. So please keep the releases coming! Thanks all!

Ps. Kristen Pol wrote up her detailed steps of working on Drupal 9 compatibility of others' projects. I suggest reading her tips for how to ensure compatibility and work with maintainers respectfully.