Self-made project planning tools on Drupal.org

We need to admit, these days, people are more reliant on contributed modules then Drupal core. What is in Drupal core is taken as a given and some requested new feature would ever come in a contributed module in a reasonable timeframe for a Drupal site builder. People are not waiting for Drupal 7 to get a feature, people are active in the issue queues to get things done sooner. Drupal core is supposed to lay the foundation and be stable for long, while contributed modules can experiment, change, and add new features on a much more active pace. So as a result, Drupal core can keep using the "will be released sometime" approach, and it does well with Drupal 7 allowing for more improvements by not keeping the "every 12 months" release cycle intact. With contributed modules on the other hand, much more frequent releases are anticipated with new features introduced through a shorter timeframe.

What's common in both, is that for longer lasting development, overseeing progress on major areas like the file API, the database layer and so on is important, while on shorter release cycles, knowing what is required for the release to be fixed, and whether the project is on track or not could be key to get contributors. So having (A) overviews of project progress and (B) checklists of things to be done is increasingly needed on drupal.org. There are a few things people did about these needs recently, and all have their advantages and disadvantages. There are some reasonably new approaches, so I decided to highlight the ways I see people approach project planning on drupal.org.

Drupal.org runs the infamous project* module family to help micro (and not so micro) projects run around the Drupal core. These include modules, themes and translations as well as installation profiles. Currently, drupal.org hosts more then four thousand (yes, more then 4000) projects, with support for a CVS space for hosting the code with branching and tagging, as well as a release system tied to those CVS concepts. Projects can also have an issue queue, which people can submit issues with, covering bug fix requests and suggestions as well as new feature requests. So a project's issue queue can be quite busy, but it is the only "core" way to track what needs to be done, who is assigned to these tasks, who is working on them, what is the current status and so on.

So eventually people started to experiment building new tools on top of project issues to help manage project planning, overviews and checklists. Here is a hopefully comprehensive list of what tools people built on top:

• Drupal core started off with the now defunct Patch spotlight. This was a page with guides on people on patch testing and test environment setup, but the core goal was to highlight the most important patch at any time for Drupal core. This worked for quite a while, but it turned out there is more then one important patch people should work on, and pointing them to one at a time does not always help.
• So the Patch Spotlight 2.0 concept was born, which is (thanks to groups.drupal.org running Panels with Organic Groups) a panel page with nodes organized in thematic blocks, so people can find the most important patches in each area.
• This kind of planning organization called for other modules to run TODO lists and thematic groups their own patch spotlights. Project module itself used pages on the groups site to have TODO lists for upcoming releases, so people know where they can help out, and the prominent patches are highlighted from the otherwise long issue queues. Check out http://groups.drupal.org/node/10865 and http://groups.drupal.org/node/16069 for examples. Groups like the WYSIWYG group started to use the same panels layout used by core for their own patch and task spotlight: http://groups.drupal.org/node/6492/summary

So far all of the above ways to help manage projects suffered from a need for manual editing. When a task was closed, people needed to remove it (or cross it over) in the patch spotlight / TODO list. When new issues were added, people needed to go and add them manually. The overviews only provided a sense of progress based on what humanly added metadata was in there. So people started experimenting with two relatively new things.

• Relatively recently, project issue tagging was added on drupal.org, so people can add tags to issues. Teams started to use this system to identify stuff to work on. There is a drupal.org upgrade tag (http://drupal.org/project/issues-term/346) and an i18n sprint tag (http://drupal.org/project/issues-term/301) for example. People can list issues across projects as shown on these two links, and can also filter down to per-project tagged issues. For example, i18n sprint issues against Drupal core are at http://drupal.org/project/issues/3060/term/301 This is all great to identify issues to work on, and gets us back to a list of automatically updated and fresh data, where we can see activity, we can see assigned people, status of issues and so on. What we lack is the kind of grouped, annotated view that the patch spotlight type of overviews gave above.
• So while I was thinking about this for the drupal.org redesign, other people were thinking about this for Drupal core improvements planning. Eventually, two new changes made it possible to somehow merge the human edited, annotated, grouped planning boards with automated status updates. Drupal.org had this nice project issue link filter already, where you can enter [#360936] in a comment / issue followup / book page / etc, and that would be converted to the title of that issue in a linked form, crossed out if it was closed or displayed as-is if it was open. If you hovered over the link, you've got to see the status. For planning purposes, having a more instant view of the status and knowing who is working on the issue is important, so I've suggested and implemented changes on drupal.org to make the issue links background use the issue status color and also add the assigned user name to the output. At the same time, other people working on core improvements decided that using drupal.org's book functionality would make a lot of sense to plan out and help people contribtue to / collaborate on core improvements. So the Community initiatives area was born on Drupal.org, and hosts the "Drupal core improvements" and the "Upgrade drupal.org from Drupal 5 to 6" project overview / planning pages.

Are we at finding the best / right tool for the job? Likely not. Tags are great because you can see status instantly for the tagged issues, and adding and removing tags is done while you change status and add comments anyway. The resulting issues lists still show a stream of equal looking issues however, without clear priorities, milestone assignment, and so on. Edited overviews are good because we can put issues in context, have a priority hierarchy and give more help for people who are looking to contribute. But they still require hand editing for new issues (even though existing issue status is automatically updated).

As you've seen people kept experimenting with the ways they can do project planning, checklists, release preparation to get people know where projects stand, where helping hands are needed. We will keep expanding on these tools. I would not expect drupal.org to run project management tools allowing for milestone management, release requirements setup or whatnot as provided by more directed project management solutions. We are figuring out tools for our own needs and will keep experimenting with them as we go along.

Did you ever wonder you need project planning and overviews for your drupal.org project? Did you use the groups wiki pages, groups panels approach or something else I missed entirely?

Ps. I run into this topic for the need of the Drupal.org upgrade. Dries has an update blog post on our progress and we still need your financial as well as development support for the process. The upgrade overview page at http://drupal.org/node/362117 provides a good overview of where we are, and what kind of tasks we see lying ahead for the upgrade alone.