Drupal 6

Website admin skins/themes clearly have a big market. Just see many different options for Drupal hosted on drupal.org or some nice commercial ones on themeforest.net (not Drupal specific). Administration themes are great when you'd like to separate your administration interface from your front-end. And several Drupal studies showed that many people need this separation to understand where content is managed and what does everybody else see of the site. Of course your mileage may vary and there are all kinds of sites where community participation is on a level that admin/front end separation is meaningless. The point of this blog post is not even administration themes, so let's shelve that discussion.

Websites often replace traditional web applications as well, when the "front end" of the website is already some kind of data input / management UI, where the data is usually not in the form of textual posts with comments tagged by topics. Navigation is not primarily search based because pages for different functionality in the site are not searchable in a traditional sense. Finally, menus are not built by the site builders but rather the application author, putting in menus and navigational elements via code suited for the application.

In Drupal, this kind of theming needs a different approach compared to traditional admin themes, because the web application will most probably not offer the usual Drupal admin functions. For this role, a theme more like the general ones of themeforest's admin looks fits more.

In first of a series of posts, I'd like to go ahead and talk about project handling functionality, one of the most important tools behind Drupal.org. At this moment, Drupal.org is running Drupal 5, and a big chunk of modules which don't have a Drupal 6 version to migrate to on Drupal.org is the project module family: project (also includes project_release and project_usage), project_issue, pift_server, cvslog and even comment_upload.

Except the comment_upload module (which allows file uploads on comments in a general way), maintenance for these modules are headed by Derek Wright and Chad Phillips. An outstanding thing about these modules is that they keep improving and being adjusted to user needs. Automated testing integration tools were developed and keep improving, so patches submitted against Drupal 7 get automated testing. This is just plain great. However, all this huge amount of motion is going on in the Drupal 5 version of the module. And given that Drupal.org needs a stable environment, it takes considerable effort to maintain a stable Drupal 5 branch with all these feature improvements and changes coming in.

While these modules do not even have a Drupal 6 branch yet, Adam Light went ahead and worked on a Drupal 6 port for project module. He hosts this in his own private Subversion repository (see http://drupal.org/node/157694#comment-891587 and the rest of the comments there). Since he started off long ago from a then current version of the module and implemented Views integration (instead of the one-off SQL based pages in the Drupal 5 version), the Drupal 6 port has a largely refactored codebase and does not carry the improvements made to the Drupal 5 version since then.

The lead maintainers however are at this point more interested in working on a new stable release for Drupal 5, given that some bigger changes they are planning to make would be easier to manage on their own instead of as part of a bigger porting and migration work to a new Drupal version and to a Views based backend. This gets us to a message of "please help with a new stable Drupal 5 release of project module before Drupal 6 work can be considered". While these patches are relatively big, they are far from how big of a monster patch is the Drupal 6 upgrade. All-in-all the possibly awkward conclusion is that maintainers look for help with the Drupal 5 version before Drupal 6 work can be started.

For concrete action items, Adam Light summarizes it well:

Let me explain the situation a little here:

There are still some things that we'd like to get fixed in the Drupal 5.x branch of the project and project issue modules before we branch for Drupal 6. See http://groups.drupal.org/node/10865 and http://groups.drupal.org/node/16069. One big issue that affects all of project* land is #98278: project* namespace bugs in $node. The project* maintainers have agreed that it would be best to fix this problem *before* branching, because if we try to fix it during the port that will make testing the port even more difficult than it will be.

Creating a D6 branch itself will not really unblock things. Hunmonk or I could also do that ourselves. However, the ported code that is currently in my SVN repository contains a *lot* of changes, and all of us agree that those changes should not just blindly be checked into the project cvs repository without at least some review.

Yes, we all realize that this is a non-ideal situation, and that the port is moving much slower than anyone of us would like. The best way anyone could help move the port forward would be by helping to write or review patches for any issue not yet finished in the list of things to do before the next 5.x release. Mostly, that means reviewing the $node namespace patches in the issue I linked to above. The unfortunate part of this issue is that it is huge, but really boring. And none of the sites run by the project* maintainers use a combination of modules that actually causes this bug to reveal itself. But at the same time, we realize that lots of project* users *do* use such a combination of modules (eg. they use pathauto with project), and so we need to fix this bug soon.

So there are numerous big issues affecting Drupal.org which will be solved as part of the Drupal 6 port, but the main issue holding back the port from even starting is an issue which does not even affect drupal.org (and therefore is not going forward on any reasonable speed).

In summary, there are difficulties in how improvements on drupal.org are expected by some people right now instead of after an upgrade, and the maintainers are taking on work on these items; and issues not affecting drupal.org holding back our most important upgrade ever. So you can help at least three ways with the project module upgrade:

• Put away some of your cool feature ideas for project modules on drupal.org for now. Let's focus on porting and bugfixing or we are not going to get over new feature requests anytime soon.
• Help test patches against the Drupal 5 version of project modules to fix long standing bugs. See http://groups.drupal.org/node/10865 and http://groups.drupal.org/node/16069
• Help test and fix existing issues in the Drupal 6 port of project module. It at least has Views integration issues coming from the RC2 API changes in Views. See my pending patch at http://drupal.org/node/157694#comment-1069892 which still needs work.

We definitely need your help in many ways. Let's do good for the drupal.org upgrade / redesign!

Update: The final design iteration is online! Check it out! Added screenshot to the post.

About five weeks ago, I blogged about that the redesign process is coming to an end soon, and that it is our job to take over and actually implement what was designed.

The first steps to get to a better drupal.org is (1) to see what we have, (2) keep what we are going to use forward, (3) implement migration paths for whatever we drop and (4) start adding functionality afterwards. In my previous blog post I referenced my report titled Where we are with Drupal.org modules vs. Drupal 6? which covered some of (1), and provided some ideas for cleaning up for decisions in (2).

At the start of the redesign, ideas of single sign on for drupal.org subsites, splitting out project management to its own subsite or merging all subsites into one were tossed around. A single unified node id space among subsites was discussed but more concrete implementation details were not made up. So there are lots of bigger scale infrastructure questions, and we need dedicated people to deal with these, drive the directions, make up solutions.

To facilitate teamwork, I decided to bring findings in my report to a wiki page and encourage all of you to come and sign up for tasks. The Drupal.org to Drupal 6 upgrade collaboration page is a wiki page which lists major module sets on drupal.org and calls out some ideas / directions we might/should take in each area. There is place for people to sign up and for relevant issues to be posted to have an overview of all the items needed.

While the focus of the page is to update drupal.org modules to Drupal 6 (some of which lag behind considerably, especially project handling related modules), the upgrade itself poses some questions. Some of the functionality is not meaningful to update given new plans. Two examples:

• The drupal.module based distributed authentication model for Drupal.org is planned to be OpenID going forward, and we will drop the old drupal.module based authentication scheme. This needs action on drupal.org to set up an OpenID server and provide migration for those using their drupal.org names on sites such as groups.drupal.org for example.
• The xapian and search modules now hosting the search functionality are target of much criticism. Jacob Singh, Robert Douglass and Peter Wolanin have a better proposal for drupal.org based on ApacheSolr, which will offer faceted search as well: http://dc2009.drupalcon.org/session/more-search-how-apachesolr-changes-… So why would we update the xapian module then?

While there are some questions, there are clearly required modules, such as the project module family, without which drupal.org will not live as happy as it is planned to be. There are numerous smaller modules in the drupalorg.module, or items like comment_upload which need attention if you can help out.

As Kieran Lal writes, today is the day when we will see the last design interation from Mark Boulton Design, and from there we are left with designs we need to build actual working functionality behind. With the risk of repeating myself from my previous post, I'd say again, that there is nothing like building a website for more then 300,000 users and 720,000 unique visitors per month. You might not catch such a project soon, if you miss this one! So get on and work with us in this exciting redesign!

Packt Publishing is at it again. They've published David Mercer's follow up to Drupal: Creating Blogs, Forums, Portals, and Community Websites, which was originally based on Drupal 4.7. The new book subtitled Build your own professional blog, forum, portal or community website with Drupal 6 tries to cater to the same audience but with greatly updated content.

David seems to be completely up to date on the Drupal 6 matters, as much as the March 2008 publication time allowed. This was one of the first Drupal 6 books on the market, and the author even managed to include a lengthy section on CCK. Hats off. Now that Views 2.0 is out for Drupal 6, many more people will consider using this new version as a base to start with. David caters to new users, not upgraders though, so this guide helps you get up to speed (and the Views covering books are still awaited on the market).

The book has a certain eye to detail in talking about things like setting up users and permissions. David even goes to note that setting up access rules for names or emails does not affect existing users. This practice was changed in recent Drupal versions, considering this a security bug instead of the way how Drupal works, and honestly, I don't think people expected to see this behavior noted in print. This attention to detail goes to extremes however in the examination of taxonomy. To my tastes, it would have been better to get down to more practical examples sooner instead of trying to organize the section around the theories of taxonomy. Same applies to coverage of HTML, where David tries to teach content producers certain HTML tags to write a feature-rich webpage. This might be a good idea for the theming section, but not where content is produced by end users.

With a book going into such details, you might think Drupal core fills up the pages in itself. This is however not the case. David goes to introduce contributed module installation right in chapter three with DHTML Menu module. Highly useful and/or popular modules such as Pathauto and Localization client are covered. So the book acknowledges that for building a website, Drupal core needs to be pimped up with contributed functionality. Another positive note in this approach is that even custom look and functionality is covered. In my humble opinion, this book does a modest but still better job in doing a custom theme then Ric Shreves' Drupal 5 themes accomplishes. JavaScript capabilities are also shown by integrating a custom JavaScript control.

All-in-all, I think this book is a good starter guide for Drupal 6 users, even if sometimes too detailed. You'll certainly need to be ready to learning a lot more from Views to CCK field modules while you actually build a more complex site, but starting off with a simpler website should be possible from the topics covered.

If being a co-lead organizer for Drupalcon Szeged 2008, getting married in three weeks, moving flats and of course building products and services with Acquia would not be enough, I thought I'd top my Drupalcon participation with a nice surprise session submission.

If my session makes it (vote!), and you come to Drupalcon Szeged (you should), I'll teach you how can you convert an existing HTML/CSS template to a Drupal 6 theme in a matter of 45 minutes, with the full live demo from the ground up included with instructions. I've managed to do this before, so I am confident it should be lots of fun. We will break our Drupal site numerous times, and learn to live with it while the tough time constraints are looming on us, and should of course get to a gorgeous end result. We will convert the Modern World template by Solucija and will get to a Drupal 6 theme with blocks, menus a theme screenshot and all.

I'll also tell you how can you contribute the theme to drupal.org or through other means if the template license does not allow you to upload to drupal.org. This is of course not a requirement, since you might as well only work for your own client. You decide!

Just make sure to vote on my session, to help me get into the program and come to Drupalcon not only for this great session, but all the other fun programs which are on offer. You definitely should not miss it!

Looks like people are finally realizing the enormous business opportunities lying in doing themes for Drupal sites. There is the http://www.topnotchthemes.com/ team building truly nice themes with support for common modules, knowing Drupals ins and outs.

At the same time http://www.templatemonster.com/ is picking up Drupal in their CMS section, selling Drupal themes for all kinds of focus areas. Although some of their demos have the "Mambo license" menu item running, which is quite frankly not a testament to their understanding of Drupal. However, starting off from a ported theme could still be nice, those buying Drupal themes might not want to fiddle as much customizing the theme further (update: and there are possibly other problems my soft blogging style did not uncover here, see: http://www.drupal4hu.com/node/146 and http://www.drupal4hu.com/node/141 for notes).

If you'd be interested in ported themes though, you might want to just start off from a theme downloaded for free. There is a new site coming up, started by a Hungarian Drupal enthusiast Ádám Boros. He is going through some of the exciting existing HTML templates and converting them to Drupal themes, providing for you to download for free. Why another theme site, you might ask? Why not just submit to Drupal.org? Well, although free to use and take, some of the HTML templates are not released under the GPL, so they are not suitable for submission on drupal.org. This requires people to either host them on their own site, or go centralize to a location. Ádám's new site, drupal6themes.com aims to not only host Ádám's work but also provide a platform for others to submit their Drupal 6 compatible themes and host them there.

I am extremely happy to see all these theming businesses and the expansion of the available themes to come together, and hope the growth is going to be even bigger going forward.

Early in 2007, I was discussing the possibility of being the Drupal 6 maintainer with Dries. Dries was just collecting candidates at that time and it was not for some months after work on Drupal 6 started, that he selected me as a maintainer. Through the discussions, we reiterated the point multiple times, that core maintainership requires a significant dedication and time from the maintainer. I never thought I will be able to work this much on Drupal 6 though.

In mid-April, when Dries announced me as the selected core committer, I was still at university, completing my MSc thesis, and that my thesis was focused on internationalization features for Drupal 6 helped my involvement a great deal. It was a perfect match. Then after finishing university, I went on to work in Google Summer of Code 2007, where my application was accepted earlier. In part, I continued the multilanguage improvements for Drupal 6 (with localization advancements) and taken over Localization Server from Bruno Massa to take it to new heights and eventually make it an official tool for Drupal translators.

A few hours after I blogged about graduating, I was approached by Dries and Jay Batson to work for Acquia full time as a Drupal core developer. Being an open source contributor for at least 7 years, I was amazed by the possibility. While it was a lifetime opportunity for me, Drupal 6 benefited most with this offer. I have been working on Drupal 6 full time after finishing Google Summer of Code, right from the next business day, August 21st (months before Acquia was even announced).

I just checked out the data about the volume of my involvement in Drupal 6, and it is fantastic to see that how this all made possible to enable the community to contribute with a higher pace then ever. My commits reflect all your combined work, multiple contributors working on each improvement or fix to Drupal 6. If Acquia's priority would not be to contribute to Drupal 6 and help me enable the improvements and fixes to land in core, we would not be here with a release of this quality in this timeframe. The difference between the parts of this commit graph when I was working on Drupal 6 in my spare time and full time really shows the difference.

So while there are lots of thanks coming my way, let me pass on some to Acquia. Let's meet in the Drupal 7 issue queue as well, we have fun stuff to do in the next release as well!

In my short free hours the last few days, I was brainstorming on new features for the translation template extractor (this little module which extracts translatable strings from Drupal modules) to make both the translators and Drupal coders life easier. Today I am proud to announce, that I released the old stable code as Potx-5.x-1.0 and Potx-6.x-1.0 (which signifies that the development code was quite stable for some time now) and wandered to implement new features for the 2.0 versions of the modules. From today, the 6.x-2.0-dev branch contains the two new features I developed the last few days:

• The module now extracts translation templates for themes too, not only modules. This was an obvious feature request, but the original implementation was quite shortsighted, so the relevant part needed a full code rethink to support themes. This is good for translators.
• The bigger news for module and theme developers is that potx now comes with (experimental) coder module integration. For those who have not heard about coder module, this little piece of software helps you to upgrade modules and ensure they conform to coding guidelines. It even helps you avoid some common security problems. But until now, it did not help you review your translatability errors. In fact, I got bug reports on the translation template extractor that if a module passed coder's review, it should not have any localization errors. Well, when used together with potx-6.x-2-dev, coder module now offers a new code review option. You can check translatability errors of your modules right there!

How can we make this even better? Well, there are still some TODO items for potx module, which will be implemented later (and I am sure people would like to see a 5.x-2.0-dev backport of the new features), but obviously people will not be better if told they make mistakes, if we don't tell them what to do instead. So I sat down and carefully crafted the Drupal 6 translation cheat sheet for your consumption. This fine piece contains the PHP and JavaScript interface translation API functions as well as the functions used in the installer (such as .install files and install profiles). I also collected the three most common errors and provided two tools to help you ensure you do as best as you can. This cheat sheet also includes explanation of the different placeholder syntaxes used in t()-ed strings, which even I have not been able to get used to still.

I hope you will find the new features and the cheat sheet useful, and take some extra time to ensure your modules are properly coded for interface translation, when you upgrade them for Drupal 6. Remember, we are going to have a "multilingual release" with all the new language features, so it becomes increasingly important that contributed modules use the interface translation API properly.

Update: Replaced the file with the 1.1 version, as I noticed that the !html placeholder needs a security warning to ensure people are aware that usage of this placeholder is not advised.

Happy hacking!

A few days ago Károly Négyesi (known to most as chx) announced that he organizes a (virtual) cirtical issue queue cleanup event for the 3rd of November, in which people gathered on the #drupal IRC channel will focus on fixing bugs in the critical issue queue.

webchick, desbeers, dvessel, davidstrauss and myself will spend part or all of November 3 with one goal in mind: empty out the critical issue queue for Drupal 6. In case we reach this goal and we have more time, well, there are a number of noncritical bugs and an even bigger number of patches that need work. Come, join the fun on #drupal on irc.freenode.net [...]

A few hours ago, Raincity Studios also announced that they are organizing a (real life) event for the 31th of October, titled the Disguised Drupalist Debugging Day. The Raincity event involves snacks and lots of help even for people starting to get their feet wet in patch rolling.

At last but not least, people from the Netherlands organize their first DrupalJam event for the 16th of November, which additionally to hosting sessions also includes a translation sprint, which admittedly is not direct code getting into Drupal 6, but nonetheless a very good opportunity for people to test every corner of Drupal 6, while translating them, and at the same time using the localization tools heavily, giving them some boost.

Things seem to line up nicely, and seeing Drupal 6 getting more and more stable by the day is very gratifying. In related news, the Drupal 6 development version is so stable, Angie Byron (known to most as webchick) just started her blog site on this version.

Well, back in 2003 when I joined the Drupal developer community (got my welcome mail from the drupal-devel list on September 15, 2003), I haven't thought I will be that involved with the system a few years down the road. It was (and is) a rather cool tool for a big Hungarian web development community website I was migrating from some ugly CMS... Now I do so many things around Drupal that it is not easy to track.