translation

As some of you might be aware, a group of talented and very determined people sprinted in Berlin about a month ago just to improve the i18n module for almost a week. A lot of great improvements made it in including tests, translatable contact forms and even some great usability improvements. Jose Reyero and Friederike Schmiedebach have great posts about the sprint.

Before continuing my article series, I wanted to touch quickly on the usability improvements, because my previous piece presented a pretty grim picture of textgroup based translation solutions such as blocks. Well, i18n module is still using that backend, but the new user interface improvements will make the translation process much more transparent and a lot easier to work with when managing Drupal configuration.

My last post where I've explained how Internationalization module re-implements some of Field API and where it does not do that it misses crucial functionality did not get much discussion. Therefore I decided to turn the key point at the end to the center of discussion: that either Drupal core will do fields for all user input (content and configuration alike, all through form your site name to your views empty text), or i18n module needs to do it in contrib. There is a clear need for input widgets, validators, permission handling, storage and output formatters and rendering used consistently. If it is not done by core, it will keep being a bolted-on half-failing approach despite best efforts in contrib. Please discuss at http://groups.drupal.org/node/154434

The other important post that we need your input on is about removing all UI strings from code. There are various issues with having them in code, while there are also various disadvantages to removing them from there. There are performance, translatability and even user experience concerns involved. This post is already getting some discussion, but we need much more. This could be a huge, fundamental change, so all your input is welcome. Don't say we did not ask you. Please discuss at http://groups.drupal.org/node/154394

Your input helps shape Drupal 8 and how Drupal supports building multilingual sites for years to come. Have your voice heard now!

There is great discussion forming on my previous posts on exportables and user provided text as well as the dangers of using t() for user editable data, and I can only hope we can keep that up! In that tradition, I'm cross-posting this piece to groups.drupal.org as well for discussion.

Regular readers could find this boring, but let's reiterate the three working modes that all objects should ideally be able to handle in Drupal to support multilingual site building.

1. Being able to mark an object as in one language.
2. Being able to mark an object as in one language and relate it to others as being a translation set. This is useful when you want to use the different language objects in different relations, track their history separately, have different permissions and workflows for them, etc.
3. Being able to translate pieces of the object that need translation and leave the rest alone. Load the right language variant of the object dynamically as needed. This is very useful for keeping external relations intact and sharing common fields between translations effortlessly.

There are certain things, where not all of these make sense. For the site's name for example, people would probably only use either (1) or (3). For a block for example, people should be able to use either based on their needs. (2) is useful to place blocks differently on translated pages, (3) is good to keep the placement consistent without effort. This can be different on a per-block basis. Same applies to nodes, menus, taxonomy, views, rules, and so on.

In my previous post titled Drupal's multilingual problem - why t() is the wrong answer posted on my blog and on groups.drupal.org for feedback, I've detailed issues with using t() as a translation tool for "user provided data". This post goes into some further details, a discussion of current solutions which could form basis for discussion of future solutions.

How can we even tell the difference between code and user provided translatables?

It is fair to assume that many multilingual sites will not have English as their default language (many not even as any of their supported languages), so we cannot assume that blocks, menus, and so on are entered in English. However, source code based strings are considered part of the user interface, and as such assumed to be written in English. What does this has to do with default configurations set up by modules and How do we reconcile this with the growing popularity of exportables and features (as in Feature module generated versioned export packages)? Let's look at these two questions.

Drupal is a great system to run foreign language websites on. The core itself is written in English and modules and themes are expected to follow suit. For developers, very simple wrapper functions are available to mark your translatable strings and let Drupal translate them to whatever language needed. These are the famous t(), the less famous format_plural() and a whole family of other functions. See my cheat sheet (PDF) and the drupal.org documentation for more on this.

Then there is "the other side", whatever does not come from code. Drupal works pretty well and very consistent if you want all of those to be in a foreign language (i.e. not English), but not in multiple languages (any of which can be English at that point). Drupal only has direct multilingual support in nodes (+ fields of entitites) and for path aliases. But life with Drupal means you work with all kinds of other objects like blocks, views, rules, content types, etc which are not "language-aware".

Unfortunately for building multilingual Drupal sites, this is the biggest problem that needs to be worked around. The contributed Internationalization module attempts to fill in the gaps, provide language associations and different workflows for translating these language-unaware objects. This works to some degree, but is really not easy without much help from the modules implementing these objects.

With the basics of node and site settings translation behind us, we are getting to the more complex parts, at least in terms of the user interfaces involved. While with node translation you get a tab on each node to translate it (regardless of setting up translation sets or using translatable fields), and with settings translation, you get quick jump links, the subsystems that work with textgroups will require a better understanding of how the Drupal systems relate.

Three ways to think about language support

When people want to have language support for their site, they typically think of one of three things:

1. Being able to mark an object as in one language. With node translation this was achieved by language enabling nodes.
2. Being able to mark an object as in one language and relate it to others as being a translation set. For nodes, this is supported by Drupal core's content translation module.
3. Finally, being able to translate pieces of the object that need translation and leave the rest alone. Load the right language variant of the object dynamically as needed. In the case of nodes, this is achieved with the contributed entity_translation module (formerly translation.module).

The first case is great when you don't need to translate the object, the second is great when you need to use translations in different contexts (for nodes, you can maintain a separate comment set, put in different menus, etc). The last is great when you want to maintain the object the same way regardless of language. This might be great for an e-commerce site. Read part 4 of my blog post series for exact details for nodes.

Applying this to blocks, the i18n module provides functionality (1) and (3), but not (2) at this point. Translation set support is being implemented for various objects (menus, paths, etc. are already covered by i18n), but not blocks yet. (1) is very simple to use, but (3) will be a real pain if you don't read this blog post...

I've intended to announce this development at Drupalcon San Francisco but unfortunately the session on this was merged with a more general i18n session which was coupled by the ash cloud above Europe, so I could not go. Evidently, collaborative software translation is not a mainstream topic. On the other hand, I keep receiving requests of the general applicability of the tools Drupal.org uses about every two weeks, and this interest always amazes me. While the localization server tool used on localize.drupal.org grew out of needs of the Drupal community, the solutions were architected to be useful in a general purpose software translation environment. While the architecture was there, it was lacking useful UI controls to just run it as a generic software translation tool.

Existing non-Drupal users like the Gallery 2/3 project and the Musescore desktop app utilize custom data connector modules, which the localization server nicely supports, allowing for custom code to gather data for translation. Gallery even uses a custom Localization client port for clients to submit translations to the server, even though their software is not Drupal based. However, translating arbitrary software without writing your custom connector code was not possible earlier.