If you ever translated Drupal software, you used the Translation Template Extractor (potx) at least indirectly, even if you never heard of it. It reads the source code of Drupal core and contributed projects, and collects the translatable strings. Every string translated on localize.drupal.org was extracted by potx. Developers and translators also use it to extract strings of their own modules and themes, on a web UI or with Drush.
potx is one of the oldest tools of our community. Originally built in 2003, I took it over from Jacobo Tarrio and Brandon Bergren in 2007. It was updated for every major version of Drupal since, but at its core it was still a set of global functions and variables from the early days, and its issue queue had bugs reported as far back as 2016. Today I released potx 3.0.0-alpha1 with a brand new UI, a brand new object oriented architecture and lots of very old issues fixed.
Why now?
Felip Manyer i Ballester has been working with a team of contributors updating localize.drupal.org to modern Drupal for a while, and I am contributing to that effort as well. potx is at the heart of how that site parses the releases of projects and it needed a serious look. On the updated site, thousands of releases should still be extracted reliably going forward with good error reporting. The site update made it timely to revisit the module architecture and UI too. I also made sure to triage the issue queue for long standing problems to make sure we are taking care of them too.
A brand new UI
The old potx UI was very bare-bones: a set of nested fieldsets of directories and a radio button for each component, with the options at the end of a very long page. The new form (inspired by one of my other projects Upgrade Status) lists the projects in tables by group, with their submodules indented under them. You can filter, select any number of projects or whole groups, and the summary at the end shows the selection and the name of the resulting file, which is now dynamic based on what you are exporting to make it easier to distinguish the contents later.
The form looks good in Claro and in the new Default Admin theme of Drupal 12, including its dark mode. After all the module still spans support for Drupal 10, 11 and 12 at the same time!
A brand new architecture
potx 2 had 6 source files with about 4,400 lines of code with most of them in one potx.inc file of global functions using global variables for state management. potx 3 has 60 classes with about 6,800 lines of code: extraction runs, separate extractors for PHP, Twig, JavaScript, YAML and shipped configuration, and recognizers for the different kinds of translatable strings. The number of test methods grew from 24 to 107, with unit, kernel, functional and JavaScript tests, which all run on Drupal 10, 11 and 12.
Extensively tested on 51 real Drupal projects
While I took care to have good test coverage, I wanted to be doubly sure. I wanted to know at least the same strings are still extracted from real code as before. This is where Claude Code was especially of great help. With that I set up a group of 51 projects to re-extract with every change: Drupal 11 core and 33 top modern contributed projects, like Group, Paragraphs, Pathauto, Token, Redirect and the Migrate tools, Drupal 7 core and 9 Drupal 7 contributed projects, like Views, Ctools and Organic Groups, and the test fixtures of potx for Drupal 5 to 8 and recipes. The original output of potx 2 was the baseline. Every step of the port was compared with it: the strings either had to be the same, or if new they needed to be a result of a bugfix. For example, the fix of the constraint messages added 12 strings for these projects: 3 from Drupal core, 8 from Group and 1 from Redirect, with no other string changes.
I also compared the output of potx 3.x with the output of the 7.x-3.x branch that runs localize.drupal.org today, on the same 51 projects. The results are the same strings plus new strings from the bugs fixed.
Lots of small but very old bugs fixed
Alongside modernizing the code, this was the right time to fix long standing bugs too. The setup of the extensive comparison tooling and new tests added along the way ensured we can fix them without regressions. So I went through the whole issue queue to see which bugs still apply. Many were already fixed or did not apply anymore, but some were open for a long time. Fixed in this release are:
- The context of the t and trans filters in Twig templates was not extracted (from 2016)
- Missing translations for constraints unless their class name exactly follows a pattern (from 2017)
- Plural translation values are not exported (from 2018)
- Not discovering profiles properly (from 2019)
- A PHP deprecation message on the extract form for modules without a .module file (from 2023)
- The PHP log() function triggered an unrelated validation error and drush potx without --folder did not extract the strings of shipped configuration (from 2024)
- Bracket syntax for arrays in Twig was not supported (from 2025)
Plus more fixes for JavaScript, YAML, shipped configuration, annotations and info files, several of them found by comparing the output of the 51 projects with the old code.
Setting a new baseline
This is the first alpha release, so please try it on your projects and report problems in the issue queue:
composer require 'drupal/potx:^3.0@alpha'The updated localize.drupal.org will also use potx 3 to parse releases on modern Drupal!
Thanks to 30+ contributors 💙
This release has more than 30 contributors credited, many of them for issues reported years ago. Thanks to adel-by, adriancid, arkener, bryandenijs, duaelfr, eelkeblok, espurnes, geek-merlin, gngn, hctom, heddn, hoanns, idebr, jmeijer, joe.p, johnatas, juliaschwarz, knyshuk.vova, lucashedding, markhalliwell, matslats, maxmendez, mpp, penyaskito, pfructuoso, ras-ben, risse, rodrigoaguilera, seemas, theanisotropic, timohuisman, volman and watergate for the reports, patches, reviews and tests that led to this release. A tool that has been around since 2003 still has lots of life in it!
