[OPEN-ILS-DEV] Re: Localization framework status

[Dan Scott]

Updates to the framework status follow inline:

On 18/11/2007, Dan Scott <denials at gmail.com> wrote:
> In the last few days, I've been working on the build/i18n/ directory
> of the Evergreen tree. I just wanted to give an update on my progress
> in getting a translation framework pulled together. Comments welcome!
>
> We will be using the Translate Toolkit [1] to convert files between
> DTD or JavaScript property file formats and PO format.
>
> My current Makefile assumes that the en-US DTD and JavaScript property
> files shall be the authoritative source from which the PO files shall
> be generated or updated. All locales other than en-US shall store the
> PO files in the Subversion repository (in the appropriate
> /build/i18n/po/ll-LL/) directory, and these locales shall generate
> their DTD and JavaScript property files generated from a set of PO
> files at build time. The release manager will be responsible for
> adding the generated files to the release tarballs.
>
> To generate a brand new set of PO files for a given locale, issue the
> following commands from the build/i18n/ directory (where ll-LL
> represents the language / region combination of IANA language tag
> subtag [2] and ISO 3166-1-alpha-code [3]):
>
> make LOCALE=ll-LL newpo
>
> This will create a new set of PO files in the build/i18n/po/ll-LL/ directory.

Note that we now have a set of POT files in build/i18n/po/ that
translators can start working with. I should point out that at this
time that only opac.dtd.pot (the translatable strings for the
catalogue) is relatively stable; the other POT files will grow
significantly over the course of the road to 1.4, as they are
associated with the staff client and there are still many strings to
extract from the XUL & JS files.

> If you have begun localizing the DTD or JavaScript property files and
> the files are in the build tree in a directory relative to the en-US
> files, such as was the case with en-CA and fr-CA for opac.dtd, you can
> then update the new set of PO files you just generated with the
> following command:
>
> make LOCALE=ll-LL updatepo
>
> This will update the set of PO files in the build/i18n/po/ll-LL/ directory.

Although this target still remains available in the Makefile for any
out-of-tree localizations that were in progress, I used it to update
the PO for the two localizations (en-CA and fr-CA) that had been
committed to Open-ILS/web/opac/locale/ and thus its primary purpose
for existence has been satisfied.

> To generate a new set of DTD and JavaScript property files containing
> whatever localizations have been made for a certain set of strings:
>
> make LOCALE=ll-LL updatemoz
>
> This will generate a set of DTD and JavaScript property files for the
> requested locale in the build/i18n/locale/ll-LL/ directory that
> contain localized strings. If a localization was not provided for a
> given string, then the en-US string will automatically be provided.

The "updatemoz" target suggested that only Mozilla files (DTD and
JavaScript property files) would be generated. I have renamed this
target "updateproject" in the Makefile to reflect the eventual
inclusion of strings from the database / other XML files.

> A basic set of unit tests for the preceding scenarios has been
> included in build/i18n/tests/testpo.py.
>
> Some thoughts:
>   * The fr-CA and en-CA opac.dtd files that are currently checked into
> trunk should be deleted if we have general agreement that this is the
> direction we want to take.

I have now deleted the fr-CA and en-CA opac.dtd files from the
repository, so starting with the 1.2.1 release these files will have
to be generated as part of the release process.

>   * This does not yet represent a complete set of localizable strings.
> Need to extend the framework to support localization of default
> database field values, fm_IDL.xml reporter labels, ils_events.xml, and
> eventually the Evergreen manual and contextual help.
>   * The Makefile will need to be extended to automatically generate
> all of the supported / up-to-date locales to avoid too much manual
> work for the release manager.
>   * The Makefile will need to be taught to place the generated files
> into the appropriate locations in the build tree to avoid too much
> manual work for the release manager.
>   * Maybe we'll need to convert to something more flexible than a
> Makefile to ease the previous two items.
>   * Still need to set up Pootle [1], use the Launchpad Translation
> Manager [4], or set up some other translation management tool to take
> full advantage of the benefits offered by the PO format.

The Launchpad Translation Manager (part of the Ubuntu project
infrastructure) requires submitted POT files to be licensed under the
BSD license. Although I think translation contributors would benefit
from the suggested localizations that result from over 200 projects
worth of translations, I don't know if we can relicense the English
strings to BSD. In the mean time, I will go ahead and try setting up
Pootle locally.

> [1] http://translate.sourceforge.net
> [2] http://www.iana.org/assignments/language-subtag-registry - note
> that we might end up supporting three-character language codes after
> all - mea culpa Mike
> [3] http://www.iso.org/iso/english_country_names_and_code_elements
> [4] https://translations.launchpad.net/
>

One other update: I have been testing the Dojo toolkit's
internationalization support [5] for use as a possible means of
providing date, time, number, and currency formatting for the
catalogue and staff client, and have been encouraged by successful
tests with Firefox 2.0.0.9 and Opera 9.2.4. Konqueror 3.5.7 suffers
from an XmlHttpRequest defect that causes it to display corrupted
characters instead of UTF-8, and I haven't fired up my Windows XP
virtual machine to try out IE as of yet (although I'm not concerned
about IE 7, actually; more concerned about IE 6).

During the course of my Dojo investigation, I filed several bugs that
resulted in corrected or clarified documentation - just the EG project
sowing the seeds of improvement where ever we go...

[5]
http://www.dojotoolkit.org/book/dojo-book-0-9/part-3-programmatic-dijit-and-dojo/i18n/globalization-guidelines/formatting-and-v

-- 
Dan Scott
Laurentian University