i18n, proposed tooling and workflow

This issue is intended to act as a catalyst for getting the Internationalization ball rolling, discussions going and decisions made.
Below are a few things I’m proposing is based on a mix of my own research and previous conversations in the #i18n Slack channel.
I obviously do not have all the answers (and there is a chance that the ones I have could be improved on) so please fill in what ever gaps you see or propose changes where ever you see fit.

Glossary of terms

• i18n: internationalization
• BLOS: big list of strings
• admin: the admin section that lives under the /ghost url
• frontend: non-admin panel section, where articles are published and publicly visible

Proposed Tooling

Transifex, for managing translations:

• Transifex has a pretty comprehensive API (http://docs.transifex.com/api/) for interacting with their resources as well as a few grunt plugins in case we need to update the dev workflow (like this, this, and this)

• It also has a mechanism for keeping the source language file in-sync with a file on GitHub , which would make it really easy to keep things updated (full details).

  As long as your source file is accessible from a public URL, such as this raw file on GitHub, Transifex will periodically check the URL for updates. You can also use a file from a private repo in Github as long as the URL has the respective token that Github provides. If there are updates in the file, Transifex automatically fetches the file and updates your content.

ember-intl, for client/Ember translations

• Part of the Format.js library collection
• I compared it to ember-i18n and ember-cli-i18n, Format.js provides a more complete feature set
• It does not officially support sideloading BLOS in the current, but it is implemented in the 2.0 branch (see issue ref)
• It goes beyond just strings, but also includes template helpers for formatting dates, numbers and time per locale
• Includes template helpers for providing custom formatting for dates, numbers and time
• Follows well-known ICU message syntax for pluralization, select and variable interpolation
• Leverages the standards-compliant window.Intl object and provides a polyfill for browsers that do not have support for it as yet.

intl-messageformat, for server-side translations

• I also looked at messageformat.js and i18next-node but they seemed for fall short in some aspects.
• Also part of the Format.js library collection
• Biggest sticking point is that it follows ICU message formats and Unicode CLDR standards
• Leverages native ECMAScript i18n API functionality for date and currency formatting (polyfill if no available in engine)
• Usage is kind of clunky, but that can be hidden behind a facade for a better developer experience

Proposed Rollout

We should be doing a staged-rollout of the i18n functionality. The ideal end-state is for Ghost to have the concept of “language packs” that the user can install on demand when ever they want. But this long-term vision is some-what dependent on (or at they ver least related to) Ghost Apps being flushed out. The phases detailed out below are intended to lead up to this ideal state.

Phase 1 - harvesting strings

Regardless of what process or workflow we end up following to distribute translations, all the hard coded strings (both in the Ember and Node app) will need to be moved in to a “source language file”. So phase 1 would be to do a basic implementation of the i18n libs mentioned above. Ember sample, Node sample
At the end of phase 1 English would still be the only language supported in Ghost and there would be no noticeable change to the end-user.

Phase 2 - start translating strings

(this can be done in parallel with Phase 1 once there is a decent amount of strings are in Transifex)
Once we have our base “source language file” (most likely en.json), the next step would be to setup Transifex and let the translators commence the translating in to the initial target languages.
This process will be on-going as the “source langue file” will continue to be updated and new languages may be introduced as well.

Phase 3 - develop locale-setting functionality

Both the Node and Ember app need enhancements to (a) ask the user for their locale during setup, (b) modify locale in “settings” and © allow for the locale value to be programmatically retrieved. This might go through a few iterations depending on how we want to allow users to control locales (blog-wide? per-user basis? only during setup?). At this point, i18n could be enabled under Labs with limited locale support (depending on progress of phase 2).
Locale files will still need to be bundled with the release (not a big deal YET, because there would be 2 or 3 locales at most)

Proposed Development Process / Workflow

The source language file (en.json for both client and server) is the only language file that will be in the repo. This file will be kept in-sync with Transfex using this process. All other language files will be downloaded as needed.
Long-term, the only language file that should be in the downloadable ghost-*.zip file should also be en.json. But short-term, until apps are ready, any other supported locale will need to be included as well (hopefully this will only be necessary for 1 or 2 locales).

Proposed Workflows and organizational structure

See the “Workflows and organizational structure” section in this gist for information related to this topic.

[jamesarosen]

I’m not a Ghost contributor, but I’ve worked on a few i18n/l10n projects. (I maintain ember-i18n, though I think ember-intl is a great choice for this flow.) I suggest two very small additions to this flow:

1. At the end of phase 1, build a glossary. There are going to be a handful of really core terms (like “post,” “user,” “editor,” and more) that have nuanced meaning. You’ll get better, more consistent translations if you compile this list and write a sentence or two about each one.
2. As you start each localization project, translate the glossary first. Make sure that translators use the established core translations when adding new strings.

[heyakyra]

I wonder if it would make sense to use Project Fluent in the future. It’s a next-gen translation format by Mozilla which improves over GNU gettext and ICU MessageFormat and is supported by i18next-fluent, which is a standard way to implement this in JavaScript. This would allow using a slick open source translation tool like Pontoon or using a common commercial tool like Locize to outsource translations.