Enhancing Internationalization Support

Summary

There have been several terra-i18n and terra-i18n-plugin requests and issues logged regarding how tranlsaitons are aggregated and loaded into an application. To address these, a major version bump will be needed to support the following are changes:

• Only load the specified aggregated-tranlsation locales. Fixes the i18nLoader which always attempts to load the terra-supported locales: https://github.com/cerner/terra-core/issues/855
• Fallback to base locale if regional locale is not present: https://github.com/cerner/terra-core/issues/854
  • Fallback to ‘en’ if base locale fails to load.
• Open up the i18nLoader restriction to terra-supported locales such that non-terra-supported locales will load.
• Allow regex serach pattern specification for indicating where translations directories are located: https://github.com/cerner/terra-core/issues/1258
• Recursively build the aggregated-translations output directory: https://github.com/cerner/terra-core/issues/1247

Accomplishing These Changes

To resolve the requests and issues outlined above, the proposed change would be to create an aggregate-translations pre-build script which would replace the use of the terra-i18n-plugin. The aggregate-translations script would be ran before startng webpack and would:

1. Aggregate the translations in the specified translation directories for the specified language locales. This step produces a 1-to-1 output to the terra-i18n-plugin, but will use globbed search patterns to build the aggregated translations instead of recursively searching on a strict pattern or searching within one specified directory.
2. Create new intlLoader and translationLoader files on each aggregate-translations execution. These files will only contain filer loader information for the specified locales. This fixes https://github.com/cerner/terra-core/issues/855 as with the current behavior, webpack will transpile this file and attempt to access the non-existent locale files and throw errors, even if the file were not currently being loaded.

The aggregate-translations script will have the following options:

• -s or --search: the regex pattern to glob search for translations. Multiple patterns can be specified and will be added to the default search patterns.
• -l or --locales: The list of locale codes to aggregate on and combine into a single, respective translation file. The default will be the terra-supported locales.
• -f or --file-system: the filesytem to use when writing the aggregated translations and loader files
• -o or --output: the output location for the aggregated translations and loader files

Then, in the webpack.config.js file, one would only need to add the aggreated-translations output directory path to the resolve.modules list. This cleans up the configuration as follows:

const webpack = require('webpack');
...// other loaders & plugins
-const I18nAggregatorPlugin = require('terra-i18n-plugin');
-const i18nSupportedLocales = require('terra-i18n/lib/i18nSupportedLocales');

module.exports = {
  entry: {
    ... //entries
  },
  module: {
    ... // loaders
  },
  plugins: [
-    new I18nAggregatorPlugin({
-      baseDirectory: __dirname,
-      supportedLocales: i18nSupportedLocales,
-    }),
    ... // other plugins
  ],
  resolve: {
    extensions: ['.js', '.jsx'],
+    // This line would remain the same and provide the aggregated-translations files and loaders as modules to terra-i18n.
    modules: [path.resolve(__dirname, 'aggregated-translations'), 'node_modules'],
  },
  ...
};

Locale Fallback

Locale fallback will be handled as follows:

• Load the es-US aggregated-translations file --> if no aggregated-translations file, load es ggregated-translations file --> then load the en aggregated-translations file --> if no file, fail

This fallback approach only supports the fallback of a missing aggreagted-translaiotns file. This does not support fallback for missing translations messages within an aggregated-translations file. If a translations message is missing, the intl message key will display.

With enabling this fallback method, terra-base will always fallback to loading english translations. The https://github.com/cerner/terra-core/issues/1149 issue was logged to allow for the use of terra-base without translations, which overall would be a bad-practice for setting up an application consumers terra component. If https://github.com/cerner/terra-core/issues/1149 remains valid, terra-base will changes to provide a non-intl-provider setup.

Opening Up Locale Restrictions

These updates will not allow non-terra-supported locales to be loaded within an application. For anyone choosing to use non-terra-supported locales, they will be notified that the locale is not supported and will responsiblity for supplying the correct message for the localized terra componnets. Failure to do so will result in the intl message key to display.

Additionally, to use non-terra-supported locales, users are responsible for ensuring intl supports the locale otherwise failure will occur when loading the locale data from intl.

[emilyrohrbough]

Proposal Updates

Initially it was thought a major version bump was necessary to handle the necessary changes needed to only load the specified aggregated-translation locales, #855. After some realization on the undertaking of what a major version bump would entail, more thought was put into how passivity could be maintained while still achieving the proposed changes outlined above:

Resolving Loader Errors:

Above, it was proposed to create the i18n pre-build aggregate-translations script/setup function, which would create the intlLoader and translationsLoader modules that are configured for the locales specified. This would remove the previous build errors that would result in the web pack statically analyzing the i18nLoader file and attempting to resolve the module path for each terra-supported locale.

To maintain passivity and remove the need for a major version bump, a postinstall write-default-loaders script will also be added. This write-default-laoders script will write intlLoader and translationsLoader modules to the node_modules directory of the terra-i18n package.

How this maintains passivity:

The current web pack resolver.modules configuration is as follows:

resolve: {
    modules: [path.resolve(__dirname, ‘aggregated_translations’), 'node_modules'],
}

This resolve.modules configuration indicates module resolving occurs in this order:

1. ./aggregated_translations
2. ./node_modules

Here is a clear, high level explanation on node’s algorithm for resolving modules. Given web pack uses the enhanced-resolve function to resolve file paths while building, and it includes the node filesystem, this resolution algorithm allows us to create these “dynamic” intlLoader and translationsLoader modules to maintain passivity with the i18n changes.

So, take loading the intlLoader module for example, these module algorithms used in web pack/node will resolve from terra-i18n as:

1. First searches for ./aggregated_translations/intlLoader.js
   • If using the pre-build step, the module is loaded from here & will only load the specified locales
2. Searches for terra-18n/node_modules/intlLoader.js
   • If using the i18n-plugin, the module is loaded from here and will load each terra-supported-locale as perviously loaded

[emilyrohrbough]

With @mjhenkes suggestion, I propose we change the -s, --s CLI flag to be -d, --directories and then provide the aggregateTranslations() setup function. This function would accept an options object as a parameter that would honor the keys:

• directories: default to []
• fileSystem: default to fs
• locales: default to terra-supported locale list
• outputPath: default to ./aggregated-translations

This would give the option within the webpack as:

const webpack = require('webpack');
...// other loaders & plugins
-const I18nAggregatorPlugin = require('terra-i18n-plugin');
-const i18nSupportedLocales = require('terra-i18n/lib/i18nSupportedLocales');
+const aggregateTranslations = require('terra-i18n').aggregateTranslations;


+aggregateTranslations({ 
+    locales: ['en', 'es'],
+    directories: [path.resolve(__dirname, 'custom_translations')]
+});
 
module.exports = {
  entry: {
    ... //entries
  },
  module: {
    ... // loaders
  },
  plugins: [
-    new I18nAggregatorPlugin({
-      baseDirectory: __dirname,
-      supportedLocales: i18nSupportedLocales,
-    }),
    ... // other plugins
  ],
  resolve: {
    extensions: ['.js', '.jsx'],
+    // This line would remain the same and provide the aggregated-translations files and loaders as modules to terra-i18n.
    modules: [path.resolve(__dirname, 'aggregated-translations'), 'node_modules'],
  },
  ...
};