[QUEST] Module Unification: Final Cut

Throughout 2017 a small crew has pushed forward the implementation of Ember’s new filesystem layout for applications and addons. This was described in RFC 143 and we’ve lovingly called the effort “module unification”.

Much of the effort up to now has been either design or exploration-heavy. It wasn’t plausible to reduce the work list into things that individual contributors could pick up without context.

But in the last weeks we have turned a corner. Our effort is behind three feature flags in Ember, Ember CLI, and ember-resolver. In order to enable these flags by default for new applications, we need your help. In this quest issue I’ve outlined the current list of known blockers as well as guidance for how to test the new features in new and migrated codebases.

Several contributors have been huddled in #st-module-unification on the Ember.js Community Slack in recent months. Please join us there when you start to pick up one of these tasks.

The Issue List

Ember.js

• In the Ember.js PR that landed namespaces behind a feature flag, we implemented parsing of {{namespace::name}} strings at runtime. This implementation should be improved to have this work happen at Ember app build time.
• Before advocating to “go” the feature, we want to get benchmarks (via https://github.com/krisselden/ember-macro-benchmark and https://github.com/eviltrout/ember-performance) to confirm there isn’t an unexpected large regression. The changes to implement these features are not expected to improve Ember’s performance, but we should understand the impact.

Ember CLI

• Ember CLI uses the presence of src/ to configure the return value of isModuleUnification(). This works well for apps and addons. The detection fails for a classic dummy app in a module unification addon, however. Although a dummy app may be in tests/dummy/app/ with no tests/dummy/src/ directory, the addon’s root level src/ confuses our logic and the dummy app is incorrectly treated as a module unification app.
• Requires design: An ember addon author may want two dummy apps for testing, a module unification src/ app and a classic app/ app. Their acceptance tests would run with both apps, or they could have two acceptance test suites.
• Module Unification blueprints and generators, mostly tracked at: https://github.com/ember-cli/ember-cli/issues/7530
  • (@cibernox https://github.com/ember-cli/ember-cli/pull/7667) The module unification Ember addon blueprint should generate a module unification dummy app, not a classic app.
  • Generators for initializers and instance-initializers are broken
  • The test_helper.js files for module unification addons (https://github.com/ember-cli/ember-cli/blob/master/blueprints/module-unification-addon/files/tests/test-helper.js#L1) should import from ../src/main.
  • The app and addon dummy app blueprints both use the fallback resolver via resolver.js and classic initializers support via main.js. Before we remove the feature flag, we need to convert each addon these apps are dependent upon to be a module unification addon. Then we should remove the fallback setup for newly generated apps and addon dummy apps.
• ember-maybe-import-generator-for-testing needs to apply to tests in src directory, see https://github.com/ember-cli/ember-maybe-import-regenerator-for-testing
• If a classic app is running a version of Ember CLI that pre-dates module unification support, then an addon using module unification (with a src/ directory)must have a way a) fail gracefully or b) bring the reexport logic we use in ember-cli with it to the old version of ember-cli.
• Module unification addons re-export their top level components to app/ for classic apps. This allows an addon author to use a src/ directory and still have their addon work for either kind of app. We would like to provide addons a build-time way to configure this re-export logic. It would permit them to re-name components and services to match legacy classic APIs while they adopt new APIs for module unification apps.
• Ember addons need an API that allows them to alter the module unification config (for example https://github.com/ember-cli/ember-resolver/blob/master/mu-trees/addon/ember-config.js) and add their own collections and types. In theory Ember (or the resolver?) would use this same API to add its own types and collections.
• Fix lib/models/project.js’s isModuleUnification method to not only rely on presence of top level src as “the project is module unification format”. Specifically, an addon may be in module unification format, but have its dummy app be in classic format.

Ember Template Compiler

• hbs helpers should accept a source argument like templates in a running app do. In: https://github.com/ember-cli/ember-cli-htmlbars-inline-precompile
• Part of the hbs AST transform should be to inline the source value. However test files for a private collection, local lookup component might not be able to use the “easy” value of their filename and still have access to resolve their subject. For example in src/ui/routes/application/-component/foo-bar/component-test.js templates created with hbs will not be able to access the {{foo-bar}} component. The simplest solution here is to pass a source to these hbs templates that is “one level up”, for example the source for templates in the test file would be src/ui/routes/application. There are some tradeoffs in taking this or other approaches, but it seems like a good place to start. https://github.com/ember-cli/babel-plugin-htmlbars-inline-precompile/pull/33

Ember Resolver

• The ember-resolver feature flag for module unification can be removed at any time. It should swap to be a toggle based on Ember CLI’s project.isModuleUnification() value. The build-time flagging means classic Ember app’s don’t get a larger JS payload for a feature (module unification) that they are not using. https://github.com/ember-cli/ember-resolver/pull/228

Migrator

• (@iezer rwjblue/ember-module-migrator#73) The migrator should update the test_helper.js file to import the app from ../src/main. Currently this mentioned in the fallback documentation but it should just be part of the conversion.
• The migrator currently renames template-only components to not have a folder, like src/ui/components/name.hbs. The resolver doesn’t yet support this, and instead always expects a folder, for example: src/ui/components/name/template.hbs. The ember-resolver can probably be tweaked to fix this at runtime, but alternatively the migrator should simply not do this flattening in the near-term.
• Migrator tries to re-export with the name helper, but also imports a thing with the name helper. https://github.com/emberjs/ember.js/issues/16361

ember-svg-jar

• This addon is the only one we’ve found with a src/ directory that is not used for module unification. We need to update it for compatibility: https://github.com/ivanvotti/ember-svg-jar/issues/56

ember-load-initializers

• Test files in the initializers directory should be ignored by the loader. We can’t strip test files from development output since ce it would break /tests/ on the dev server, but we can easily have the initializer ignore them. Tracked at https://github.com/ember-cli/ember-load-initializers/issues/47

Several contributors have been huddled in #st-module-unification on the Ember.js Community Slack in recent months. Please join us there when you start to pick up one of these tasks.

Creating a new module unification app

Install Ember CLI master:

npm install -g https://github.com/ember-cli/ember-cli.git

Generate a new app with the module unification env variable:

MODULE_UNIFICATION=true EMBER_CLI_MODULE_UNIFICATION=true ember new my-app

Out of the box a newly generated application will use the ember-resolver “fallback” resolver. This allows the module unification app to use classic app/ addons. Much later, once we’ve converted the default addons that ship with new Ember apps, we can drop the “fallback” resolver and use the MU-only “glimmer wrapper” resolver.

The implementation of module unification uses a configuration of collections and types. In the classic addon app/ layout, addons could define new types of factories without any explicit definition. Before removing the feature flags we need an API for them to explicitly do so against the module unification config. Temporarily you may want to mutate the config for your app to add a type, for example in src/resolver.js.

If you have an issue after generating a new application you can likely find or file an issue:

• If something does seem like it works out of the box it likely related to the blueprints and should be filed at https://github.com/ember-cli/ember-cli/issues
• To generate MU files use the feature flag: EMBER_CLI_MODULE_UNIFICATION=true ember g component x-button. If there is an issue using a generator, you can find or file it at https://github.com/emberjs/ember.js/issues
• If an addon seems not to work properly and is you get errors around the resolver, issues can be found and filed at https://github.com/ember-cli/ember-resolver/issues

Migrating an existing Ember app

Install the module migrator:

npm install -g ember-module-migrator jscodeshift

Run the migrator on your app codebase:

cd ~/project/path
ember-module-migrator

Install Ember CLI master:

npm install --save-dev https://github.com/ember-cli/ember-cli.git

And update your files to match the latest blueprint, similar to what you do for any upgrade:

ember init

You’ll want to:

• Keep the ember-resolver feature flag in config/environment.js.
• Keep the Ember.js feature flag in config/environment.js.
• Keep the canary version of Ember.js in package.json.
• Keep the addition of loadInitializers in src/main.js. There are two of these lines, one for src/ and one for classic addons exporting into app/.
• Keep the change to the fallback resolver in src/resolver.js.
• Keep the change to import the app from ../src/main in test-helper.js.

If you have an issue after migrating an application you can likely find or file an issue:

• If you run into issues with the migrator (bad file names, bad file contents) you can find or file an issue at https://github.com/rwjblue/ember-module-migrator.
• The blueprint step may also be part of setup issues https://github.com/ember-cli/ember-cli/issues.
• See the section on creating a module unification app for more information which applies to migrated apps.

Creating a new module unification addon

Install Ember CLI master:

npm install -g ember-cli/ember-cli#master

Generate a new app with the module unification env variable:

MODULE_UNIFICATION=true ember addon my-addon

You can find a file issues for the experience of building an addon:

• Issues testing an addon are mostly found at https://github.com/ember-cli/ember-cli/issues although there are known issues around https://github.com/ember-cli/ember-cli-htmlbars-inline-precompile.
• If you have an unexpected issue using your add in a classic app or module unification app you should look to https://github.com/ember-cli/ember-cli/issues.

[GavinJoyce]

The instructions for creating a MU app have changed, @mixonic perhaps you could update the quest issue description with these instructions?

Creating a new module unification app

Install Ember CLI master:

npm install -g https://github.com/ember-cli/ember-cli.git

Generate a new app with the module unification env variables:

MODULE_UNIFICATION=true EMBER_CLI_MODULE_UNIFICATION=true ember new my-mu-app

Generate a component:

EMBER_CLI_MODULE_UNIFICATION=true ember g component x-button

Run the app:

EMBER_CLI_MODULE_UNIFICATION=true ember serve

[aalasolutions-zz]

Hmm, not sure where to put these, but for me writing EMBER_CLI_MODULE_UNIFICATION=true each time is hassle. I have changed my package.json file with these

"scripts": {
    "build": "EMBER_CLI_MODULE_UNIFICATION=true ember build",
    "lint:hbs": "ember-template-lint .",
    "lint:js": "eslint .",
    "start": "EMBER_CLI_MODULE_UNIFICATION=true ember serve",
    "g": "EMBER_CLI_MODULE_UNIFICATION=true ember g",
    "d": "EMBER_CLI_MODULE_UNIFICATION=true ember d",
    "test": "EMBER_CLI_MODULE_UNIFICATION=true ember test"
  },

and now I can just run npm run g component todo-item and it will generate properly …