Async configuration / gulpfiles (feature request)

Background

AFAIK, when gulp loads configuration / discovers your tasks via a user project’s gulpfile.js, it relies on side effects on the gulp object to decide which tasks were defined. This is very nice from a simplicity standpoint. And gulp has existing support for implementing tasks that execute asynchronously.

However, this makes it hard to define/declare tasks if the act of doing so relies on asynchronous APIs. For example, I have worked in some projects where we want to define our tasks deterministically from repository assets, but programmatically. To do so, we invariably call Node filesystem APIs, or execute a few child processes.

Right now, we can only do this on versions of Node that have synchronous versions of File I/O and child processes… and even then it would not use the ideal degree of possible concurrency. gulp is awesome about asynchrony and concurrency elsewhere, please let us do the same thing while defining tasks 😀

Similar proposal for grunt: https://github.com/gruntjs/grunt/issues/783

Suggested Possible Design

Support asynchronous gulpfile.js in two forms:

Form 1: exported thenables (Promise)

module.exports = ... some thenable ...;

Form 2: exported function (that can return a thenable)

module.exports = function () {
    return ... some thenable ...;
}

If the exported value from gulpfile.js is a thenable (Form 1), then that thenable cooperatively signals when configuration is complete. If the exported value is a function (Form 2), then that function also gets invoked exactly once and if the return value is thenable then that value represents the completion of configuration.

The reason why Form 2 is important is because exporting a thenable by assigning to module.exports is tricky to do asynchronously. It’s much more intuitive to return a promise chain from a function, because you can use the return statement in branches, loops, etc.

Using thenables makes gulp agnostic to whether the project is running on a V8 version with builtin Promise vs. using one of the many polyfills.

Optionally support a configuration timeout to help users realise that their async configuration code may have a bug when it doesn’t complete before the timeout.

Compatibility

Technically changing the interpretation of gulpfile.js by observing returned functions/thenables is a compatibility break because existing versions of gulpfile.js may already be doing that and relying on gulp to ignore them. However, this is probably unlikely.

If you want to support this without a breaking change, you can add a config step API like gulp.config that can accept a thenable-returning function.

[phated]

I’m not really interested in solving everyone’s problems (that’s what StackOverflow is for); however, this can easily be solved by thinking outside the box a tiny bit.

function injectConfig(task) {
  return function(cb) {
    getConfig().then(function(config) {
      task(config, cb);
    });
  };
}

gulp.task('somethingThatNeedsConfig', injectConfig(function(config, cb) {
  // use config
  // do something async
  cb();
}));

[phated]

Going to end this thread.

There’s a few solutions but this isn’t going into gulp core:

• Use my solution above
• Implement a custom CLI