Asynchronous initialization of Feathers services

This is real issue that I’ve recently had, but I guess this concerns the framework in whole. I think that it is natural for Feathers services to be asynchronously initialized. They can pick their configuration from asynchronous sources, or it can be something else.

In most cases synchronous application initialization is presumed, but I’ve encountered design problems when started using Sequelize.

While Mongoose does some good job on accumulating promises inside for initialization actions (connection, index creation), so queries are automatically chained to initial promises, Sequelize does nothing like that, e.g. when sync() is called.

I would expect that SQL database is ready at the moment when web server has been started, but when the application is defined in synchronous manner, it will not. And this is what happens with typical Feathers setup. In some cases async initialization may take several ms, in other cases it may take seconds, while the server will pollute logs with errors that should have been never happen normally.

Another problem is error handling. If there are uncaught errors during async service initialization, the reasonable move would be to halt app initialization and catch errors in a single place.

Here’s a small example that explains the idea and is based on the structure proposed by Feathers generator template.

app.js:

const app = require('feathers')();

// THIS!
app.set('feathersServicePromises', []);

app
.use(require('body-parser').json())
.configure(require('feathers-hooks'))
.configure(require('feathers-rest'))
.configure(require('./middleware'))
.configure(require('./services'));

// THIS!
Promise.all(app.get('feathersServicePromises'))
.then(() => app.listen(3000), console.error);

services/index.js:

module.exports = () => {
    const app = this;

    app
    .configure(require('./foo-async-service')
    .configure(require('./bar-async-service')
    .configure(require('./baz-sync-service')
};

services/foo-async-service.js:

const sequelize = require('./common/sequelize');

module.exports = () => {
    const app = this;
    const fooModel = sequelize.model('foo');

    // AND THIS!
    const initPromise = fooModel.sync();
    app.get('feathersServicePromises').push(initPromise);

    app.use('/foo', require('feathers-sequelize')({ Model: fooModel  }));
    app.service('/foo');
};

The places of interest here are initPromise and Promise.all(...). It would be convenient to just return initPromise from service function, like it is done in hooks. And Promise.all(...) could probably be lazily executed with app object toPromise() method or promise getter property. So it could be

module.exports = () => {
    ...
    return fooModel.sync();
};

in service function, and

app
...
.configure(require('./services'))
.promise
.then(() => app.listen(3000), console.error);

in entry point.

I suppose that the pattern I’ve used for async initialization is already simple and clean, but it would be great if the framework would take this job.

My suggestion is to place this feature under consideration.

I would appreciate any thoughts on the subject.

[daffl]

This has become a fairly long discussion but here is the proposal to solve the issue of asynchronous application setup. As it turns out, Feathers already has a pretty established way of handling asynchronous method workflows, namely Hooks. Based on #924 by @bertho-zero to allow adding hook functionality to any asynchronous method, we now have a way to make setting up an application or specific service much easier.

Asynchronous service.setup, app.setup and app.listen

This will be the breaking change making it Feathers core (@feathersjs/feathers) version 4. service.setup, app.setup and app.listen will return a Promise and run asynchronously.

Before:

const logger = require('./logger');
const app = require('./app');
const port = app.get('port');
const server = app.listen(port);

server.on('listening', () =>
  logger.info('Feathers application started on http://%s:%d', app.get('host'), port)
);

Now:

const logger = require('./logger');
const app = require('./app');
const port = app.get('port');

app.listen(port).then(server => {
  logger.info('Feathers application started on http://%s:%d', app.get('host'), port)
});

Service setup hooks

Also tracked in https://github.com/feathersjs/feathers/issues/853, service.setup will now be asynchronous (return a promise) allowing to add hooks to it:

app.service('myservice').hooks({
  before: {
    async setup(context) {
      await createDatabaseTableIfNotExists();

      return context;
    }
  }
});

Application setup hooks

Application setup hooks will be part of the already existing app.hooks. The difference for setup hooks will be that they only run once when app.setup() or app.listen() is called instead of for every service call.

app.hooks({
  before: {
    async setup(context) {
      await createDatabaseTableIfNotExists();

      return context;
    }
  },

  error: {
    async setup(context) {
      if(isSomeRecoverableError(context.error)) {
        restartInTenMinutes();
      }

      return context;
    }
  }
});

Differences for setup hooks

setup is slightly different than other service methods in that

• It can not be called externally
• It will only be called once during the application lifecycle
• It will never have a specific user context
• It has no params, data or id

This means that:

• As already mentioned, application setup hooks will only run once instead of for every service like other application level hooks
• all hooks will not be applied to setup to avoid breaking existing hooks

[daffl]

This functionality will not affect how middleware runs. It is possible to register middleware in a setup hook after other asynchronous things have happened though:

setup: [ doAsyncStuff, context => context.app.use(errorHandler) ]

@claustres I was debating that as well, something looking like:

app.service('myservice').hooks({
  before: {},
  after: {},
  error: {},
  setup: {
    before(context) {
      await createDatabaseTableIfNotExists();

      return context;
    }
  }
});

app.hooks({
  before: {},
  after: {},
  error: {},
  setup: {
    before: createDatabaseTableIfNotExists(),
    after: startMonitoring(),
    error: restartInTenMinutes()
  }
});

It adds some inconsistencies when it comes to retrieving and registering hooks and doing codemods but it is more explicit than having some internal exceptions you have to know about.