Proposal for populates++ hook

An attempt at a design to start discussion.

• @ekryski 's proposal is basically used when we get to populating one item with another.
• Populate definitions may be stored together. This way they can easily refer to one another.
• Once an item or group of items is added, they can be further populated using ‘expandBy’. This is recursive.
• This may eventually be related to models for feathers services.

I’m hoping for

• Comments on concept and packaging
• Improvements to naming.

const populates = {

  // a straight forward set of populates taken from MichaelErmer/feathers-populate-hook
  message: {
    user: { // what @ekryski proposed. Temp renames to localField & foreignField for clarity.
      service: 'users', localField: '_id', foreignField: 'userId',
      on: 'result', multiple: false, query: {}
    },
    comments: {
      service: 'comments', localField: '_id', foreignField: 'commentIds' /* is array */,
      on: 'result', multiple: true, query: {}
    },
    categories: {},
    creator: {},
  },

  // populate a room with all its messages, further populating each message.
  room: {
    creator: {},
    messages: {
      service: 'messages', localField: '_id', foreignField: 'messageIds',
      on: 'result', multiple: true, query: {},
      expandBy: populates.message, // expand every message with populates.message
    },
    owners: {},
  },

  // populate an organization with its rooms, partially further populating each room
  organization: {
    owners: {},
    rooms: {
      service: 'rooms', localField: '_id', foreignField: 'roomIds',
      on: 'result', multiple: true, query: {},
      expandBy: [populates.room, 'messages', 'owners'], // don't expand using populates.room.creator
    }
  },
};

const organizations = app.service('organizations');

organizations.after({
  find: [
    // the fancy populating
    hook.population(populates.organization),

    // You can continue to populate an item the same way as with the current populate
    hook.population({
      creator: {
         service: 'users', localField: '_id', foreignField: 'creatorId',
         on: 'result', multiple: false, query: {}
      }
    }),
  ]
});

[kc-dot-io]

@eddyystop nice! Any chance you can add some performance timing? In the past, I’ve found a big issue with populating data on json and key value stores is the performance of making multiple queries, especially at the 3rd to Nth levels, as each population requires and additional call. This can be pretty heavy if the data structure is an array of arrays, so it would be ideal for us to have some consideration around bench marks when it comes to this so people are thinking about it up front when they design their data models. Food for thought, but great work here!

[eddyystop]

Test of initial populate hook, with trace logs.

https://github.com/eddyystop/feathers-test-populate-etc