[FEATURE] Testing support

Summary:

Testing OLTP applications that use DynamoDB as the primary database is a bit awkward right now and requires extra bells and whistles to make sure there are no data collisions. Imagine a hypothetical scenario for a game application where high score is stored on the user object. You have one test which updates user’s high score and sends a notification letting the user know they’re #1 on the leaderboard. You have another test that populates the leaderboard - scans the table and returns the users with the highest score. It’s really tricky to write these tests in a way to prevent all possible data collisions and make them succeed deterministically.

In Rails there’s a popular FactoryBot library that works nicely with Rspec (testing framework) and makes it really easy to work with mock data. You define your mock data in the global scope at the top of your test suite, and then just reference these variables in each of your tests, but the variables will actually contain different copies of data inside of each test and data collision almost never happens.

Other:

• I have read through the Dynamoose documentation before posting this issue
• I have searched through the GitHub issues (including closed issues) and pull requests to ensure this feature has not already been suggested before
• I have filled out all fields above
• I am running the latest version of Dynamoose

---

Describe the solution you’d like Something like this can be done with DynamoDB, though due the specifics of its architecture the only reliable way to avoid collisions is to create a separate table for each test suite. Here’s how I think this could work.

First, there should be a config option (not sure if it already exists or not) to assume all database tables are fully set up and skip checks when we call dynamoose.model(). We won’t need these tables in the test environment.

Second, we can have a function, something like dynamoose.factory(), that takes a Dynamoose model as the first argument and a function as the second. It creates a table prefixed with Date.now() timestamp and returns a wrapper for that model that for all intents and purposes behaves like the actual reference to the Dynamoose model, except that you should be able to just run Model.create() with no arguments and it will create a perfectly valid instance for you using this factory function. The factory function takes two arguments provided by Dynamoose, create function (don’t confuse with the aforementioned Model.create()) and index.

Here’s how this would look in practice:

// test/factories.ts
import * as _ from 'lodash';
import * as dynamoose from 'dynamoose';
import { User, Note } from '../src/models'; 
export const userFactory: dynamoose.FactoryFunc<User> = async (create, index) => {
  const item = await create({
    name: `User ${index+1}`,
    company: [
      _.sample(['Silver', 'Crystal', 'Ocean']),
      _.sample(['Enterprises', 'Productions', 'Security', 'Technologies']),
    ].join(' ');
    await item.customMethod();
});
export const noteFactory: dynamoose.FactoryFunc<Note> = (create, index) => create({ name: `Note #${index+1}` }));
export const nullFactory = () => null;

// test/my-test.ts
import * as dynamoose from 'dynamoose';
import { User, Note } from '../src/models'; 
import { userFactory, noteFactory, nullFactory } from './factories';
let UserFactory: dynamoose.Factory<User>;
let NoteFactory: dynamoose.Factory<Note>;
let CustomNoteFactory: dynamoose.Factory<Note>;
let NullFactory: dynamoose.Factory<Note>;
let userA: User;
let userB: User;
let noteA: Note;
let noteB: Note;
let noteC: Note;

beforeEach(() => {
  UserFactory = dynamoose.factory(User, userFactory); // creates '1587779245702_User' table
  NoteFactory = dynamoose.factory(Note, noteFactory); // creates '1587779245756_Note' table
  CustomNoteFactory = dynamoose.factory(Note, (create, index) => create({ name: `Note ${String.fromCharCode(65 + index)}` }), `${Date.now()}-CustomNote` ); // creates '1587779245805-CustomNote' table
  NullFactory = dynamoose.factory(Note, nullFactory); // creates '1587779245913_Note' table
  userA = new UserFactory(); // User { name: 'User 1', company: 'Ocean Enterprises' }
  userB = new UserFactory({ name: 'Custom User Name' }); // User { name: 'Custom User Name', company: 'Crystal Technologies' }
  noteA = new NoteFactory(); // Note { name: 'Note #1' }
  noteB = new NoteFactory(); // Note { name: 'Note #2' }
  noteC = new CustomNoteFactory(); // Note { name: 'Note A' }
  new NullFactory({ name: 'My Note' }); // returns null and never creates an instance of the item, because the `create` function never gets called inside the factory function. This shows that if the factory function returns something other than undefined, the factory will be set to that value.
});

afterEach(() => {
  NullFactory.destroy(); // Deletes the table created for this factory
});

it('works', async () => {
  noteA.name = 'New Note';
  await Promise.all([UserFactory.sync(), NoteFactory.sync(), CustomNoteFactory.sync()]); // Note that this is where the items actually get saved to the database. This lets us overwrite values for individual tests, like what we did here with noteA.
  console.log(noteA); // Note { name: 'New Note' });
  console.log(noteB); // Note { name: 'Note #2' });
});

it('also works', async () => {
  await dynamoose.factory.sync(UserFactory, NoteFactory, CustomNoteFactory); // Alternative syntax for saving everything to the database.
  console.log(noteA); // Note { name: 'Note #1' }
  console.log(noteB); // Note { name: 'Note #2 } <- Note that even though this object has the same data as in the first test, it is stored in a brand new table, created in the beforeEach hook iteration for this specific test
});

afterAll(() => {
  dynamoose.factory.destroyAll(); // Deleted tables for all remaining factories
});

Describe alternatives you’ve considered This can either be implemented as a core Dynamoose functionality, or it can be its own library, something like ‘dynamoose-factory’, whatever makes most sense. I would actually probably prefer it to be a separate library as this will reduce the size of the production bundle. However, in order to implement this as an external library, it should be possible to interact with Dynamoose’s internals in a way that can achieve this.

[fishcharlie]

@dolsem Amazing summary here. Few questions that initially come to mind:

1. You mention Note that this is where the items actually get saved to the database and also mention about creating tables. But is there actually a DynamoDB Local instance running in this case? Or is this more of a mock situation where it’s saving to some mock array or something that acts as the database?
2. There is a dynamoose.factory.destroyAll method you mention, what about a dynamoose.factory.syncAll method? Would that be useful?
3. For noteA = new Note(); is that supposed to be noteA = new NoteFactory();?
4. In the userFactory, is item then an instance of a User Document? Or what is the type of item?
5. What happens if you want to test query, scan, update, or get functionality? The example provided only seems to go over creating new items/documents.

---

Then replying to a few things:

First, there should be a config option (not sure if it already exists or not) to assume all database tables are fully set up and skip checks when we call dynamoose.model(). We won’t need these tables in the test environment.

Couldn’t we just recommend setting dynamoose.model.defaults = {"create": false}; in test model? Or are we still concerned that the model level options will overwrite the defaults here?

This can either be implemented as a core Dynamoose functionality, or it can be its own library, something like ‘dynamoose-factory’, whatever makes most sense. I would actually probably prefer it to be a separate library as this will reduce the size of the production bundle. However, in order to implement this as an external library, it should be possible to interact with Dynamoose’s internals in a way that can achieve this.

Totally agree with it being a separate library. dynamoose is normally considered a dependency and dynamoose-factory would likely be considered a devDependency. I don’t see the value in adding blot to a production build for functionality that is limited to testing.

I do think this project should be managed by the dynamoose GitHub organization tho to ensure compatibility and such.

[fishcharlie]

@dolsem If you have time, I’d love to see your ideas for a simpler API here. For me, it seems like a great idea. I don’t think it should be part of the core library tho since it has nothing to do with production level code.

The only consideration here, is how it’ll work, and if it’s something that will have enough benefits to where I’d be willing to maintain it. I’m struggling currently to see that it provides enough benefits to warrant the LOE of maintaining it.

If you’d be interested tho, I’d totally consider creating a repo under this organization and having you run this project.

I guess my point is that this is low on my priority list as of right now (especially since there are other ways to do it natively). So although it’s a cool idea, and I’d love to see it done, it doesn’t really fit into my current roadmap and plans I have at this time. If someone else would like to take charge of making this happen, totally willing to support that tho.