[BUG] Performance issues when schema has data Buffer type

Summary:

The DDB records have a binary data size of 200KB approximately. When I use Dynamoose the scan took 18 seconds, when, I use AWS DDB client, it takes ~100ms.

I suspect that Dynamoose has some performance issues in a Buffer type serialization. When I remove a field with Buffer type from the schema, it starts working fast as expected.

Code sample:

console.time('scan');
const result = await myModel.scan().exec();
console.timeEnd('scan');
console.log(result);

Schema

const mySchema = {
  site: {
    type: Object,
    required: true,
    schema: {
      pages: {
        type: Buffer,
        required: true
      },
      ...
    }
  }
}

Model

const DEFAULT_OPTIONS = {create: false, waitForActive: {enabled: false}};

const myModel: ModelType<MyEntity> & IMyModelMethods = dynamoose.model<MyEntity>(
  'MyTable',
  mySchema,
  DEFAULT_OPTIONS
) as ModelType<MyEntity> & IMyModelMethods;

Current output and behavior (including stack trace):

scan: 19.177s // <---- That's the issue

{
...
count: 53,
scannedCount: 53,
timesScanned: 1,
}

Expected output and behavior:

The scan should take much less than 19 seconds.

Environment:

Operating System: MacOS Operating System Version: 11.3.1 Node.js version (node -v): v14.16.0 NPM version: (npm -v): 6.14.11 Dynamoose version: 2.7.3

Other:

• I have read through the Dynamoose documentation before posting this issue
• I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
• I have searched the internet and Stack Overflow to ensure this issue hasn’t been raised or answered before
• I have tested the code provided and am confident it doesn’t work as intended
• I have filled out all fields above
• I am running the latest version of Dynamoose

[fishcharlie]

@bushev That is helpful. I’ll try to run this sometime soon. Sadly I’m going out of town and won’t be back until the 24th. I’ll see if I can carve out sometime during that trip to look into this more, but no guarantees on that front.

Sorry you are running into this, and hopefully I’ll be able to look into this more soon.

[fishcharlie]

Fixed by #1225