Getting unexpected results array when using FilterExpression with scan

Summary:

I’m trying to get some items returned if the value of one of their attributes is contained within an array of values that I’m passing over.

Code sample:

General

issues = await Model.scan({
  FilterExpression: 'contains(:ids, itemId)',
  ExpressionAttributeValues: { ':ids': params.ids } // params.ids looks like ['id1', 'id2', 'id3']
}).exec()

Current output and behavior:

Within the results I’m getting the correct items back, however, what I’m getting is not an object that includes a property “Items” with those results, instead, I get an array for which when I use Object.keys over, it looks like this:

[ '0',
'1',
'2',
'3',
 '4',
'5',
'6',
'7',
'8',
'lastKey',
 'count',
'scannedCount' ]

Were the value to each of the “numeral” keys is an actual item (object) and then the last three are the “count” props you would expect.

This, of course, makes it hard to use the data (manipulate it somehow, such as mapping over the items), because of those last three props.

Expected output and behavior:

I would expect something similar to what you’d receive from a native DynamoDB result. Something that looks kind of like:

{'items': [...] // with this being a list of the results
'lastKey': 0, // or any number
 'count': 0, // or any number,
'scannedCount: 0, // or any number' }

I can see that this is happening here:

https://github.com/dynamoosejs/dynamoose/blob/155729cda824cb4136e5e0de1a070d9de87b98e8/lib/Scan.js#L85-L103

However, I’m still wondering if I’m doing something wrong with the query or missing something here in order to get a hold of the data in a clearer way.

Is this intended behaviour? Any suggestions for good practices on how to “properly” get those items back helps. Thanks a lot!

[fishcharlie]

@manuelxaguilar Oh one more thing. If DynamoDB is storing it as a string, it’s harder to do filters and such on it. So in that case, yes, it is probably easier to do it on your end. There is a lot of information in other issues (see #445 and #446). We also made a breaking change in version 1.0.0 that kinda changed this behavior.

That being said, you have to consider what that means. DynamoDB charges for every GB sent out to the internet (I believe it is). So the more data that gets returned to you, the more expensive it will be. Overall there are reasons you’d want to let DynamoDB handle it for you. But it really depends on a lot of factors that only you will know.

---

At some point I’d love to go and clean up the whole code related to maps vs objects and lists vs arrays. I personally don’t see the difference between the two. Yet they are different in Dynamoose. It would be a major breaking change and a huge undertaking.

It would also require a massive update to the documentation. I’d also love to update the documentation to give more information about your original question.

That all being said, this is open source. And while I’d love to get around to those things, there is a limited amount of time I have to get stuff done on this project. Any help would be much appreciated! 😃

[fishcharlie]

@manuelxaguilar Mind creating a PR with those changes? And make sure Allow edits from maintainers is checked so I can play around with it?