Help on examples on using `.query` and the query object (not the chainable methods)

Summary:

I’m trying to do a generic query builder that transforms URL queries to a REST API to the inner syntax that Dynamoose query object expects, and for now I was able to implement the sorting and the sparse fieldsets, but for some reason I cannot do the same with even simple queries using filter, and, or, and since there is little to none documentation or examples whatsoever on how to do it, I’m having a hard time making it to work, I even looked into the tests but all examples are related to the chainable methods, and the Query source code didn’t helped me a lot too, so any help or guidance will be appreciated.

Note: I wanted to use the query options variant instead of the chainable methods since it would be easier for me to build and transform the API requests into the query options, and this is how in the past I’ve do it with some other ORMs like Sequelize, for instance.

https://dynamoosejs.com/api/query

Thanks

Code sample:

Schema

const schema: any = new orm.Schema({
      id       : {
        type    : String,
        hashKey : true,
        required: true,
        default : () => {
          return uuid.v4();
        }
      },
      type     : {
        type        : String,
        default     : 'user',
        forceDefault: true,
        index       : [
          {
            global    : true,
            name      : 'typeUsernameGlobalIndex',
            rangeKey  : 'username',
            project   : true,
            throughput: defaultThroughput
          },
          {
            global    : true,
            name      : 'typeCreatedAtGlobalIndex',
            rangeKey  : 'createdAt',
            project   : true,
            throughput: defaultThroughput
          }
        ]
      },
      username : {
        type    : String,
        required: true
      },
      createdAt: {
        type: String
      }
    }, Object.assign({}, ORM_SCHEMA_DEFAULT_OPTIONS, {
      throughput: defaultThroughput
    }));

Model

orm.model(process.env.SERVICE_DYNAMODB_TABLE_USERS, schema);

General

I’m calling the query method like the following:

users = await User
        .query(...filterQuery)
        .exec();

which filterQuery is an array with to items, the first is the query object and the second one the options object.

Current output and behavior:

I’ve tried with a lot of filterQuery sets and variations foe example, filter by the username field, like (first array item is the query object, second one the options object, and I tried the filter on both maps without any results):

filterQuery [ { type: { eq: ‘user’ } }, { and: { filter: { username: eq: “some” } } } ] filterQuery [ { type: { eq: ‘user’ } }, { { filter: { username: eq: “some” } } } ] filterQuery [ { type: { eq: ‘user’ } }, { { username: { eq: “some” } } ] filterQuery [ { type: { eq: ‘user’ } }, { { username: { filter: { eq: “some” } } } } ] filterQuery [ { type: { eq: ‘user’ } }, { { username: { filter: { eq: “some” } } } } ]

And no matter how I build the query, the result is always the same, an unfiltered array with all users.

Expected output and behavior:

That the filter works and return a filtered array.

Environment:

Operating System: Kubuntu Operating System Version: 18.04 64bits Node.js version (node -v): v10.0.0 NPM version: (npm -v): 6.9.0 Dynamoose version: 1.7.2 Dynamoose Plugins: none

Dynamoose Plugins:

• Yes, I believe that one or multiple 3rd party Dynamoose plugins are effecting this issue
• No, I believe this issue is independent of any 3rd party Dynamoose plugins I’m using
• Unknown, I’m unsure if Dynamoose plugins are effecting this issue
• I am not using any Dynamoose plugins

Other information (if applicable):

None

Type (select 1):

• Bug report
• Feature suggestion
• Question
• Other suggestion
• Something not listed here

Other:

• I have read through the Dynamoose documentation before posting this issue
• I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
• I have searched the internet and Stack Overflow to ensure this issue hasn’t been raised or answered before
• I have tested the code provided and am confident it doesn’t work as intended
• [not applicable] I have ensured that all of my plugins that I’m using with Dynamoose are listed above
• I have filled out all fields above
• I am running the latest version of Dynamoose

[diosney]

@fishcharlie Any help on this would be awesome 😃

Just to let you know, I plan to spend some time with Dynamoose once I can set up all my needed basic features so after that I will add some PRs to improve documentation and the validations on what I’ve stumbled until now.

[diosney]

@uzochukwuonuegbu Sorry for being late, kind of bussy, you know how it is.

I did it for a generic Serverless JSON:API library I was working on, so expect the code to be kind of convoluted.

See this function at https://gitlab.com/vifros/serverless/serverless-json-api/-/blob/master/src/classes/jsonapi.class.ts#L382

There you are going to find how I built the queries step by step depending on the input entry, for instance:

...
   // This is the initial setup for the query object.
   let query: any = model
      .query('type')
      .using(`type${uppercasedRangeKey}GlobalIndex`)
      .eq(type);

    // This is a complex function that builds the query depending on the input filter object, see below.
    if ('filter' in reqQuery) {
      buildQueryFromFilter(query, reqQuery.filter, schemaObject, rangeKey);
    }

    // This is how I queued the sorting.
    switch (reqQuery.sort[0]) {
      case '+':
        query.ascending();
        break;

      case '-':
        query.descending();
        break;

      default:
        query.ascending();
        break;
    }

...

    // How setup a limit.
    query.limit(pageSizeOptimizationForLists);

    // And pagination.
    if (reqQuery.page.after
        && reqQuery.page.after !== 'null') { // Due the base64 encoding.

      // TODO @diosney: Validate this.
      query.startAt(JSON.parse(Buffer.from(reqQuery.page.after, 'base64').toString()));
    }

...
   // And sparse fieldsets.
   query.attributes(fieldsToReturn);

...

And the buildQueryFromFilter function you can find it at https://gitlab.com/vifros/serverless/serverless-json-api/-/blob/master/src/classes/jsonapi.class.ts#L488

There you will see how I built the filters from an input object, I first do some checks and normalizations and later in the function I really use the query object again, starting with the last if...else:

 if (fieldName === rangeKey) {
          query.where(rangeKey);

          if (hasNotOperator) {
            query.not()[operator.replace('$', '')](...operatorValue);
          }
          else {
            query[operator.replace('$', '')](...operatorValue);
          }
        }
        else {
          // index === 0 is when no `where` was added (no filter on rangeKey).
          // index === 1 is when a `where` was added (filter on rangeKey).
          //
          // On those cases, the `and` is not needed since it is already implicit.
          if (index >= 1 && !hasOrOperator) {
            query.and();
          }

          if (index > 0 && hasOrOperator) {
            query.or();
          }

          query.filter(fieldName);

          // This ugly repeated if-else is due `this` is lost if we cache the query.
          if (hasNotOperator) {
            if (operator === '$in') {
              query.not()[operator.replace('$', '')](operatorValue);
            }
            else {
              query.not()[operator.replace('$', '')](...operatorValue);
            }
          }
          else {
            if (operator === '$in') {
              query[operator.replace('$', '')](operatorValue);
            }
            else {
              query[operator.replace('$', '')](...operatorValue);
            }
          }
        }

Hope this helps you a little bit, I only spent a month or two doing the library for a project and then moved on, but managed to do everything I needed with the chaining approach.