Collection Variant Resolver: Performance Issues

Describe the bug The variant field resolver is causing us memory and blocking issues, on brief inspection it looks like some queries are returning a lot of rows, this could be related to some queries are still unnecessarily joining on collections despite being the result of the select in patch.

Environment (please complete the following information):

• @vendure/core version: latest stable
• Nodejs version: 16
• Database (mysql/postgres etc): postgres

Additional context Add any other context about the problem here.

Here’s the output of getQuery() - there’s no limit, even though one is specified when calling the query via options on the field resolver (not the outer collection options)

SELECT *
FROM "product_variant" "productvariant"
         LEFT JOIN "product_variant_channels_channel" "productvariant_productvariant__channels"
                   ON "productvariant_productvariant__channels"."productVariantId" = "productvariant"."id"
         LEFT JOIN "channel" "productvariant__channels"
                   ON "productvariant__channels"."id" = "productvariant_productvariant__channels"."channelId"
         LEFT JOIN "product_variant_translation" "productvariant_translations"
                   ON "productvariant_translations"."baseId" = "productvariant"."id"
         LEFT JOIN "collection_product_variants_product_variant" "collection_productvariant"
                   ON "collection_productvariant"."productVariantId" = "productvariant"."id"
         LEFT JOIN "collection" "collection" ON "collection"."id" = "collection_productvariant"."collectionId"
         LEFT JOIN "product" "product" ON "product"."id" = "productvariant"."productId"
WHERE 1 = 1
  AND "productvariant__channels"."id" = :channelId
  AND "product"."deletedAt" IS NULL
  AND "productvariant"."deletedAt" IS NULL
  AND "collection"."id" = :collectionId

Obviously that’s quite different from what actually gets executed via getManyAndCount() - which looks to be a series of select (distinct id) from and select from where id in (…) and the get distinct id queries do look to have the limit passed though

SELECT *
FROM "product_variant" "productvariant"
         LEFT JOIN "product_variant_channels_channel" "productvariant_productvariant__channels"
                   ON "productvariant_productvariant__channels"."productVariantId" = "productvariant"."id"
         LEFT JOIN "channel" "productvariant__channels"
                   ON "productvariant__channels"."id" = "productvariant_productvariant__channels"."channelId"
         LEFT JOIN "product_variant_translation" "productvariant_translations"
                   ON "productvariant_translations"."baseId" = "productvariant"."id"
         LEFT JOIN "collection_product_variants_product_variant" "collection_productvariant"
                   ON "collection_productvariant"."productVariantId" = "productvariant"."id"
         LEFT JOIN "collection" "collection" ON "collection"."id" = "collection_productvariant"."collectionId"
         LEFT JOIN "product" "product" ON "product"."id" = "productvariant"."productId"
WHERE (1 = 1 AND "productvariant__channels"."id" = $1 AND "product"."deletedAt" IS NULL AND
       "productvariant"."deletedAt" IS NULL AND "collection"."id" = $2)
  AND ("productvariant"."id" IN (88))

I’m thinking it shouldn’t be joining back on collections there because we don’t need them at this point due to having already retrieved the distinct ids of the productVariants in the collection

Here’s the corresponding distinct id query:

SELECT DISTINCT "distinctAlias"."productvariant_id" as "ids_productvariant_id"
FROM (SELECT *
      FROM "product_variant" "productvariant"
               LEFT JOIN "product_variant_channels_channel" "productvariant_productvariant__channels"
                         ON "productvariant_productvariant__channels"."productVariantId" = "productvariant"."id"
               LEFT JOIN "channel" "productvariant__channels"
                         ON "productvariant__channels"."id" = "productvariant_productvariant__channels"."channelId"
               LEFT JOIN "product_variant_translation" "productvariant_translations"
                         ON "productvariant_translations"."baseId" = "productvariant"."id"
               LEFT JOIN "collection_product_variants_product_variant" "collection_productvariant"
                         ON "collection_productvariant"."productVariantId" = "productvariant"."id"
               LEFT JOIN "collection" "collection" ON "collection"."id" = "collection_productvariant"."collectionId"
               LEFT JOIN "product" "product" ON "product"."id" = "productvariant"."productId"
      WHERE 1 = 1
        AND "productvariant__channels"."id" = $1
        AND "product"."deletedAt" IS NULL
        AND "productvariant"."deletedAt" IS NULL
        AND "collection"."id" = $2) "distinctAlias"
ORDER BY "productvariant_id" ASC
LIMIT 1 

[xrev87]

@xrev87 can you provide the GraphQL query that is causing that SQL query?

Sure:

fragment BreadcrumbsProps on Collection {
  breadcrumbs {
    id
    name
    slug
    __typename
  }
  __typename
}

fragment CollectionProps on Collection {
  createdAt
  description
  id
  isPrivate
  name
  position
  slug
  productVariants(options: { take: 1 }) {
    totalItems
    __typename
  }
  __typename
}

fragment CustomFieldsProps on CollectionCustomFields {
  Featured
  __typename
}

fragment ChildrenProps on Collection {
  children {
    ...CollectionProps
    ...BreadcrumbsProps
    customFields {
      ...CustomFieldsProps
      __typename
    }
    __typename
  }
  __typename
}

fragment ParentProps on Collection {
  parent {
    ...CollectionProps
    __typename
  }
  __typename
}

query Collections($options: CollectionListOptions) {
  collections(options: $options) {
    items {
      ...CollectionProps
      ...BreadcrumbsProps
      ...ParentProps
      ...ChildrenProps
      customFields {
        ...CustomFieldsProps
        __typename
      }
      __typename
    }
    totalItems
    __typename
  }
}

// Options: sort: { position: ASC }

[michaelbromley]

Thanks, I think I discovered the root cause. This is quite a heavy query anyway due to the unbounded recursive nature of parent/children selection. But I have made a change that in my testing dataset reduces the query time from ~15s to ~3s (did not measure memory consumption).

Will put this fix in the upcoming v1.7.