[Azure Search] Cannot 'select' fields in schemas containing Edm.ComplexType fields using the TypeScript syntax

• Package Name: @azure/search-documents
• Package Version: 11.3
• Operating system: n/a
• nodejs
  • version:
• browser
  • name/version:
• typescript
  • version: 4.8
• Is the bug related to documentation in
  • README.md
  • source code documentation
  • SDK API docs on https://docs.microsoft.com

Describe the bug According to https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/search/search-documents/README.md#querying-with-typescript , it should be possible to strongly type queries made using SearchClient. However, the current types only support a subset of fields. It is not possible to query sub-fields of Edm.ComplexType or Collection(Edm.ComplexType).

To Reproduce Here’s a modified version of the documentation that triggers the compilation error:

// An example schema for documents in the index
interface HotelDetails {
  Description: string;
  ParkingIncluded: boolean;
  LastRenovationDate: Date;
  Rating: number;
}
interface Hotel {
  HotelId: string;
  HotelName: string;
  HotelDetails: HotelDetails;  //< Edm.ComplexType
}

const client = new SearchClient<Hotel>(...);

const searchResults = await client.search("wifi -luxury", {
  select: [
    "HotelId",             //< ok
    "HotelName",           //< ok
    "HotelDetails/Rating"  //< compilation error while the reference grammar says it should be ok.
                           //< https://learn.microsoft.com/en-us/azure/search/search-query-odata-select
  ],
});

Expected behavior The client should expect the proper list of fields and produce the correct return type, including when dealing with nested objects and collections.

Additional context This bug is caused by the following definition:

search<Fields extends keyof T>(searchText?: string, options?: SearchOptions<Fields>): Promise<SearchDocumentsResult<Pick<T, Fields>>>;

which uses keyof T to build the list of fields. It should be possible to create a type that deeply traverses T and builds the proper list of keys. In fact, this post on StackOverflow already covers most cases and only needs minor modifications to change the separator character when it comes to generating the list of fields. It would not, however, be enough to deal with the return type of the search function.

EDIT: Here’s a complete version that handles arrays correctly that you can use as the basis to fix this issue:

type Join<K, P> = K extends string | number ?
    P extends string | number ?
    `${K}${"" extends P ? "" : "/"}${P}`
    : never : never;

type Prev = [never, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
    11, 12, 13, 14, 15, 16, 17, 18, 19, 20, ...0[]]

type Paths<T, D extends number = 10> = D extends never
    ? never
    : T extends object
        ?
            {
                [K in keyof T]-?: K extends string | number
                    ? T[K] extends Array<infer AT>
                        ? `${K}` | Join<K, Paths<AT, Prev[D]>>
                        : `${K}` | Join<K, Paths<T[K], Prev[D]>>
                    : never
            }[keyof T]
            : never

type Foo = {
    foo: number;
    bar: string;
    biz: () => {};
    baz: Foo;
    buzz: { a: string }[];
}

const select: Paths<Foo>[] = ['foo', 'baz/foo', 'buzz/a'];   //< compiles

[witemple-msft]

I came up with an alternative that I think could address the situation without requiring too much type machinery. They are specialized for the Search $select grammar.

There are two utility types:

• UnionToIntersection<U>, which converts a union T1 | T2 | ... | TN to an intersection T1 & T2 & ... & TN.
• ExcludedODataTypes, which is just a union of types that shouldn’t be recursively mapped, currently only Date but might add others as you’ve suggested.

Then the two operative types that appear in the search signature:

• SearchSelectPaths<T extends object>, which does a post-order traversal of T to form a union of all property paths.
• SearchDeepPick<T extends object, Paths extends SearchSelectPaths<T>>, which performs the deep pick by recursively creating a union of interfaces each containing a single refined field of T and then converting the whole thing to an intersection.

Heavily-commented TypeScript Playground here. This solution has some nice properties, such as all the paths and fields being IntelliSense auto-completable and without destroying the editor hints with huge serialized types.

@kawazoe It would be awesome if you could try these types with some of the examples that you said were throwing 7056 (type too long) and any other recursion limit errors. I’d love to know how the editor experience looks & whether or not you run into any type errors. I think these definitions should prevent those recursion limit errors by actually convincing the type serializer not the expand the type in the first place. Also, how deeply nested are the types that you are working with?

CC @xirzec @bterlson @joheredi wdyt?

[kawazoe]

@witemple-msft For the array part, it’s just doing a map on each value. Basically, if your type look like this:

type People {
  addresses: [
    { part: string, city: string, country: string, kind: string }
  ]
}

A select query like ['addresses/city', 'addresses/kind'] should produce:

{
  addresses: [
    { city: string, kind: string }
  ]
}

And a query like ['addresses'] should return the entire original array.

In all cases, you cannot limit which element in the array you will be getting. You can only control their structure. Obviously, this is also recursive since you can have an array of complex type in an array of complex type.