[BUG] Azure Search - Cannot $filter by fields containing special characters, even when escaped

Describe the bug When a filter query includes special characters that have been escaped with backslashes, as described in the SDK docs, it does not match fields containing the same text. (Also, single quote ’ is apparently escaped by doubling it: ‘’.) In this case, the test document contains a field like:

"FirmName": "Crazy Charz Inc. ' + - && ! ( ) { } [ ] ^ \" ~ * ? : \\ /"

Expected behavior The document above should be returned.

Actual behavior (include Exception or Stack Trace) Instead, zero results are returned.

To Reproduce

1. Create an index with the document mentioned in the description.
2. Using the SDK, set SearchParams.Filter to search.in(FirmName, 'Crazy Charz Inc. '' \+ \- \&\& \! \( \) \{ \} \[ \] \^ \" \~ \* \? \: \\ \/', '|'). (That’s how the value looks when I inspect in in VS.)
3. Observe that no results are returned (not an error; see SO link below for details on what happens in the Search Explorer with the same query).

Environment:

• Microsoft.Azure.Search v10.1.0
• Windows 10
• Search is performed in a .NET Standard 2.0 library and consumed by a .NET Core 3.1 library.
• Visual Studio 16.4.3

Also see this SO post.

Not to be confused with this feature request; the issue is that the filter doesn’t work even when escaping the characters, not the need for a character-escaping method. Unless I’m doing something wrong, in which case, please tell me how to pass this test case. Thanks!

[brjohnstmsft]

@PrimeHydra I agree it isn’t discoverable enough. It’s probably in a note somewhere. I’ll change the filter docs to make it more obvious.

We’re currently working on a new generation of client libraries for Azure Cognitive Search, and one of our goals is to provide a mechanism for encoding each syntax so you don’t have to deal with it anymore.

[brjohnstmsft]

@PrimeHydra I’ve updated the documentation. The changes should go live some time later today. I added a note at the top of the “simple” and “full Lucene” syntax pages explaining that they’re not the same as the filter syntax, then added a section here explaining how string escaping works in OData.

Thanks for reporting this – your feedback helps us make our service better!