question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Azure.Search throws when retreiving a SearchIndexer

See original GitHub issue

Describe the bug Retrieving a SearchIndexer using Azure.Search.Documents.Indexes.SearchIndexerClient.GetIndexerAsync throws an exception

Expected behavior No exception should be thrown

You can see the problem is “mappingFunction”: { “name”: “base64Encode”, “parameters”: null } which has “parameters”: null

and: https://github.com/Azure/azure-sdk-for-net/blob/Azure.Search.Documents_11.1.1/sdk/search/Azure.Search.Documents/src/Generated/Models/FieldMappingFunction.Serialization.cs

if (property.NameEquals("parameters"))
{
    Dictionary<string, object> dictionary = new Dictionary<string, object>();
    foreach (var property0 in property.Value.EnumerateObject())
    {

has this accessed without a check, which throws the exception below. (In the version on the master branch there is an additional check which will still throw an exception).

This file is tagged as autogenerated, but I’m not clear how so I haven’t sent a PR (feel free to point me in the right direction).

Actual behavior (include Exception or Stack Trace) System.InvalidOperationException HResult=0x80131509 Message=The requested operation requires an element of type ‘Object’, but the target element has type ‘Null’. Source=System.Text.Json StackTrace: at System.Text.Json.JsonElement.EnumerateObject() at Azure.Search.Documents.Indexes.Models.FieldMappingFunction.DeserializeFieldMappingFunction(JsonElement element) at Azure.Search.Documents.Indexes.Models.FieldMapping.DeserializeFieldMapping(JsonElement element) at Azure.Search.Documents.Indexes.Models.SearchIndexer.DeserializeSearchIndexer(JsonElement element) at Azure.Search.Documents.IndexersRestClient.<GetAsync>d__19.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Azure.Search.Documents.Indexes.SearchIndexerClient.<GetIndexerAsync>d__48.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()

To Reproduce I do not know exactly how to reproduce this – the indexer was created using the old Microsoft.Azure* libraries. Since this is an actual object that exists in Azure I believe I should be able to retrieve it however it was created.

Here is the request and response as logged for the GetIndexerAsync call: [Informational] Azure-Core: Request [a01498a3-1a94-47c0-a207-4aaa9d976cb6] GET https://xxx.search.windows.net/indexers(‘file-upload-indexer’)?api-version=2020-06-30 Accept:application/json; odata.metadata=minimal api-key:REDACTED x-ms-client-request-id:a01498a3-1a94-47c0-a207-4aaa9d976cb6 x-ms-return-client-request-id:true User-Agent:azsdk-net-Search.Documents/11.1.1,(.NET 5.0.2; Microsoft Windows 10.0.19042) client assembly: Azure.Search.Documents [Informational] Azure-Core: Response [a01498a3-1a94-47c0-a207-4aaa9d976cb6] 200 OK (00.3s) Cache-Control:no-cache Pragma:no-cache ETag:W/“0x8D85A6D49BB7052” client-request-id:a01498a3-1a94-47c0-a207-4aaa9d976cb6 x-ms-client-request-id:a01498a3-1a94-47c0-a207-4aaa9d976cb6 request-id:a01498a3-1a94-47c0-a207-4aaa9d976cb6 elapsed-time:228 OData-Version:4.0 Preference-Applied:REDACTED Strict-Transport-Security:REDACTED Date:Thu, 21 Jan 2021 16:43:04 GMT Content-Type:application/json; odata.metadata=minimal Expires:-1 Content-Length:1204

[Verbose] Azure-Core: Response [a01498a3-1a94-47c0-a207-4aaa9d976cb6] content: {“@odata.context”:“https://xxx.search.windows.net/$metadata#indexers/$entity",“@odata.etag”:“\“0x8D85A6D49BB7052\””,“name”:“file-upload-indexer”,“description”:null,“dataSourceName”:“file-upload-datasource”,“skillsetName”:null,“targetIndexName”:“file-upload-index”,“disabled”:false,“schedule”:{“interval”:“PT2H”,“startTime”:“0001-01-01T00:00:00Z”},“parameters”:{“batchSize”:null,“maxFailedItems”:-1,“maxFailedItemsPerBatch”:null,“base64EncodeKeys”:null,“configuration”:{“failOnUnsupportedContentType”:false,“failOnUnprocessableDocument”:false,“indexStorageMetadataOnlyForOversizedDocuments”:true}},“fieldMappings”:[{“sourceFieldName”:“metadata_storage_size”,“targetFieldName”:“fileSize”,“mappingFunction”:null},{“sourceFieldName”:“metadata_storage_path”,“targetFieldName”:“key”,“mappingFunction”:{“name”:“base64Encode”,“parameters”:null}},{“sourceFieldName”:“metadata_storage_content_type”,“targetFieldName”:“contentType”,“mappingFunction”:null},{“sourceFieldName”:“metadata_storage_last_modified”,“targetFieldName”:“lastModified”,“mappingFunction”:null},{“sourceFieldName”:“metadata_storage_name”,“targetFieldName”:“name”,“mappingFunction”:null}],“outputFieldMappings”:[],"encryptionKey”:null}

Environment:

  • Name and version of the Library package used: Azure.Search.Documents 11.1.1, Azure.Storage.Blobs 12.8.0
  • Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects):

The project is targeting .NET Core 5

PS C:\Users\mark> dotnet --info .NET SDK (reflecting any global.json): Version: 5.0.102 Commit: 71365b4d42

Runtime Environment: OS Name: Windows OS Version: 10.0.19042 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\5.0.102\

Host (useful for support): Version: 5.0.2 Commit: cb5f173b96

.NET SDKs installed: 3.1.100 [C:\Program Files\dotnet\sdk] 3.1.101 [C:\Program Files\dotnet\sdk] 3.1.400 [C:\Program Files\dotnet\sdk] 3.1.404 [C:\Program Files\dotnet\sdk] 5.0.101 [C:\Program Files\dotnet\sdk] 5.0.102 [C:\Program Files\dotnet\sdk]

.NET runtimes installed: Microsoft.AspNetCore.All 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

  • IDE and version : Visual Studio 16.8.4

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:11 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
blushingpenguincommented, Jul 20, 2021

You can’t work around retrieving an existing search indexer that way though – this bug was actually encountered by trying to fetch a search indexer that had been created by the previous version of the Azure wrappers (v11, Microsoft.Azure.Storage) after upgrading to the current version. The only solution I found there was to drop and recreate the search indexer (for me, not a big issue, but potentially expensive for a big index).

0reactions
msftbot[bot]commented, Sep 23, 2021

Hi @blushingpenguin, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Indexer errors and warnings - Azure Cognitive Search
Indexers emit this error when the property flagged as the 'key' can't be found on a particular document. The document key is invalid,...
Read more >
Indexer troubleshooting guidance - Azure Cognitive Search
This article provides indexer problem and resolution guidance for cases when no error messages are returned from the service search.
Read more >
Why does azure search throw IndexBatchException with no ...
The batch size is always 1 as one event message only contain one entity that I need to merge or upload in search....
Read more >
Using Azure Functions to Detect Search Indexer Errors
The problem I was trying to solve was that Azure Search Indexers don't alert you when they fail to index a document.
Read more >
Azure Search Indexer Succeeded with 0/0 Documents
Azure Search Indexer Succeeded with 0/0 Documents · Azure Search Indexer Clears Documents · Azure Search Indexer fails to create when trying to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found