Serialization exception on remote schema error

Describe the bug As part of introducing test seams into our application we’ve added a delegating handler to the http clients used to communicate with remote schemas. This allows us to simulate them being unavailable. The core code for this handler looks like the following:

if (IsServiceUnavailable(request)) {

	return new HttpResponseMessage {
		StatusCode = HttpStatusCode.ServiceUnavailable,
		Content = new StringContent(""),
		ReasonPhrase = $"Service '{_schemaName}' unavailable due to test configuration"
	};
} else {
	return await base.SendAsync(request, cancellationToken);
}

When the unavailable response is triggered we see the initiating request to the the stitched schema fail with a 500 error.

The stack trace in the logs includes the following:

Newtonsoft.Json.JsonSerializationException: Self referencing loop detected for property 'ManifestModule' with type 'System.Reflection.RuntimeModule'. Path 'errors[0].extensions.remote.Exception.TargetSite.Module.Assembly'.   
at bool Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference(JsonWriter writer, object value, JsonProperty property, JsonContract contract, JsonContainerContract containerContract, JsonProperty containerProperty)
at bool Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, out JsonContract memberContract, out object memberValue)  
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)  
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) 
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
at void Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, object value, Type objectType)  at void Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, object value, Type objectType)
at string Newtonsoft.Json.JsonConvert.SerializeObjectInternal(object value, Type type, JsonSerializer jsonSerializer)
at async Task HotChocolate.Execution.JsonQueryResultSerializer.SerializeAsync(IReadOnlyQueryResult result, Stream stream) in C:/hc/src/Core/Core/Execution/JsonQueryResultSerializer.cs:line 38 
at async Task HotChocolate.AspNetCore.QueryMiddlewareBase.WriteResponseAsync(HttpResponse response, IExecutionResult executionResult) in C:/hc/src/Server/AspNetCore/QueryMiddlewareBase.cs:line 221
at async Task HotChocolate.AspNetCore.QueryMiddlewareBase.HandleRequestAsync(HttpContext context, IQueryExecutor queryExecutor) in C:/hc/src/Server/AspNetCore/QueryMiddlewareBase.cs:line 208
at async Task HotChocolate.AspNetCore.QueryMiddlewareBase.InvokeAsync(HttpContext context) in C:/hc/src/Server/AspNetCore/QueryMiddlewareBase.cs:line 117

Expected behavior I would expect the stitched schema to return a 200 response with the details of the unavailable remote schema included in the errors collection and the results of any other remote schema to be returned.

Additional context This is with Hot Chocolate 0.8.1.

[michaelstaib]

After looking at it … I see the error now. The problem is that we are not removing the error from the error in the extensions. That is why your clear extension fix remedies this issue.

[nigel-sampson]

I haven’t been able to recreate the issue in your unit tests yet but I’ve found that adding an error filter that clears the extensions works around this.

.AddExecutionConfiguration(excution =>
{
	excution.AddErrorFilter(error => {
		return error.Exception is Exception ex
			? ErrorBuilder.FromError(error).ClearExtensions().Build()
			: error;
	});
})

On a related note, it seems that we’re potentially leaking exception information in the extension.remote.Exception. path of this exception such as stack trace.