question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

IAsyncEnumerable support

See original GitHub issue

I have created a converter to add support for IAsyncEnumerable<T> to Newtonsoft.Json. I explored doing this directly in the Newtonsoft.Json codebase, but because it targets much older framework versions, I deemed this to be infeasible. Instead, I have created a separate library that contains a converter that can be registered to provide bidirectional conversion. As a separate library, it can target .NET Standard 2.0 and use C# 8 and not have to worry about the older framework versions that the main library supports (implementing IAsyncEnumerable is impossible in pretty much everything in the list other than netstandard2.0).

I think it would be good to publish this as an easily-discoverable NuGet package, e.g. with the name Newtonsoft.Json.AsyncEnumerable. Is there any chance that it might be accepted into the Newtonsoft.Json codebase in order that it be published in this way?

If not, what would the etiquette be around publishing a NuGet package with that ID under my own ownership? Making it perfectly clear of course in all descriptions & documentation that I was not affiliated with James Newton-King or the .NET Foundation.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:9 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
logiclrdcommented, Jul 5, 2020

I just Googled this, and found a site outlining the following scenario:

  • The thread pool has a limited number of threads. On my system this limit is not tiny (4K for my memory/architecture) but it is nevertheless quite finite.
  • With sync over async, the thread initiating the operation might itself be a thread pool thread. Each new operation that comes in pulls a thread from the pool.
  • It invokes the async operation, which sets up an underlying I/O and then returns. When that I/O is completed, it queues up a thread pool action to process it.
  • While the async operation is running, the sync part at the top has to wait for it, which means blocking that thread pool thread it is occupying.
  • If you manage to get as many overlapping I/O operations as there are thread pool threads, then the entire thread pool will be occupied by those sync top ends blocking on the completion of the underlying I/O operations. When those I/O operations complete, their continuations won’t ever get scheduled because there are no free threads.

Thus, a deadlock arises.

So, my converter is only a band-aid – it works in constrained testing scenarios, but wouldn’t be a reliable scenario in a service scenario.

0reactions
logiclrdcommented, Jul 5, 2020

Then I fear ASP.NET Core will never properly support IAsyncEnumerable. 😦 I’ll have to think about whether there might be some architecture to make it discoverable whether a formatter properly supports IAsyncEnumerable.

Read more comments on GitHub >

github_iconTop Results From Across the Web

IAsyncEnumerable<T> Interface
Exposes an enumerator that provides asynchronous iteration over values of a specified type. generic <typename T> public interface class IAsyncEnumerable. C#
Read more >
IAsyncEnumerable with yield in C# - Code Maze
Let's learn about IAsyncEnumerable and the way to iterate over an IEnumerable collection asynchronously with the yield keyword.
Read more >
Where's IAsyncEnumerable<T>?
I am using Unity 2020.3 with enabled support for C# 8. I am able to use await foreach, switch expressions and other fancy...
Read more >
ASP.NET Core 6 and IAsyncEnumerable - Async Streamed ...
NET 6 is support for async streaming of IAsyncEnumerable . In .NET 6, System.Text.Json can serialize incoming IAsyncEnumerable in ...
Read more >
Asynchronous coroutines with C# and IAsyncEnumerable
In 2018, C# 8.0 introduced support for asynchronous streams with new language and runtime features like: IAsyncEnumerable · IAsyncEnumerator ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found