question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to gracefully shutdown server

See original GitHub issue

How to gracefully close stream during host shutdown? i am always getting " The operation was canceled." exception on CTRL+C (Asp.NET core 3.1 gRPC template). `

        try
        {
            await foreach (var response in requestStream.ReadAllAsync(context.CancellationToken))
            {
                _ = Task.Run(() =>
                {
                    ProcessResponse(response);
                });
            }
        }

        catch (Exception) when (context.CancellationToken.IsCancellationRequested)
        {
            _logger.LogInformation("Server shutdown");
            
            //Graceful shutdown
        }

`

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
JamesNKcommented, Jun 7, 2020

Yes its client streaming scenario. When I press Ctrl+C on server gRPC server console, its taking more time to stop server since cancellationToken is not triggered from client. I modified code as follows to close stream immediately (Using IHostApplicationLifeTime / IHostLifetime)

When you Ctrl+C Kestrel will attempt to gracefully shutdown. That means it will wait some time for requests that are in-progress to end. That is why you see a delay.

var cancellationSource = CancellationTokenSource.CreateLinkedTokenSource(
    context.CancellationToken, _lifetime.ApplicationStopping, _lifetime.ApplicationStopped);

This is good. The server will end reading from the stream when ApplicationStopping is triggered. The call then ends on the server and Kestrel stops quickly.

I don’t think you need ApplicationStopped:

var cancellationSource = CancellationTokenSource.CreateLinkedTokenSource(
    context.CancellationToken, _lifetime.ApplicationStopping);
1reaction
rajeshaz09commented, Jun 6, 2020

Hello @rajeshaz09, I am a gRPC user, not an author of gRPC but let me try to handle the question.

Answering your question is not related to the specific use case, but based on code I am assuming you have:

  • client streaming scenario,
  • an application that is closed using CTRL+C is a client application,

I believe that you are not required to call any method on stream, to have it closed or disposed as it is already aware of being cancelled. Moreover, the stream was responsible for throwing an OperationCanceledException you have catched. Let me know if it solves your concerns.

I think that the log information (“Server shutdown”) is a bit misleading as you are not doing server application shutdown, but rather reacting to RPC termination.

Thanks @wicharypawel Yes its client streaming scenario. When I press Ctrl+C on server gRPC server console, its taking more time to stop server since cancellationToken is not triggered from client. I modified code as follows to close stream immediately (Using IHostApplicationLifeTime / IHostLifetime)

`

        var cancellationSource = CancellationTokenSource.CreateLinkedTokenSource(
            context.CancellationToken, _lifetime.ApplicationStopping, _lifetime.ApplicationStopped);

        try
        {
            await foreach (var response in _requestStream.ReadAllAsync(cancellationSource.Token))
            {
                _ = Task.Run(() =>
                {
                    ProcessResponse(response);
                });
            }
        }

        catch (Exception) when (cancellationToken.IsCancellationRequested)
        {
            //Graceful closing of streaming in following cases
            //1) host stopping
            //2) client ended stream
        }

`

Read more comments on GitHub >

github_iconTop Results From Across the Web

Gracefully Shutdown a Server - Hyper
hyper Server s have the ability to “gracefully” shutdown. This means stopping to accept new requests, and shutting down once all in-progress requests...
Read more >
Graceful shutdown in Go http server | by Sam Wang
Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return ...
Read more >
Gracefully Shut Down Go Web Servers | by Halil Yıldırım
If your web server is waiting your active requests to be handled before shutting down the server, it means you are gracefully shutting...
Read more >
How to gracefully shutdown your Linux system
The shutdown command will trigger the proper Init level resulting in the OS gracefully stopping all running services. Any stuck processes will be...
Read more >
Shutdown server gracefully
Under Manage > Remote Task, you can create a power task selecting any of "Reboot, Power Cycle & Power Off" with "Shutdown OS...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found