question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

App Service Graceful Shutdown / Migrate Connections help

See original GitHub issue

Framework: .net5.0 Microsoft.Azure.SignalR: v1.8.0

I have an Azure App Service which I am using as the “Hub Server” in “Default Mode” and I deploy to a staging slot on the service to then swap in the new version.

I have found the documentation on Graceful Shutdown and also read the settings documentation around GracefulShutdown.Mode.

My deployment Pipeline deploys the new version to the staging slot then swaps it in to the production. At this point my clients may be “connected” (proxied through the Azure SignalR service) to the old version in the staging slot.

I’ve configured, as per the documentation, the Graceful Shutdown to Migrate the connections over:

options.GracefulShutdown.Mode = GracefulShutdownMode.MigrateClients;
options.GracefulShutdown.Timeout = TimeSpan.FromSeconds(90);
options.GracefulShutdown.Add<Hubs.NotificationHub>(c =>
{
    Trace.WriteLine("GRACEFUL SHUTDOWN");
    return c.Clients.All.SendAsync("GracefulShutdown");
});

And after the swap my Pipeline stops the staging slot.

But from what I can see - there is no migration, or even graceful shutdown - the client connections get disconnected immediately and reconnect (sometimes the Disconnect is actually logged after the Reconnect, but this is a side issue).

Is there something I am doing wrong? Is this the correct approach?

If I don’t shut down the staging slot then clients remain connected to the old version in the staging slot and new clients can even connect to it. In the documentation it talks about “Set the Server Offline” - but it isn’t clear about the approach for this. Should I be adding a sticky slot setting to do this - something like removing the Azure SignalR connection string? But as I understand on the swap, this would cause a settings change and an application reload therefore terminating the connections anyway?

Could someone give me some guidance on this? Firstly why I see no appearance of the Graceful Shutdown, and then any advice on the recommended way to do this?

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
terencefancommented, Apr 20, 2021

I’ll see if it works well, sorry for any inconvenience.

0reactions
terencefancommented, Sep 2, 2021

https://github.com/Azure/azure-signalr/pull/1376 The bug is fixed in 1.10.0. It was broke by one of our latest changes that allowed the application task to stand alone for a while after the connection has been closed.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Server graceful shutdown - SignalR Service
This article provides information about gracefully shutdown SignalR app server.
Read more >
How to graceful shutdown web app when it exceeds start ...
Azure App Service does not send a SIGTERM signal to gracefully shut down a web app when it exceeds the start time limit....
Read more >
Graceful shutdown and zero downtime deployments in ...
In this article, you will learn how to prevent broken connections when a Pod starts up or shuts down. You will also learn...
Read more >
Graceful shutdown with Node.js and Kubernetes
Graceful shutdown means that the OS (operating system) can safely shut down its processes and close all connections, however long that takes.
Read more >
Kubernetes Pod Graceful Shutdown — How? | by FoxuTech
This setting should help the application to process all the requests and close the connections. This will avoid the forceful shutdown. Command line....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found