SqlClient's transient error handling strategy going forward?

Is your feature request related to a problem? Please describe.

Recently saw https://github.com/dotnet/SqlClient/pull/307 being discussed but was closed in an effort to add some retry support to SqlClient.

This is very much needed, as I feel there is no out of the box retry support for SqlConnection / SqlClient today to combat transient connectivity issues.

Focusing specifically on connecting to Azure SQL, which has pretty terrible connectivity issues, where connections can break and commands can fail mid flight during maintenance operations. Just google “SQL Azure retries” and you will find an abundance of deprecated, copy/paste code, DIY strategies, etc.

As SqlClient / DbConnection / DbCommand / etc classes are either not inheritable or behind interfaces there are problems in implementing retry support today and handing off “resilient” connection instances to application and library code, where they don’t need to make any code changes.

There seems to be some support in EF as per https://docs.microsoft.com/en-us/ef/ef6/fundamentals/connection-resiliency/retry-logic but otherwise you are on your own, as documented by https://docs.microsoft.com/en-us/azure/architecture/best-practices/retry-service-specific#sql-database-using-adonet

All examples of providing retry logic on Azure / MSDN for example are incomplete, not practical, or forget about library code. This unfortunately is due to SqlClient not being flexible enough to support this kind of logic.

Describe the solution you’d like

Simple, out of the box, “pit of success”, connection and command execution resiliency support for SqlClient.

Automatically handle the generic and SQL Azure specific connectivity issues, with proper backoff support, etc, and optional events to notice when a connection reconnect was required to let the application make a decision and provide logging.

A perfect solution to me feels like simply passing in an IErrorHandlingStrategy interface (example name) to the constructor of SqlConnection, which is invoked when there are SQL errors during connection/command execution that require a decision to be made (retry? backoff?) based on the exception.

By default, we could have No Strategy (existing behavior), or Azure Strategy, which knows to handle its bunch of connectivity errors (of which there are many).

Then downstream code has no idea about the retry strategy. It needs no code changes. Applications dont need further changes either. It just works.

Describe alternatives you’ve considered

1. Manually wrapping our code in Polly retry blocks when hitting the database (via extension methods such that command.ExecuteAsync becomes command.ExecuteAndHandleErrorsAsync etc). This makes our code messier to write and also means downstream library code we use is not fixed as we cannot change what method they call.

2. Attempting to inherit SqlClient / SqlConnection / DbCommand / etc and implement retry logic by overriding appropriate methods. This is not possible because some classes are sealed, and some methods are not virtual.

Additional context

There is unfortunately no clear way to implement reliable resiliency logic today that flows between app code and library code seamlessly due to SqlClient and friends being sealed or not having appropriate methods override-able.

It would be fantastic to discuss what a reasonable approach could look like to implement retry support.

I appreciate it is very nuanced, for example, what if a command fails mid-transaction – retrying via reconnect is not necessary a correct thing to do, but with perhaps appropriate hook points the application can make a decision.

[roji]

See the ongoing discussion about resilience in database drivers in general: https://github.com/dotnet/runtime/issues/34817

Although it’s very understandable to want a driver solution that “just works” without any application changes, it’s important to realize that there are some real complications here. See especially https://github.com/dotnet/runtime/issues/34817#issuecomment-613031842 and https://github.com/dotnet/runtime/issues/34817#issuecomment-613051180. Specifically:

if you have issued 3 commands and the 4th fails due to error 1205 (result streaming) you have to retry all 4, if they’re in the same transaction.

A driver-only solution would have to internally track all commands which have been executed against the transaction, and be able to retry them upon transient failure. Even then, what happens if the user has already read the results for the first three commands (and half of the fourth)? Does the driver read and discard all that information in the second pass, and somehow resume in the right place in the middle of the fourth resultset? What happens if the actual data in those resultsets has changed, since a new transaction was started and the database has been changed in the meantime?

These problems seem extremely complicated, and possibly insurmountable at the driver level, without any sort of application knowledge/handling.

[cheenamalhotra]

Closing issue as Configurable Retry logic support is now released with v3.0.0-preview1.

Furthermore, seamless reconnection (connection resiliency) abilities during command execution are not supported by SqlClient as it also requires support from server, which is not available at the moment.