SqlDataReader.Close blocks in SNIReadSyncOverAsync after early breaking from reading large dataset

SqlDataReader is taking unproportionaly long time to Dispose if you are early breaking enumeration through large data set.

I can observer order of magnitude difference in Dispose time depending on number in top clause in select statement, but regardless of the number of rows actually selected.

Let’s have the following code:

    public static void Test()
    {
        IEnumerable<int> en1 = EnumerateLargeData(1000);
        // log time here
        foreach (int i in en1)
        { }
        // log time here

        IEnumerable<int> en2 = EnumerateLargeData(2000000000);
        foreach (int i in en2)
        { }
        // log time here
    }

    public static IEnumerable<int> EnumerateLargeData(int maxCount)
    {
        using (SqlConnection sqlConnection = new SqlConnection(connectionString))
        {
            sqlConnection.Open();
            using (SqlCommand getDataCommand = new SqlCommand("[dbo].[GetLargeData_SP]", sqlConnection))
            {
                getDataCommand.CommandTimeout = 0;
                getDataCommand.CommandType = CommandType.StoredProcedure;
                getDataCommand.Parameters.AddWithValue("@MaxCount", maxCount);

                using (var reader = getDataCommand.ExecuteReader())
                {
                    int recordsCount = 0;
                    while (reader.Read())
                    {
                        yield return 1;
                        if (recordsCount++ > 100)
                        {
                            //Here is where issue happens
                            break;
                        }
                    }
                }
            }
        }
    }

Where SP is selecting data from some very large table:

CREATE procedure [dbo].[GetLargeData_SP]
	@MaxCount INT
AS
BEGIN
	SELECT TOP (@MaxCount)
		[DataPoint]
	FROM 
		[dbo].[LargeDataTable] WITH(NOLOCK)
END

You will see a very large difference in Dispose time depending on the maxCount argument - especially when pulling data over slower netwrok. In my scenario I’m done fetching data in few hundreds milliseconds but then stuck in Dispose for 2 minutes. Exactly in this stack:

[roji]

The correct behavior of a client talking to SQL Server over TDS is to either Cancel the command or consume the results. In the repro, the driver is cleaning up for the application by consuming the results, which is the safe option since it does not know what the executed statement is doing. Failing to consume the results is unsafe and can result in inconsistent application behavior. Calling Cancel would be a choice the application has to make, if it is safe to do so.

FWIW the same happens in Npgsql, and I’d expect it to be somewhat standard behavior across database drivers.

@divega we have #113 to track implementation of the new async APIs. However, while close/dispose of the reader could help not block the thread while results are consumed, it would still let the query run to completion, possibly using server and network resources, which isn’t ideal. The only way around that would be to trigger cancellation when the reader is disposed, but as @David-Engel wrote, it’s not right for the driver to do this since we don’t want what’s running and whether it’s safe to cancel.

My general response to this would be to not request a huge amount of results if one isn’t sure they’re going to be needed. SQL paging, cursors and similar mechanisms can be used to fetch results in chunks, providing natural “exit” points. Otherwise the application can trigger cancellation itself as mentioned above.

[divega]

@David-Engel new CloseAsync and DisposeAsync APIs are being added to the ADO.NET provider model in .NET Core 3.0. I think these could help avoid blocking in a a future version of SqlClient, when you are able to target .NET Standard 2.1. Perhaps having a general issue in the backlog for implementing the new async surface would be a good idea (unless this already exists).

cc @roji