System.IO.IOException: Unable to write data to the transport connection: Broken pipe.

When running a .NET Core 2.1 app in Docker, after a few minutes, the app exits with the following exception:

identity-api         |    Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to write data to the transport connection: Broken pipe.) ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe. ---> System.Net.Sockets.SocketException: Broken pipe
identity-api         |    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
identity-api         |    --- End of inner exception stack trace ---
identity-api         |    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
identity-api         |    at System.Data.SqlClient.SNI.SslOverTdsStream.WriteInternal(Byte[] buffer, Int32 offset, Int32 count, CancellationToken token, Boolean async)
identity-api         |    --- End of inner exception stack trace ---
identity-api         |    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
identity-api         |    at System.Threading.Tasks.Task.Wait()
identity-api         |    at System.Data.SqlClient.SNI.SslOverTdsStream.Write(Byte[] buffer, Int32 offset, Int32 count)
identity-api         |    at System.Net.Security.SslStreamInternal.WriteSingleChunk[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
identity-api         |    at System.Net.Security.SslStreamInternal.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
identity-api         |    at System.Net.Security.SslStreamInternal.Write(Byte[] buffer, Int32 offset, Int32 count)
identity-api         |    at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
identity-api         |    at System.Data.SqlClient.SNI.SNITCPHandle.Send(SNIPacket packet)
identity-api         |    at System.Data.SqlClient.SNI.SNIMarsConnection.Send(SNIPacket packet)
identity-api         |    at System.Data.SqlClient.SNI.SNIMarsHandle.SendControlPacket(SNISMUXFlags flags)
identity-api         |    at System.Data.SqlClient.SNI.SNIMarsHandle.Dispose()
identity-api         |    at System.Data.SqlClient.SNI.TdsParserStateObjectManaged.Dispose()
identity-api         |    at System.Data.SqlClient.TdsParserSessionPool.Dispose()
identity-api         |    at System.Data.SqlClient.TdsParser.Disconnect()
identity-api         |    at System.Data.SqlClient.SqlInternalConnectionTds.Dispose()
identity-api         |    at System.Data.ProviderBase.DbConnectionPool.DestroyObject(DbConnectionInternal obj)
identity-api         |    at System.Data.ProviderBase.DbConnectionPool.CleanupCallback(Object state)
identity-api         |    at System.Threading.TimerQueueTimer.CallCallback()
identity-api         |    at System.Threading.TimerQueueTimer.Fire()
identity-api         |    at System.Threading.TimerQueue.FireNextTimers()

There are two containers that access SQL, they both exit around the same time.

This happens consistently. I’ve tried Alpine and non-Alpine images. The apps are both MVC and the DataContexts have been unused other than running the migrations.

[johnsimons]

Getting the same error running the following:

Netcoreapp2.2 / Linux container (mcr.microsoft.com/dotnet/core/runtime:2.2) / Kubernetes 0.13.5 / SQL Azure ( Elastic Pools, Standard Plan )

Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to write data to the transport connection: Broken pipe.) ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe. ---> System.Net.Sockets.SocketException: Broken pipe
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Data.SqlClient.SNI.SslOverTdsStream.WriteInternal(Byte[] buffer, Int32 offset, Int32 count, CancellationToken token, Boolean async)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Data.SqlClient.SNI.SslOverTdsStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Security.SslStreamInternal.WriteSingleChunk[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
   at System.Net.Security.SslStreamInternal.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
   at System.Net.Security.SslStreamInternal.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Data.SqlClient.SNI.SNITCPHandle.Send(SNIPacket packet)
   at System.Data.SqlClient.SNI.SNIMarsConnection.Send(SNIPacket packet)
   at System.Data.SqlClient.SNI.SNIMarsHandle.SendControlPacket(SNISMUXFlags flags)
   at System.Data.SqlClient.SNI.SNIMarsHandle.Dispose()
   at System.Data.SqlClient.SNI.TdsParserStateObjectManaged.Dispose()
   at System.Data.SqlClient.TdsParserSessionPool.Dispose()
   at System.Data.SqlClient.TdsParser.Disconnect()
   at System.Data.SqlClient.SqlInternalConnectionTds.Dispose()
   at System.Data.ProviderBase.DbConnectionPool.DestroyObject(DbConnectionInternal obj)
   at System.Data.ProviderBase.DbConnectionPool.CleanupCallback(Object state)
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireNextTimers()

This causes the pods to restart with a 139 exit code, and we can’t prevent it 😞

Currently, this is a show stopper for us, are there any workarounds?

[David-Engel]

I noticed a very suspicious throw introduced long ago that would easily cause this issue if disposing a broken, mars-enabled connection: https://github.com/dotnet/corefx/pull/4594/files#diff-a8e6146000f306b97ce4e48207f7cd9fR69 I plan to remove the throw for the next M.D.SqlClient preview.