Intermittent `MongoSocketWriteException` using the reactive driver

Hi team, we are facing intermittent MongoSocketWriteException

Click to see error stacktrace

at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186)
   at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:195)
   at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
   at reactor.core.publisher.FluxUsingWhen$UsingWhenSubscriber.deferredError(FluxUsingWhen.java:411)
   at reactor.core.publisher.FluxUsingWhen$RollbackInner.onComplete(FluxUsingWhen.java:488)
   at reactor.core.publisher.Operators.complete(Operators.java:136)
   at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:45)
   at reactor.core.publisher.Mono.subscribe(Mono.java:4150)
   at reactor.core.publisher.FluxUsingWhen$UsingWhenSubscriber.onError(FluxUsingWhen.java:377)
   at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172)
   at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onError(MonoCollectList.java:113)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106)
   at reactor.core.publisher.Operators.error(Operators.java:197)
   at reactor.core.publisher.MonoError.subscribe(MonoError.java:52)
   at reactor.core.publisher.Mono.subscribe(Mono.java:4150)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
   at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onError(MonoFlatMapMany.java:255)
   at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onError(FluxConcatMap.java:262)
   at com.mongodb.reactivestreams.client.internal.AbstractSubscription.onError(AbstractSubscription.java:142)
   at com.mongodb.reactivestreams.client.internal.MongoIterableSubscription.lambda$requestInitialData$0(MongoIterableSubscription.java:47)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.async.client.OperationExecutorImpl$1$1$1.onResult(OperationExecutorImpl.java:92)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.operation.FindOperation$3.onResult(FindOperation.java:750)
   at com.mongodb.internal.operation.OperationHelper$ReferenceCountedReleasingWrappedCallback.onResult(OperationHelper.java:531)
   at com.mongodb.internal.operation.CommandOperationHelper$TransformingReadResultCallback.onResult(CommandOperationHelper.java:543)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor$2.onResult(DefaultServer.java:285)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.CommandProtocolImpl$1.onResult(CommandProtocolImpl.java:82)
   at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection$2.onResult(DefaultConnectionPool.java:530)
   at com.mongodb.internal.connection.UsageTrackingInternalConnection$2.onResult(UsageTrackingInternalConnection.java:142)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.InternalStreamConnection$2.onResult(InternalStreamConnection.java:435)
   at com.mongodb.internal.connection.InternalStreamConnection$2.onResult(InternalStreamConnection.java:429)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.InternalStreamConnection$3.failed(InternalStreamConnection.java:542)
   at com.mongodb.connection.netty.NettyStream$2.operationComplete(NettyStream.java:204)
   at com.mongodb.connection.netty.NettyStream$2.operationComplete(NettyStream.java:200)
   at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
   at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
   at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
   at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
   at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
   at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
   at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:1021)
   at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:882)
   at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367)
   at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
   at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
   at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071)
   at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
   at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   at java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: org.springframework.dao.DataAccessResourceFailureException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message
   at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:85)
   at org.springframework.data.mongodb.core.ReactiveMongoTemplate.potentiallyConvertRuntimeException(ReactiveMongoTemplate.java:2827)
   at org.springframework.data.mongodb.core.ReactiveMongoTemplate.lambda$translateException$91(ReactiveMongoTemplate.java:2810)
   at reactor.core.publisher.Flux.lambda$onErrorMap$28(Flux.java:6677)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
   ... 42 common frames omitted\nCaused by: com.mongodb.MongoSocketWriteException: Exception sending message
   at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:619)
   at com.mongodb.internal.connection.InternalStreamConnection.access$1200(InternalStreamConnection.java:78)
   ... 22 common frames omitted\nCaused by: io.netty.channel.StacklessClosedChannelException: null
   at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source)\n"}

Version details:

org.springframework.boot:spring-boot-starter-data-mongodb-reactive -> 2.4.5 org.springframework.data:spring-data-mongodb:3.1.8 org.springframework.boot:spring-boot-starter-actuator -> 2.4.5 org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.32

Can you please help us understand what could be the reason for this intermittent failure.

[ashishkujoy]

Thanks @christophstrobl for quick reply. I will try to provide a sample application.

[ashishkujoy]

For others reference: https://stackoverflow.com/questions/48464455/connection-to-cosmosdb-through-mongo-api-fails-after-idle

This is the same issue which we are observing in our application, earlier in non prod we were using Bitmani Mongo DB docker images and not seen the issue, after switching to AWS document db we started observing this issue. We will tryout setting max idle time out, hopefully that will resolve our issue.