MDC context not passed when using custom ForkJoinPool

Describe the bug

I have a spring boot application in which I have created a custom fork join pool as given below -

@Bean(ThreadPoolConstants.FORK_JOIN_POOL)
@Lazy
public Executor getCustomForkJoinPool() {
    final ForkJoinPool.ForkJoinWorkerThreadFactory factory = pool -> {
        final ForkJoinWorkerThread worker = ForkJoinPool.defaultForkJoinWorkerThreadFactory.newThread(pool);
        worker.setName(ThreadPoolConstants.FORK_JOIN_POOL + worker.getPoolIndex());
        return worker;
    }; 

    return new LazyTraceExecutor(beanFactory,
            new ForkJoinPool(Runtime.getRuntime().availableProcessors(), factory, null, true));
}

The reason for doing this is to increase the thread pool count which I will do by providing min and max pool size (since they were not available in JDK8)

Using this custom pool, am doing a parallelStream() operation on a Map as shown below -

ThreadPoolFactory.getInstance(ThreadPoolType.FORK_JOIN_POOL)
        .execute(() -> myMap.values().parallelStream().forEach(handler -> {
            log.debug("HANDLER :: {}", handler);
            handler.handleEvent(event);
        })
);

Above parallelStream() operation is done in a service method implementation which has @Async() annotation as shown below -

@Async(ThreadPoolConstants.EVENT_HANDLER)
@Override
public void handleEvent(@NonNull final String event) {
}

Now @Async(ThreadPoolConstants.EVENT_HANDLER) is also a custom thread pool created like below -

@Bean(ThreadPoolConstants.EVENT_HANDLER)
@Lazy
public Executor getEventHandlerExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(threadPoolConfig.getHandlerCorePoolSize());
    executor.setMaxPoolSize(threadPoolConfig.getHandlerMaxPoolSize());
    executor.setQueueCapacity(threadPoolConfig.getHandlerQueueCapacity());
    executor.setWaitForTasksToCompleteOnShutdown(true);
    executor.setAllowCoreThreadTimeOut(true);
    executor.setAwaitTerminationMillis(threadPoolConfig.getAwaitTerminationMillis());
    executor.setThreadNamePrefix(ThreadPoolConstants.EVENT_HANDLER);
    executor.initialize();

    return new LazyTraceExecutor(beanFactory, executor);
}

When using this, and printing logs, the MDC key is getting changed across the EVENT_HANDLER and FORK_JOIN_POOL thread pools. I was expecting that the MDC key would remain same across pools when using LazyTraceExecutor.

Expected Behaviour - MDC keys passed, should not be reset across pools.

Version -

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-dependencies</artifactId>
  <version>2020.0.4</version>
  <type>pom</type>
  <scope>import</scope>
</dependency>

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

Using adoptOpenJDK11 for docker base image and local IDE environment.

[marcingrzejszczak]

ThreadPoolTaskExecutor.setWaitForTasksToCompleteOnShutdown(true); is not honoured when Sleuth is used VS when its not used. Is this behaviour expected ? If so, how do we tear down the executor after finishing all its submitted tasks gracefully ?

what do you mean it’s not supported? We do support it https://github.com/spring-cloud/spring-cloud-sleuth/blob/main/spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/async/LazyTraceThreadPoolTaskExecutor.java#L147

[tejaswibm]

No it wouldn’t. It’s the first time ever I hear of anyone put @Async on a controller. You can have even a simple void controller that delegates work to an @Async service.

– My understanding of Async was wrong I suppose. For the void return type, I could have just called the async service alone. I had read about this from this spring blog

Specifically -

In some cases you can return to the client immediately while a background job completes processing. For example sending an email, kicking off a database job, and others represent fire-and-forget scenarios that can be handled with Spring’s @Async support or by posting an event to a Spring Integration channel and then returning a confirmation id the client can use to query for the results.

I have removed parts of the demo app. Hope this helps demo.zip