Subscriptions are processing after Spring graceful shutdown

Describe the bug Not sure if this is some problem in how we use pubsub, or an actual bug. Let me know if this does not belong here and I can close it. But we have a Subscriber wrapper class that contains a bunch of subscribers in a collection. And when spring shuts down we do the following to shutdown and await each one:

@PreDestroy
override fun stop() {
    subscribers.onEach { (subscriptionName, subscriber) ->
        logger.info("Stopping {}", kv("subscription", subscriptionName))
        subscriber.stopAsync()
    }
    subscribers.onEach { (subscriptionName, subscriber) ->
        logger.info("Waiting for {} to terminate", kv("subscription", subscriptionName))
        subscriber.awaitTerminated(5, TimeUnit.SECONDS)
    }
}

The problem is we get exceptions like these: class java.lang.reflect.InvocationTargetException null, Cause: class org.springframework.transaction.CannotCreateTransactionExceptionCould not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: EntityManagerFactory is closed in our subscribers. Not all, but some.

Spring is configured like this in terms of shutdown:

server.shutdown=graceful
spring.lifecycle.timeout-per-shutdown-phase=30s

Our hunch is that maybe the shutdown times out and EntityManagerFactory is closed before the subscriptions are shutdown properly.

Does our shutdown logic look sane, or should we handle this differently? Is it possible that PubSub executors are still running after the timeout given our setup?

Very thankful for any input or guidance on this topic.

[eriklagerholm]

For future reference we opted to use Spring’s SmartLifecycle API in order to achieve what we want. It was able to execute the code before the EntityManager was closed.

[meltsufin]

Closing due to inactivity. Please re-open if you are still experiencing the issue.