Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The agent is unable to instrument ManagedExecutorService and crashes the thread

See original GitHub issue

The APM agent is unable to instrument the ManagedExecutorService. Whenever a Callable is submitted invocation crashes with:

SEVERE [org.glassfish.enterprise.concurrent] (EE-ManagedExecutorService-default-Thread-1) java.lang.ClassCastException: co.elastic.apm.agent.impl.async.ContextInScopeRunnableWrapper cannot be cast to org.glassfish.enterprise.concurrent.internal.ManagedFutureTask

Environment OpenJDK 1.8.0_191 (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12) Wildfly 9.0.2.Final APM agent 1.5.0, 1.6.0-SNAPSHOT at fb3c7cd5f15ee19f2c6d4ed4d5a5a2378e83412e

Default Wildfly EE configuration:

Click to expand

<subsystem xmlns="urn:jboss:domain:ee:3.0">
    <spec-descriptor-property-replacement>false</spec-descriptor-property-replacement>
    <concurrent>
        <context-services>
            <context-service name="default" jndi-name="java:jboss/ee/concurrency/context/default" use-transaction-setup-provider="true"/>
        </context-services>
        <managed-thread-factories>
            <managed-thread-factory name="default" jndi-name="java:jboss/ee/concurrency/factory/default" context-service="default"/>
        </managed-thread-factories>
        <managed-executor-services>
            <managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" core-threads="5" max-threads="25" keepalive-time="5000"/>
        </managed-executor-services>
        <managed-scheduled-executor-services>
            <managed-scheduled-executor-service name="default" jndi-name="java:jboss/ee/concurrency/scheduler/default" context-service="default" hung-task-threshold="60000" core-threads="2" keepalive-time="3000"/>
        </managed-scheduled-executor-services>
    </concurrent>
    <default-bindings context-service="java:jboss/ee/concurrency/context/default" datasource="java:jboss/datasources/ExampleDS" jms-connection-factory="java:/activemq/ConnectionFactory" managed-executor-service="java:jboss/ee/concurrency/executor/default" managed-scheduled-executor-service="java:jboss/ee/concurrency/scheduler/default" managed-thread-factory="java:jboss/ee/concurrency/factory/default"/>
</subsystem>

Nothing to see in debug logs, except for loads of these:

ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - co.elastic.apm.api.CaptureTransaction was loaded by org.jboss.modules.ModuleClassLoader
ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - co.elastic.apm.api.CaptureSpan was loaded by org.jboss.modules.ModuleClassLoader
ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - javax.ws.rs.Path was loaded by org.jboss.modules.ModuleClassLoader
ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - javax.jws.WebService was loaded by org.jboss.modules.ModuleClassLoader
ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - javax.faces.lifecycle.Lifecycle was loaded by org.jboss.modules.ModuleClassLoader
ERROR [stderr] (ServerService Thread Pool -- 73) [ServerService Thread Pool -- 73] DEBUG co.elastic.apm.agent.bci.bytebuddy.CustomElementMatchers - javax.faces.lifecycle.Lifecycle was loaded by org.jboss.modules.ModuleClassLoader

Issue Analytics

State:
Created 4 years ago
Comments:9 (7 by maintainers)

Top GitHub Comments

1reaction

felixbarnycommented, Apr 18, 2019

Oh, right. It’s not enough to exclude ManagedThreadPoolExecutor from instrumentation. This class is actually already not instrumented as we only instrument the execute and submit methods. These methods are inherited from the superclass java.util.concurrent.ThreadPoolExecutor.

A temporary workaround is to disable context propagation for thread pools altogether by setting disable_instrumentations=incubating,executor.

I’ll think about a proper solution.

0reactions

eyalkorencommented, Jul 8, 2021

@btsinfo this have been fixed in #1864 but not released yet. Until it is released, you can use our latest published snapshot.

Top Results From Across the Web

ManagedExecutorService (Java(TM) EE 7 Specification APIs)

Tasks are run in managed threads provided by the Java™ EE Product Provider and are run within the application component context that submitted...

ASM - can't instrument java/lang/Thread - Stack Overflow

Thread ")); Ran into JVM crashes, so modified the code to just instrument the Thread class and init (not ) method.

IntelliJ IDEA 2019.2 EAP (192.4205.1 build) Release Notes

Feature IDEA‑210929 Provide ability to copy a TBX link to a settings Feature IDEA‑210961 Implement action for Slow Startup Profiling Feature IDEA‑111333 Provide Sublime Text...

Jersey 2.37 User Guide - GitHub Pages

This is user guide for Jersey 2.37. We are trying to keep it up to date as we add new features. When reading...

2013 - Java Agent Hangs, Crashes, OOM, High CPU, Agent conf

CA Tuesday Tip: 2013 - Java Agent Hangs, Crashes, OOM, High CPU, ... Generate a series of 5 thread dumps on the application...