elasticapm.label() "No transaction currently active." in celery integration. elasticapm.set_custom_context works.

Describe the bug: Using Python Celery contrib implementation, i’m trying to use elasticapm.label() API instead of elasticapm.set_custom_context() in the same point in the code (in a transaction specifically) doesn’t seem to work and return an error.

This works as intended:

import elasticapm
from elasticapm.contrib.celery import register_exception_tracking, register_instrumentation

from .celery_app import app

elasticapm.instrument()
apm_client = elasticapm.Client()
register_instrumentation(apm_client)
register_exception_tracking(apm_client)

@app.task(bind=True, base=CustomBaseTaskClass)
def MyCoolFunction(self, instance_id, payload: str) -> int:
    elasticapm.set_custom_context({
        'instance_id': instance_id,
        'queue_name': self.request.delivery_info['routing_key'],
        'job_id': self.request.id
    })

    ...

This, isn’t working:

import elasticapm
from elasticapm.contrib.celery import register_exception_tracking, register_instrumentation

from .celery_app import app

elasticapm.instrument()
apm_client = elasticapm.Client()
register_instrumentation(apm_client)
register_exception_tracking(apm_client)

@app.task(bind=True, base=CustomBaseTaskClass)
def MyCoolFunction(self, instance_id, payload: str) -> int:
    elasticapm.label(
        instance_id = instance_id,
        queue_name = self.request.delivery_info['routing_key'],
        job_id = self.request.id
    )

    ...

returning the following error:

[2021-09-16 09:40:07,162: WARNING/ForkPoolWorker-1] Ignored labels instance_id, queue_name, job_id. No transaction currently active.

As reported in the documentation APM Overview [7.14] » Data Model » Metadata it should apply at the same entities as set_custom_context API, plus on Spans.

Environment

• OS: Ubuntu 20.10
• Python version: 3.8.10
• Framework and version: Celery 5.1.2
• APM Server version: 7.12.0
• Agent version: 6.4.0

[riccardomassullo]

@zikphil thanks for the info, however this doesn’t explain the difference in behavior between set_custom_context and label API. However, the auto instrumentation for celery seems to work good, until i found this label API problem with Celery 5.1.0. Look at the following open issue, i commented how i achieved auto instrumentation for Celery Issue 685, official documentation about standalone celery auto instrumentation wasn’t really clear.

[basepi]

Excellent! Thanks for the update.