question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

OpenTelemetry gRPC exporter fails because SpanKind is None

See original GitHub issue
  • Package Name: azure-core-tracing-opentelemetry
  • Package Version: 1.0.0b9
  • Operating System: Windows Server 2012 R2
  • Python Version: 3.8.10

Describe the bug When trying to use opentelemetry-exporter-otlp-proto-grpc to export OpenTelemetry traces created by Azure Python SDK, some traces fail to get exported because the generated OpenTelemetry Spans are not created with a valid SpanKind (is set to None)

Exception while exporting Span batch.
Traceback (most recent call last):
  File "F:\Repo vm-deploy\venv\lib\site-packages\opentelemetry\sdk\trace\export\__init__.py", line 358, in _export_batch
    self.span_exporter.export(self.spans_list[:idx])  # type: ignore
  File "F:\Repo vm-deploy\venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\trace_exporter\__init__.py", line 291, in export
    return self._export(spans)
  File "F:\Repo vm-deploy\venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\exporter.py", line 293, in _export
    request=self._translate_data(data),
  File "F:\Repo vm-deploy\venv\lib\site-packages\opentelemetry\exporter\otlp\proto\grpc\trace_exporter\__init__.py", line 277, in _translate_data
    f"SPAN_KIND_{sdk_span.kind.name}",
AttributeError: 'NoneType' object has no attribute 'name'

The spans can be exported to ConsoleSpanExporter and they look like this:

{
    "name": "VirtualMachinesOperations.get",
    "context": {
        "trace_id": "0xc2b6ac1e000327beb3a5d257d1baaaa3",
        "span_id": "0x2517d83d04756b74",
        "trace_state": "[]"
    },
    "kind": "None",
    "parent_id": null,
    "start_time": "2022-04-26T11:18:07.412171Z",
    "end_time": "2022-04-26T11:18:07.809196Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {},
    "events": [],
    "links": [],
    "resource": {
        "service.name": "vm-deploy",
        "service.version": "1.0.0",
        "deployment.environment": "dev"
    }
}

The issue can be traced back to this line:

https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/azure/core/tracing/decorator.py#L81

Where no kind is specified, and when the OpenTelemetrySpan is instantiated, None is assigned to the kind in this line:

https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core-tracing-opentelemetry/azure/core/tracing/ext/opentelemetry_span/__init__.py#L67

And then this check passes because value exists (although is assigned to None)

https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core-tracing-opentelemetry/azure/core/tracing/ext/opentelemetry_span/__init__.py#L69

I think that when the OpenTelemetrySpan is instantiated, SpanKind should never default to None, since it is not a SpanKind supported by OpenTelemetry API: https://opentelemetry.io/docs/reference/specification/trace/api/#spankind

To Reproduce I will try to create a minimal example of the error and link it to the issue as soon as possible.

Expected behavior Spans should be exported to gRPC exporter with no errors

Screenshots N/A

Additional context OpenTelemetry documentation states in this link that SpanKind should default to SpanKind.Internal.

If I change this line

https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core-tracing-opentelemetry/azure/core/tracing/ext/opentelemetry_span/__init__.py#L67

to OpenTelemetrySpanKind.INTERNAL

the exporter works as expected.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:13 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
lmazuelcommented, May 26, 2022

PR has been merged, will be released next week (probably Wednesday)

1reaction
cpimentcommented, May 21, 2022

Hi @lmazuel, sorry for the delay. I have just tested it and it works flawlessly 😄 Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

open-telemetry/opentelemetry-java - Gitter
Begin running and processing data , but data never arrives from the OTLP exporter. i added some error logging to the OTLP exporter...
Read more >
Send a custom span event - New Relic Developers
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import ... SpanKind. ... Your code will no longer save an exception span event on your spans.
Read more >
Getting Started | OpenTelemetry
/tmp/otel-collector-config.yaml receivers: otlp: protocols: grpc: ... By default, opentelemetry-instrument exports traces and metrics over ...
Read more >
python - error code: StatusCode.UNIMPLEMENTED - Jaeger
UNIMPLEMENTED - Jaeger - Opentelemetry ... and setup that I'm using, currently failing with Failed to export traces, error code: StatusCode.
Read more >
Source code for opentelemetry.instrumentation.requests
... import http_status_to_status_code from opentelemetry.trace import SpanKind, ... ERROR, ) result = getattr(exc, "response", None) finally: ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found