question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

AzureExporter not working with multiprocessing

See original GitHub issue

Describe your environment. MacOS 10.14.6 Python 3.7.5 opencensus-ext-azure==1.0.4 opencensus-ext-requests==0.7.3

Steps to reproduce. I have code that I want to monitor the dependency calls to Azure DevOps APIs. Our code is running multiprocessing using the Process class. When the exporter is ran outside of multiprocessing, it sends telemetry to App Insights. When ran inside a multiprocessing Process, it doesn’t. I added a callback to print the spandata and it doesn’t get called when using Process.

from azure.devops.connection import Connection
from msrest.authentication import BasicAuthentication

from multiprocessing import Process, Pool, Queue
from base_insights import BaseInsights
from opencensus.common.runtime_context import RuntimeContext


class TestInsights:
    def __init__(self):
        self.tracer = BaseInsights.tracer

    def process(self):
       
        procs = []
        organization_url = 'https://dev.azure.com/org'
        credentials = BasicAuthentication('', '')
        
        p1 = Process(target=self.my_loop, args=[organization_url, credentials])
        p1.start()
        p1.join()

    def my_loop(self, organization_url, credentials, parent_span=None):
      
        with self.tracer.span(name='TestLoopProcessThreadingInside'):

            connection = Connection(base_url=organization_url, creds=credentials)
            core_client = connection.clients.get_core_client()

            org = core_client.get_project_collection("test")


TestInsights().process()

BaseInsights:

import os

from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace import file_exporter

from opencensus.trace import config_integration
from opencensus.trace.samplers import ProbabilitySampler, AlwaysOnSampler
from opencensus.trace.tracer import Tracer

config_integration.trace_integrations(['requests'])


def singleton(cls):
    return cls()


@singleton
class BaseInsights:
    def __init__(self):
        exporter = AzureExporter()
        exporter.add_telemetry_processor(self.callback_function)

        self.tracer = Tracer(exporter=exporter, sampler=AlwaysOnSampler())


    def callback_function(self, envelope):
        print(envelope)

What is the expected behavior? Span data gets sent to Application Insights

What is the actual behavior? Span data is not sent to Application Insights

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:2
  • Comments:19 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
dpgrodriguezcommented, Aug 5, 2020

I tried running this code on 4 different environments.

CentOS 7.8 - Python 2.7.5 CentOS 7.8 - Python 3.6.8 macOS 10.14.6 - Python 2.7.16 macOS 10.14.6 - Python 3.7.5

I got the same results. Only INPROC: TestOutside is being logged in Application Insights.

Screen Shot 2020-08-05 at 10 43 46 AM

1reaction
giulianabarattocommented, Dec 13, 2022

Any news on this? 👀 We’re having the same problem with Celery and would really appreciate to have an official solution

Read more comments on GitHub >

github_iconTop Results From Across the Web

pypi opencensus-ext-azure not functioning properly (sends ...
Running as a normal Python module should not cause this behaviour since state is not kept within subsequent runs. If you are not...
Read more >
Error using python multiprocessing in Azure Machine Learning
I want to do some parallel data manipulation in Azure ML but I can't seem to get multiprocessing to work. So when I...
Read more >
open-telemetry/opentelemetry-python - Gitter
Hey, just a heads up, seems like master is failing with lint, because of isort issues. For this to happen to me I...
Read more >
opencensus 0.11.0 - PythonFix.com
Stars: 631, Watchers: 631, Forks: 234, Open Issues: 202 ... AzureExporter not working with multiprocessing; AzureLogHandler does not seem to ...
Read more >
Is There A Workaround To Make Opencensus Work With Mlflow?
I have problem when deleting an entity using deleteById. ... We have a simple python AzureExporter not working with multiprocessing #928.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found