AzureExporter not working with multiprocessing
See original GitHub issueDescribe your environment. MacOS 10.14.6 Python 3.7.5 opencensus-ext-azure==1.0.4 opencensus-ext-requests==0.7.3
Steps to reproduce.
I have code that I want to monitor the dependency calls to Azure DevOps APIs. Our code is running multiprocessing using the Process
class. When the exporter is ran outside of multiprocessing, it sends telemetry to App Insights. When ran inside a multiprocessing Process, it doesn’t. I added a callback to print the spandata and it doesn’t get called when using Process
.
from azure.devops.connection import Connection
from msrest.authentication import BasicAuthentication
from multiprocessing import Process, Pool, Queue
from base_insights import BaseInsights
from opencensus.common.runtime_context import RuntimeContext
class TestInsights:
def __init__(self):
self.tracer = BaseInsights.tracer
def process(self):
procs = []
organization_url = 'https://dev.azure.com/org'
credentials = BasicAuthentication('', '')
p1 = Process(target=self.my_loop, args=[organization_url, credentials])
p1.start()
p1.join()
def my_loop(self, organization_url, credentials, parent_span=None):
with self.tracer.span(name='TestLoopProcessThreadingInside'):
connection = Connection(base_url=organization_url, creds=credentials)
core_client = connection.clients.get_core_client()
org = core_client.get_project_collection("test")
TestInsights().process()
BaseInsights:
import os
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace import file_exporter
from opencensus.trace import config_integration
from opencensus.trace.samplers import ProbabilitySampler, AlwaysOnSampler
from opencensus.trace.tracer import Tracer
config_integration.trace_integrations(['requests'])
def singleton(cls):
return cls()
@singleton
class BaseInsights:
def __init__(self):
exporter = AzureExporter()
exporter.add_telemetry_processor(self.callback_function)
self.tracer = Tracer(exporter=exporter, sampler=AlwaysOnSampler())
def callback_function(self, envelope):
print(envelope)
What is the expected behavior? Span data gets sent to Application Insights
What is the actual behavior? Span data is not sent to Application Insights
Issue Analytics
- State:
- Created 3 years ago
- Reactions:2
- Comments:19 (7 by maintainers)
Top Results From Across the Web
pypi opencensus-ext-azure not functioning properly (sends ...
Running as a normal Python module should not cause this behaviour since state is not kept within subsequent runs. If you are not...
Read more >Error using python multiprocessing in Azure Machine Learning
I want to do some parallel data manipulation in Azure ML but I can't seem to get multiprocessing to work. So when I...
Read more >open-telemetry/opentelemetry-python - Gitter
Hey, just a heads up, seems like master is failing with lint, because of isort issues. For this to happen to me I...
Read more >opencensus 0.11.0 - PythonFix.com
Stars: 631, Watchers: 631, Forks: 234, Open Issues: 202 ... AzureExporter not working with multiprocessing; AzureLogHandler does not seem to ...
Read more >Is There A Workaround To Make Opencensus Work With Mlflow?
I have problem when deleting an entity using deleteById. ... We have a simple python AzureExporter not working with multiprocessing #928.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I tried running this code on 4 different environments.
CentOS 7.8 - Python 2.7.5 CentOS 7.8 - Python 3.6.8 macOS 10.14.6 - Python 2.7.16 macOS 10.14.6 - Python 3.7.5
I got the same results. Only
INPROC: TestOutside
is being logged in Application Insights.Any news on this? 👀 We’re having the same problem with Celery and would really appreciate to have an official solution