question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Web Worker postMessage failing using numpy

See original GitHub issue

pyodide version: 0.17.0 i’m using pyodide in my react app. I use the off-the-shelf webworker provided from the example. It more or less works, however, there’s one “interesting” caveat that I thought I’d bring to the attention of the maintainers:

say I have the following python code:

import pandas as pd
pd.DataFrame([1,2,3])

If I open up a python REPL or even use the Pyodide Terminal Emulator

I get the output:

   0
0  1
1  2
2  3

However, whenever

const results = await self.pyodide.runPythonAsync(python);
self.postMessage({ results })

gets called, I get the following:

failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': [object Object] could not be cloned

the googles and the overflowing stack lead me to believe it might have something to do with this but seems like a red herring.

After banging my head on the keyboard for a while I realized you could sort of work around this by doing:

self.postMessage({results: `${results}`})

except if we change the initial example to:

import pandas as pd
pd.DataFrame([1,2,3])
print("SOMETHING")

now result is undefined. I can get the SOMETHING by redirecting stdout and returning that, but overall, this is kind of not an ideal situation.

I tried splitting the statements line-wise and iterating (ala a repl), but that doesn’t work either.

I did a fair amount of diligence on this and realized this might be solved by https://github.com/pyodide/pyodide/pull/1563, but it’d be great if someone could provide a working example of how this should work or maybe point me in the direction of what i’m doing wrong as i’m basically doing exactly what the docs said and can’t get this seemingly trivial example to work.

maybe update the docs? happy to provide more context + an example.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
hoodmanecommented, Sep 8, 2021

pickling in Python should also work I think

But only if you have a Python on the other side of the connection too.

1reaction
acreskeyMozcommented, Sep 8, 2021

Thanks @hoodmane - that was indeed the problem and using .toString() if the results are isPyProxy() is working for my test cases.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Chrome can't load web worker - Stack Overflow
This works fine in Safari, but Chrome reports the following error: Uncaught SecurityError: Failed to create a worker: script at '(path)/worker.
Read more >
Using Web Workers - Web APIs - MDN Web Docs
Web Workers are a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering ......
Read more >
Is postMessage slow? - surma.dev
In my experience, a web worker's core responsibility is, at the very least, managing your app's state object. State often only changes when...
Read more >
Web workers in action: why they're helpful, and how you ...
Communication between main thread and worker thread happens via postMessage method and onmessage event handler. Dedicated web worker. In case of ...
Read more >
Multithreading in JavaScript with Web Workers - Honeybadger.io
There are many problems associated with JavaScript's single-threaded behavior. Among them is combining long-running tasks with the UI ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found