question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

HLG serialization bug

See original GitHub issue

What happened: Serialization fails with TypeError: can not serialize 'numpy.uint16' object when using master.

What you expected to happen: Serialization should succeed.

Minimal Complete Verifiable Example:

import dask.array as da
import numpy as np
from distributed.protocol.highlevelgraph import highlevelgraph_pack

def fn(x, dt):
    return x.astype(dt)

arr = da.blockwise(fn, "x", da.ones(1000), "x", np.uint16(0), None, dtype=np.uint16)

highlevelgraph_pack(arr.__dask_graph__(), None, None)

Anything else we need to know?: This happens for a number of numpy data types.

Environment:

  • Dask version: 2021.02.0+3.ga14e47f
  • Distributed version: 2021.02.0+1.g7d2a22f
  • Python version: 3.7.6
  • Operating System: PoP!_OS 18.04 LTS
  • Install method (conda, pip, source): pip

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:31 (17 by maintainers)

github_iconTop GitHub Comments

4reactions
JSKenyoncommented, Mar 15, 2021

This seems to have resolved the problem! Thanks @madsbk! I will be doing further testing over the coming week and will let you know if I encounter any further issues.

1reaction
jrbourbeaucommented, Mar 15, 2021

Awesome, thanks @JSKenyon

Read more comments on GitHub >

github_iconTop Results From Across the Web

Broadcaster Serialization - GuitarsByLeo.com
has a bug in it, so that the serial numbers are not listed in proper sort sequence. :BadPC: ] Hope this helps.
Read more >
Changelog — Dask.distributed 2022.12.1 documentation
Handle empty memoryviews of bytearrays when (de)serializing (GH#6576) Benjamin Zaitlen. Ensure steal requests from same-IP but distinct workers are rejected ...
Read more >
Deserialization Bugs in the Wild - Medium
Insecure deserialization is a type of vulnerability that arises when an attacker is able to manipulate the serialized object and cause ...
Read more >
High Dynamic Range and Wide Gamut Color on the Web
On the web, SDR, and HDR content using both HLG and PQ is expected to ... That migration is in progress, with CSS...
Read more >
GStreamer 1.18 release notes - Freedesktop.org
The latest bug-fix release in the 1.18 series is 1.18.6 and was released on 2 ... if any and map bt2020-10, PQ and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found