question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

msgpack.exceptions.ExtraData: unpack(b) received extra data.

See original GitHub issue

Crashes out with msgpack.exceptions.ExtraData: unpack(b) received extra data.

  • Python 3.9.12
  • macOS 12.4
  • 2021 Macbook Pro M1 Pro
python image_from_text.py --text='alien life' --seed=7

/Users/samm/miniconda3/lib/python3.9/site-packages/jax/_src/lib/__init__.py:34: UserWarning: JAX on Mac ARM machines is experimental and minimally tested. Please see https://github.com/google/jax/issues/5501 in the event of problems.
  warnings.warn("JAX on Mac ARM machines is experimental and minimally tested. "
Namespace(mega=False, torch=False, text='alien life', seed=7, image_path='generated', image_token_count=256)
parsing metadata from ./pretrained/dalle_bart_mini
tokenizing text
['Ġalien']
['Ġlife']
text tokens [0, 8925, 742, 2]
loading flax encoder
encoding text tokens
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
loading flax decoder
sampling image tokens
image tokens [6965, 6172, 1052, 14447, 6172, 12062, 15771, 2193, 10710, 4147, 1052, 6172, 2528, 14447, 5772, 8447, 6965, 14447, 14447, 14447, 11665, 6879, 15798, 9479, 910, 15303, 5605, 7542, 1052, 14447, 14447, 2528, 6965, 1052, 14447, 6078, 3386, 2519, 12838, 16017, 867, 8447, 11993, 12426, 11196, 14447, 14447, 2528, 6965, 14447, 14447, 7491, 16147, 13512, 8269, 271, 10397, 15945, 15945, 4903, 12892, 14447, 14447, 2528, 6965, 14447, 14447, 351, 358, 10362, 6001, 8612, 14037, 7864, 14246, 5201, 2810, 14447, 14447, 2528, 6965, 14447, 14447, 10549, 15618, 11792, 13401, 16223, 1464, 12861, 6992, 572, 601, 14447, 14447, 2528, 6965, 14447, 14447, 14447, 13183, 194, 14633, 1994, 10912, 2778, 5495, 12187, 2528, 14447, 14447, 2528, 6965, 14447, 14447, 14447, 2528, 14068, 4054, 5071, 1948, 5286, 7771, 12062, 12016, 14447, 14447, 2528, 6965, 14447, 14447, 14447, 7504, 15433, 7781, 4816, 12062, 663, 3812, 8447, 8173, 14447, 14447, 2528, 6965, 14447, 14447, 6078, 13401, 6790, 2813, 10121, 4301, 4811, 5984, 3851, 8493, 14447, 14447, 2528, 6965, 14447, 14447, 4465, 12509, 4238, 12290, 10543, 8222, 11348, 13909, 5919, 6965, 14447, 14447, 2528, 11591, 14447, 6172, 11665, 9501, 2810, 9570, 7781, 910, 10549, 4395, 10639, 16147, 8173, 14164, 2528, 11591, 14164, 11993, 11610, 15891, 6242, 1936, 14602, 4903, 3583, 11574, 7516, 12892, 8173, 14447, 2528, 11591, 7467, 5243, 13157, 2810, 6790, 16017, 7236, 4301, 11725, 10689, 11941, 12659, 8173, 1052, 2528, 6965, 6598, 4465, 4816, 2895, 11820, 3132, 15917, 1811, 4904, 6933, 6690, 4811, 7504, 2528, 11605, 7467, 4815, 351, 6948, 10228, 7771, 9479, 9213, 11196, 6628, 9897, 12480, 5885, 14247, 5772, 5772]
detokenizing image
Traceback (most recent call last):
  File "/Users/samm/git/min-dalle/image_from_text.py", line 44, in <module>
    image = generate_image_from_text(
  File "/Users/samm/git/min-dalle/min_dalle/generate_image.py", line 74, in generate_image_from_text
    image = detokenize_torch(image_tokens)
  File "/Users/samm/git/min-dalle/min_dalle/min_dalle_torch.py", line 107, in detokenize_torch
    params = load_vqgan_torch_params(model_path)
  File "/Users/samm/git/min-dalle/min_dalle/load_params.py", line 11, in load_vqgan_torch_params
    params: Dict[str, numpy.ndarray] = serialization.msgpack_restore(f.read())
  File "/Users/samm/miniconda3/lib/python3.9/site-packages/flax/serialization.py", line 350, in msgpack_restore
    state_dict = msgpack.unpackb(
  File "msgpack/_unpacker.pyx", line 201, in msgpack._cmsgpack.unpackb
msgpack.exceptions.ExtraData: unpack(b) received extra data.

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:14
  • Comments:12 (1 by maintainers)

github_iconTop GitHub Comments

20reactions
BenEastoncommented, Jun 28, 2022

Looks like the file at pretrained/vqgan/flax_model.msgpack isn’t actually the right file. If you replace it with the file downloaded from here it’ll work! https://huggingface.co/dalle-mini/vqgan_imagenet_f16_16384/blob/main/flax_model.msgpack

13reactions
OrDuancommented, Jun 28, 2022

Ok I got it working by wget https://huggingface.co/dalle-mini/vqgan_imagenet_f16_16384/resolve/main/flax_model.msgpack and move that file into the ./pretrained/vqgan folder

Read more comments on GitHub >

github_iconTop Results From Across the Web

msgpack.exceptions.ExtraData: unpack(b) received extra data.
I try to learn and deploy sampel code for falcon tutorial in readthedocs. in this section when testing app by pytest tests get...
Read more >
Unpacking msgpack from respond in python - Stack Overflow
ExtraData : unpack(b) received extra data. Part of my code: r1=requests.get('http://localhost:3000/fs?path=' ...
Read more >
Source code for msgpack.exceptions
extra = extra def __str__(self): return "unpack(b) received extra data." # Deprecated. Use Exception instead to catch all exception during packing.
Read more >
msgpack Documentation - Read the Docs
ExtraData is raised when there is trailing data. This exception is raised while only one-shot (not streaming) unpack. exception msgpack.
Read more >
msgpack.exceptions.ExtraData: unpack(b) received extra data.
Now i got the exepction: msgpack.exceptions.ExtraData: unpack(b) received extra data. The Spider writes now messages in frontier-done.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found