question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

chainerx.array is not picklable

See original GitHub issue

It is often convenient if we can pickle chainerx arrays when we save/restore/communicate them.

>>> import pickle
>>> import chainerx as chx
>>> a = chx.array(10)
>>> a
array(10, shape=(), dtype=int64, device='native:0')
>>> pickle.dumps
<built-in function dumps>
>>> pickle.dumps(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: return_value_policy = copy, but the object is non-copyable!

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
niboshicommented, Aug 20, 2019

Introducing a Python wrapper class (#7983) does seem to fix the problem.

arr = chainerx.array([1, 2], chainerx.float32, device='cuda:0')
s = pickle.dumps(arr)

Now this variant works with the patch.

0reactions
emcastillocommented, Aug 20, 2019

The problem seems to be this py::return_value_policy::reference, probably is setting something in the python side of the wrapper the first time the corresponding python object is created.

A simple solution to this problem is to return the name of the device when serializing, and reconstruct the device from the name upon deserialization. I quickly tested this and worked. But if we can find a way to make the C++ to be directly pickable it will be better.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Object array is not picklable - python - Stack Overflow
Verifyed, pickle can handle object array given in Question. The following works without failure. pick_array = pickle.dumps(array) ...
Read more >
ChainerX Tutorial — Chainer 7.8.1 documentation
Graph construction and backpropagation is built into the array, meaning that any function, including the NumPy-like functions, can be backpropagated through. In ...
Read more >
Chainer v7.0.0a1 をリリースしました - Google Groups
Implement array vs array functionality to chainerx.minimum (#6541, thanks @aksub99!) ... Make InvalidType picklable (#6884, thanks @zaltoprofen!)
Read more >
Python Examples of numpy - ProgramCreek.com
PZERO assert chainerx.e is numpy.e assert chainerx.euler_gamma is ... print 'cannot apply bellman_ford, graph is not oriented' return dist = np.array([np.
Read more >
Qiitaのタグ一覧(アルファベット順)
... "ARR",E "array",C "array_multisort",E "ArrayList",D "arraymancer" ... "chainerRL",D "ChainerX",E "ChainOfResponsibility",E "ChakraCore" ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found