question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

python 3.6: cdef classes with cpdef methods cannot be pickled

See original GitHub issue

Hello,

In python 3.6.0, the cdef classes with cpdef methods cannot be pickled. The problem doesn’t appears with python 3.5.3.

I’m using cython 0.25.2.

You can find below a package structure to reproduce it.

Thanks a lot for your help. Best regards.

Package structure:

├── lib
│   ├── cy.pxd
│   ├── cy.pyx
│   ├── __init__.py
|── main.py

Code:

######################### lib/cy.pxd #########################

cdef class WithoutC:
    def hello(self):
        return "Hello, World"
cdef class WithCPDef:
    cpdef str hello(self):
        return "Hello, World"
cdef class WithCDefWrapper:
    def hello(self):
        return _helloC(self)
cpdef _helloC(object caller):
    return "Hello, World"

######################### lib/cy.pxd #########################

cdef class WithoutCPDef:
    pass
cdef class WithCPDef:
    cpdef str hello(self)
cdef class WithCDefWrapper:
    pass
cpdef _helloC(object caller)

######################### lib/init.py #########################

import pyximport
import numpy
pyximport.install(
    pyximport=True,
    pyimport=False,
    build_dir=None,
    build_in_temp=True,
    reload_support=False,
    load_py_module_on_import_failure=False,
    inplace=False,
    language_level=3,
    setup_args={'include_dirs': numpy.get_include()}
)

######################### main.py #########################

#!/usr/bin/env python3

import pickle as pkl
import os
from lib.cy import WithoutC, WithCPDef, WithCDefWrapper
def tryThis(obj):
    print("Pickling %s ..." % obj.__class__.__name__)
    try:
        pkl.dump(obj, open("test.pkl", "wb"))
        print("\t... OK")
    except Exception as e:
        print("\t... KO: %s" % str(e))
for t in WithoutC(), WithCPDef(), WithCDefWrapper():
    tryThis(t)
if os.path.exists("test.pkl"):
    os.remove("test.pkl")

Output with python 3.5.3:

Pickling WithoutC ...
	... OK
Pickling WithCPDef ...
	... OK
Pickling WithCDefWrapper ...
	... OK

Output with python 3.6.0:

Pickling WithoutC ...
	... OK
Pickling WithCPDef ...
	... KO: can't pickle lib.cy.WithCPDef objects
Pickling WithCDefWrapper ...
	... OK

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
scodercommented, Mar 12, 2018

Thanks for the test case. It works for me in Py3.[567], so I’ll close this ticket.

0reactions
jakirkhamcommented, May 25, 2018

@cxh123, could you please raise a new issue with an MRE and versions used (Python, Cython, and anything else relevant)?

Read more comments on GitHub >

github_iconTop Results From Across the Web

pickle cython class - python - Stack Overflow
All cdef classes that do not contain pointers or unions can automatically be pickled. For classes containing structs automatic pickling is ...
Read more >
Extension types (aka. cdef classes) - Cython's Documentation
Normal Python classes can inherit from cdef classes, but not the other way ... a cdef method, a cpdef method is fully overridable...
Read more >
Issue 26519: Cython doesn't work anymore on Python 3.6
I tried to run numpy test suite on Python 3.6 compiled in debug mode. ... TypeError: can't pickle Argument objects --- Argument class...
Read more >
Pickling Cython classes - snorfalorpagus dot net
In order to support pickling of cdef classes you must implement the ... also instruction on handling objects that can't be directly pickled....
Read more >
cython Changelog - pyup.io
Pickling unbound methods of Python classes failed. Patch by Pierre Glaser. ... Calling ``cpdef`` functions from cimported modules crashed the compiler.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found