PyPy compatibility

This was raised on twitter, and may be important for some users. Currently msgspec is written as a c extension using some private cpython apis, making it incompatible with pypy. It would be good to have a pypy compatible build, whether through changes to the c extension or through a separate pure python release. Either seems fine to me, provided the result doesn’t decrease cpython performance noticeably.

[jcrist]

This is not something I plan on supporting in a volunteer-only basis (if someone wanted to pay for this, I might be willing to expand support).

A few reasons:

• msgspec is heavily tied to the CPython c-api for performance. PyPy does support a subset of the CPython c-api, but it does so in a way that is not very performant, and isn’t 100% compatible. We could add shims everywhere that’s needed to make msgspec PyPy compatible, but that would muddy up the code, increasing our maintenance burden.
• Due to how PyPy’s c-api layer works, performance on PyPy would be worse than on CPython. Whether this matters is application dependent.
• CPython is getting faster, reducing the advantages of PyPy
• Other “fast” libraries (e.g. asyncpg, orjson, …) also don’t support PyPy, and people generally seem fine with that.

[Goldziher]

This is not something I plan on supporting in a volunteer-only basis (if someone wanted to pay for this, I might be willing to expand support).

A few reasons:

• msgspec is heavily tied to the CPython c-api for performance. PyPy does support a subset of the CPython c-api, but it does so in a way that is not very performant, and isn’t 100% compatible. We could add shims everywhere that’s needed to make msgspec PyPy compatible, but that would muddy up the code, increasing our maintenance burden.
• Due to how PyPy’s c-api layer works, performance on PyPy would be worse than on CPython. Whether this matters is application dependent.
• CPython is getting faster, reducing the advantages of PyPy
• Other “fast” libraries (e.g. asyncpg, orjson, …) also don’t support PyPy, and people generally seem fine with that.

Thanks for clarifying this. I think we are fine not having pypy support.