[C++ API] use of `std::function` ABI may not be stable

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

These are the function signatures that acts as the FFI between the application code and ray runtime (C++ worker runtime)

using RemoteFunction = std::function<msgpack::sbuffer(const ArgsBufferList &)>;
using RemoteMemberFunction =
    std::function<msgpack::sbuffer(msgpack::sbuffer *, const ArgsBufferList &)>;

However, it is known that std::function does not have a stable ABI, e.g. https://stackoverflow.com/questions/60291845/undocumented-abi-changes-of-stdfunction-between-gcc-4-and-gcc-5-6-7-8-9-how-t The same could be true of msgpack::sbuffer(const ArgsBufferList &)…

Use case

Not breaking unexpectedly

Recommendation

Convert function type signatures to be C-ABI-compatible, which is more stable and the typical standard for FFI.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[mwtian]

@jon-chuang If this is about a Ray-internal C API which wraps core worker C++ API, that other language workers (Rust, even C++) can build upon, it would be more feasible. The benefit of the C API (Core worker C++ API -> C API -> language workers) is to provide compatibility with multiple languages’ FFI facilities, instead of ABI stability (there is no intention to offer core worker API stability yet). @iycheng has investigated this before too.

The value of C API on top of C++ API seems less obvious. We can be more productive by having a minimum example (start Ray, run a task, get the result etc) of how the proposed set of C APIs would look like, and how users should interact with them. Then issues about API ergonomics, efficiency and ABI stability would be clearer.

Hence, I want to see if we can get consensus on whether in the long-term, we want to move to a C-ABI.

Where are we moving from, the existing C++ API?

[jon-chuang]

Hmm, I agree its not an urgent priority. However:

but it imposes restrictions on the C++ API, e.g. no tempated args like std::vector<>.

Yes, but its easy to fix with serialization, which in this case is quite lightweight. Or simply raw pointers.

Further, this is just an internal FFI implementation, and has no restriction on the user whatsoever, who can use all C++ features and STL in their remote function signatures.

but it seems easier to document and less likely to go out of date compared to an ABI stability guarantee.

I agree the FFI might change, but I think C ABI is a good starting point to build future development upon.

I think the FFI breakage with C ABI is very explicit - as it requires actual code changes, not just build system change, as could happen with C++ ABI.

Hence, I don’t think it is harder to document or will go out of date, as the extern "C" FFI signature is 1. internal, and does not require documentation 2. tied to a particular release. Both changing the extern "C" FFI or C++ FFI would both equally warrant warning users for the breaking-changes in the release. (and make sure to put a comment to do so next to the FFI definition). On the contrary, if one wants to tie C++ API to a specific compiler etc., I believe that requires much more documentation.

Perhaps since the C++ FFI is also currently unstable, it might be the best time to make some changes, so that one doesn’t have to change the FFI later on when users expect more stability.

---

The nice thing about extern "C" FFI is that one does not ever need to worry about recompiling application .so files even if user upgrades ray installation (barring FFI changes).

---

Hence, I want to see if we can get consensus on whether in the long-term, we want to move to a C-ABI.