scale/federate the rendezvous server?

Looking towards the future: magic-wormhole uses a single central rendezvous server to allow clients to find each other based upon just a small integer. If/when it becomes more popular, the traffic on this server might be an issue. How could we use multiple server hosts to avoid this bottleneck?

(note that this is independent of the “transit relay” server, which is morally equivalent to a TURN server and helps file transfers to work when both clients are behind NAT boxes, by gluing two TCP streams together)

There are two components to the rendezvous server. The first is channel allocation: the most common workflow is for wormhole send to allocate a channel by asking the server to assign one, and this needs to be unique (so the server needs to know a complete list of all channel IDs currently in use). These are called “nameplates” in the protocol and the code.

The second is “mailbox” management: a semi-persistent set of messages, and a publish-subscribe protocol for both sides to add and retrieve those messages. Each nameplate maps to a mailbox, each mailbox has a queue/set, and each client connects to a mailbox.

I think the channel-allocation part requires some sort of singular central table. Even if we get multiple servers involved, I think they’ll need to talk to a single memcached or something.

The mailboxes could be sharded out among multiple machines, as long as there’s a way for every client to get connected to the right machine. For example, we could use DNS round-robin or something to point to a fleet of servers, all of which can use that memcached instance to find out where a given mailbox lives. If a client connects to the wrong server, that server looks up the mailbox in the central table, learns about the correct server, then redirects the client to them. I think that means one non-sharded message per lookup, but everything else scales with the number of servers we use.

[anarcat]

i could have sworn i mentionned this before, but couldn’t this be implemented through the DHT?

there are a few concerns with this:

1. the DHT may not be able to assert unicity of nameplates. could other mechanisms be found for this, for example using UUIDs or something similar?
2. mailboxes may be much more complex to implement and slower

Are you familiar with the DHT?

[piegamesde]

Some really naive idea: there are ten rendezvous servers, relay0.magic-wormhole.io to relay9.magic-wormhole.io (can we please all agree on that “relay” is a misnomer for the rendezvous server?). Clients decide based on the last digit of their nameplate which server to take, and the each relay server only gives out nameplates ending with the corresponding digit. The old relay.magic-wormhole.io does some forwarding to the other ones for backwards compatibility.