Create EVM Process for Trinity

What is wrong?

When Trinity is running a chain it will likely have the following processes.

• DevP2P
  • Handles networking
  • Makes requests to the EVM process to read existing chain state.
  • Submits new data for import to the EVM process.
• JSON-RPC
  • Serves requests to the JSON-RPC server over the IPC pipe
  • Makes requests to the EVM process to read existing chain state.
  • Submits new transactions to the DevP2P process for broadcast to the network.
  • Submits requests for EVM execution for things like eth_call and eth_estimateGas.
• EVM
  • Serves requests from the JSON-RPC process for thing like eth_call and eth_estimateGas.
  • Serves requests from the DevP2P process for reading chain data.
  • Handle new chain data submitted by the DevP2P process.

There may be a fourth process for the transaction pool as well as others we haven’t thought of yet.

How can it be fixed

Each of these process will be it’s own multiprocessing.Process instance. We have limited communication options between these processes using the primatives from the multiprocessing module (Pipe/Queue/..?..).

Exactly how this all looks still needs to be hashed out, after which we can fill this section in further.

[hwwhww]

I draw a brief diagram: the three main processes are illustrated at the top of the diagram and some possible sharding components/modules I suppose we need are at the bottom.

• Shard Manager
  • Upon receiving new collations, checks VMC status and applies fork choice rules.
  • Makes requests to full nodes via web3.py - could be local-mode or remote-mode
    • local-mode: connects directly to Chain of its local main chain data
    • remote-mode: connects to other main chain node (geth, parity, etc.)
  • Caches the lately blockhashes and CollationAdded logs
  • [As a shard validator] Upon receiving “sampled-as-collator” notification from VMC Listener, applies CREATE_COLLATION algorithm.
    • Performs stateless sync via multiple interactions with DevP2P process and web3.py.
    • Collects transactions into a new collation via DevP2P.
    • Sends add_header tx to VMC via web3.py.
  • [TBD] I’m not sure about should Shard Manager be a standalone process or it could be divided into DevP2P process and EVM process. It will be clear after we figure out the interactions between DevP2P process and EVM process.
• VMC Listener
  • Polls the VMC status and notifies Shard Manager.

srd is just a possible ID for Sharding Wire Protocol. (It can’t be too similar to shh.)

Please don’t hesitate to point out any mistake.

[pipermerriam]

@gsalgado it’s probably worth looking into this as a minimal initial solution as well as a baseline for benchmarking whatever we think a better solution may be.

I do know that spinning up multiprocess.Process instances has a high-ish overhead which is one reason to prefer a long running process, however it doesn’t mean a short lived process is not a viable initial step.