V2 Architecture proposal (continuation of #111)

Since @area and @onbjerg’s discussion of an overhaul in #111 I’ve done some work on a gas analytics mocha plugin for Truffle and believe a similar pattern for solidity-coverage might resolve some of the technical challenges noted there.

Third-party mocha reporters provide hooks for every part of the test cycle. Operating within one would allow us to escape the issues presented by Truffle’s chain reversion. Reporters also run inside Truffle’s execution context so the web3 and artifacts objects are globally exposed there.

In this model SC might look as follows:

Preparation

• Using abi-decoder, create a map of function signatures to file names / paths. (Example here)
• Using the Context object developed for the truffle debugger, create a program-counter to line-column position map for each file.
• Using logic @area wrote as the core of the current implementation, generate an AST based map of ‘points of interest’ to line-column position data for each file.

Runtime: For each test, block, transaction:

• identify the file being run by extracting the function signature from the tx’s input field and locating it in the table we’ve constructed with abi-decoder.
• call debug_traceTransaction and collect the opcode/program counter data for the tx.
• discover and record the meaning of the executed instructions via the file’s Context and AST maps.

Some issues to consider:

• assert and require will need special processing. We can easily identify regular conditional branches because they occupy different physical space in the text files. But branching for these built-ins will have to be detected in another way.
• view and pure: We could pre-process the files by stripping these modifiers off and running all methods as transactions. A little hacky and means that explicit uses of the .call postfix will not get covered. It might be nice to press truffle and testrpc to implement tracing for .call since it would be useful for their debugger anyway. Not sure how to actually do this though.
• We will need to throw away our test suite and find a new way of validating the tool.
• We will be a Truffle plugin and look like this:

module.exports = {
  networks: {
    ...etc...
  },
  mocha: {
    reporter: 'solidity-coverage'
  }
};

Overall benefits:

• No forks or special deps. Even parsing. We should use the solc AST provided by the Truffle artifact because it guarantees the column position data is identical, among other things.
• Resolution of most bugs.
• Long term stability, close integration with solc etc.

Please rip this apart. Still blue-skies as far as I am concerned.

cc: #146

[area]

Tests can generate contracts outside of those deployment steps though, which wouldn’t be captured. I think we’d have to look at the code at the address and figure out which contract it related to.

[cgewecke]

Tangentally: Remix has some great code for handling the solc ast / doing source-map stuff. Eminently steal-able.