Disarm fails when instrumented code fails to compile

If you have a contract that compiles before being instrumented but fails to compile after being instrumented, scribble will successfully arm the contract but fail to disarm it. This can leave a bit of a mess of armed files to clean up before you can fix or diagnose the issue.

Example

Shadowed.sol

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

interface IShadower {
    function shadowed(uint256 a) external;
    function shadowed(uint256 a, uint256 b) external;
}

/// #invariant {:msg "I0"} 0 == 0;
contract Scribbled {
    function entry() external {
        IShadower(address(0)).shadowed(1);
    }
}

solc Shadowed.sol

Succeeds.

scribble Shadowed.sol --output-mode files --arm

Succeeds, but leaves an instrumented Shadowed.sol that fails to compile.

In this case, it is because the function pointer passed to the _callsite_ method cannot discern between the two versions of IShadower.shadowed(). As far as I know, this is a limitation of solidity rather than a scribble bug, but used here as an example.

scribble Shadowed.sol --output-mode files --disarm

Because the newly instrumented Shadowed.sol won’t compile, this fails and leaves armed files that must be sorted out manually or reverted to a previous commit.

Feature Request

• If it’s not necessary to compile the armed contracts before disarming, that seems like an easy fix. Maybe this could be behind a --disarm --force flag.
• Attempt to compile the armed contracts and revert instrumenting if they fail to compile.

[blitz-1306]

@wolflo

What is the reason for compiling the armed contracts before disarming?

I would say major reason is (and always been a big pain) an import resolution. We can resolve imports by relying on Solc callbacks while compiling files (unless breadcrumbs idea, suggested by @cd1m0). There are a lot of effort done to make that work:

• https://github.com/ConsenSys/solc-typed-ast/blob/daa2922c9fac23b97987c862adc424316f75a3c1/src/compile/utils.ts#L218-L249
• https://github.com/ConsenSys/solc-typed-ast/blob/daa2922c9fac23b97987c862adc424316f75a3c1/src/compile/utils.ts#L356-L371
• https://github.com/ConsenSys/solc-typed-ast/blob/daa2922c9fac23b97987c862adc424316f75a3c1/src/compile/utils.ts#L309-L310

[JoranHonig]

@cd1m0 maybe we can include a series of comments on top of instrumented files indicating that they’re auto generated and that they shouldn’t be edited directly (only after it has been disarmed).