Error decoding bytes parameter for logs emitted by external functions (solc 0.4.x)

(Cross-reporting from web3 3544. Apologies in advance if this has already been reported here.)

Using the v5 ABI package, this error

Error: data out-of-bounds (length=36, offset=64, code=BUFFER_OVERRUN, version=abi/5.0.0-beta.153)
      at Logger.makeError (node_modules/@ethersproject/logger/lib/index.js:178:21)
      at Logger.throwError (node_modules/@ethersproject/logger/lib/index.js:187:20)
      ...

is thrown when decoding events with bytes params when

• contracts are compiled with solc 0.4.x
• emitting methods have external visibility

pragma solidity ^0.4.24;

contract A {

    event lockCallback(uint256 amount, uint256 allowance, bytes data);

    // Fine
    function public_fireEvent(bytes sig) public {
      emit lockCallback(5,5, sig);
    }
    // Errors
    function external_fireEvent(bytes sig) external {
        emit lockCallback(5,5, sig);
    }
}

Example data for contract above

public_fireEvent

0x
0000000000000000000000000000000000000000000000000000000000000005
0000000000000000000000000000000000000000000000000000000000000005
0000000000000000000000000000000000000000000000000000000000000060
0000000000000000000000000000000000000000000000000000000000000004
abe985cb00000000000000000000000000000000000000000000000000000000

external_fireEvent

0x
0000000000000000000000000000000000000000000000000000000000000028
0000000000000000000000000000000000000000000000000000000000000078
0000000000000000000000000000000000000000000000000000000000000060
0000000000000000000000000000000000000000000000000000000000000004
abe985cb

external’s bytes data is not right-padded due to a bug reported in https://github.com/ethereum/solidity/issues/3493. The fix shipped with solc 0.5.x.

@ricmoo Do you have any views about what could/should be done for this case? 🙂

[ricmoo]

After quite a bit of experimentation and playing around with possible scenarios, I have come to the conclusion that it seems safe to support events (and only events) using a “loose” Reader. Any extraneous bytes read from an adjacent (non-word aligned) ABI word will be discarded, and the offsets are always loaded from a base offset within each nest of the Coder, which in itself is popped off one the way back up the call stack (implicitly, since the base reader retains its location in the calling frame).

I have added support in 5.0.12, but only for the bytes and string types. All other dynamic types in ABIv1 have word-aligned sizes, so this is not an issue.

In Solidity 0.4 the ABIv2 was experimental, so any output with nested dynamic structures may contain the same issue, and are not supported as they may still trigger the OP issue. If someone out there has a strong need for support for ABIv2 nested structures in legacy Solidity external functions, please let me know, but I suspect no one was using this feature, and a lot more effort will be required to prove it is safe to process those with a loose Reader.

Please try this out and let me know if it works for you. I’ve tried it against the various examples opened up in all the issues I could find related to non-word-aligned bytes and string event data.

Thanks! 😃

[ricmoo]

This error is intentional.

It means that there is an invalid UTF-8 string that is being returned by the contract and that it is is unsafe to use for most purposes (such as hashing, using as a key or showing in a UI).

If you catch the error, there should be enough information to choose a lossy method of decoding the string, or you may wish to address the root cause. The toUtf8String method offers an API to perform lossy conversion either using a replacement strategy or an ignore strategy.

Make sense? 😃