Paths in metadata

(Following up on discussion in the Jan 8 source-verify meeting)

Am wondering about how the metadata is hashed and the importance of the source file paths. (Apologies in advance if I’ve misunderstood this…)

In the solidity docs there is a warning:

Since the bytecode of the resulting contract contains the metadata hash by default, any change to the metadata might result in a change of the bytecode. This includes changes to a filename or path, and since the metadata includes a hash of all the sources used, a single whitespace change results in different metadata, and different bytecode.

For example, A.sol, compiled with truffle compile and deployed using the data in the truffle artifact, has metadata source keys like:

"sources": {
  "/Users/cgewecke/code/ef/sv-truffle/contracts/A.sol": {
   "keccak256": "0xfc68a9ffdb0d7d3ec5456f5e2d2c4f855e684892f6c78fcb0dc1e482696d9bb7",
   "urls": [
    "bzz-raw://4d22457fe6215d956a57fcd0cb63f745d8fd718ba8d85a20d65c72232038779f",
    "dweb:/ipfs/QmPmUahXAjtq5mJTW9Qc6oNUVGEuocxb1un5371zLJAAb5"
   ]
  }
},

Running solc --metadata contracts/A.sol over the same source produces

"sources": {
  "contracts/A.sol": {
   "keccak256": "0xfc68a9ffdb0d7d3ec5456f5e2d2c4f855e684892f6c78fcb0dc1e482696d9bb7",
   "urls": [
    "bzz-raw://4d22457fe6215d956a57fcd0cb63f745d8fd718ba8d85a20d65c72232038779f",
    "dweb:/ipfs/QmPmUahXAjtq5mJTW9Qc6oNUVGEuocxb1un5371zLJAAb5"
   ]
  }
},

It seems like the file system context and tooling of the deployment and metadata compilations are not allowed to differ even when the sources are identical.

Not sure this would work but perhaps if the file’s keccak hash was used as a key instead, metadata could be matched to sources from anywhere as long as paths were associated with the files when uploading to the service for JSON-IO compilation.

If metadata was pathless, you could:

• have a source-verify cli tool set up to run in CI on merge to a release branch
• have a bot that searched Github for Solidity projects, cloning and compiling them, uploading all the metadata to SV to build a large sources DB.

[christoph2806]

In the meantime (tested with truffle@5.4.8), they seem to have updated truffle’s metadata generator to create reasonable paths e.g. project:/contracts/shared/Delegator.sol so there should be no need to use the cumbersome workaround mentioned above.

[christoph2806]

One possible workaround is to run truffle in a docker:

1. Create an empty folder
2. npm init -y
3. npm i truffle
4. create a Dockerfile with the following content:

FROM node:12
WORKDIR /usr/src/truffle
COPY package*.json ./
RUN npm install
COPY . .
WORKDIR /solc
CMD [ "/usr/src/truffle/node_modules/truffle/build/cli.bundled.js", "compile", "--all" ]

1. Build docker with docker build -t truffle-docker
2. run the docker container with docker run -v [your contracts folder]:/solc truffle-docker

In the resulting build the source paths are starting with /solc/… (thx to @chriseth for the tip)