Empty mixHash values resulting in failing blockchain tests

mixHash is on many tests just filled with 0s, can be seen on the following search, which leads to many failing blockchain tests.

I explicitly tested with notxs.json test over on ethereumjs-vm which is now failing due to differing values:

1639a35bab884e5d9b095bd860d6f776768b9be157275792a758e1e3909308e0
0000000000000000000000000000000000000000000000000000000000000000

An older version of the tests repo (tested the snapshot on 3f5febc) still has the correct value.

[holgerd77]

Ok, I would write a summary of the latest changes in the repo every two weeks and publish this as a tagged release, if there are no objections from Dimitry. I think I could take half a day of for this every two weeks, can also commit to stick on for this for at least 6+ months. Would also in parallel then do a short post on Reddit for people interested.

I will also try to do accompanying PRs for the docs, so that changes apparent in the releases are reflected to some extend in the documentation, maybe I can develop some regularity here as well, we’ll see. Will be some longer process though.

[holgerd77]

Stuff like this really needs to be documented, otherwise people just have no chance than to stumble over this if they are not following every single commit/PR in the repo (this would even justify some “Attention! Attention!” note in the first lines of the README). Is all described in this issue https://github.com/ethereum/tests/issues/464 now implemented? Then I can prepare a PR towards the docs with the changes.

Have you seen the suggestion on releases https://github.com/ethereum/tests/issues/531? This would also help very much on stuff like this, since people then have a descent chance to follow up on release notes.

One way to do it would be to just do maybe bi-weekly releases not really tied to some major changes, this would already have the advantage to sum up the recent changes, one could also accompany this with a post on Reddit to raise some awareness every two weeks. Other way would be to tie releases to major changes (Constantinople EIP xxx tests ready, test format change,…). I would probably prefer the first version, seems easiest to start with and has some more regularity.

Release would just be an optional tagged release on GitHub, nothing published or something, so just an additional offer for people to stick onto this rather than some arbitrary commit snapshot, this would already give teams a more common ground to exchange on test problems.