Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Teku + Besu, attestations randomly missed since the merge

See original GitHub issue

besu 22.7.2 teku 22.9.0

Sep 15 15:06:00 eth-consensus teku[522445]: 2022-09-15 15:06:00.673 INFO - Attestation performance: epoch 146952, expected 6, produced 6, included 5 (83%), distance 1 / 1.00 / 1>

teku log besu log

It’s not clear why produced is 6 and included 5.

The end result is 80% performance.


Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:1
  • Comments:19 (6 by maintainers)

github_iconTop GitHub Comments

ajsuttoncommented, Sep 18, 2022

@totocz A 2 second arrival is relatively slow - ideally it should be under 1s, but it’s not generally the end of the world. The main problem with those numbers is execution_payload_result_received which is the time that Teku is waiting on besu to respond and that’s what’s caused those blocks to complete importing after the 4s mark. Some of them would have imported late even if they’d been received quickly.

To give an idea of block arrival times. In the last day, 11% of blocks were received within 500ms, 50% of blocks were received between 500ms and 1s, 25% between 1s and 1.5s and 8% between 1.5s and 2s. Then about 5% of blocks were received later than that.

So with the time besu is taking most blocks would probably be imported just in time, but quite frequently blocks would be imported late and you’d see an empty slot message when really the block was received with enough time that it should have been imported in time. This is the common pattern we’re seeing with 22.9.1 where it’s the execution client that is now dominating the block import time and there’s not much more Teku can do to improve things.

iamkoacommented, Sep 16, 2022

Having the same issue with Teku + Geth. Attestation performance has gone from 95%+ pre-merge to around 80% post-merge.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Missed validations since merge (Besu/Prysm) : r/ethstaker
Since the merge, I've been getting pretty bad participation stats. Lots of missed validations and incorrectly voted head.
Read more >
Besu Execution timeout/missed attestations #4400 - GitHub
I've seen this problem both with and without the JSON-RPC error reported. Initially post merge with the instance still running from pre-merge, I ......
Read more >
Troubleshooting - Teku - ConsenSys
Occasional missed attestations are normal on a P2P network, but they should be a few percent. If all recent attestations are marked as...
Read more >
Nethermind on Twitter: "This means, that after the painful ...
#TheMerge is around the corner, and with Sedge, you can set up a validator ... Otherwise, your validator may not have the best...
Read more >
Client diversity on Ethereum's consensus layer — jmcook.eth
A sync committee is a group of 512 randomly chosen validators that ... that is maximally penalized for missed attestations loses 3/4 of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found