Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
[BUG] Signage points out of order
See original GitHub issueDescribe the bug I’m seeing a lot of signage points out of order. Sometimes they are skipped (as in #1796), but mostly they are logged out of order, with some signage points repeated several times (so this seems to be different to #1796). This is on v1.1.3 so it seems that #1978 has not fixed this issue.
$ grep "/64" ~/.chia/mainnet/log/debug.log.5
...
2021-05-03T02:05:00.943 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 48/64: 61c8d7f3fb96df19d8215c28701e743eca99e21d4443ae7bf23b83f2fd196085
2021-05-03T02:05:09.084 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 49/64: 2a64f5b329d52a4588f7c8301005cb6a879693d38b995ef0bd86e7250e47006d
2021-05-03T02:05:17.421 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 50/64: 45e80cfbd7eb16594722b2011397b8b04dc6577cc140f5f7ac5dcc266974f2b1
2021-05-03T02:05:27.213 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 51/64: 545ca33c3ac0ab3420bb7de0dbbcfa19306e734bea06ff70bc18fa79982e1d43
2021-05-03T02:05:37.114 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 50/64: 45e80cfbd7eb16594722b2011397b8b04dc6577cc140f5f7ac5dcc266974f2b1
2021-05-03T02:05:37.302 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 49/64: 2a64f5b329d52a4588f7c8301005cb6a879693d38b995ef0bd86e7250e47006d
2021-05-03T02:05:37.354 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 51/64: 545ca33c3ac0ab3420bb7de0dbbcfa19306e734bea06ff70bc18fa79982e1d43
2021-05-03T02:05:37.557 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 13/64: 579a292227c4c77627a30d50c0a8dc3a69fe48054ee7ee310b4d1c52155b4867
2021-05-03T02:05:37.651 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 46/64: 539296c03d531b65c6bbd2f5a68c20fb1d187a8d0317df519cdc8cb1c4ced8a1
2021-05-03T02:05:38.096 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 52/64: 3022e5751a3175a048b3f26443084a36ab352408269fd53478b9b808101bb15f
2021-05-03T02:05:38.396 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 8/64: 18b5d73a29ffb81bb9b8014529df44068c4d0fa53eb1d536d091e32ff158ddec
2021-05-03T02:05:38.424 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 14/64: cb865a715b8c8092d2c252dfa998d7e20c1bd70fd3dbd26d91c48ebe6fe85580
2021-05-03T02:05:38.432 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 20/64: 314c49c26f6b6c9964e66c3f84d71f689d7710c3a360eec32125a5b7c0e6c222
2021-05-03T02:05:38.434 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 47/64: b4b6617e99d24dd026887b49139011ccc17e6cf84435b9a62004502e3b190155
2021-05-03T02:05:38.522 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 15/64: 3005782b0d0b42097d3181b179d17ce6cad75e55042f21c9cfaf23a7ad699021
2021-05-03T02:05:39.248 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 9/64: 80014f1474e128c0cf214f6eda5e7bed4162b20e26b6951306fd1c1e0224938d
2021-05-03T02:05:39.331 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 10/64: 7810e18fb543e14875d71b506278b72fcf13f1a340890473128f85fcd1fc6c59
2021-05-03T02:05:39.387 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 22/64: c26706b0a2429aa05e4d674b4fa86078a9573796cbcf1299467be41e7ac4c9f4
2021-05-03T02:05:39.442 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 23/64: 7e74c58278ee80881402c4cc02739b1168e88cad7e8b2bf059e3d06545c11d30
2021-05-03T02:05:39.470 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 16/64: 92453e2f541d0d92c509be0a5e81d6cc40e3883e976f07525088f78284960966
2021-05-03T02:05:39.654 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 21/64: 045c7752a2eb06aa55acd0977c80e93389b4286340e670d1f7cff662fb37cea4
2021-05-03T02:05:39.912 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 43/64: 2a95f6414e88d67d924a3de131789ad0654a9a399e1553208f6d6ba88e66fb7f
2021-05-03T02:05:40.068 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 12/64: 34025f55846fce82b421e59c30a986bea0e18c53cb970a4c9a73ac1642ae6b4f
2021-05-03T02:05:40.312 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 17/64: 7f1ee80736e16d8e1da00632e18f116c6e7e2858a37b10bab9540ea64d2e070a
2021-05-03T02:05:40.342 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 44/64: a62064887deee8eccc4a61097b4830fc9ca208cfa7e8f832ef411951df8af445
2021-05-03T02:05:40.356 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 45/64: 4fe09e49da5a68281eba591dd2a1d2346be9f36760ebc85402e3845e7e49158b
2021-05-03T02:05:40.616 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 18/64: d9a7aca27cb2d265cde9ad4a6856fbdccd668d426a5e34de6cf4b623cb412cb0
2021-05-03T02:05:40.663 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 19/64: 2f2d9e654f8feea501ee8ea44aad12ade76264b0c54a0f6d747e11e9e6fac53b
2021-05-03T02:05:40.678 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 33/64: 69d5697f9cbe266fba4552833d77379367ff684abb8f06f6d048d9c19859f3e6
2021-05-03T02:05:40.841 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 34/64: 2f37c02eace7829eee18f53348fe5ca7e71e7a68699fa339d5e6e777582f4784
2021-05-03T02:05:40.843 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 35/64: 33e34333282a21cc403889281be70276215f3f9b1fe284a4f3f58a86da982541
2021-05-03T02:05:40.844 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 36/64: a131fe95c68b996bafe630b8123eaaff0f641031ef2d41227c82198c92f75529
2021-05-03T02:05:40.885 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 37/64: 4127e2bc5bfdb8f8e937108a5199796df0c130301093cd930a021b485b2e8680
2021-05-03T02:05:40.905 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 38/64: da21ce2b969cf7e96c5290e477d6f66aeca2a52e4c5297701cae31f374a35204
2021-05-03T02:05:41.129 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 11/64: 7ef4887e0056f932bc95cfe510b13221c24f8a97efc9d73e760e0cc94d0bef81
2021-05-03T02:05:41.163 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 39/64: f5cedafc0628a4bcb147405e7fb6d9524063bf3a6fc2f240d1fc2bdcbf858369
2021-05-03T02:05:41.164 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 40/64: 2fac70d7fcd251fa41f74985aa7f6be68edf7bcf99419105cc102eb983799df2
2021-05-03T02:05:41.166 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 41/64: 91c8470ffb90d682cec40b5c5848b8d49f629a49761dca0a06ebd95cc9d2da6f
2021-05-03T02:05:41.167 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 42/64: c065556e48dcaf659d5dea4ddcb1f6112b419cf724bb1e5a1332f51e2da86a75
2021-05-03T02:05:41.540 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 31/64: 6244b88777580cf850d87c0850e20201ef248cf67f33d7c9110730b2c8454a06
2021-05-03T02:05:41.556 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 32/64: 60d05c1a9ccb3e6e694a7dd33282db31689a71dcce6ed27772869d1605c35ac9
2021-05-03T02:05:42.371 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 26/64: d54f6f6c0316668504ee78a7955cdb4c04e1b88bf861f7d1b5188c71689f2985
2021-05-03T02:05:42.414 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 27/64: ba36b67a05acc0dc386e00796a608be375622fd341b0b8bd2da6d849fbc18b69
2021-05-03T02:05:43.641 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 4/64: c0b887e8c83bb0ceae7aeb9bec6d85b18dd48f92e10feab283c706c33afdbe3a
2021-05-03T02:05:43.721 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 5/64: 14c3f02fb60d603f8e3d7918ad449a24edf797afe988533076ae19426c3d6dd8
2021-05-03T02:05:43.785 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 6/64: bbf4c82b46f56f04507bd389a06a87a643dc63ef00bfaf94a0b738282891902a
2021-05-03T02:05:44.158 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 7/64: ac5fee597c17f6851418c7db99351168d477fe1818c946f08eee74ec71682c06
2021-05-03T02:05:44.863 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 24/64: 246fbc09d90e17a7c146087ce80420399b2f830a958ada8c982862a9d668ae02
2021-05-03T02:05:45.107 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 25/64: 4ecdafadbb826250b934898d88a0537a13e4e81e991c8ed9a30e39dce03b3e10
2021-05-03T02:05:45.336 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 28/64: 166879f47d93bf684bd90543ec5daae27c6b95c801aa7b4a3972bf5669183383
2021-05-03T02:05:45.703 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 29/64: 38cfb88fd03f5b0b7eb270cae04a65a4c044873c6671b0514bab77708c9afc9d
2021-05-03T02:05:45.804 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 30/64: 30e98412e2f4fa2dc5943cac081ddac7162790db0a1b00a51c590c74c9b50ea2
2021-05-03T02:05:48.524 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 53/64: 57017a1d2897af93338225280511b44de8d996ade3d8d4cc1efd8e0dfb9dcaec
2021-05-03T02:05:56.552 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 54/64: f8d08b3808fd99838a68963b22df4a71c4e6e75fa4bd87c15eb7f8c101f97a2c
2021-05-03T02:06:02.151 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 3/64: 462353ae1eb92c843f67a7da1e3328141cc46d2d70d0ce4240d07485afcd2c4f
2021-05-03T02:06:02.211 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 1/64: 1b89c6d5c0784e20f48875811b84dfdd0fd8261a1a81cf30ec413be0265e7e4f
2021-05-03T02:06:02.783 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 2/64: c6efa4b56dee8065805776261dd5def4772bcee08028831d6ff2c90e0f5ac497
2021-05-03T02:06:06.236 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 55/64: 43b62e39abb3046429e2ea6b39dd49836d3802071c0b92df2beee85277fc0b7d
2021-05-03T02:06:14.701 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 56/64: c4fec8cda2cb07f4510b9a754043889d14e858c663b616d021e9bb8f63f8b58f
2021-05-03T02:06:23.987 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 57/64: 3e5af3a5e176e8440be8d23e361689b1a4bade3e90c0d08fa2c438cf2f136bd5
2021-05-03T02:06:31.932 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 58/64: 579ede9a18b9697c1880db31e50fe4470183917366bb610125e5c4f31c1b522b
2021-05-03T02:06:40.332 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 59/64: a165ae4ddf6a06388d4df98a9d3fb1e20e42a2e1a6177c0c7b8b350850fa1baf
2021-05-03T02:06:48.636 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 60/64: d2eb40f3bf3d519357bfe1e2eedbe78faa3a4379ab5c2d4a9121dc2a2483f8f2
2021-05-03T02:06:57.382 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 61/64: fe69399b15a59722696cf91477a5f06e9dc3b255c162a374896323c3ca96ac58
2021-05-03T02:07:07.221 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 62/64: 682d4740af1a29e9333732d6f294db2945a83ec9365c71f28f8bd4071b36cf65
2021-05-03T02:07:15.572 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 63/64: f753db0f6d70be659a420cb0761ea168154bec48eedafdc0da07268918f59ebb
2021-05-03T02:07:24.182 full_node chia.full_node.full_node: INFO ⏲️ Finished sub slot, SP 64/64, c76ec5679b1fbe36d69cb049d0c66142d4403b9762f0cc37c9df9ab865ac5985, number of sub-slots: 2, RC hash: 070061755aa13f14da56db866e8e028a5f458b13e0bf559a0a50a3fc40460f58, Deficit 16
2021-05-03T02:07:33.946 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 1/64: 326684d071647bfe86bd426ecdabb75a2f0f81cf0f45559c52187122c2b29f63
2021-05-03T02:07:42.670 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 2/64: 487f24cc5fa88b7def653f4c81bac86575a86e3ca04916661ef8c8e3a6046594
2021-05-03T02:07:50.436 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 3/64: 8d427c4736179771bc5b5e338543e8641b18cb0967917926d45730f56c79c241
2021-05-03T02:07:58.964 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 4/64: cfa970d25190cde401d8b08e4751dcf34b7f7cf1c7bbcd56792aa373d439a2e9
2021-05-03T02:08:07.803 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 5/64: faeca45eb2229ea0480f63945e8ea20c9c70ee274e480ec5c0ad831dcff394b6
2021-05-03T02:08:15.817 full_node chia.full_node.full_node: INFO ⏲️ Finished signage point 6/64: af397ca6b21db8fc3fea590a10e916603684a14c3ad356c20eedfc639ae27227
...
Expected behavior All signage points to appear in order and only once.
Desktop (please complete the following information):
- OS: Linux
- OS Version/Flavor: CentOS 7.9
- Chia version: 1.1.3
Additional context This error is occurring on a machine set up as a farmer only. It has some plots stored locally, but most of the plots are on remote harvesters.
The machine has plenty of resources (quad core, 16GB RAM, OS & Chia running on an SSD, solid fibre internet connection) and very little running at the times when this happens.
The reason I was checking my logs is that I have not won a block in 7 days, despite having an average of over 2,000 plots during that time. The probability for this by my calculations is less than 1%, so something seems likely to be wrong here.
Issue Analytics
- State:
- Created 2 years ago
- Comments:12 (2 by maintainers)
Top Results From Across the Web
Issues with signage points (out of order) : r/chia - Reddit
Issues with signage points (out of order). Anyone else seeing problems with signage points/challenge indexes?
Read more >Signage Point 'Bursts'? - Support - Chia Forum
Typically they come in pairs where the signage point is written out (local machine and remote harvester) and then receives likewise paired ...
Read more >8 Telltale Signs of Bug Infestation - Pegasus Pest Control
Inspect the areas of your house where you find them to see if you can identify their access point and try to figure...
Read more >Research (bug?) :: Age of History II 일반 토론 - Steam Community
If I distribute my research points using the plus signs for the different areas, all is fine. ... In order to decrease research...
Read more >Insect Anatomy - Purdue Extension Entomology
This gives the insect the structure to which muscles can attach and operate, ... as a basis for separating insects into their respective...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I also updated to 1.1.4 but unfortunately am still seeing the same.
In this case there are two signage points at 38 both with different hashes:
In this case 20 and 21 are out of order:
In this case 64 is missing entirely:
I’ve seen the same. I was only having a few signage points out of order initially. Now, I’ve just seen a minute and a half delay, and a burst of signage points coming in and out of order. Looks like it recovers, but it’s never good to lose opportunities to respond to challenges. Long story short, 1.1.4 did not resolve my issue either.
Update: I traced my issue back to my remote harvester randomly disconnecting and reconnecting. I think that’s a different issue than what @wriches was seeing. I still see the occasional out of order signage point (nothing missing, but if my remote harvester stays connected, I have no issues (in 1.1.3 my remote harvester was fine but I was seeing a lot more missed signage points, so 1.1.4 resolved those issues for me.) I’ve since restarted my harvester and will be on the look out if it randomly disconnects again.