Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Node shut down without notice and deleted my wallet

See original GitHub issue

What happened?

I’ve been making a rescan for days and ironfish node shut down without notice. I restart the node but ironfish created a new default wallet. My original wallet is lost right now. I did not export my account before. while I’ve been rescaning, node was not synced but I was shown as SYNCED.

Screen Shot 2022-09-13 at 13 14 35

ironfish.log file does not have timestamps, hence I don’t think it will be useful however I shared relevant section of the log output.

I can also give you a ssh access to the node if you are curious for further research.

Version

0.1.45

Debug output

Database in use, skipping output that requires database.
Stop the node and run the debug command again to show full output.

Iron Fish version               0.1.45 @ 0725b77
Iron Fish library               0.0.22 @ 0725b77
Operating system                Linux x64
CPU model(s)                    Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
CPU threads                     8
RAM total                       15.52 GiB
Heap total                      4.01 GiB
Node version                    v16.17.0
ironfish in PATH                true
Garbage Collector Exposed       false
Telemetry enabled               true
Node name                       Thanos
Block graffiti                  pentafenolin

Relevant log output

Requesting 20 blocks starting at 00000...11161 (178910) from T7rLfcr (dontyouwr)
Added block seq: 178920, hash: 00000...0e7e0, txs: 143, progress: 98.21%, time: 2222.7ms
Requesting 20 blocks starting at 00000...2e5a1 (178930) from T7rLfcr (dontyouwr)
Added block seq: 178940, hash: 00000...266e4, txs: 2, progress: 98.22%, time: 65.0ms
Requesting 20 blocks starting at 00000...be1af (178950) from T7rLfcr (dontyouwr)
Added block seq: 178960, hash: 00000...9d43f, txs: 301, progress: 98.23%, time: 4176.0ms
Requesting 20 blocks starting at 00000...a05e3 (178970) from T7rLfcr (dontyouwr)
Added block seq: 178980, hash: 00000...29212, txs: 301, progress: 98.25%, time: 4031.1ms
Requesting 20 blocks starting at 00000...ff767 (178990) from T7rLfcr (dontyouwr)
Added block seq: 179000, hash: 00000...75088, txs: 301, progress: 98.26%, time: 3413.9ms
Requesting 20 blocks starting at 00000...76888 (179010) from T7rLfcr (dontyouwr)
Added block seq: 179020, hash: 00000...3e9ce, txs: 34, progress: 98.27%, time: 384.0ms
Requesting 20 blocks starting at 00000...ba2f0 (179030) from T7rLfcr (dontyouwr)
Added block seq: 179040, hash: 00000...d7b2e, txs: 301, progress: 98.28%, time: 4845.0ms
Requesting 20 blocks starting at 00000...f9298 (179050) from T7rLfcr (dontyouwr)
Added block seq: 179060, hash: 00000...460ce, txs: 57, progress: 98.30%, time: 1109.3ms
Requesting 20 blocks starting at 00000...02fe5 (179070) from T7rLfcr (dontyouwr)
Added block seq: 179080, hash: 00000...79f91, txs: 15, progress: 98.31%, time: 283.4ms
Requesting 20 blocks starting at 00000...fe2df (179090) from T7rLfcr (dontyouwr)
Added block seq: 179100, hash: 00000...a7416, txs: 101, progress: 98.32%, time: 1824.7ms
Requesting 20 blocks starting at 00000...4a0bf (179110) from T7rLfcr (dontyouwr)
Added block seq: 179120, hash: 00000...08fff, txs: 19, progress: 98.33%, time: 237.4ms
Requesting 20 blocks starting at 00000...36d07 (179130) from T7rLfcr (dontyouwr)
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer 3sAcTx6 (aigelovb3) at 179576
Found peer 3sAcTx6 (aigelovb3) ancestor 00000...9b188, syncing from 177916 -> 179576 (1660) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from 3sAcTx6 (aigelovb3)
Peer 3sAcTx6 (aigelovb3) disconnected (DISCONNECTED) while syncing.
Starting sync from uh0YR9Y (dao-nodes-node1). work: +328353558021244198, ours: 177916, theirs: 179036
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer uh0YR9Y (dao-nodes-node1) at 179036
Found peer uh0YR9Y (dao-nodes-node1) ancestor 00000...9b188, syncing from 177916 -> 179036 (1120) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from uh0YR9Y (dao-nodes-node1)
Added block to fork seq: 129522, head-seq: 177916, hash: 00000...c8c85, head-hash: 00000...9b188, work: 3038370087339454705, head-work: 6398124311587320091, w
ork-diff: 3359754224247865386
Peer uh0YR9Y (dao-nodes-node1) disconnected (DISCONNECTED) while syncing.
Starting sync from 4RIabMo (deckamy). work: +485908455930853494, ours: 177916, theirs: 179598
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer 4RIabMo (deckamy) at 179598
Found peer 4RIabMo (deckamy) ancestor 00000...9b188, syncing from 177916 -> 179598 (1682) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from 4RIabMo (deckamy)
Peer 4RIabMo (deckamy) disconnected (DISCONNECTED) while syncing.
Starting sync from cKEuYZh (air_rider). work: +314883622515434496, ours: 177916, theirs: 179596
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer cKEuYZh (air_rider) at 179596
Found peer cKEuYZh (air_rider) ancestor 00000...9b188, syncing from 177916 -> 179596 (1680) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from cKEuYZh (air_rider)
Peer cKEuYZh (air_rider) disconnected (DISCONNECTED) while syncing.
Starting sync from tHSjlyA (krivajaraketa13). work: +346929688369253488, ours: 177916, theirs: 179216
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer tHSjlyA (krivajaraketa13) at 179216
Found peer tHSjlyA (krivajaraketa13) ancestor 00000...9b188, syncing from 177916 -> 179216 (1300) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from tHSjlyA (krivajaraketa13)
Peer tHSjlyA (krivajaraketa13) disconnected (DISCONNECTED) while syncing.
Starting sync from kEwUxkP. work: +246990639084911019, ours: 177916, theirs: 178755
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer kEwUxkP at 178755
Found peer kEwUxkP ancestor 00000...9b188, syncing from 177916 -> 178755 (839) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from kEwUxkP
Peer kEwUxkP disconnected (DISCONNECTED) while syncing.
Starting sync from t3AZPUa (roadswithoutend). work: +343398532426378412, ours: 177916, theirs: 179965
Finding ancestor using linear search on last 3 blocks starting at 00000...9b188 (177916) from peer t3AZPUa (roadswithoutend) at 179965
Found peer t3AZPUa (roadswithoutend) ancestor 00000...9b188, syncing from 177916 -> 179965 (2049) after 1 requests
Requesting 20 blocks starting at 00000...9b188 (177916) from t3AZPUa (roadswithoutend)
Added block to fork seq: 129595, head-seq: 177916, hash: 00000...1cba1, head-hash: 00000...9b188, work: 3040624152501437330, head-work: 6398124311587320091, w
ork-diff: 3357500159085882761
Added block to fork seq: 158566, head-seq: 177917, hash: 00000...a31b4, head-hash: 00000...02582, work: 4245758240530065874, head-work: 6398411081969217993, w
ork-diff: 2152652841439152119
Peer t3AZPUa (roadswithoutend) disconnected (DISCONNECTED) while syncing.
Starting sync from SMFHUrD (Nether). work: +593146876863683068, ours: 177917, theirs: 180431
Finding ancestor using linear search on last 3 blocks starting at 00000...02582 (177917) from peer SMFHUrD (Nether) at 180431
Found peer SMFHUrD (Nether) ancestor 00000...02582, syncing from 177917 -> 180437 (2520) after 1 requests
Requesting 20 blocks starting at 00000...02582 (177917) from SMFHUrD (Nether)
Added block to fork seq: 162592, head-seq: 177918, hash: 00000...d7fce, head-hash: 00000...56501, work: 4535084462774821434, head-work: 6398697152228113215, w
ork-diff: 1863612689453291781
Added block to fork seq: 164080, head-seq: 177918, hash: 00000...3bc2f, head-hash: 00000...56501, work: 4656581762074511119, head-work: 6398697152228113215, w
ork-diff: 1742115390153602096
Added block seq: 164080, hash: 00000...3bc2f, txs: 81, progress: 97.94%, time: 616.6ms
WebSocket server started at :::9033
Connected to the Iron Fish network
Not connected to the Iron Fish network
Connected to the Iron Fish network
Not connected to the Iron Fish network
Connected to the Iron Fish network
Not connected to the Iron Fish network
Connected to the Iron Fish network
Not connected to the Iron Fish network
Connected to the Iron Fish network
Starting sync from +gA3crb (a-belikov9). work: +1222720393406244546, ours: 177918, theirs: 182034
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer +gA3crb (a-belikov9) at 182034
Found peer +gA3crb (a-belikov9) ancestor 00000...56501, syncing from 177918 -> 182034 (4116) after 1 requests
Requesting 20 blocks starting at 00000...56501 (177918) from +gA3crb (a-belikov9)
Peer +gA3crb (a-belikov9) disconnected (DISCONNECTED) while syncing.
Starting sync from mezkHPB (roman). work: +1157286465022433558, ours: 177918, theirs: 181823
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer mezkHPB (roman) at 181823
Peer mezkHPB (roman) disconnected (DISCONNECTED) while syncing.
Starting sync from lKtliQY (tetsugyo). work: +1177270582451382788, ours: 177918, theirs: 181887
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer lKtliQY (tetsugyo) at 181887
Found peer +gA3crb (a-belikov9) ancestor 00000...56501, syncing from 177918 -> 182034 (4116) after 1 requests
Requesting 20 blocks starting at 00000...56501 (177918) from +gA3crb (a-belikov9)
Peer +gA3crb (a-belikov9) disconnected (DISCONNECTED) while syncing.
Starting sync from mezkHPB (roman). work: +1157286465022433558, ours: 177918, theirs: 181823
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer mezkHPB (roman) at 181823
Peer mezkHPB (roman) disconnected (DISCONNECTED) while syncing.
Starting sync from lKtliQY (tetsugyo). work: +1177270582451382788, ours: 177918, theirs: 181887
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer lKtliQY (tetsugyo) at 181887
Found peer lKtliQY (tetsugyo) ancestor 00000...56501, syncing from 177918 -> 181887 (3969) after 1 requests
Requesting 20 blocks starting at 00000...56501 (177918) from lKtliQY (tetsugyo)
Peer lKtliQY (tetsugyo) disconnected (DISCONNECTED) while syncing.
Starting sync from tB79svQ (rhNode). work: +1154790332513985444, ours: 177918, theirs: 181815
Finding ancestor using linear search on last 3 blocks starting at 00000...56501 (177918) from peer tB79svQ (rhNode) at 181815
Found peer tB79svQ (rhNode) ancestor 00000...56501, syncing from 177918 -> 181815 (3897) after 1 requests
Requesting 20 blocks starting at 00000...56501 (177918) from tB79svQ (rhNode)
Added block seq: 177920, hash: 00000...56535, txs: 301, progress: 97.68%, time: 3637.0ms
Requesting 20 blocks starting at 00000...556e4 (177938) from tB79svQ (rhNode)
Added block seq: 177940, hash: 00000...0c5a8, txs: 21, progress: 97.69%, time: 282.0ms

Graffiti

pentafenolin

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:21 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
hughycommented, Sep 15, 2022

It is confusing! The accountName config option should really be called something like accountsDatabaseName.

Setting accountName to default changes to the database that has your account, penta.

Thanks for your patience with this issue!

1reaction
isezencommented, Sep 13, 2022

Unfortunately, I see only default account. Actually, at the first beginning, I created another account named penta and deleted default one 😃

Incident happened when I was making a rescan operation. It was stuck and my node was far behind the current height. For instance, at the date 13.09.2022 10:08 UTC, current height was 182133 but my node’s height was 177918. After that, suddenly node stopped (of course, rescan command stopped, too). Then I started node by ironfish start --datadir=/home/iron/.ironfish and I got the result that I shared with you.

Some outputs about the issue:

iron@tesla ~> ironfish accounts:list
default

iron@tesla ~> ironfish config:show
{
   "bootstrapNodes": [
      "test.bn1.ironfish.network"
   ],
   "databaseName": "default",
   "databaseMigrate": false,
   "defaultTransactionExpirationSequenceDelta": 15,
   "editor": "",
   "enableListenP2P": true,
   "enableLogFile": true,
   "enableRpc": true,
   "enableRpcIpc": true,
   "enableRpcTcp": false,
   "enableRpcTls": true,
   "enableSyncing": true,
   "enableTelemetry": true,
   "enableMetrics": true,
   "getFundsApi": "https://api.ironfish.network/faucet_transactions",
   "ipcPath": "/home/iron/.ironfish/ironfish.ipc",
   "logLevel": "*:info",
   "logPeerMessages": false,
   "logPrefix": "[%time%] [%level%] [%tag%]",
   "miningForce": false,
   "blockGraffiti": "pentafenolin",
   "nodeName": "Thanos",
   "nodeWorkers": -1,
   "nodeWorkersMax": 6,
   "p2pSimulateLatency": 0,
   "peerPort": 9033,
   "rpcTcpHost": "localhost",
   "rpcTcpPort": 8020,
   "rpcTcpSecure": false,
   "tlsKeyPath": "/home/iron/.ironfish/certs/node-key.pem",
   "tlsCertPath": "/home/iron/.ironfish/certs/node-cert.pem",
   "maxPeers": 50,
   "minimumBlockConfirmations": 12,
   "minPeers": 1,
   "targetPeers": 50,
   "telemetryApi": "https://api.ironfish.network/telemetry",
   "accountName": "penta",
   "generateNewIdentity": false,
   "blocksPerMessage": 20,
   "minerBatchSize": 25000,
   "poolName": "Iron Fish Pool",
   "poolAccountName": "default",
   "poolBanning": true,
   "poolBalancePercentPayout": 10,
   "poolHost": "0.0.0.0",
   "poolPort": 9034,
   "poolDifficulty": "15000000000",
   "poolAttemptPayoutInterval": 900,
   "poolSuccessfulPayoutInterval": 7200,
   "poolStatusNotificationInterval": 1800,
   "poolRecentShareCutoff": 7200,
   "poolDiscordWebhook": "",
   "poolMaxConnectionsPerIp": 0,
   "poolLarkWebhook": "",
   "jsonLogs": false,
   "explorerBlocksUrl": "https://explorer.ironfish.network/blocks/",
   "explorerTransactionsUrl": "https://explorer.ironfish.network/transaction/"
}

Output of config.json:

iron@tesla ~/.ironfish> cat config.json
{
    "enableTelemetry": true,
    "nodeName": "Thanos",
    "blockGraffiti": "pentafenolin",
    "enableLogFile": true,
    "accountName": "penta",
    "logPrefix": "[%time%] [%level%] [%tag%]"
}

iron@tesla ~> ironfish status

Version              0.1.45 @ 0725b77
Node                 STARTED
Node Name            Thanos
Block Graffiti       pentafenolin
Memory               Heap: 93.45 MiB -> 134.36 MiB / 4.01 GiB (2.3%), RSS: 2.16 GiB (13.9%), Free: 8.86 GiB (42.9%)
P2P Network          CONNECTED - In: 57.49 KB/s, Out: 500.03 KB/s, peers 42
Mining               STARTED - 0 miners, 0 mined
Mem Pool             Size: 175 tx, Bytes: 260.33 KiB
Syncer               IDLE - avg time to add block 947.69 ms
Blockchain           SYNCED @ HEAD 0000000000004719cd200745ea86b4d901660e81615917bc5dc267af30fca734 (182425)
Accounts             000000000002d296c5b4ad92b76899796d249bec13ddc174e71a18d74af2494b (35267)
Telemetry            STARTED - 1920 <- 2 pending
Workers              STARTED - 0 -> 1 / 6 - 0.49 jobs Δ, 233.85 jobs/s
Read more comments on GitHub >

github_iconTop Results From Across the Web

How to restore your node wallet (updated) - Guides
Here is a separate chapter about managing your Umbrel node, about methods/ways to restore/recover your Umbrel LND wallet in case of SHTF.
Read more >
Importing an Existing Wallet for your Node
You MUST CONFIRM that they have shut down validation for your node and will NEVER resume it, and manually confirm that your validators...
Read more >
grin_community/support
Does anyone else's grin node stop syncing? I have to clear my peer list and restart from time to time.
Read more >
Frequently Asked Questions
Launch Zelcore, then the full node wallet. Zelcore will show a message that it found an unencrypted wallet file. This is good, click...
Read more >
Guardian Node is taking a long time to sync to the Theta ...
@clutchnshift try resetting your node database. Using the app's menu, click Node -> Delete Node Database and follow the prompts.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

ironfish node stop syncing on ironfish@0.1.49

Next page

accounts:remove results in noteToNullifier error