Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error: Coudn't complete data download

See original GitHub issue

Hi! Today I updated the arweave-js version on my project and notice an issue in my dApp.

I have this TX: A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0

And I’m using getData to get the data from the tx: const data = await arweave.transactions.getData('A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0', {decode: true, string: true} );

Before the update everything was fine, the data was fetched, but now I’m getting these errors from the console:

[chunk] Failed to fetch chunk at offset 11882184452909 chunks.js:67 [chunk] This could indicate that the chunk wasn't uploaded or hasn't yet seeded properly to a particular gatway/node Error: Error: Coudn't complete data download at 0/1563

I think that maybe the problem comes from the offset calculation and the arweave-js new update.

Please help! haha. Thanks in advance 😃

Issue Analytics

State:
Created 2 years ago
Comments:11 (9 by maintainers)

Top GitHub Comments

3reactions

rosmcmahoncommented, Jan 3, 2022

I did some digging around and came up with this, you can just feed it your txid 😃

const txid = 'A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0'

const { data } = await axios.get('https://arweave.net/' + txid, {
	responseType: 'arraybuffer',
})

const uploader =  await arweave.transactions.getUploader(txid, data)
		
while(!uploader.isComplete){
	await uploader.uploadChunk()
	console.log(`chunkUploaded: ${uploader.uploadedChunks}/${uploader.totalChunks} %${uploader.pctComplete}`)
	console.log(`lastResponseStats: ${uploader.lastResponseStatus}`)
}

3reactions

hlollicommented, Jan 2, 2022

If you take a look at

https://arweave.net/tx/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/offset

you can track down where the chunks for a give transaction is located at.

The offset ends at 11882184454471 but begins at 11882184454471 - dataSize + 1. So it starts at: 11882184454471 - 1563 + 1 = 11882184452909

Now if you try to download that chunk

curl https://arweave.net/chunk/11882184452909
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Not Found</pre>
</body>
</html>

it seems it isn’t found.

this transaction was mined into block 756664 at 26th of August 2021 (writing this as I’m thinking out loud).

And indeed it is visible from the gateway cache: https://an7bym2iyfh2bmjinrfqykwm5sdjrqwos6kdgn7buhj6ku2m6twq.arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/

This means in simple terms, sadly, that the gateway received the chunks but didn’t succeed in sending them to our nodes (that is a serious problem if true).

To verify this hypothesis: try to curl that chunk from peers in the network https://arweave.net/peers

e.g. curl http://110.52.217.148:1984/chunk/11882184452909 curl http://220.200.166.227:1984/chunk/11882184452909 curl http://122.100.147.8:1984/chunk/11882184452909 curl http://52.199.181.113:1984/chunk/11882184452909

To fix this:

download the data and the transaction headers.
Run the following code snippet as is described in README.md

const Arweave = require("arweave");
const ArweaveTransaction = require("arweave/lib/transaction.js");
const fs = require("fs");

const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

let data = fs.readFileSync(
  "./data_from_the_gateway.ext"
);

let txHeaders = require("./tx_header.json"); 

(async () => {
  const tx = new ArweaveTransaction.default(resumeObject);

  let uploader = await arweave.transactions.getUploader(tx, data);

  while (!uploader.isComplete) {
    await uploader.uploadChunk();
  }

})();

feel free to replace

const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

with a node or gateway of your choosing (given that arweave.net likely failed here, but I’m sure it will work the second time).

Alternative to all of this, you can choose to fetch the data from the gateway-cache instead of relying on getData:

const data = fetch("https://arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0");

since the gateway cache isn’t permanent, I strongly encourage to re-upload the chunks to make fully sure the nodes have them!

We are planing to re-upload all the chunks in coming weeks, but until then, we can only be as good as our weakest link, and it seems we clearly made a mistake here.