index.json in S3 grows indefinitely and causes errors

Description

Publishing a package that is 3Mb will be saved to s3 on its own but also to the index.json:

(src/put/publish.js)

    json['dist-tags'][tag] = version;
    json._attachments[`${name}-${version}.tgz`] = pkg._attachments[`${name}-${version}.tgz`];
    json.versions[version] = versionData;

    ...

    await storage.put(
      `${name}/${version}.tgz`,
      json._attachments[`${name}-${version}.tgz`].data, // eslint-disable-line no-underscore-dangle
      'base64',
    );

    await storage.put(
      `${name}/index.json`,
      JSON.stringify(json),
    );

If you publish 100 times, the index.json will be roughly 300Mb, which will fail or be grossly inefficient:

    const pkgBuffer = await storage.get(`${name}/index.json`);
    json = JSON.parse(pkgBuffer.toString());

Would the solution be to clear out json._attachements prior to saving to index.json? What recommendations do you have since I have already run into the issue, can I simply delete the entirety of the bucket in s3 if I don’t care about past releases?

[jonsharratt]

So this existed as I had to piece together how npm works. Turns out they used couchdb so this was why I copied over that functionallity.

They hit the same perf issues and look to have taken the attachments out of the package.json themselves.

Merged your proposed fix @ganapativs and tagged a release, give it a go and let me know how you get on.

Sorry for the delay on this been a crazy past couple of months.

[k-k]

@jonsharratt Im guessing the get request removed attachments, but does the PUT request attempt to return the patched index.json as a response - and does not remove attachments?