Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Each added image is stored as a separate file to disk

See original GitHub issue

Cluster environments often restrict the number of inodes per account/mount point. For example, the cluster of my organization has a limit of 1M files that can be stored in the partition for experiment artifacts. While the tensorboard client writes all resources (such as images upon add_image) to pb files, wandb client seems to store every image as a separate file to disk. This implementation causes the out-of-quota errors very quickly when running experiments on the scale of tens and hundreds in a cluster environment. It would be great to either add a flag to delete such temporary images upon syncing them to the cloud, or write resources into a single (or less) files.

Issue Analytics

State:
Created 3 years ago
Reactions:3
Comments:6

Top GitHub Comments

2reactions

issue-label-bot[bot]commented, Oct 19, 2020

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.79. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

0reactions

toshascommented, Apr 20, 2021

Hi, is the issue on the roadmap?