Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Remove datasets from ClearML's file server when deleting them through the UI

See original GitHub issue

Proposal Summary

I’m saving my ClearML datasets directly to the file server (I’m not using AWS, Azure, or whatever and I’m self hosted). This is very handy to us because we can easily download them from ClearML, and it’s faster over our local network than the cloud. However, when deleting a dataset through the UI interface, it’s doesn’t remove the files located in /opt/clearml/data/fileserver/project/.dataset/....

Motivation

We have multiple hundreds of GB of data, and this will cause our storage to expand pretty quickly. Manually deleting the datasets is a bit troublesome, and I would like avoiding that.

Related Discussion

Initial slack thread: https://clearml.slack.com/archives/CTK20V944/p1665736071874939.

Issue Analytics

State:
Created a year ago
Comments:12 (1 by maintainers)

Top GitHub Comments

1reaction

erezalgcommented, Nov 16, 2022

@aurelien-m,

This will be a part of the next server release, planned later this week or early next week 😄

1reaction

evg-allegrocommented, Oct 18, 2022

Thanks, I think that we identified the issue. Let me check what can be done

Top Results From Across the Web

Cleanup Service - ClearML

APIClient class to implement a service that deletes old archived tasks and their associated files: model checkpoints, other artifacts, and debug samples. Modify ......

Automatically deleting artifacts and models files from UI #144

Whenever I'm deleting a task using the web UI, it gives me the list of files that have not been deleted on my...

Dataset still in the server even when deleted in the web interface

Hello! Be sure to select the `Drop Data` radio button when trying to delete data. It should look as follows: -----.

In Watson Studio Local, unable to delete data sets without ...

Open a project. Click on Data sets. Click on Add a Data Set Browse and upload a file that has no extension (created...

Load and delete a data set within environment through Shiny ...

I save the airquality one as a csv file and then load it into the workspace via the fileInput function. Here is what...