question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Temporary files are never cleaned

See original GitHub issue

While doing cross-validation for hyperparameters tuning, I noticed the filesystem usage growing indefinitely. During processing some directories are created in /tmp, but they are never cleaned.

Looking at the code I think the issue is in the fit method in models.py:

def fit(self, stan_init, stan_data, **kwargs):
        (stan_init, stan_data) = self.prepare_data(stan_init, stan_data)

        if 'inits' not in kwargs and 'init' in kwargs:
            kwargs['inits'] = self.prepare_data(kwargs['init'], stan_data)[0]

        args = dict(
            data=stan_data,
            inits=stan_init,
            algorithm='Newton' if stan_data['T'] < 100 else 'LBFGS',
            iter=int(1e4),
            output_dir = mkdtemp(),
        )
        args.update(kwargs)

        try:
            self.stan_fit = self.model.optimize(**args)

The mkdtemp function requires the user to manually delete the directory once it is no longer needed (see the doc) but I don’t see where it is cleaned here.

The cmdstanpy doc also says the following:

:param output_dir: Name of the directory to which CmdStan output
            files are written. If unspecified, output files will be written
            to a temporary directory which is deleted upon session exit.

So I’m wondering if output_dir = mkdtemp() is really necessary here ?

Thank you!

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:2
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
tcuongdcommented, Sep 4, 2022

Cmdstanpy cleans up its own files using atexit

Yep nice, just tested and it’s working as expected for CV. Will do a quick PR to bump the minimum version for cmdstanpy.

1reaction
WardBriancommented, Sep 4, 2022

Cmdstanpy cleans up its own files using atexit

Read more comments on GitHub >

github_iconTop Results From Across the Web

Fix Temporary Files not Deleting [Automatically, Disk Cleanup]
Windows 10 stores all sorts of temporary files on your PC, and this guide will help you to fix the temporary files won't...
Read more >
Top 3 Ways to Fix Windows 10 Temporary Files Not Deleting
Are you unable to delete temporary files on your Windows 10 computer ... Step 5: On the Disk Cleanup window, tap the 'Clean...
Read more >
Disk Cleanup Hanging / Not Completing - Temporary Files
Hi Microsoft's 'Disk CleanUp' Utility frequently 'hangs' and does not (appear) to complete clearance of Temporary Files.
Read more >
When is a Windows User's Temp directory cleaned out?
Windows never automatically cleans the %TEMP% directory by default. In Windows 10 you have to enable this feature in Settings, ...
Read more >
Why Aren't Windows Temp Files Deleted Automatically?
Windows doesn't know whether every application is done with its temporary files when you run the Disk Cleanup, so it errs on the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found