Temporary files are never cleaned

While doing cross-validation for hyperparameters tuning, I noticed the filesystem usage growing indefinitely. During processing some directories are created in /tmp, but they are never cleaned.

Looking at the code I think the issue is in the fit method in models.py:

def fit(self, stan_init, stan_data, **kwargs):
        (stan_init, stan_data) = self.prepare_data(stan_init, stan_data)

        if 'inits' not in kwargs and 'init' in kwargs:
            kwargs['inits'] = self.prepare_data(kwargs['init'], stan_data)[0]

        args = dict(
            data=stan_data,
            inits=stan_init,
            algorithm='Newton' if stan_data['T'] < 100 else 'LBFGS',
            iter=int(1e4),
            output_dir = mkdtemp(),
        )
        args.update(kwargs)

        try:
            self.stan_fit = self.model.optimize(**args)

The mkdtemp function requires the user to manually delete the directory once it is no longer needed (see the doc) but I don’t see where it is cleaned here.

The cmdstanpy doc also says the following:

:param output_dir: Name of the directory to which CmdStan output
            files are written. If unspecified, output files will be written
            to a temporary directory which is deleted upon session exit.

So I’m wondering if output_dir = mkdtemp() is really necessary here ?

Thank you!

[tcuongd]

Cmdstanpy cleans up its own files using atexit

Yep nice, just tested and it’s working as expected for CV. Will do a quick PR to bump the minimum version for cmdstanpy.

[WardBrian]

Cmdstanpy cleans up its own files using atexit