MatplotlibWriter overwrite list of figures does not work.

Description

The latest MatplotlibWriter (which has not been released as stable yet) has an error here: (https://github.com/quantumblacklabs/kedro/blob/b76707669cde0b860389cd24a4397cbce001d13c/kedro/extras/datasets/matplotlib/matplotlib_writer.py#L149)

Where self._filepath is an object of ‘PurePosixPath’ which is not iterable.

Context

I wanted to output a list of matplotlib figures using this dataset and using the new attribute overwrite=True.

Steps to Reproduce

Here is a snippet that reproduces the error:

from kedro.extras.datasets.matplotlib import MatplotlibWriter
import matplotlib.pyplot as plt

# We run save twice so that the figure is overwritten the second time:
for i in range(2):

    dataset = MatplotlibWriter(filepath='test', overwrite=True)
    figs = []
    for i in range(3):
        fig,ax=plt.subplots()
        ax.plot([1,2,3])
        figs.append(fig)
        
    dataset.save(figs)

TypeError: 'PurePosixPath' object is not iterable

Solution suggestion:

Convert the PurePosixPath object to string solves the problem: self._fs.rm(str(self._filepath), recursive=True)

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• Kedro (latest) (current stable 0.17.5 does not have overwrite attribute):
• Python version used 3.7
• Operating system and version: Win10

[deepyaman]

@deepyaman Yeah, I think that sounds correct. If you have both versioning and overwrite on then it should check to see whether the specific timestamped directory you’re trying to save to already exists, and if it does then remove it. As you say, this will only actually delete anything if you’ve manually specified a version timestamp that already exists. It’s quite an edge case but this sounds safest overall rather than deleting the whole directory hierarchy.

https://github.com/quantumblacklabs/kedro/blob/ae80b129cb4f1973a554af964d48d8af0c355bb9/kedro/io/core.py#L582-L586 makes it seem that, by default, you couldn’t write to an already existing directory. In that case, rather than implementing custom logic, should we just raise a warning (something like “overwrite can’t be set when versioning”) and set overwrite to False?

[AntonyMilneQB]

@deepyaman Yeah, I think that sounds correct. If you have both versioning and overwrite on then it should check to see whether the specific timestamped directory you’re trying to save to already exists, and if it does then remove it. As you say, this will only actually delete anything if you’ve manually specified a version timestamp that already exists. It’s quite an edge case but this sounds safest overall rather than deleting the whole directory hierarchy.