SmartCache Wrapper Component

Is your feature request related to a problem? Please describe. To leverage SmartCache, what user need to do is simply wrap a Dataset with SmartCacheDataset as below. However, it is a repeating task for any Dataset in the exact same way.

class SmartCacheMyDataset(SmartCacheDataset):
    """
    Add SmartCache functionality to MyDataset
    """
    def __init__(
        self,
        data,
        other_parameters,
        transform,
        replace_rate,
        cache_num,
        cache_rate,
        num_init_workers,
        num_replace_workers,
    ):
        extractor = MyDataset(data, other_parameters)
        super().__init__(
            data=extractor,
            transform=transform,
            replace_rate=replace_rate,
            cache_num=cache_num,
            cache_rate=cache_rate,
            num_init_workers=num_init_workers,
            num_replace_workers=num_replace_workers,
        )

Describe the solution you’d like It would be nice to have SmartCache wrapper (as a handler, transformer, or anything else) that can be added into the pipeline to leverage SmartCache functionality without the need to write a new dataset. In this way, SmartCache can be turned on and off by just adding or removing this component without changing any other part of the code.

I think the wrapper itself is easy and don’t have much more than what I described above but the only challenge is that it should be called either before DataLoader or inside it.

[Nic-Ma]

Hi @behxyz ,

Already added to our internal Clara roadmap.

Thanks.

[drbeh]

Hi @Nic-Ma , I agree, it would be great if we can add this feature request to the Clara Train roadmap.

Thanks