some accelerator enhancements

Hi,

My main code base have been switched to be based on accelerate for a while, it works grate! Thank you guys so much!

There are a few things I think it’s a common need.

The first is .distsibuted attributes, which is basically .num_processes > 1. I mean, it’s not a lot of works but … I’m an extremely lazy person.

def distsibuted(self):
    """
    Runs in distributed mode
    """
    return self.num_processes > 1

Besides, these two decorators are often used in my code when saving models and downloading datasets, I think it could be useful.

def on_main_process(func):
    """
    Run func on main process
    """
    @wraps(func)
    def wrapper(self, *args, **kwargs):
        if self.is_main_process or not self.distributed:
            return func(self, *args, **kwargs)
    return wrapper


def on_local_main_process(func):
    """
    Run func on local main process
    """
    @wraps(func)
    def wrapper(self, *args, **kwargs):
        if self.is_local_main_process or not self.distributed:
            return func(self, *args, **kwargs)
    return wrapper

[sgugger]

I don’t mind having wrappers on the Accelerator for those. As long as the naming is clear, it’s not a bad thing to have multiple properties/methods/context managers for different use cases 😃

[ZhiyuanChen]

I believe everything has been merged now, so I’m gonna close this issue.

Thanks again for this grate work!