Add support for MacOS Apple Metal "mps" backend

System Info

MacOS, M1 architecture, Python 3.10, Pytorch 1.12 nightly, Transformers latest (4.19.2)

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, …)
• My own task or dataset (give details below)

Reproduction

Try to set backend to newly released “mps” backend (Apple Metal) in Pytorch.

from transformers import pipeline

classifier = pipeline("sentiment-analysis")
classifier.device = "mps"

classifier("We are very sad to mps backend is not supporter in Transformers.")

Expected behavior

Transformers should run on the GPU.

Instead, an error is thrown.

File ~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/transformers/pipelines/base.py:826, in Pipeline.device_placement(self)
    824         yield
    825 else:
--> 826     if self.device.type == "cuda":
    827         torch.cuda.set_device(self.device)
    829     yield

AttributeError: 'str' object has no attribute 'type'

[Narsil]

Hi, you can now do:

classifier = pipeline("sentiment-analysis", device=torch.device("mps"))

Not sure how/when TF is going to add support, but we’ll figure out a way to enable this cross library too afterwards.

[danieldk]

>>> classifier = pipeline("sentiment-analysis")
>>> classifier.device = torch.device("mps")
>>> classifier.model.to("mps")
>>> classifier("We are very sad to mps backend is not supporter in Transformers.")
[{'label': 'NEGATIVE', 'score': 0.5000945329666138}]

Unfortunately, as evidenced in the output, the PyTorch MPS backend is still very much broken. Even a lot of basic operations do not work correctly. For example:

>>> torch.arange(10, device="mps")
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], device='mps:0')
>>> torch.ones(10, device="mps").type(torch.int32)
tensor([1065353216, 1065353216, 1065353216, 1065353216, 1065353216, 1065353216,
        1065353216, 1065353216, 1065353216, 1065353216], device='mps:0',
       dtype=torch.int32)

So, it’s unlikely that you can use it yet, until the Torch maintainers shake out some bugs.