MPS not working

[jakobmeusburger]

If specify mps as my --device option I get the following error:

Traceback (most recent call last):
  File "/Users/jakob/.pyenv/versions/3.10.13/bin/speechcatcher", line 8, in <module>
    sys.exit(main())
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/speechcatcher/speechcatcher.py", line 645, in main
    speech2text = load_model(tag=tag, device=args.device, beam_size=args.beamsize, quiet=quiet or progress, cache_dir=args.cache_dir)
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/speechcatcher/speechcatcher.py", line 74, in load_model
    return Speech2TextStreaming(**espnet_model_downloader.download_and_unpack(tag, quiet=quiet),
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/espnet_streaming_decoder/asr_inference_streaming.py", line 80, in __init__
    asr_model, asr_train_args = ASRTask.build_model_from_file(
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/espnet_streaming_decoder/espnet2/tasks/abs_task.py", line 1872, in build_model_from_file
    model.to(device)
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 857, in _apply
    self._buffers[key] = fn(buf)
  File "/Users/jakob/.pyenv/versions/3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

Python: 3.10.13
Torch: 2.1.2
Speechcatcher: 0.3.1

[jakobmeusburger]

My fault, thought in the documentation is written Mac M1 / mps support. But if I scroll to the right it it completes to Mac M1 / mps support isn't available yet.

When is it planned to be supported?

[bmilde]

See also: espnet/espnet#4710

Pytorch doesn't support all needed ops on the MPS device (yet). There's also couple of hard coded "cuda" / "cpu" checks in espnet that can be fixed, but I wasn't able to get correct outputs. That might also be due to faulty mps implementations somewhere in pytorch. Due to all the missing ops, inference is slower on MPS than on CPU, so I've given up on it.

[bmilde]

Note that speechcatcher doesn't depend on espnet anymore and uses a new distilled package https://github.com/speechcatcher-asr/espnet_streaming_decoder that only does inference for streaming models. If you want to retry my MPS experiment with a newer pytorch version, you would need to apply the fixes in https://github.com/speechcatcher-asr/espnet/tree/mac_m1_mps to https://github.com/speechcatcher-asr/espnet_streaming_decoder.