cannot playback webcam on MacOS

[gabor-borbely]

• OS: MacOS Sequoia
• hardware: MacBookPro, with Apple M1 Pro chip
• python==3.12 (via homebrew)
• numpy==2.2.4
• opencv-python==4.11.0.86
• pyav==14.2.1
• ffmpeg==7.1.1 (via homebrew)

The following code has a strange behaviour:

import av
import cv2 as cv

with av.open(
   '0',
   format='avfoundation',
   options={"framerate":"30"}
) as container:
   for frame in container.decode(video=0):
       cv.imshow('frame', frame.to_rgb().to_ndarray()[:, :, ::-1])
       if cv.waitKey(1) != -1:
           break

cv.destroyAllWindows()

displays a couple of frames then shuts down with error

Traceback (most recent call last):
  File "/Users/gabor-borbely/test.py", line 9, in <module>
    for frame in container.decode(video=0):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "av/container/input.pyx", line 215, in decode
  File "av/container/input.pyx", line 174, in demux
  File "av/container/core.pyx", line 303, in av.container.core.Container.err_check
  File "av/error.pyx", line 423, in av.error.err_check
av.error.BlockingIOError: [Errno 35] Resource temporarily unavailable: '0'

I tried with OpenCV (works as expected):

from contextlib import contextmanager

import cv2 as cv


def playback(*args, **kwargs):
    with video_capture(*args, **kwargs) as cap:
        while True:
            ret, cv_frame = cap.read()
            if not ret:
                break
            timestamp = int(cap.get(cv.CAP_PROP_POS_MSEC))
            yield timestamp, cv_frame


@contextmanager
def video_capture(*args, **kwargs):
    cap = cv.VideoCapture(*args, **kwargs)
    try:
        if not cap.isOpened():
            raise ValueError(f"Cannot open {args=}, {kwargs=}")
        yield cap
    finally:
        cap.release()

for ts, frame in playback(0+cv.CAP_AVFOUNDATION):
    cv.imshow('frame', frame)
    if cv.waitKey(1) != -1:
        break

cv.destroyAllWindows()

Also the CLI ffmpeg works:

ffmpeg -f avfoundation -framerate 30 -video_size 1280x720 -i 0 test.mpg

[zak-45]

Maybe this should provide some help, as look similar problem : macOS screen capture

[gabor-borbely]

This "works". Starts a new recording after every 4-6 frames. In this way the program doesn't fail explicitly but the playback is choppy 😿
By the way, I could use the screen capture device without this hack. My problem appeared with the webcam.