Description
Is your feature request related to a problem? Please describe.
The current export_to_video function in Hugging Face's Diffusers library exports a compressed video, but it's not straightforward for users to obtain raw, lossless PNG frames from a list of frames. This can be a problem for users who need to work with individual frames or want to export them in a specific format as part of a workflow.
Describe the solution you'd like.
I propose introducing a new function, export_to_frames, in huggingface/diffusers/utils/export_utils.py. This function would take a the frames (either NumPy arrays or PIL Image objects) and export each frame as a separate PNG file in a specified output directory. The function would also allow users to specify the frame rate and output directory.
Describe alternatives you've considered.
While users can currently solve this problem on their own by using other libraries or writing custom code, it would be beneficial to provide a simple and standard method for exporting raw, uncompressed PNG frames. This would save users time and effort, and make the Diffusers library more user-friendly.
Additional context.
I've included very rough example implementation of the proposed export_to_frames function below:
`
def export_to_frames(
video_frames: Union[List[np.ndarray], List[PIL.Image.Image]], output_dir: str = None, fps: int = 10
) -> str:
"""
Export each frame in a list of frames to a directory.
Args:
video_frames (Union[List[np.ndarray], List[PIL.Image.Image]]): A list of frames.
output_dir (str, optional): The directory where the frames will be saved. Defaults to None.
fps (int, optional): The frame rate. Defaults to 10.
Returns:
str: The path to the output directory.
"""
try:
imageio.plugins.ffmpeg.get_exe()
except AttributeError:
raise AttributeError(
(
"Found an existing imageio backend in your environment. Attempting to export frames with imageio. \n"
"Unable to find a compatible ffmpeg installation in your environment to use with imageio. Please install via pip install imageio-ffmpeg"
)
)
print( "video_frames",len(video_frames) )
if isinstance(video_frames[0], np.ndarray):
print( "numpy")
video_frames = [(frame * 255).astype(np.uint8) for frame in video_frames]
elif isinstance(video_frames[0], PIL.Image.Image):
print( "PIL")
video_frames = [np.array(frame) for frame in video_frames]
print( "video_frames",len(video_frames) )
for i, frame in enumerate(video_frames):
print( "frame", i )
filename = f"frame_{i:04d}.png"
if isinstance(frame, np.ndarray):
print("wrote via np")
imageio.imwrite(os.path.join(output_dir, filename), frame)
elif isinstance(frame, PIL.Image.Image):
print("wrote via PIL")
frame.save(os.path.join(output_dir, filename))
return output_dir`
This rough function was tested briefly but should be rewritten I'm just using it for illustrative purposes since it worked. Please let me know if this idea is worth considering further and if we could proceed with something like this in the standard utilities in future?