Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Would anyone consider a diffusers export_to_frames utility fuction? #10683

Open
lovetillion opened this issue Jan 29, 2025 · 0 comments
Open

Comments

@lovetillion
Copy link

lovetillion commented Jan 29, 2025

Is your feature request related to a problem? Please describe.
The current export_to_video function in Hugging Face's Diffusers library exports a compressed video, but it's not straightforward for users to obtain raw, lossless PNG frames from a list of frames. This can be a problem for users who need to work with individual frames or want to export them in a specific format as part of a workflow.

Describe the solution you'd like.
I propose introducing a new function, export_to_frames, in huggingface/diffusers/utils/export_utils.py. This function would take a the frames (either NumPy arrays or PIL Image objects) and export each frame as a separate PNG file in a specified output directory. The function would also allow users to specify the frame rate and output directory.

Describe alternatives you've considered.
While users can currently solve this problem on their own by using other libraries or writing custom code, it would be beneficial to provide a simple and standard method for exporting raw, uncompressed PNG frames. This would save users time and effort, and make the Diffusers library more user-friendly.

Additional context.
I've included very rough example implementation of the proposed export_to_frames function below:

`
def export_to_frames(
video_frames: Union[List[np.ndarray], List[PIL.Image.Image]], output_dir: str = None, fps: int = 10
) -> str:
"""
Export each frame in a list of frames to a directory.

Args:
    video_frames (Union[List[np.ndarray], List[PIL.Image.Image]]): A list of frames.
    output_dir (str, optional): The directory where the frames will be saved. Defaults to None.
    fps (int, optional): The frame rate. Defaults to 10.

Returns:
    str: The path to the output directory.
"""

try:
    imageio.plugins.ffmpeg.get_exe()
except AttributeError:
    raise AttributeError(
        (
            "Found an existing imageio backend in your environment. Attempting to export frames with imageio. \n"
            "Unable to find a compatible ffmpeg installation in your environment to use with imageio. Please install via pip install imageio-ffmpeg"
        )
    )
print( "video_frames",len(video_frames) )

if isinstance(video_frames[0], np.ndarray):
    print( "numpy")
    video_frames = [(frame * 255).astype(np.uint8) for frame in video_frames]

elif isinstance(video_frames[0], PIL.Image.Image):
    print( "PIL")
    video_frames = [np.array(frame) for frame in video_frames]

print( "video_frames",len(video_frames) )

for i, frame in enumerate(video_frames):
    print( "frame", i )
    filename = f"frame_{i:04d}.png"
    if isinstance(frame, np.ndarray):
        print("wrote via np")
        imageio.imwrite(os.path.join(output_dir, filename), frame)
    elif isinstance(frame, PIL.Image.Image):
        print("wrote via PIL")
        frame.save(os.path.join(output_dir, filename))

return output_dir`

This rough function was tested briefly but should be rewritten I'm just using it for illustrative purposes since it worked. Please let me know if this idea is worth considering further and if we could proceed with something like this in the standard utilities in future?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant