Add libass native decoder #2324
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The fonts are sent to the renderer via the format metadata.
This news native decoder use libass to parse and render the ssa/ass subtitle. Co-authored-by: Vincent Zauhar <[email protected]> Co-authored-by: SonnyUplavan <[email protected]> Co-authored-by: Raphael Dupuis <[email protected]>
The "VSFilter mangled colors" is requested by libass. See: https://github.com/libass/libass/blob/5b44c4d690254a7bd8de2e048fa23820036b671b/libass/ass_types.h#L157-L262 The math formula are from: https://en.wikipedia.org/wiki/YCbCr#R'G'B'_to_Y%E2%80%B2PbPr Co-authored-by: Vincent Zauhar <[email protected]>
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
I'm afraid we are unlikely to merge this PR as the maintenance burden is likely to be too high for the benefit it brings, as I previously mentioned in this comment: google/ExoPlayer#8435 (comment) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new decoder for
.asssubtitles to fix this issue: google/ExoPlayer#8435It also partially fixes the following issue: #691
It is currently a draft because the following issues still need to be addressed:
.asssubtitles muxed in MKV files. It would also needs to support "raw" .ass file.AssRendereris added inSubtitleTranscodingTrackOutput.javaandDefaultRenderersFactory.javais hacky.draw_ass_rgbais too slow for 4K (3840x2160) bitmaps where the subtitle completely covers the screen. Neon optimization is needed (or maybe openmp?).Note: We are no longer working on this PR. If anyone wants to use it as a base, feel free to improve it and open a non-draft PR.
Why should Media3 provide a native decoder using libass?
It’s practically impossible to replicate libass behavior accurately.
The first commit that introduced SSA/ASS subtitle support in Media3 is this one, made in 2017 — that was 8 years ago, and Media3 still lacks support for many elements of the ASS/SSA spec.
Here's a screenshot showing the video with and without subtitles. Implementing this in SsaParser.java is far from trivial.
Also, here’s a video that contains a
.asssubtitle (hardsubbed). The subtitle is the text: "CIBLE DE LA QUÊTE".https://github.com/user-attachments/assets/af2b54b5-baf8-41d9-bc05-f97702757402
Again, implementing all the feature in java needed just to display the subtitle accuratly is basically impossible.
Finally, note that this PR is the work of Raphael Dupuis, Sonny Uplavan, Vincent Zauhar and me.