MasterMob Transcribing Changes #44
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Mark Reid <[email protected]>
Signed-off-by: Mark Reid <[email protected]>
- Only MasterMob tracks can become otio clips - CompositionMob referenced by SourceClip become Stacks - SourceMobs become MediaReferences - More aggressively simpify - Cache prevously processed Mobs Signed-off-by: Mark Reid <[email protected]>
- OperationGroup metadata only stored on effect - Improved naming of OperationsGroups Signed-off-by: Mark Reid <[email protected]>
Signed-off-by: Mark Reid <[email protected]>
Signed-off-by: Mark Reid <[email protected]>
|
@markreidvfx does this PR supersede this old one? AcademySoftwareFoundation/OpenTimelineIO#1063 |
|
FYI, we spent some time comparing new/old results for the time warp test suite, and some simple production examples. The changes made to the timecode look great, and the changes to the media references and metadata seem easy to accommodate in our (Pixar) pipeline. We need to run some more complex examples to understand the nesting changes, and we plan to run a large set of older production example AAFs through to look for any problems, but in general this new approach looks great. In this week's OTIO TSC meeting, I believe that @reinecke said Netflix is already using this updated code in their pipeline with good results. Thanks @markreidvfx! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR proposes an overhaul to how the adapter transcribes AAF files. This CHANGES the structure of the OTIO file being generated in a few ways. This WILL most likely break tools rely on the old metadata structure and therefore should be a major version increase. I believe this new structure is a better mapping of the AAF to OTIO.
Improvements included:
walkmethod. Files that previously failed due to this should work nowThis rework follows a few guiding principals. ONLY SourceClips on MasterMobs become OTIO Clips. SourceClips on SourceMobs become MediaReferences.
Here a Diagram of transcribing a MasterMob hierarchy. It shows what AAF objects get converted to what OTIO objects.
Only SourceClips on MasterMobs become OTIO Clips
ONLY SourceClips on MasterMobs become OTIO Clips.
There are strict rules in the AAF Spec that define what MasterMob slots can contain. They are only allowed to contain one or more EssenceGroups or SourceClips. These components are often inside a Sequence object. I've seen some case where MasterMobs contain Filler even though not in the spec. This PR should handle all these cases.
A SourceClip in CompositionsMob is only allowed to reference slots on MasterMobs (or another CompositionsMob in some cases). They aren't allowed to reference SourceMobs. When transcribing a CompositionMob and a SourceClip is encountered, the referenced MasterMob is completely converted into a separate OTIO Timeline. Then track they are referencing by SourceClip's slot_id is the cloned into the timeline being built. The clip(s) on the cloned track will contain all the MasterMob's metadata.
With the previous behaviour, it is a little messy on what becomes a OTIO Clip. Typically the MasterMob metadata would end up on a MediaReference. Now MasterMob metadata will ALWAYS be on the OTIO Clip. This makes it very simple and consistent to find UserComments and other bin metadata.
SourceClips on SourceMobs become MediaReferences
A SourceMob referenced by a MasterMob now becomes its own media reference. More SourceMob metadata is persevered including
EssenceDescriptorobjects. ANetworkLocatorwill get converted to atarget_url. If the SourceMobs EssenceDescriptor doesn't have aNetworkLocatora OTIOMissingReferenceis created.Each MediaReference has a
available_rangebase on the timecode found on the SourceMobs. This timecode propagated back to the OTIO Clip, so it too should have the correct start timecode.I'm also converting the MasterMob's
UNC PathUserComment into a MediaReference too because this was the old behaviour, I'm considering removing that behaviour or changing it to an option. I've seen people use other bin metadata to specify media reference urls, it might be more useful for this to a user supplied list of keys to look for additional urls.SubClip CompositionMobs become stacks
A SourceClip in a CompositionMob can also reference another CompositionMob. This is often referred to as a SubClip. The Mob these SourceClips reference are converted to a OTIO Timelines and the referenced Track gets cloned in a similar fashion to MasterMobs.
A SubClip can contain useful user metadata. To preserve this data, SourceClips that reference other CompostionMobs become OTIO Stacks. Special care is also taken to preserve any Markers found SubClips.
If the SubClip has only have 1 track they could be flatten to Track instead of Stack, but I choose not to do this. I was thinking it was better keep SubClips consistent.
Changes to OperationGroup
The other structural change that might break things is all OperationGroup metadata is now stored in the OTIO Effect. This allows flattening to preserve all effect data. OperationGroup are only flattened if their effect has one input. If the effect requires multiple inputs they remain a OTIO Stack.
Changes to Simplify
More attempts are made to simplify the file after transcribing is complete. This includes some additional flattening of OTIO tracks and combining effects if possible. Simplify also makes sure all stacks have tracks. I still think this code is pretty complex since it involves a lot of recursion. More testing and documenting is required here.
Still TODO
ComponentAttributeListClip Example
Here is a small condensed example from ALab of what the changes look like to an OTIO Clip.
Old Structure
{ "OTIO_SCHEMA": "Clip.2", "metadata": { "AAF": { "ClassName": "SourceClip", "ComponentAttributeList": { "_IMAGE_BOUNDS_OVERRIDE": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\" ?><Bounds> <Framing>-800/1 -450/1 1600/1 900/1</Framing> <Valid>-800/1 -450/1 1600/1 900/1</Valid> <Essence>-800/1 -450/1 1600/1 900/1</Essence> <Source>-800/1 -450/1 1600/1 900/1</Source></Bounds>" }, "DataDefinition": { "Description": "Picture data", "Identification": "01030202-0100-0000-060e-2b3404010101", "Name": "Picture" }, "Length": 34, "Name": "mk020_0060", "SourceID": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.f5e701d0.98459605.de43f493.9ff4a1bd", "SourceMobSlotID": 1, "SourceMobUsage": "", "StartTime": 0 } }, "name": "mk020_0060", "source_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "effects": [], "markers": [], "enabled": true, "media_references": { "DEFAULT_MEDIA": { "OTIO_SCHEMA": "ExternalReference.1", "metadata": { "AAF": { "ClassName": "MasterMob", "ConvertFrameRate": false, "CreationTime": "2022-09-23 05:23:43", "LastModified": "2022-09-23 07:30:31", "MobAttributeList": { "_COLOR_B": 38144, "_COLOR_G": 11776, "_COLOR_R": 14592, "_GEN": 1663918231, "_IMPORTSETTING": "__AttributeList", "_USER_POS": 0 }, "MobID": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.f5e701d0.98459605.de43f493.9ff4a1bd", "Name": "mk020_0060", "Slots": {}, "UserComments": { "UNC Path": "C:\\Users\\nathanla\\Desktop\\mk020_shots\\mk020_0060\\mk020_minicut_comp_scene_cut_intentcut_v001.0561.png" } } }, "name": "", "available_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "available_image_bounds": null, "target_url": "file://C:/Users/nathanla/Desktop/mk020_shots/mk020_0060/mk020_minicut_comp_scene_cut_intentcut_v001.0561.png" } }, "active_media_reference_key": "DEFAULT_MEDIA" }New Structure
{ "OTIO_SCHEMA": "Clip.2", "metadata": { "AAF": { "ClassName": "MasterMob", "ConvertFrameRate": false, "CreationTime": "2022-09-23 05:23:43", "LastModified": "2022-09-23 07:30:31", "MediaKind": "Picture", "MobAttributeList": { "_COLOR_B": 38144, "_COLOR_G": 11776, "_COLOR_R": 14592, "_GEN": 1663918231, "_IMPORTSETTING": "__AttributeList", "_USER_POS": 0 }, "MobID": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.f5e701d0.98459605.de43f493.9ff4a1bd", "Name": "mk020_0060", "Slots": {}, "UserComments": { "UNC Path": "C:\\Users\\nathanla\\Desktop\\mk020_shots\\mk020_0060\\mk020_minicut_comp_scene_cut_intentcut_v001.0561.png" } } }, "name": "mk020_0060", "source_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "effects": [], "markers": [], "enabled": true, "media_references": { "DEFAULT_MEDIA": { "OTIO_SCHEMA": "ExternalReference.1", "metadata": {}, "name": "UNC Path", "available_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "available_image_bounds": null, "target_url": "file://C:/Users/nathanla/Desktop/mk020_shots/mk020_0060/mk020_minicut_comp_scene_cut_intentcut_v001.0561.png" }, "mk020_minicut_comp_scene_cut_intentcut_v001.0561.png": { "OTIO_SCHEMA": "ExternalReference.1", "metadata": { "AAF": { "ClassName": "SourceMob", "CreationTime": "2022-09-23 05:40:44", "EssenceDescription": { "ClassName": "ImportDescriptor", "Locator": {} }, "LastModified": "2022-09-23 05:40:44", "MobAttributeList": { "_PJ": "Polaris" }, "MobID": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.f8ecb58d.98459605.c930f493.9ff4a1bd", "Name": "mk020_minicut_comp_scene_cut_intentcut_v001.0561.png", "Slots": {} } }, "name": "mk020_minicut_comp_scene_cut_intentcut_v001.0561.png", "available_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "available_image_bounds": null, "target_url": "file:////C/Users/nathanla/Desktop/mk020_shots/mk020_0060/mk020_minicut_comp_scene_cut_intentcut_v001.0561.png" }, "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.01c3e84a.98459605.b414f493.9ff4a1bd": { "OTIO_SCHEMA": "ExternalReference.1", "metadata": { "AAF": { "ClassName": "SourceMob", "CreationTime": "2022-09-23 07:30:31", "EssenceDescription": { "BlackReferenceLevel": 16, "ClassName": "CDCIDescriptor", "CodingEquations": "CodingEquations_ITU709", "ColorPrimaries": "ColorPrimaries_ITU709", "ColorRange": 225, "ColorSiting": "CoSiting", "ComponentWidth": 8, "Compression": "04010202-7113-0000-060e-2b340401010a", "DataOffset": 393216, "DisplayHeight": 1080, "DisplayWidth": 1920, "DisplayXOffset": 0, "DisplayYOffset": 0, "EssenceBox": { "Height": "900", "PositionX": "-800", "PositionY": "-450", "Width": "1600" }, "FrameIndexByteOrder": 18761, "FrameLayout": "FullFrame", "FrameSampleSize": 0, "HorizontalSubsampling": 2, "ImageAlignmentFactor": 4096, "ImageAspectRatio": "16/9", "ImageSize": 6406144, "Length": 34, "Locator": {}, "OffsetToFrameIndexes64": 6541799, "ResolutionID": 1253, "SampleRate": "24", "SampledHeight": 1080, "SampledWidth": 1920, "SampledXOffset": 0, "SampledYOffset": 0, "SourceBox": { "Height": "900", "PositionX": "-800", "PositionY": "-450", "Width": "1600" }, "StoredHeight": 1080, "StoredWidth": 1920, "TransferCharacteristic": "TransferCharacteristic_ITU709", "ValidBox": { "Height": "900", "PositionX": "-800", "PositionY": "-450", "Width": "1600" }, "VerticalSubsampling": 1, "VideoLineMap": {}, "WhiteReferenceLevel": 235 }, "LastModified": "2022-09-23 07:30:31", "MobAttributeList": { "_ORIGINAL_FILEMOB_ID": "060a2b340101010501010f1013-000000-f5e7025598459605-a27cf4939ff4-a1bd", "_PJ": "ALAB_project" }, "MobID": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.01c3e84a.98459605.b414f493.9ff4a1bd", "Name": "", "Slots": {} } }, "name": "urn:smpte:umid:060a2b34.01010105.01010f10.13000000.01c3e84a.98459605.b414f493.9ff4a1bd", "available_range": { "OTIO_SCHEMA": "TimeRange.1", "duration": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 34.0 }, "start_time": { "OTIO_SCHEMA": "RationalTime.1", "rate": 24.0, "value": 1002.0 } }, "available_image_bounds": null, "target_url": "file://Pixar-Nexis/Tools_Media/Avid%20MediaFiles/MXF/octave.2/V01.632D60B2_1C3E801C3E84AV.mxf" } }, "active_media_reference_key": "DEFAULT_MEDIA" }