Skip to content

Proposal: Extended Channel Pixel Types for OpenEXR #2365

Description

@ssh4net

1. Objective

Extend OpenEXR channel typing from the current limited set to a general scalar type system, while preserving three guarantees:

  1. Existing EXR files remain fully readable.
  2. New EXR files with new scalar types are encoded so conforming older decoders reject them rather than silently misread them.
  3. Updated libraries can provide explicit compatibility conversion for legacy applications, but only inside updated libraries that understand the new format revision.

This is not a proposal to redefine EXR as a generic image container. It is a proposal to extend the numeric storage model in a controlled way.

2. Proposed Channel Types

Instead of adding only a few isolated types, define a coherent scalar family:

  • UINT8
  • UINT16
  • UINT32 (existing UINT)
  • INT8
  • INT16
  • INT32
  • HALF
  • FLOAT
  • DOUBLE

This is materially cleaner than an unsigned-only extension. Once 8-bit and 16-bit integers exist, signed integer support should be designed in immediately, otherwise the type system becomes asymmetric and the implementation work will be repeated later.

3. Non-Goals

This proposal does not define that integer channels are:

  • normalized
  • color-managed
  • display-ready
  • gamma-encoded
  • raw-camera-interpreted
  • video-range or full-range by default

The scalar type defines binary representation only. Interpretation remains separate.

4. Most Critical Requirement: Safe Behavior with Older Decoders

This is the central compatibility rule:

A decoder that does not understand the new scalar-type extension must fail deterministically before it allocates pixel buffers or attempts decode or transcode.

That must be guaranteed by the file format, not left to implementation quality.

4.1 Mandatory New File-Version Feature Flag

Introduce a new file-version feature bit for extended scalar channel types.

A file containing any of:

  • UINT8
  • UINT16
  • INT8
  • INT16
  • INT32
  • DOUBLE

must set this flag.

Recommended rule: the flag may also be set for future scalar-type extensions so the guard remains format-level, not enum-level.

This follows an existing OpenEXR compatibility pattern: version-field flags are already used to prevent older readers from misinterpreting files they cannot fully understand, for example the long-name extension.

4.2 Failure Behavior of Older Readers

Older readers that do not know this flag must reject the file as unsupported file version or unsupported feature, and must not proceed into header parse, channel setup, or pixel decode.

Deeper review of the OpenEXR codebase shows that the reference readers already fail closed in two relevant places:

  • unknown version flags are rejected before header parse
  • unknown channel pixel types are rejected during header validation rather than silently decoded

The new flag is still essential because it keeps the safety rule at the file-format level, mirrors the existing version-flag compatibility model, and avoids relying on every third-party reader independently validating channel enums correctly.

Without a required feature bit, non-conforming or custom decoders may:

  • treat unknown channel type as corrupt header and continue partially
  • map unknown type to an existing enum by accident
  • compute wrong bytes per sample
  • under-allocate or overrun buffers
  • miscompute chunk strides
  • trigger leaks during partial error unwinding
  • decode garbage while appearing successful

This proposal requires that silent fallback by decoders that do not support the extension be forbidden by construction.

5. Decoder Safety Model

5.1 Old Decoders

Old decoders must not attempt compatibility conversion. They do not understand the new types, so the only safe behavior is early rejection.

5.2 Updated Decoders

Only a decoder that explicitly supports the new file-version feature may perform:

  • raw decode of new types
  • explicit conversion to legacy types for application compatibility

Compatibility conversion is therefore a feature of updated libraries, not of legacy decoders.

6. Library Compatibility for Legacy Applications

A new OpenEXR library may support old applications through explicit compatibility policy.

That means:

  • the library understands the new file format
  • the application may only understand old destination types
  • the library performs requested conversion safely and deliberately

Possible policy modes:

  • Error — reject unsupported channel types
  • Hide — omit unsupported channels from the exposed image view
  • Convert — convert unsupported on-disk channel types to application-supported destination types

The default for a modern low-level API should be Error, not implicit conversion.

The default for legacy convenience APIs may be Convert, but only in APIs already defined as conversion-oriented.

7. API Model: Separate Physical Type from Delivered Type

The API must not blur these concepts.

Each channel should expose at least:

  • filePixelType — actual on-disk type
  • requestedPixelType — destination type requested by caller, if any
  • deliveredPixelType — actual type delivered after conversion

This avoids the dangerous model where the library lies about the stored type. Legacy code can still operate through converted buffers, but modern code can always inspect the true file representation.

8. Mandatory Conversion Specification

If new types are added, conversion semantics must be fully specified in the format and library contract. Otherwise the C path, C++ path, Python bindings, and codec-specific code will drift.

8.1 Integer to Integer

  • exact if representable
  • otherwise clamp to destination range
  • never wrap
  • never reinterpret sign bit

Examples:

  • INT16 -> UINT8: clamp to [0, 255]
  • UINT16 -> INT16: clamp to [0, 32767]
  • INT32 -> INT16: clamp to [-32768, 32767]

8.2 Integer to Float

  • numeric conversion only
  • no implicit normalization
  • exact where representable, otherwise nearest representable value

8.3 Float to Integer

  • NaN converts to 0
  • +inf clamps to destination maximum
  • -inf clamps to destination minimum for signed integers, or 0 for unsigned integers
  • finite values truncate toward zero
  • then clamp to destination range

This preserves existing OpenEXR half/float -> uint conversion behavior for legacy conversion-oriented APIs.

8.4 Float Family Conversions

  • HALF, FLOAT, and DOUBLE follow ordinary representational conversion
  • NaN is preserved where supported
  • infinity is preserved where supported
  • overflow follows target format behavior

8.5 No Metadata-Driven Numeric Conversion

Optional metadata such as min and max, offset, or normalization hints must not alter these core conversion rules. That belongs to interpretation and display logic, not decode primitives.

9. Interpretation Metadata

To resolve the question of what integer values mean, keep meaning separate from storage.

Without metadata, integer channels are only integer channels.

Optional metadata may describe intended interpretation, for example:

  • sampleInterpretation
    • generic
    • normalized
    • linear
    • nonlinear
    • depth
    • mask
    • label
    • raw_sensor
  • valueRangeMin
  • valueRangeMax
  • offset
  • blackLevel
  • whiteLevel
  • colorSpace
  • transferFunction
  • activeBits

These attributes are advisory. They never change decode validity.

Because this metadata is advisory rather than decode-critical, it does not need the new scalar-type feature bit. Older readers already preserve unknown attributes opaquely.

10. Implementation Requirements

10.1 Remove All Size Assumptions

The codebase must no longer assume:

  • HALF == 2 bytes
  • everything else equals 4 bytes

A single authoritative function or table must define byte size for every pixel type:

  • UINT8 -> 1
  • INT8 -> 1
  • UINT16 -> 2
  • INT16 -> 2
  • HALF -> 2
  • UINT32 -> 4
  • INT32 -> 4
  • FLOAT -> 4
  • DOUBLE -> 8

All packing, unpacking, chunk iteration, slice validation, and buffer stride calculations must use that table.

10.2 Replace Trio-Specific Logic with Generic Dispatch

The current implementation often assumes a small hardcoded pixel-type matrix. That must be replaced by:

  • centralized type traits
  • centralized size lookup
  • centralized conversion functions
  • codec-independent scalar-type handling where possible

This matters for correctness and for long-term maintainability.

11. Compression Support

Compression support must be explicit per codec.

Before codec rollout, the generic container plumbing has to be generalized. The current C and C++ implementations still hardcode the three-type model in public enums, frame-buffer defaults, chunk sizing, and encode/decode pipeline negotiation. Therefore codec enablement must come after refactoring the common type and size infrastructure, not before it.

11.1 Rule

A codec either:

  • supports a scalar type correctly, or
  • rejects that scalar type with a clear error

It must never reinterpret a new type through an old path silently.

11.2 Rollout Strategy

The safest plan is staged support.

Phase 1

Allow new scalar types only for codecs validated to be element-size and type safe, for example:

  • uncompressed
  • RLE
  • ZIP
  • ZIPS
  • possibly PIZ after audit

Phase 2

Add audited support for more specialized codecs:

  • PXR24
  • HTJ2K
  • DWA

These are likely blockers because they often encode assumptions about type class, signedness, or float behavior.

11.3 PXR24 Note

PXR24 may not be sensible for every new scalar type. It is acceptable for the specification or library to reject unsupported type and codec combinations rather than forcing universal support.

12. Deep Data

Deep sample counts remain UINT32.

This should be stated explicitly. The scalar-type extension must not alter the semantics of deep sample count channels.

Deep sample values may be evaluated separately, but sample counts are fixed.

13. Required Safety Behavior During Decode

A robust design should also prescribe decoder behavior, not only file syntax.

13.1 Preflight Validation Before Allocation

Before allocating per-channel decode buffers, the decoder must validate:

  • file-version feature bits
  • channel type enum validity
  • bytes per sample for all channel types
  • chunk size consistency
  • slice stride and bounds
  • codec support for each channel type
  • total buffer size overflow

Only after successful validation may allocation and decode proceed.

13.2 Fail Closed on Unsupported Types

If any channel type is unsupported in the current mode:

  • low-level API: fail before decode
  • compatibility API: fail, hide, or convert according to explicit policy

The decoder must never continue with guessed interpretation.

13.3 Memory Safety

The proposal should also recommend RAII and structured cleanup discipline in all new paths so that early errors in extended-type decode do not leak partially allocated state.

This is implementation guidance, but it is relevant because wrong bytes-per-sample handling is exactly the kind of issue that produces buffer corruption and cleanup bugs.

14. Backward Compatibility Matrix

Existing Files with Old Types

  • readable by old libraries
  • readable by new libraries

New Files with Extended Scalar Types

  • rejected early by conforming old libraries due to the file-version feature bit
  • readable by new libraries
  • convertible by new libraries for legacy applications if policy allows

Old Applications on Top of a New Library

Possible through explicit compatibility conversion, for example:

  • UINT8, UINT16, INT8, INT16, INT32 -> UINT32, FLOAT, or HALF where requested
  • DOUBLE -> FLOAT or HALF where requested

But this is a property of the new library, not of old decoders.

15. Recommended Design Position

The strongest technical position is:

OpenEXR should generalize scalar channel typing rather than add isolated special cases. The extension should include signed integers from the beginning, because once 8-bit and 16-bit integer storage is admitted, signed and unsigned integer support belongs to the same architectural change. However, backward safety is more important than type breadth. Therefore, the format must introduce a mandatory new file-version feature flag that forces conforming older decoders to reject extended-type files before decode begins. Compatibility conversion is allowed only in updated libraries that understand the extension and can apply explicit policy safely.

16. Compatibility Limits for Historical Readers

The requirement for older readers to reject extended-type files must be understood as a format-level compatibility rule, not as a claim that already-shipped binaries can be fixed retroactively.

If a historical OpenEXR library is already deployed and the user has no access to its source code, its behavior cannot be changed. Therefore, the proposal cannot guarantee safe rejection for every old binary ever shipped. It can only guarantee that extended-type files are encoded in a way that conforming older readers are expected to reject before decode.

This means:

  • files using extended scalar types must set a mandatory file-version feature flag
  • the flag must live in an extension point that readers check before channel-type-dependent allocation or decode
  • updated readers must reject files that use extended scalar types without this flag
  • compatibility with closed-source or non-conforming historical binaries is necessarily best-effort only

In practice, historical readers fall into three categories:

  1. Readers that validate unknown required file-version features early. These will reject extended-type files safely.
  2. Readers that reject such files during early version or header validation, even if the error message is generic. This is still acceptable if failure occurs before channel decode.
  3. Readers that ignore unknown required features or proceed far enough to mis-handle new channel types. These cannot be made safe without changing the binary.

The OpenEXR reference implementation is in categories 1 and 2 here, not category 3: current and historical releases already reject unknown required version flags, and the reference readers also reject invalid channel pixel types during header validation.

Accordingly, the proposal should make the following narrower and technically accurate claim:

“Files using extended scalar channel types must set a mandatory file-version feature flag that conforming readers check before decoding. Readers that do not support this feature must reject the file. This guarantees safe failure for conforming older implementations, but cannot retroactively correct historical binaries that ignore required feature flags or otherwise violate forward-compatibility rules.”

A direct consequence is that extended scalar types must not be introduced merely as additional pixelType enum values inside an otherwise old-format file. They must be guarded by a top-level required feature indicator encountered before channel parsing, allocation, and decode logic.

17. Concise RFC-Style Conclusion

OpenEXR is extended to support a generalized scalar channel type system consisting of unsigned integers, signed integers, and floating-point types, including UINT8, UINT16, INT8, INT16, INT32, and DOUBLE, in addition to existing UINT, HALF, and FLOAT. Files using any extended scalar type must set a new file-version feature flag so that decoders lacking support reject such files deterministically and before pixel decode in conforming implementations. Updated libraries may provide explicit compatibility conversion for legacy applications, but only after validating the new file feature and actual on-disk channel types. Scalar type defines numeric storage only; interpretation such as normalization, display intent, raw-sensor semantics, or color meaning remains optional metadata and is not implied by the type itself. Compatibility with non-conforming historical binaries remains best-effort only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions