Whisper Redesigned Solution

[kunal-vaishnavi]

Description

This PR re-designs how Whisper is created and supported in ONNX Runtime GenAI. The new solution is designed to be used in conjunction with this work in ONNX Runtime.

Some of the added changes include:

• Re-designed GenAI config that separates the encoder model and decoder model
  • Removes the encoder-decoder-init section
  • Creates a new encoder section
  • Separates session options, EP options, and model properties to be per-model instead of re-using the decoder's options for all components
  • Re-assigns pre-computed cross-attention KV caches as outputs to encoder model instead of inputs to decoder model
• Re-designed runtime support that makes the states and steps much clearer
  • Creates AudioEncoder, WhisperDecoder (i.e. TextDecoder), and WhisperState as separate states
  • Creates AudioFeatures class that can be re-used for other speech models
  • Adds generic support for FP32 CPU, FP32 CUDA, FP16 CUDA, and any quantized versions
  • Removes temporary workarounds for past-present buffer sharing due to restrictions from both the exported ONNX model and ONNX Runtime
  • Handles models with and without the following: buffer sharing, DecoderMaskedMultiHeadAttention, and alignment heads

Known Issues

• This branch still has to be synced with the latest changes in the main branch of ONNX Runtime GenAI and active dev branches that will materially change this PR.
• The cross QK kernels do not have parity with the alternative, more-accurate approach to compute the cross QKs as a separate inference pass. Currently, it is recommended to use the alternative approach for calculating word-level timestamps.
• The cross QK kernels are only supported for CUDA.
• The end-to-end working example is still under development here. Once working, a copy of those scripts will be added as a sub-folder in the Python examples.

Motivation and Context

The original implementation of Whisper was added in ONNX Runtime GenAI to create an initial foundation. This new approach is more flexible and more customizable for users. It also introduces an encoder-decoder architecture setup that can be used for other encoder-decoder models or other speech models.