feat: add MiniMax as alternative LLM provider for data preprocessing#39
Open
octo-patch wants to merge 1 commit intoPKU-YuanGroup:mainfrom
Open
feat: add MiniMax as alternative LLM provider for data preprocessing#39octo-patch wants to merge 1 commit intoPKU-YuanGroup:mainfrom
octo-patch wants to merge 1 commit intoPKU-YuanGroup:mainfrom
Conversation
Add configurable LLM provider support to the data preprocessing scripts (frame captioning and video captioning). Users can now choose between OpenAI (default) and MiniMax via --provider flag, or use any OpenAI-compatible API via --base_url and --model. Changes: - Add data_preprocess/llm_provider.py: shared provider config module with PROVIDER_PRESETS, create_client(), get_model_name() - Modify step2_1_GPT4V_frame_caption.py: use configurable client/model - Modify step3_1_GPT4V_video_caption_concise.py: same - Modify step3_1_GPT4V_video_caption_detail.py: same - Update run.sh: add PROVIDER variable - Add 26 unit tests + 3 integration tests - Update README with MiniMax usage docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add configurable LLM provider support to the data preprocessing captioning scripts. Users can now use MiniMax (MiniMax-M2.7, MiniMax-M2.5) as an alternative to OpenAI GPT-4V for frame and video captioning, or point to any OpenAI-compatible API.
Changes
data_preprocess/llm_provider.py: Shared provider configuration module withPROVIDER_PRESETS,create_client(),get_model_name(), andclamp_temperature()step2_1_GPT4V_frame_caption.py,step3_1_GPT4V_video_caption_concise.py,step3_1_GPT4V_video_caption_detail.py— all now accept--provider,--base_url,--modelargumentsrun.sh: AddedPROVIDERvariable for easy switchingUsage
Backward Compatibility
openai— no changes needed for existing workflows--api_keyargument still works as beforeOPENAI_API_KEYis used as fallback for any providerTest Plan