Add Granite 4.1 Vision (granite4_vision)#45597

Open

artem-spector wants to merge 47 commits intohuggingface:mainfrom

artem-spector:add-gv41

Commits on May 3, 2026

Add Granite 4.1 Vision model (granite4_vision)

artemspector
and
claude
committed
Fix auto-registration after upstream auto_mappings refactor

artemspector
and
claude
committed
Fix conflict marker in image_processing_auto.py

artemspector
and
claude
committed
Fix check-repo: remove spatial_stride (unused in modeling), fix auto_mappings duplicate

artemspector
and
claude
committed
Fix duplicate legacy key in conversion_mapping.py

artemspector
and
claude
committed
Fix check-repo failures for granite4_vision

artemspector
and
claude
committed
Regenerate auto_mappings.py after rebase onto upstream/main

artemspector
and
claude
committed
Fix CI failures after upstream rebase

artemspector
and
claude
committed
Fix Sam3 auto_mappings.py entries corrupted by rebase
artemspector
committed
Restore hy_v3, openai_privacy_filter, slanet entries dropped by bad rebase regeneration
artemspector
committed
Revert dependency_versions_table.py to match setup.py (upstream state)
artemspector
committed
Fix bad rebase: remove hy-v3/slanet/openai-privacy-filter entries from auto_docstring and check_repo

artemspector
and
claude
committed
Remove merge_lora_adapters and PEFT adapter-toggling generate override

artemspector
and
claude
committed
Add Granite4VisionTextModel with deepstack injection, replace manual layer loop

artemspector
and
claude
committed
Add Granite4VisionTextConfig, fix missing TextConfig import in generated file

artemspector
and
claude
committed
Fix Granite4VisionTextConfig to inherit PreTrainedConfig, add missing import

artemspector
and
claude
committed
Fix class ordering: define Granite4VisionPreTrainedModel before TextModel

artemspector
and
claude
committed
Fix inv_freq corruption in Granite4VisionTextRotaryEmbedding during from_pretrained

artemspector
and
claude
committed
Inline downsampling into modular, add qformer_config sub-config, convert to pure functions

artemspector
and
claude
committed
Return Granite4VisionImageFeaturesOutput from get_image_features

artemspector
and
claude
committed
Drop Granite4Vision image processor re-definitions, delegate to LlavaNext

artemspector
and
claude
committed
Address medium PR review items 8-14

artemspector
and
claude
committed
Address nit PR review items 15-23

artemspector
and
claude
committed
Address remaining review items 20 and 29

artemspector
and
claude
committed
Fix test failures found by Granite4VisionModelTest

artemspector
and
claude
committed
Use lazy import for Blip2QFormerConfig; qformer_config sub_configs uses AutoConfig

artemspector
and
claude
committed
Add Granite4VisionTextModel to check_repo ignore lists; document missing public classes

artemspector
and
claude
committed
Fix missing imports in modular: math, AutoConfig, select_best_resolution

artemspector
and
claude
committed
Pop output_attentions/output_hidden_states from **kwargs in Granite4VisionModel.forward

artemspector
and
claude
committed
Fix check_modeling_structure violations (TRF002, TRF009, TRF010)

artemspector
and
claude
committed
Fix ruff I001 import ordering and processing consistency check

artemspector
and
claude
committed
Remove converter-regenerated files that should not exist

artemspector
and
claude
committed
Remove autodoc entries for ImageProcessor classes that don't exist in this model

artemspector
and
claude
committed
Fix model card date for add_dates.py check

artemspector
and
claude
committed
Fix dependency_versions_table.py: sync mlinter version with setup.py

artemspector
and
claude
committed
Address review round 3: config, capture_outputs, hidden states, projector naming

artemspector
and
claude
committed
Nits: rename one-letter vars, AttributeError() for unused inherited config attrs

artemspector
and
claude
committed
Fix qformer_config: build fully-specified at init, no post-super field patching

artemspector
and
claude
committed
Move _can_record_outputs and _deepstack_inject to Granite4VisionPreTrainedModel

artemspector
and
claude
committed
Address remaining review items: capture_outputs, output class, DynamicCache

artemspector
and
claude
committed
Regenerate modeling/processing from updated modular; fix copyright header

artemspector
and
claude
committed
Fix ruff formatting in modular_granite4_vision.py

artemspector
and
claude
committed
Fix add_dates.py: update granite4_vision model card date to 2026-04-30

artemspector
and
claude
committed
Remove incorrect skip from test_can_init_all_missing_weights

artemspector
and
claude
committed
Fix add_dates.py: update granite4_vision model card date to 2026-05-03

artemspector
and
claude
committed
Fix _init_weights: add nn.Embedding, nn.LayerNorm, Granite4VisionTextRMSNorm branches

artemspector
and
claude
committed
Fix ruff F821: detect RMSNorm by attribute pattern instead of class name

artemspector
and
claude
committed