This repository was archived by the owner on Nov 12, 2025. It is now read-only.
Model input and generation logic improvements #22
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… and automation for the EO-1 project (#5) (#6) * Add initial project structure with configuration files, datasets, and example scripts * Update .gitignore to include new demo data paths, modify pre-commit configuration to exclude additional directories, and enhance README with more examples and installation instructions. Adjust dataset handling in pipeline configuration and dataset classes for improved training flexibility. Remove deprecated demo scripts and refine evaluation scripts for clarity. * Update .gitignore to include demo data paths, enhance README with additional examples, and modify Libero benchmark configuration files for improved clarity and structure. Adjust training scripts and evaluation settings across various experiments for consistency. * Remove fast testing workflow configuration from GitHub Actions * Update pre-commit configuration to refine exclusions, enhance README with structured examples, and remove unused imports in the EO model script. Co-authored-by: dlqu_0010 <[email protected]>
…d in eo1-dev branch
…y and performance.
…rations for improved clarity, and adjust training scripts for consistency across experiments. Enhance README documentation for better guidance on dataset preparation and training processes.
…dit checks for improved security analysis.
…ure and functionality. Updated EO1VisionFlowMatchingConfig to inherit from PretrainedConfig, streamlined initialization, and added keys_to_ignore_at_inference. Enhanced EO1VisionProcessor to support new text processing capabilities and improved handling of robot inputs and outputs. Adjusted class names for consistency and clarity.
… with integration details for EO-1 with LERobot. Refactor dataset handling in MultimodaLeRobotDataset and adjust model architecture in EO1VisionFlowMatchingModel for improved functionality. Update training utilities for better configuration management and streamline processor methods for action selection.
…a environment directly.
Update 'freeze_lm_head' option in TrainPipelineConfig for enhanced training flexibility. Refactor training utilities to align with new configuration settings.
…r exclusion of model development files.
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces improvements to model input handling and training configuration, along with code cleanup and refactoring. The changes enhance the flexibility of training workflows by adding finer control over model component freezing and fixing several bugs in action sampling logic.
- Added
freeze_lm_headoption for independent control over language model head training - Fixed state handling and indexing bugs in model generation and action sampling logic
- Removed unused code and cleaned up imports to reduce maintenance burden
Reviewed Changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/test_hf_model.py | Removed obsolete test script |
| tests/test_vlm.py | Enhanced VLM test script with proper multi-image handling and state tracking |
| experiments/*/train.sh | Removed unnecessary environment script sourcing |
| experiments/3_simpler/simpler_env/eval_simpler.sh | Removed unnecessary environment script sourcing |
| eo/train/train_utils.py | Updated LM head freezing to use new configuration option |
| eo/train/pipeline_config.py | Added freeze_lm_head option and fixed vision_lora logic |
| eo/model/modeling_qwen2_5_vl.py | Cleaned up imports and added states reset logic |
| eo/model/modeling_eo1.py | Fixed action sampling indexing bugs and added states parameter |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
EO-Robotics
approved these changes
Sep 24, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces several improvements and bug fixes across the codebase, primarily focusing on refining model input handling, freezing model components for training, and cleaning up unused or redundant code. The changes enhance the flexibility and correctness of training and inference workflows, especially for vision-language models.
Model input and generation logic improvements:
statesinput in botheo/model/modeling_eo1.pyandeo/model/modeling_qwen2_5_vl.py, ensuring thatstatesare correctly passed and reset when necessary, which helps prevent unintended behavior during generation. [1] [2]sample_actionsto use a localchunk_sizevariable for clarity and consistency, and fixed several indexing bugs related to cache and prefix lengths. This makes the code more robust and easier to maintain. [1] [2] [3] [4]Training configuration and freezing enhancements:
freeze_lm_headoption toTrainPipelineConfigand updated theconfigure_llmutility to support freezing the language model head independently from the rest of the LLM, giving finer control over which parts of the model are trainable. [1] [2]__post_init__to ensure thatvision_lorais disabled whenlora_enableis false, preventing configuration mismatches.Code cleanup and testing improvements:
_samplemethod and its associated output dataclass fromeo/model/modeling_qwen2_5_vl.py, streamlining the codebase and reducing maintenance burden.scripts/test_vlm.pytotests/test_vlm.pyand enhancing it to correctly handle multiple image inputs and maintain state across turns.tools/test_hf_model.pyscript, further cleaning up redundant code.Miscellaneous script updates:
env.shfrom various experiment and evaluation scripts, simplifying environment setup. [1] [2] [3] [4] [5]Dependency and import adjustments:
eo/model/modeling_qwen2_5_vl.pyby removing unused typing and generation-related imports, making the file more concise.# What does this PR do?Fixes # (issue)
Before submitting