Skip to content

feat: release 4.0 (npu workflow) - #39

Merged
maciejmajek merged 73 commits into
mainfrom
development
Jul 15, 2026
Merged

feat: release 4.0 (npu workflow)#39
maciejmajek merged 73 commits into
mainfrom
development

Conversation

@maciejmajek

@maciejmajek maciejmajek commented Jul 1, 2026

Copy link
Copy Markdown
Member

Development → main

This PR replaces the manual multi-step setup with a pixi-managed workflow: environments per deployment profile and tasks covering the full lifecycle, from pixi run setup to pixi run demo. It also adds a config-driven inference dispatcher with NPU support, agentic conversation tracing, a custom spawn service in the sim, and CI.

Notable changes

  • Pixi workflow: per-profile environments (sim, hmi, hil, single-pc gpu/npu) with tasks for the whole lifecycle: setup, build-sim, build-ros2, build-llama, build-fastflowlm, download-models, then sim, stack, inference, agents, hmi, plus demo, demo-trace, smoke-test, and dev tasks (lint, test, kill). ROS 2 deps come from pixi packages instead of submodules; O3DE switched to the SDK; versions locked in pixi.lock.
  • Inference dispatcher (rai_app/inference/serve.py): resolves [endpoints.*] from config.toml into backend launch commands (llama.cpp for GPU/CPU, FastFlowLM for NPU), including health checks and weight downloads.
  • NPU support: FastFlowLM submodule plus backend-aware structured output falling back to GBNF grammar where function calling is unavailable.
  • Agentic trace: ConversationFileCallback records agent conversations to a run dir; demo-trace runs the full demo unattended and collects the trace, also dockerized (Dockerfile.demo).
  • Sim: custom ROS 2 SpawnEntity service handler in the gem, prefab/asset updates, unused level removed.
  • CI: workflow to build, publish, and smoke test Docker images.
  • Docs: rewritten around the pixi flow.

Note:
28K additions come from pixi.lock file

maciejmajek and others added 30 commits May 14, 2026 15:09
fix: run ros2 stack
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
* refactor: pixi

* fix: inference

* fix: multithreaded llama build

* fix: reranker model name

* feat: smoke test colors

* chore: update links

* docs: update

* docs: update

* docs: update
fix: run ros2 stack

* feat: demo.sh

* feat: single command demo run

* docs: update
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
jhanca-robotecai and others added 15 commits June 24, 2026 15:09
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
* Initial changes

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* ROS 2 deps from submodules to pixi packages

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Co-authored-by: Maciej Majek <46171033+maciejmajek@users.noreply.github.com>

* Pre-commit hook changes

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Add dev pixi env for pre-commit

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Fix lint env

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Code review fixes (AI-bloat)

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Mirror pixi setup from amrcv blueprint

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Fix pixi issues: Editor and GameLauncher work

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Remove unused level

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Code review fixes

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Add .dockerignore

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

---------

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
Co-authored-by: Maciej Majek <46171033+maciejmajek@users.noreply.github.com>
Co-authored-by: maciejmajek <maciej.majek@robotec.ai>
* Bump robotec-o3de-tools submodule

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

* Update kairos prefab to new WheelAnimTool

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>

---------

Signed-off-by: Jan Hanca <jan.hanca@robotec.ai>
* feat: agentic trace

* feat: pixi demo-trace task

* refactor: demo trace

* chore: increase inference server setup timeout

* fix: demo trace log handling

* docs: agentic trace
@maciejmajek maciejmajek changed the title Merge development into main Release 4.0 NPU workflow Jul 8, 2026
@maciejmajek maciejmajek changed the title Release 4.0 NPU workflow feat: release 4.0 (npu workflow) Jul 8, 2026
maciejmajek and others added 8 commits July 9, 2026 10:31
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
* Catch moveit errors in housekeep tool

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Reset simulation on housekeep fail

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* pre-commit

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

---------

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
* Catch llm errors

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* pre-commit

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Use formulate prompt

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

---------

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
* Populate scene using batch spawning

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Add custom spawn service handler for SpawnEntities

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* pre-commit

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Remove unused dependency

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Use batch spawning in housekeep scenario

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Move random rotation to spawn function

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Remove magic number and use self.logger

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Use constant for spawn request timeout

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Recursive path for build-sim input files

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Add spawn_on_spot method back

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Bump llama.cpp submodule

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

---------

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
* Remove humanworker from scene when testing

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

* Ensure topic exists before despawning humanworker

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

---------

Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>
@boczekbartek
boczekbartek marked this pull request as ready for review July 14, 2026 20:47
Signed-off-by: Kacper Dąbrowski <kacper.dabrowski@robotec.ai>

@knicked knicked left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested everything locally and in the docker pulled from the dockerhub.
The demo runs great with the NPU hardware and local LLMs.

@maciejmajek
maciejmajek merged commit 170ebc1 into main Jul 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants