Agent As A Service (A3S)

This repository implements the functionalities developed as part of Task 3.1 (Human-in-the-Loop) within the AI4REALNET project.

🚧 Work in progress toward Deliverable D3.2 – Beta Software Release

📅 Official due date: April 2026

---

Supported By

This repository combines two complementary components: a backend “agent as a service” that can restore and continue simulations, and a tracing/visualization tool to inspect and interact with recorded decision trajectories.

agent-as-a-service (Task 3.1) A lightweight backend service that hosts an RL agent alongside a simulation environment and exposes it over Redis. It can restore an environment from a serialized state plus an action sequence, simulate forward for a configurable number of steps (optionally returning rendered frames), and report action spaces to clients—enabling “what-if” rollouts and HITL overrides from external tools.

trace-rl (Task 2.3) A trajectory tracking + visualization package that records an episode as a tree of decision blocks (supporting branching via cloning/overrides) and provides a Dash web app to explore it. Users can load a saved trajectory, inspect actions/KPIs/events and renderings per step, and optionally connect to a live agent-as-a-service backend to override actions and generate alternative futures directly from the UI.

---

Setup

1. Clone and Navigate to the Repository

git clone https://github.com/enlite-ai/agent-as-service-trace-rl.git
cd agent-as-service-trace-rl

2. Create the Conda Environment

Create the environment from the provided environment.yml file.

Note: The default environment name is ai4realnet-a3s.

conda env create --file environment.yml

3. Activate the Environment

conda activate ai4realnet-a3s

4. Install the Local Package

pip install -e .

---

⚠️ Compatibility Note

Using a different version of dash_cytoscape may result in losing background node colors when hiding and restoring nodes.

---

Example Workflow

Below is an example workflow using the Taxi Gym environment.

---

1️⃣ Generate a Trajectory

Run the trajectory collector script:

python trace-rl/scripts/gym_trajectory_collector.py

This will create an example trajectory for the Taxi Gym environment.

📌 Take note of where the trajectory file is stored.

---

2️⃣ Start the Live Agent (Optional but Recommended)

The live agent allows you to compute additional rollouts and answer what-if queries.

A separate Hydra configuration system is used to define agent and environment setups.

• Configuration files are located in:

agent-as-a-service/agent_as_a_service/conf/

• Example configuration:

agent-as-a-service/agent_as_a_service/conf/taxi-v3_random.yaml

Start the live agent with:

python agent-as-a-service/agent_as_a_service/src/runner.py -cn taxi-v3_random

---

3️⃣ Start the Dash Application

In a separate terminal window, start the visualization app:

python trace-rl/trace_rl/app/app.py

Click the link displayed in the console to open the Dash application in your browser.

---

4️⃣ Load a Trajectory

1. In the Dash application, click "Select Trajectory".
2. Navigate to the trajectory file.
3. Wait for the trajectory to load.

Note: The default view may appear misaligned after loading. Click "Reset View" (bottom left) to center the visualization.

---

5️⃣ Inspect the Trajectory

You should now see the trajectory tree that looks like this:

You can:

• Navigate through the tree structure
• Inspect individual blocks
• View renderings
• Examine actions taken by the agent

---

6️⃣ Connect the Live Agent

⚠️ The live agent must already be running.

1. Click "Connect Agent" (bottom left).
2. Test the connection.

---

7️⃣ Ask What-If Questions

Once connected to a live agent, you can override actions and simulate alternative futures.

1. Select a block in the trajectory tree.
2. Click "Override Action".
3. Choose:
   • The action to override
   • The simulation depth

The system will then compute and visualize the resulting trajectory.

Note: Currently the system is setup such that it returns the rendering for all timesteps. This can be limited with MAX_RENDERING_IMAGES in trace-rl/trace_rl/app/config.py

---

Important Conceptual Notes

• Each block contains the action that transitions to the next block. This is in the right panel below the rendering.
• The final state is not visualized because the agent does not take any decision there.

Reachability of Final States

It is not possible for a human user to reach a final state from a leaf node unless that leaf node already represents a final state.

In other words:

• If a node is a non-final leaf, no further actions can be taken to reach a terminal state.
• Only nodes that are already terminal represent final states.

---

References

maze-rl GitHub

maze-rl docs

maze-flatland GitHub

flatland-rl GitHub

flatland Website

AI4RealNet Website

---

Contacts

🌐 Website: enlite.ai 🔗 LinkedIn: link

---

Acknowledgements

This work is partially supported by the AI4REALNET project, which has received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement No. 101119527. Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.