Use Case: Using Praxis as the Infrastructure Layer for Skillberry-Agent

[eranra]

Hi @usize and @shaneutt,

@shaneutt, yesterday we had a very productive discussion with @usize and presented some of the work we are doing at IBM Research around AI skill optimization (The skillberry project https://github.com/skillberry-ai/ ).

One area we are exploring is treating customer agents as black boxes and providing optimized tools, prompts, and reusable snippets through a proxy-agent layer. We developed a standalone implementation of this here:

https://github.com/skillberry-ai/skillberry-agent

Based on our discussion and @usize's explanation of Praxis, I think that there is a strong opportunity for collaboration. Rather than building and maintaining our own proxy infrastructure, we could potentially leverage the Praxis proxy to intercept and manipulate LLM requests and responses.

Today, our system exposes a /chat/completions API. Requests are processed by an internal LangGraph + MCP component that communicates with a Skills backend (the skillberry-store) to retrieve optimized assets. Based on the user's prompts and interactions, we dynamically enhance and transform requests before they are sent to the LLM when doing so is expected to improve efficiency or quality.

@shaneutt, could you take a look at the repository and share your thoughts on whether Praxis could be a good fit for this architecture?

Possible next steps:

1. Continue the discussion in this issue.
2. Schedule a meeting to dive deeper into the integration possibilities.
3. Explore other collaboration options.

Ref:

https://github.com/praxis-proxy/praxis/issues/354

CC'ing the relevant IBM Research team members:

@OsherElhadad @aviweit @ronenl-ibm

[shaneutt]

We appreciate the details @eranra 👍

The Skillberry approach of treating customer agents as black boxes and optimizing through a proxy layer is interesting, and there's clear overlap with what we're building.

A few questions:

1. Where would you draw the boundary? Your system does both request interception (proxy) and agentic orchestration (LangGraph + MCP tool execution). Are you looking for Praxis to replace just the proxy/routing layer while Skillberry handles orchestration as a backend service? Or would you want skill resolution logic running inside Praxis as filters?
2. What are the request/response transformations? Specifically is it system prompt injection, tool list augmentation, response post-processing, or all of the above? Understanding the exact modifications helps determine whether these fit as Praxis filters or are better handled externally via ext_proc (Epic: External Processing #17).
3. How latency-sensitive is skill resolution? If the Skillberry Store lookup adds meaningful latency, that affects whether it should be inline (filter) or async. What are typical round-trip times to the store?
4. I imagine our ext_proc support would work for you, but we're generally going to document advisories against using anything but native Rust filters whenever possible. Does this present a challenge for you?

We've been having a weekly sync, I'll send you an invite if you would like to add this as an upcoming agenda topic.

Thanks!

[eranra]

Hi @shaneutt,

Sorry for the late response—I missed the notification in my email. 🙂

Where would you draw the boundary? Your system does both request interception (proxy) and agentic orchestration (LangGraph + MCP tool execution). Are you looking for Praxis to replace just the proxy/routing layer while Skillberry handles orchestration as a backend service? Or would you want skill resolution logic running inside Praxis as filters?

From what I understood from @usize, and from reading the documentation (I haven't installed Praxis yet), there is an intention to move higher up the stack and provide both the facilities and the extension points needed to support more sophisticated pipelines like the ones we're building in the SKillberry-proxy with MCP and LangGraph. However, I see this more as an architectural direction than something that's fully realized today.

I think a practical approach would be to start with a pluggable model for the MCP/LangGraph integration and use Praxis initially for request interception and context manipulation, with the heavy lifting handled by an external process. Over time, I hope (and would suggest) that Praxis evolves to allow more of this functionality to be embedded directly into the core.

Another reason for this approach is the difference in development velocity between Python and Rust. It's much easier to iterate quickly in Python, although in 2026, with Claude and friends, that gap is becoming much smaller.

What are the request/response transformations? Specifically is it system prompt injection, tool list augmentation, response post-processing, or all of the above? Understanding the exact modifications helps determine whether these fit as Praxis filters or are better handled externally via ext_proc.

Unfortunately, it's all of the above. 🙂

What we do is "learn from the trajectories"—understanding what works well and what doesn't—and use that knowledge to build a collection of reusable snippets (text) and tools (executables). These may be entirely new or modifications of the originals. As a result, we dynamically modify both the available tool list and the execution context. We also allow ourselves to remove or replace parts of the context when we believe it will improve the agent's behavior (or even deliberately "confuse" the agent when appropriate for experimentation).

How latency-sensitive is skill resolution? If the Skillberry Store lookup adds meaningful latency, that affects whether it should be inline (filter) or async. What are typical round-trip times to the store?

That's a very good question.

In most cases, we're dominated by LLM latency. The calls to the LLM are significantly slower than any of the essentially deterministic processing happening in the proxy (currently implemented in Python). That said, latency is very important, and if we can demonstrate that this "black-box" approach is also efficient from a latency perspective, I think that would be valuable.

The current requirement to access the Skillberry-Store which is an external remote service - may naturally push us toward using ext_proc, but I'm not sure about that, we need to learn more and decide.

I imagine our ext_proc support would work for you, but we're generally going to document advisories against using anything but native Rust filters whenever possible. Does this present a challenge for you?

Yes, that's what I suspected - that we'd likely need to use the ext_proc approach. I'm not sure I fully understand the reasoning behind the recommendation to prefer native Rust filters, so I'll ask @aviweit to take a closer look.

Thanks for the very informative and helpful response! 🙂

Also, if possible, could you forward the invitation for the weekly meeting also to @aviweit ?

@OsherElhadad @aviweit @ronenl-ibm

[shaneutt]

Also, if possible, could you forward the invitation for the weekly meeting also to @aviweit ?

Yes, please ping me on Slack and send me the email of everyone you want added 👍

I think a practical approach would be to start with a pluggable model for the MCP/LangGraph integration and use Praxis initially for request interception and context manipulation, with the heavy lifting handled by an external process. Over time, I hope (and would suggest) that Praxis evolves to allow more of this functionality to be embedded directly into the core.
What we do is "learn from the trajectories"—understanding what works well and what doesn't—and use that knowledge to build a collection of reusable snippets (text) and tools (executables). These may be entirely new or modifications of the originals. As a result, we dynamically modify both the available tool list and the execution context. We also allow ourselves to remove or replace parts of the context when we believe it will improve the agent's behavior (or even deliberately "confuse" the agent when appropriate for experimentation).

ACK

The current requirement to access the Skillberry-Store which is an external remote service - may naturally push us toward using ext_proc, but I'm not sure about that, we need to learn more and decide.
Yes, that's what I suspected - that we'd likely need to use the ext_proc approach. I'm not sure I fully understand the reasoning behind the recommendation to prefer native Rust filters, so I'll ask @aviweit to take a closer look.

Understood. Avoiding it will be preferable when feasible: we've mostly added it as fallback position for the integration of existing systems.

I would like to start enumerating and fleshing out the critical needs, to see if they are generally applicable. If so, we would be open to adopting them natively in Praxis, and you might be able to avoid ext-proc?

I'm going to consider this issue triage/accepted, but here's what I want to do:

I want to consider it a spike for the moment, the goal being to do some rounds of research and build POCs and work up a strong narrative of what this use case means for Praxis. This should result in a proposal where we get aligned on the "What?" and "Why?" that describes the goals and motivations herein.

Let me know your thoughts, and if it's OK for me to assign you to this one and consider it "in progress" according to the approach above?

[aviweit]

Hi @eranra , @shaneutt

I wanted to share a quick update on my progress. I’ve cloned and set up Praxis on my Ubuntu environment, and implemented a simple HTTP filter that logs incoming “human” message requests and then forwards them to a RITS proxy server (OpenAI-compatible).
I was able to successfully interact with Praxis end-to-end. You can review my work here:
https://github.com/aviweit/praxis/tree/litellm-proxy-with-message-logging
Below is a sample output from my test demonstrating the flow and response from the proxy:

User/agent

$ ./test-litellm-praxis.sh | jq .

{
  "id": "chatcmpl-1c2cf0fd-92d0-4f39-b3ca-453994b4deb5",
  "created": 1781793611,
  "model": "rits/openai/gpt-oss-120b",
  "object": "chat.completion",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Sure thing! Here’s a light‑hearted one for you:\n\n**Why don’t scientists trust atoms anymore?**\n\n*Because they make up everything… but they’re also great at throwing a party—after all, they love to bond!* 😄",
        "role": "assistant",
        "reasoning_content": "We need to respond with a happy joke. No policy issues. Just a joke.",
        "provider_specific_fields": {
          "refusal": null,
          "reasoning_content": "We need to respond with a happy joke. No policy issues. Just a joke."
        }
      },
      "provider_specific_fields": {
        "stop_reason": null,
        "token_ids": null
      }
    }
  ],
  "usage": {
    "completion_tokens": 77,
    "prompt_tokens": 77,
    "total_tokens": 154
  }
}

Praxis console

026-06-18T14:39:47.831860Z  INFO praxis::server: starting server
2026-06-18T14:39:47.831890Z  INFO pingora_core::server: Server starting
2026-06-18T14:39:47.831895Z  INFO pingora_core::server: Starting services in dependency order: ["http-proxy:litellm-gateway"]
2026-06-18T14:39:47.831898Z  INFO pingora_core::server: Starting service: http-proxy:litellm-gateway
2026-06-18T14:39:47.832921Z  INFO praxis::watcher: config file watcher started path=config-litellm.yaml

============================================================
[HUMAN MESSAGE] Hello! Tell me a NEW joke.
============================================================

2026-06-18T14:40:09.817395Z  INFO praxis::reload: building new pipelines from reloaded config
2026-06-18T14:40:09.817523Z  INFO praxis::reload: config reload complete swapped=["litellm-gateway"] skipped=[]

============================================================
[HUMAN MESSAGE] Hello! Tell me a happy joke.
============================================================

^C2026-06-18T15:20:53.203617Z  INFO pingora_core::server: SIGINT received, exiting
2026-06-18T15:20:53.203653Z  INFO pingora_core::server: Waiting for runtimes to exit!
2026-06-18T15:20:53.203748Z  INFO pingora_core::server: Waiting for service runtime http-proxy:litellm-gateway to exit
2026-06-18T15:20:53.204343Z  INFO pingora_core::server: All runtimes exited, exiting now

Also, I noticed the reasoning_content field in the response and wanted to ask—how is this field generated, and what determines whether a request is allowed or not?
I am copying @OsherElhadad
Thanks,
Avi

[aviweit]

Hi @shaneutt,

I would like to share the current progress of the POC I'm working on. I took the decision to start and integrate our agentic logic - using Praxis filters. Can you please let us know if it is a good direction?

Please find questions below:

1. Does Praxis handle HTTP fragmentation (i.e., cases where incoming/outgoing requests are split into chunks)?
2. What is the best practice to "extend" Praxis with additional logic? Is it through filters?
3. Does Praxis support loading external filters, for example filters that are defined and maintained in separate repositories? Should we follow this approach when integrating our logic into Praxis?

Our POC is preliminary and is currently located at: https://github.com/aviweit/praxis/tree/feature/phase3-mcp-tools-integration

The Praxis YAML file that defines the filter flow can be found here: https://github.com/aviweit/praxis/blob/feature/phase3-mcp-tools-integration/examples/configs/ai/phase3-mcp-tools-integration.yaml

If would be very good if you can review and provide us feedback.

We noticed that there are predefined phases of filter invocation that affect their order and dependencies ("Body" and "Request" phases and corresponding ones for the response) that we will have to better understand.

We would also like to share a diagram flow of the POC:

┌─────────────────────────────────────────────────────────────────┐
│                     Incoming Request                             │
│         POST /v1/chat/completions + Authorization Header         │
│         Body: {"model": "...", "messages": [...]}                │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│  context_extractor Filter                                        │
│  - Extracts env_id from x-skillberry-env-id header               │
│  - Stores in ctx.filter_metadata["env_id"]                       │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│  skill_resolver Filter                                           │
│  - Resolves skill UUID from SKILL_UUID or SKILL_NAME env vars    │
│  - Stores in ctx.filter_metadata["skill_uuid"]                   │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│  vmcp_manager Filter                                             │
│  1. Creates VMCP server via skillberry-store API                 │
│  2. Connects to VMCP server via SSE to fetch MCP tools           │
│     SSE: http://localhost/:{vmcp_port}/sse                        │
│     Uses MCP protocol to list_tools()                            │
│  3. Converts MCP tools to OpenAI format                          │
│  4. Stores in ctx.filter_metadata:                               │
│     - "vmcp_uuid", "vmcp_name", "vmcp_port"                      │
│     - "mcp_tools" (JSON string of OpenAI-formatted tools)        │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│  mcp_tools_enricher Filter                                       │
│  1. Reads mcp_tools from ctx.filter_metadata["mcp_tools"]        │
│  2. Parses OpenAI chat completion request body                   │
│  3. Enriches request with tools schema:                          │
│     - Adds "tools" array (already in OpenAI format)              │
│     - Sets "tool_choice": "auto" (if not already set)            │
│  4. Updates request body with enriched content                   │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│  router + load_balancer                                          │
│  - Forwards enriched request to LLM backend                      │
│  - LLM receives tools schema and can call them                   │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    LLM Backend Response                          │
│  - May include tool_calls in response                            │
│  - Tool execution handled by client/agent                        │
└─────────────────────────────────────────────────────────────────┘

and high level overview of our filters :

Filter Name	Role	Input	Output
context_extractor	Extract environment context from headers	Headers: x-skillberry-env-id, x-skillberry-user-id, session-id	Metadata: env_id, user_id, session_id
skill_resolver	Resolve skill UUID	Metadata: env_id + env vars (SKILL_UUID / SKILL_NAME)	Metadata: skill_uuid
vmcp_manager	Create VMCP server + fetch MCP tools	Metadata: env_id, skill_uuid + Skillberry Store API	Metadata: vmcp_uuid, vmcp_name, vmcp_port, mcp_tools
mcp_tools_enricher	Inject MCP tools into OpenAI request	Metadata: mcp_tools + OpenAI request body	Modified request with tools array + tool_choice
router	Route request to backend	Request path	Metadata: target cluster
load_balancer	Forward request to LiteLLM endpoint	Cluster metadata + enriched request	HTTP request to backend endpoint

Thanks.

[usize]

Hi @aviweit, it's really awesome to see a POC like this in action.

1. Does Praxis handle HTTP fragmentation?

Yes. Pingora handles chunked transfer encoding and HTTP/2 framing transparently. By the time body data reaches a Praxis filter's on_request_body / on_response_body hooks, chunked framing has already been stripped so your filter receives raw Bytes chunks.

Praxis adds a body-mode layer on top of this (BodyMode):

Your POC already uses StreamBuffer mode correctly in the filters that need the full body. So you don't need to worry about fragmentation.

2. Best practice to extend Praxis?

Yes, through filters. The way you've approached it in the POC looks good.

3. External filters from separate repositories?

Yes, and this is what we'd recommend for Skillberry-specific filters.

Generated example 👇🏻

Praxis has a build-time auto-discovery mechanism designed exactly for this use case. The export_filters! macro lets external crates self-register into Praxis with zero Rust code changes to the Praxis repo.

Here's what a separate skillberry-praxis-filters crate would look like:

Cargo.toml:

[package]
name = "skillberry-praxis-filters"
version = "0.1.0"

# This marker tells Praxis's build script to auto-discover this crate
[package.metadata.praxis-filters]

[dependencies]
async-trait = "0.1"
praxis-proxy-filter = "0.3"
serde = { version = "1", features = ["derive"] }
serde_yaml = { package = "yaml_serde", version = "0.10" }
# Your dependencies — only compiled when your crate is used:
reqwest = { version = "0.12", features = ["json"] }
mcp-client = "0.1"

src/lib.rs:

use praxis_filter::export_filters;

mod skill_resolver;
mod vmcp_manager;
mod mcp_tools_enricher;

export_filters! {
    http "skill_resolver"     => skill_resolver::SkillResolverFilter::from_config,
    http "vmcp_manager"       => vmcp_manager::VmcpManagerFilter::from_config,
    http "mcp_tools_enricher" => mcp_tools_enricher::McpToolsEnricherFilter::from_config,
}

Operator usage — add one line to the Praxis server's Cargo.toml:

[dependencies]
skillberry-praxis-filters = { git = "https://github.com/skillberry-ai/skillberry-praxis-filters" }

Then reference filters by name in YAML exactly as you do now. Rebuild and run — Praxis's build.rs auto-discovers marked crates via cargo metadata and generates the registration code.

Why this matters for your POC:

Your current branch adds reqwest and mcp-client as unconditional dependencies to filter/Cargo.toml. Every Praxis user would compile and link these, even if they never use Skillberry. The external crate approach means your dependencies are only pulled in by operators who opt in.

It also decouples your release cadence from Praxis's — you can iterate on filter logic, update MCP client versions, and release independently.

---

Things to look at in the POC

context_extractor and skill_resolver don't need body buffering. Both filters only read headers and environment variables, but they declare StreamBuffer { max_bytes: 10MB } and do their work in on_request_body. We have an on_request hook that only works on headers and avoids the extra buffering.

context_extractor recompiles the global regex per-request. In validate_value(), the global validation pattern is recompiled with Regex::new() on every header value check. It should be compiled once in from_config and stored alongside the filter. This is a similar pattern to how you already handle per-rule patterns.

skill_resolver reads env vars per-request. std::env::var() on every request. Consider reading them once in from_config and storing the result. If they genuinely need to be dynamic, a KV store (ctx.kv_stores) would be a more idiomatic mechanism.

context_extractor may be generally useful. Unlike the Skillberry-specific filters. It may be worth contributing back upstream into praxis.

Re: Filter Phases

Earlier Claude assisted version of this comment managed to explain it incorrectly, which is a sign to me that we should include comments that spell it out better for agents.

In short, as with e.g., https://github.com/praxis-proxy/praxis/blob/main/filter/src/pipeline/http.rs#L124 the filters are run against each phase in serial.

Request filters are run in the order they were specified in.

Response filters run in reverse order as per: https://github.com/praxis-proxy/praxis/blob/main/filter/src/pipeline/http.rs#L86

[aviweit]

Hi @usize ,

Thanks a lot for the reply and comments. I started to move some of the work to a new repository: https://github.com/skillberry-ai/skillberry-praxis-filters. I will take your recommendations into account and will update on how it goes.

Thanks.