genai-function-calling: uses OpenInference to propagate trace IDs over mcp stdio #64
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Adrian Cole <[email protected]>
Contributor
Author
|
to make this easier, I folded telemetry.js into main.js and turned Then, I see a patch failed like this which fails like this "module TypeError: Cannot add property openInferencePatched, object is not extensible" |
Signed-off-by: Adrian Cole <[email protected]>
Contributor
Author
|
ok I changed to manually wire instead of using EDOT import and the MCP propagation works now.
@trentm any ideas on this one? notably the comment I made here which latest commit works around? "Cannot add property openInferencePatched, object is not extensible" https://github.com/Arize-ai/openinference/tree/main/js/packages/openinference-instrumentation-mcp is the source of the experimental propagation instrumentation we're using to support an MCP spec discussion |
anuraaga
reviewed
Apr 22, 2025
Collaborator
|
The debug log is here All the openinference instrumentation includes it. |
Contributor
Author
|
right cjs probably was a red herring, I'll try to revert esm now |
Signed-off-by: Adrian Cole <[email protected]>
Signed-off-by: Adrian Cole <[email protected]>
Contributor
Author
|
@anuraaga @trentm here's my summary so far. To recap My test case is running Explicit NodeSDK initialization worksok I got it working with explicit NodeSDK initialization with cjs (no split trace between MCP client and server, so Detailsexplicit SDK initialization means run via const { NodeSDK } = require("@opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require("@opentelemetry/exporter-trace-otlp-proto");
const { MCPInstrumentation } = require("@arizeai/openinference-instrumentation-mcp");
const MCPClientModule = require("@modelcontextprotocol/sdk/client/index.js");
const MCPServerModule = require("@modelcontextprotocol/sdk/server/index.js");
const mcpInstrumentation = new MCPInstrumentation();
// MCP must be manually instrumented as it doesn't have a traditional module structure
mcpInstrumentation.manuallyInstrument({
clientModule: MCPClientModule,
serverModule: MCPServerModule,
});
const sdk = new NodeSDK({
traceExporter: new OTLPTraceExporter(),
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
process.on("SIGTERM", async () => {
try {
await sdk.shutdown();
} catch (err) {
console.warn("warning: error shutting down OTel SDK", err);
}
process.exit();
});
process.once("beforeExit", async () => {
// Flush recent telemetry data if about to shutdown.
try {
await sdk.shutdown();
} catch (err) {
console.warn("warning: error shutting down OTel SDK", err);
}
});Here's the package.json {
"name": "genai-function-calling",
"version": "1.0.0",
"private": true,
"type": "commonjs",
"engines": {
"node": ">=22"
},
"scripts": {
"start": "node --env-file .env -r ./telemetry.js index.js"
},
"dependencies": {
"ai": "^4.3.9",
"@ai-sdk/azure": "^1.3.17",
"@ai-sdk/openai": "^1.3.16",
"@modelcontextprotocol/sdk": "^1.10.1",
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/auto-instrumentations-node": "^0.58.0",
"@opentelemetry/sdk-node": "^0.57.2",
"@opentelemetry/sdk-trace-node": "^1.30.1",
"@arizeai/openinference-instrumentation-mcp": "^0.1.0"
}
}EDOT has an initialization failure with Arize instructionsThen I switched back to EDOT ( Then I get an error in Detailsswitch back to EDOT means run via const { MCPInstrumentation } = require("@arizeai/openinference-instrumentation-mcp");
const MCPClientModule = require("@modelcontextprotocol/sdk/client/index.js");
const MCPServerModule = require("@modelcontextprotocol/sdk/server/index.js");
const mcpInstrumentation = new MCPInstrumentation();
// MCP must be manually instrumented as it doesn't have a traditional module structure
mcpInstrumentation.manuallyInstrument({
clientModule: MCPClientModule,
serverModule: MCPServerModule,
});Here's the package.json {
"name": "genai-function-calling",
"version": "1.0.0",
"private": true,
"type": "commonjs",
"engines": {
"node": ">=22"
},
"scripts": {
"start": "node --env-file .env -r @elastic/opentelemetry-node -r ./telemetry.js index.js"
},
"dependencies": {
"ai": "^4.3.9",
"@ai-sdk/azure": "^1.3.17",
"@ai-sdk/openai": "^1.3.16",
"@modelcontextprotocol/sdk": "^1.10.1",
"@elastic/opentelemetry-node": "^1",
"@arizeai/openinference-instrumentation-mcp": "^0.1.0"
}
}Then, I changed to initialize
Detailschanged to initialize // Import and initialize MCPInstrumentation before importing the MCP modules.
const { MCPInstrumentation } = require("@arizeai/openinference-instrumentation-mcp");
const mcpInstrumentation = new MCPInstrumentation();
// Then, manually instrument MCP, as it doesn't have a traditional module structure.
mcpInstrumentation.manuallyInstrument({
clientModule: require("@modelcontextprotocol/sdk/client/index.js"),
serverModule: require("@modelcontextprotocol/sdk/server/index.js"),
});Here's the package.json {
"name": "genai-function-calling",
"version": "1.0.0",
"private": true,
"type": "commonjs",
"engines": {
"node": ">=22"
},
"scripts": {
"start": "node --env-file .env -r @elastic/opentelemetry-node -r ./telemetry.js index.js"
},
"dependencies": {
"ai": "^4.3.9",
"@ai-sdk/azure": "^1.3.17",
"@ai-sdk/openai": "^1.3.16",
"@modelcontextprotocol/sdk": "^1.10.1",
"@elastic/opentelemetry-node": "^1",
"@arizeai/openinference-instrumentation-mcp": "^0.1.0"
}
}I'm not sure if this is something I am doing wrong or a limitation but hope the details help! I will raise a PR to update Arize docs about initialization order in a minute. https://github.com/Arize-ai/openinference/tree/main/js/packages/openinference-instrumentation-mcp/src#usage |
trentm
reviewed
Apr 22, 2025
Signed-off-by: Adrian Cole <[email protected]>
Signed-off-by: Adrian Cole <[email protected]>
Member
|
Some behaviour is different between:
In the second case (when using A guess is that Node.js runs the UpdateYah. I get the same behaviour (w.r.t. to the log output) as (2.) above if I run: |
Contributor
Author
|
fyi this is a pending PR to fix the usage in openinference docs Arize-ai/openinference#1551 |
Member
FWIW, I think you can ignore those errors. I looked at the try {
// This can fail if the module is made immutable via the runtime or bundler
module.openInferencePatched = true;
} catch (e) {
diag.debug(
`Failed to set ${CLIENT_MODULE_NAME} patched flag on the module`,
e,
);
}I'd expect to get that debug message with ESM usage and perhaps with some bundlers. |
Member
|
Possibly you could change "telemetry.js" to this: Then you just need the one |
Contributor
Author
|
thanks for trying, but the last approach didn't join the trace :( That said, I think I may have crossed wires at some point because I can't get an older commit to join the trace anymore (regardless of cjs ESM). I should have caught a screen capture. I'll look at our internal demo and see if I can make a simpler reproduction. |
Contributor
Author
|
ok this is the closest port I could make from the internal copy, and it isn't joining traces either... hmm // First, configure and start your OpenTelemetry NodeSDK per documentation.
const {NodeSDK} = require("@opentelemetry/sdk-node");
const {OTLPTraceExporter} = require("@opentelemetry/exporter-trace-otlp-proto");
const {diag, DiagConsoleLogger, DiagLogLevel} = require('@opentelemetry/api');
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
// Then, import and initialize MCPInstrumentation before importing the MCP modules.
const {MCPInstrumentation} = require("@arizeai/openinference-instrumentation-mcp");
const mcpInstrumentation = new MCPInstrumentation();
// Now, manually instrument MCP, as it doesn't have a traditional module structure.
mcpInstrumentation.manuallyInstrument({
clientModule: require("@modelcontextprotocol/sdk/client/index.js"),
serverModule: require("@modelcontextprotocol/sdk/server/index.js"),
});
// Finally, configure any other instrumentation and start the sdk
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const sdk = new NodeSDK({
traceExporter: new OTLPTraceExporter(),
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
process.on("SIGTERM", async () => {
try {
await sdk.shutdown();
} catch (err) {
console.warn("warning: error shutting down OTel SDK", err);
}
process.exit();
});
process.once("beforeExit", async () => {
// Flush recent telemetry data if about to shutdown.
try {
await sdk.shutdown();
} catch (err) {
console.warn("warning: error shutting down OTel SDK", err);
}
});Detailslogs |
Contributor
Author
Contributor
Author
|
update: seems to be more about the path vercel is using mcp being different than tested paths. @anuraaga will chase it up tomorrow |
Signed-off-by: Adrian Cole <[email protected]>
Contributor
Author
|
ok all good with the tiniest diff! |
Signed-off-by: Adrian Cole <[email protected]>
anuraaga
approved these changes
Apr 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds OpenInference instrumentation for MCP, which connects the trace context between the MCP client and the server it launches as a sub-process.
Technically, this injects trace identifiers into the request.params._meta field on the client side, and extracts them on the server.