fix(mcp): route interactive Cloud API reads through the Config API - #1090
fix(mcp): route interactive Cloud API reads through the Config API#1090Aaron ("AJ") Steers (aaronsteers) wants to merge 2 commits into
Conversation
Interactive OIDC logins produce a user-realm bearer token with no client credentials. The public API (api.airbyte.com) rejects such tokens, so workspace and connection metadata reads returned 401. Route bearer-only reads through the Config API (which accepts user tokens) while keeping the public API path for application client credentials. The Config API connection response embeds the source/destination objects and sync catalog, so connector names and stream names resolve from one response instead of extra public API lookups. Co-Authored-By: AJ Steers <aj@airbyte.io>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This PyAirbyte VersionYou can test this version of PyAirbyte using the following: # Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1784919112-cloud-mcp-config-api-user-token' pyairbyte --help
# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1784919112-cloud-mcp-config-api-user-token'PR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful ResourcesCommunity SupportQuestions? Join the #pyairbyte channel in our Slack workspace. |
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughCloud workspace and connection reads now route bearer-only authentication through internal Config API endpoints. Response models preserve embedded names and selected streams, while client-credential flows continue using the public API. ChangesCloud Config API routing
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant CloudWorkspace
participant CloudConnection
participant ConfigAPI
Client->>CloudWorkspace: request workspace information
CloudWorkspace->>ConfigAPI: fetch workspace for bearer-only auth
ConfigAPI-->>CloudWorkspace: workspace mapping
Client->>CloudConnection: request connection metadata
CloudConnection->>ConfigAPI: fetch connection withRefreshedCatalog false
ConfigAPI-->>CloudConnection: embedded connection payload
CloudConnection-->>Client: names, streams, and connection details
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@airbyte/cloud/models.py`:
- Around line 189-193: Update the stream_names comprehension in CloudConnection
to include only entries whose config indicates selected=True, while preserving
the existing stream mapping and nonempty-name checks. Extend the relevant
fixture with a deselected stream and assert it is excluded from selected_streams
and the MCP connection description.
In `@airbyte/cloud/workspaces.py`:
- Around line 229-253: Update CloudWorkspace.connect() to reuse
get_workspace_info() instead of calling api_util.get_workspace() directly,
ensuring bearer-only credentials follow the Config API routing already
implemented by get_workspace_info(). Preserve the existing connection setup and
workspace metadata behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c700cd47-097d-4787-a29b-4a5694bcf96a
📒 Files selected for processing (6)
airbyte/_util/api_util.pyairbyte/cloud/connections.pyairbyte/cloud/models.pyairbyte/cloud/workspaces.pyairbyte/mcp/cloud.pytests/unit_tests/test_cloud_config_api_routing.py
| stream_names = [ | ||
| entry["stream"]["name"] | ||
| for entry in stream_entries | ||
| if isinstance(entry.get("stream"), Mapping) and entry["stream"].get("name") | ||
| ] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Filter out deselected catalog streams?
Could we require entry["config"]["selected"] here, wdyt? This currently exposes all catalog streams as selected_streams, so disabled streams are reported by CloudConnection.stream_names and the MCP connection description.
Proposed fix
stream_names = [
entry["stream"]["name"]
for entry in stream_entries
- if isinstance(entry.get("stream"), Mapping) and entry["stream"].get("name")
+ if (
+ isinstance(entry.get("stream"), Mapping)
+ and entry["stream"].get("name")
+ and isinstance(entry.get("config"), Mapping)
+ and entry["config"].get("selected") is True
+ )
]Please add a deselected stream to the fixture and assert it is excluded.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| stream_names = [ | |
| entry["stream"]["name"] | |
| for entry in stream_entries | |
| if isinstance(entry.get("stream"), Mapping) and entry["stream"].get("name") | |
| ] | |
| stream_names = [ | |
| entry["stream"]["name"] | |
| for entry in stream_entries | |
| if ( | |
| isinstance(entry.get("stream"), Mapping) | |
| and entry["stream"].get("name") | |
| and isinstance(entry.get("config"), Mapping) | |
| and entry["config"].get("selected") is True | |
| ) | |
| ] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@airbyte/cloud/models.py` around lines 189 - 193, Update the stream_names
comprehension in CloudConnection to include only entries whose config indicates
selected=True, while preserving the existing stream mapping and nonempty-name
checks. Extend the relevant fixture with a deselected stream and assert it is
excluded from selected_streams and the MCP connection description.
There was a problem hiding this comment.
☑️ Resolved in 8dd2ebf. Agreed — filtered stream_names to selected streams. I use a _is_stream_selected helper rather than a strict config.selected is True: when config is present I honor selected, but a missing config defaults to selected (matching the platform default) so unusual payloads don't silently drop streams. Added a deselected stream to the fixture and assert it's excluded.
Address review findings: - from_config_api_response now filters syncCatalog to selected streams (config.selected), matching public API stream_names behavior; a missing config defaults to selected. - CloudWorkspace.connect() now uses get_workspace_info() so bearer-only (interactive OIDC) credentials are validated via the Config API. Co-Authored-By: AJ Steers <aj@airbyte.io>
|
|
Closing — this approach is wrong. It presumed the interactive user token is rejected by the public API and reroutes reads to the internal Config API, but that premise was never verified and is incorrect. The real cause of the interactive 401 is token piping (the wrong token being sent downstream), not the API root. Abandoning this in favor of fixing the token that reaches the public API. |
Code Coverage OverviewLanguages: Python Python / code-coverage/pytest-fastThe overall coverage in commit 8dd2ebf in the Show a code coverage summary of the most impacted files.
Python / code-coverage/pytest-no-credsThe overall coverage in commit 8dd2ebf in the Show a code coverage summary of the most impacted files.
Python / code-coverage/pytestThe overall coverage in commit 8dd2ebf in the Show a code coverage summary of the most impacted files.
Updated |
There was a problem hiding this comment.
Pull request overview
This PR fixes authenticated Airbyte Cloud MCP reads after interactive OIDC login by routing “bearer-only” metadata reads (user token without client credentials) through the Cloud Config API, while keeping the existing public API path for headless client-credential auth.
Changes:
- Add a bearer-only auth detector on
CloudWorkspaceand route workspace/connection reads through Config API helpers when applicable. - Extend
CloudConnectionInfo/CloudConnectionto consume embedded source/destination names and sync-catalog stream names from the Config API response, reducing extra public API lookups. - Add unit tests covering the routing behavior and Config API response parsing.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
tests/unit_tests/test_cloud_config_api_routing.py |
Adds unit tests for bearer-only routing and Config API response parsing. |
airbyte/mcp/cloud.py |
Switches MCP tools to rely on CloudWorkspace.get_workspace_info() and new connection name properties. |
airbyte/cloud/workspaces.py |
Adds _uses_bearer_only_auth and get_workspace_info() routing logic. |
airbyte/cloud/models.py |
Adds Config API parsing for connections and makes configurations optional when using Config API. |
airbyte/cloud/connections.py |
Routes connection reads via Config API for bearer-only auth and adds name/stream fallbacks. |
airbyte/_util/api_util.py |
Introduces Config API helper functions for workspace/connection reads. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| stream_names = [ | ||
| entry["stream"]["name"] | ||
| for entry in stream_entries | ||
| if isinstance(entry.get("stream"), Mapping) and entry["stream"].get("name") | ||
| ] |
| """Get a workspace via the Config API endpoint `POST /v1/workspaces/get`. | ||
|
|
||
| Unlike `get_workspace`, this uses the internal Config API rather than the public API. | ||
| The Config API accepts user-realm bearer tokens (from an interactive OIDC login), | ||
| which the public API rejects because it only accepts application-client tokens. |
| """Get a connection via the Config API endpoint `POST /v1/web_backend/connections/get`. | ||
|
|
||
| Unlike `get_connection`, this uses the internal Config API rather than the public API. | ||
| The Config API accepts user-realm bearer tokens (from an interactive OIDC login), | ||
| which the public API rejects because it only accepts application-client tokens. |
| configurations=None, | ||
| prefix=connection.get("prefix"), | ||
| status=_enum_value(connection["status"]), | ||
| source_name=source.get("name"), | ||
| destination_name=destination.get("name"), | ||
| stream_names=stream_names, |
| assert connection.destination_name == "My Snowflake" | ||
| assert connection.stream_names == ["users", "orders", "legacy"] | ||
| assert connection.table_prefix == "raw_" | ||
| assert calls == ["config"] |
Summary
Interactive OIDC logins to the hosted Cloud MCP produce a user-realm bearer token with no client credentials. The public API (
api.airbyte.com/v1) only accepts application-client tokens, so authenticated cloud tool calls (check_airbyte_cloud_workspace,describe_cloud_connection) returned 401 after a successful interactive login. This routes those bearer-only reads through the Config API (cloud.airbyte.com/api/v1), which accepts user tokens, while leaving the public API path unchanged for headless application credentials.Routing key: a workspace/connection is "bearer-only" when it has a
bearer_tokenand noclient_id/client_secret.Both new
api_utilhelpers reuse the existing_make_config_api_request()(which forwards a bearer token directly). No Airbyte-specific realm/deployment values are introduced — PyAirbyte stays provider-neutral; concrete realm/issuer/audience values remain owned by the deployment repo.The Config API
WebBackendConnectionReadresponse embeds the fullsource/destinationobjects andsyncCatalog, sodescribe_cloud_connectionnow resolves connector names and stream names from that single response instead of issuing extra public API connector lookups.CloudConnectionInfogains optionalsource_name,destination_name, andstream_namesfields (populated only on the Config API path);configurationsis now optional (Noneon the Config API path). Thesource_name/destination_name/stream_namesproperties onCloudConnectionprefer the embedded values and fall back to the prior public API lookups.Test plan
tests/unit_tests/test_cloud_config_api_routing.py:_uses_bearer_only_authtruth table;CloudConnectionInfo.from_config_api_responseparsing (ids, names, streams, prefix, status, workspace-id fallback);CloudWorkspaceInfo.from_mapping; and routing assertions that bearer-only auth hits the Config API helpers while client-credential auth hits the public API helpers.ruff format,ruff check, andpyrefly checkclean on touched files; existingtest_mcp_cloud.py,test_cloud_api_util.py,test_cloud_credentials.py,test_cloud_api_roots.pypass.cloud-mcp-previewvia Goose Desktop (interactive OIDC asairbyte-support-bot@airbyte.io) is pending as a follow-up on the deployment side.Link to Devin session: https://app.devin.ai/sessions/a5b9501ef92c412aad0408b7c74ef9c8
Requested by: Aaron ("AJ") Steers (@aaronsteers)
Summary by CodeRabbit