You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support attached Fabric Spark compute in azure-ai-ml - #49501
FabricSpark attachments already work through the AML ARM service, but the Python SDK treats their computeType as unsupported. SDK get/list and the v2 CLI therefore omit the Fabric IDs and pool details.
Change and compatibility
Add a FabricSparkCompute entity, YAML schema, ARM discriminator/models, and typed dispatch. Attach sends the four Fabric resource IDs, while get/list return the pool name, identity, attachment status, and provisioning data. Detach uses the existing begin_delete(name, action="Detach") path; the default Delete behavior for other compute types is unchanged. No Vienna or portal changes are included. Generated ARM models require the API spec/generation pipeline to retain this discriminator in future regenerations.
Dependencies and merge order
The source TypeSpec contract in Azure/azure-rest-api-specs#47205 must clear published-version compatibility review and support reproducible generated models before merge. The CLI pin in Azure/azure-cli-extensions#10459 must follow an SDK release containing this change.
Validation
Focused SDK compute tests: 42 passed, 1 skipped; Black and API digest checks passed. A FabricSpark response without optional nested details now deserializes to a typed compute, while attach requires all four IDs. Local generated-client regeneration stalled before output; model reproducibility remains unverified. Latest PR gate is running. No live detach was attempted.
Add typed Fabric Spark ARM models, YAML conversion, and compute dispatch so attach, get, and list expose the existing service contract. Preserve the existing delete default; attached compute is detached with action=Detach.
Authored-by: GitHub Copilot CLI v1.0.88`nModel: GPT-6 Sol (gpt-6-sol)`nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>`nCopilot-Session: 31c47880-d3a1-4cc6-a893-ab264315c7db
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.
Document and snapshot the new compute entity; preserve ARM resource IDs and normalize provisioning states from get and list. Pin the authoritative TypeSpec source for later client regeneration.
Authored-by: GitHub Copilot CLI v1.0.88
Model: GPT-6 Sol (gpt-6-sol)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 31c47880-d3a1-4cc6-a893-ab264315c7db
⚠️PR Size Warning: This PR changes 448 lines across 14 files (441 insertions, 7 deletions). Consider splitting into smaller, focused PRs for easier review. A possible boundary is ARM client/spec model support first, then the public compute entity, YAML schema, API snapshot, and tests; the dependent pieces must still ship together for functional support.
FabricSpark.properties is optional in both the generated model and the pinned TypeSpec contract, but this loader dereferences it unconditionally. A contract-valid get/list response with no nested details will raise AttributeError instead of returning a compute. Either make properties required upstream and regenerate, or handle None here and reconcile the entity/schema's required IDs.
Permit read-only Fabric Spark compute responses without optional nested properties. Continue to require all four resource IDs when attaching a pool, and keep the public API snapshot in sync.
Authored-by: GitHub Copilot CLI v1.0.88
Model: GPT-6 Sol (gpt-6-sol)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 31c47880-d3a1-4cc6-a893-ab264315c7db
Addressed the previously missed Copilot review-summary finding in 1280790: get/list now deserialize FabricSpark when the optional nested properties object is absent, returning a typed resource with unavailable Fabric IDs omitted. Attach remains strict: YAML requires the four IDs and serializing an incomplete read-only entity raises ValidationException rather than sending an invalid PUT. Added a regression test; focused compute tests pass (42 passed, 1 skipped). No inline review threads remain open.
Serialize pool IDs without an invalid ARM resourceId, permit sparse read-only YAML responses, and enforce complete IDs when attaching. Correct API metadata using the snapshot tool's normalized digest.
Authored-by: GitHub Copilot CLI v1.0.88
Model: GPT-6 Sol (gpt-6-sol)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 31c47880-d3a1-4cc6-a893-ab264315c7db
pool_name is read-only in the REST model (FabricSparkProperties.pool_name has read visibility only) and _to_rest_object() never sends it, but this signature advertises it as a caller-supplied attach option. A user can pass a value that is silently ignored. Remove it from the public constructor signature and populate the returned attribute through the internal REST-loading path (for example via **kwargs), then regenerate the API snapshot.
The reason will be displayed to describe this comment to others. Learn more.
Confirmed: this source commit is not yet reachable from Azure/azure-rest-api-specs/main. The authoritative FabricSpark contract is in Azure/azure-rest-api-specs#47205, whose published-version Swagger BreakingChange gate is still failing. We cannot replace this pin with a main-reachable commit containing FabricSpark until that upstream PR is approved and merged. I am leaving this thread open rather than claiming the SDK release-validation requirement is met; once the spec lands, I will update the pin to its main-reachable commit, verify the generator contract, and resolve this thread.
This comment was generated by an AI assistant on behalf of the PR author.
Preserve poolName in get/list output without accepting it as an attach input. Match existing compute identity-helper lint conventions and regenerate the public API snapshot.
Authored-by: GitHub Copilot CLI v1.0.88
Model: GPT-6 Sol (gpt-6-sol)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 31c47880-d3a1-4cc6-a893-ab264315c7db
Pipeline build 6951821 — python - pullrequest (Build Test windows2022_312) failed, with two unrelated test failures on windows2022_312:
sdk.ml.azure-ai-ml.tests.batch_services.e2etests.test_batch_endpoint.TestBatchEndpoint.test_batch_endpoint_create (whl artifact) — failed on setup with azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'None' while the test-proxy attempted to start a playback session (PLAYBACK_START_URL returned a non-200 status).
sdk.ml.azure-ai-ml.tests.utils.unittests.test_artifact_cache.test_concurrent_cache_writer_waits_for_checksum[set] (sdist artifact) — failed with FileExistsError: [WinError 183] Cannot create a file when that file already exists at azure/ai/ml/_utils/_artifact_utils.py:572 (os.rename(download_path, artifact_package_path)). This is a known Windows-specific os.rename semantics difference (POSIX overwrites the destination silently; Windows raises if it exists) surfacing in a concurrency test designed to race two cache writers.
Relevant pipeline output
E azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'None'
..\..\..\.venv\azure-ai-ml\.venv_whl\Lib\site-packages\devtools_testutils\proxy_testcase.py:117: HttpResponseError
E FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\cloudtest\\...\\package' -> 'C:\\Users\\cloudtest\\...\\1.2.3'
..\..\..\.venv\azure-ai-ml\.venv_sdist\Lib\site-packages\azure\ai\ml\_utils\_artifact_utils.py:572: FileExistsError
Both failing tests live in tests/batch_services/ and tests/utils/unittests/test_artifact_cache.py, and the second failure's underlying code is in azure/ai/ml/_utils/_artifact_utils.py — none of which this PR touches. The PR's diff is scoped to FabricSpark compute entity/schema code (azure/ai/ml/_schema/compute/, azure/ai/ml/entities/_compute/, azure/ai/ml/_restclient/arm_ml_service/, constants/_compute.py) and compute unit tests.
Recommended next steps
Re-run the pipeline — failure 1 looks like a transient test-proxy playback/infrastructure issue (invalid status 'None') unrelated to this PR's changes.
Failure 2 is a pre-existing Windows-platform race in test_concurrent_cache_writer_waits_for_checksum/_artifact_utils.pyos.rename usage; track it separately as flaky/platform-specific test debt — it is not something this PR's diff caused or can fix.
Copilot detected the failing pipeline and generated the analysis above. To have it attempt a
fix automatically, reply with @copilot please fix the failing pipeline on this PR.
The reason will be displayed to describe this comment to others. Learn more.
appreciate that you added unit tests. can you also add e2e test cases and add the recording of those test cases in this PR?
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background and defect
FabricSpark attachments already work through the AML ARM service, but the Python SDK treats their computeType as unsupported. SDK get/list and the v2 CLI therefore omit the Fabric IDs and pool details.
Change and compatibility
Add a FabricSparkCompute entity, YAML schema, ARM discriminator/models, and typed dispatch. Attach sends the four Fabric resource IDs, while get/list return the pool name, identity, attachment status, and provisioning data. Detach uses the existing begin_delete(name, action="Detach") path; the default Delete behavior for other compute types is unchanged. No Vienna or portal changes are included. Generated ARM models require the API spec/generation pipeline to retain this discriminator in future regenerations.
Dependencies and merge order
The source TypeSpec contract in Azure/azure-rest-api-specs#47205 must clear published-version compatibility review and support reproducible generated models before merge. The CLI pin in Azure/azure-cli-extensions#10459 must follow an SDK release containing this change.
Validation
Focused SDK compute tests: 42 passed, 1 skipped; Black and API digest checks passed. A FabricSpark response without optional nested details now deserializes to a typed compute, while attach requires all four IDs. Local generated-client regeneration stalled before output; model reproducibility remains unverified. Latest PR gate is running. No live detach was attempted.