feat: v3 app builder for Asset Export (Basic) + migration pointers on all legacy packages (AICHAT-1588) - #1004
Merged
Merged
Conversation
…ic (AICHAT-1588) Tenants migrated to v3 native apps no longer carry the legacy Argo template csa-asset-export-basic, so pyatlan.model.packages. AssetExportBasic + client.workflow.run() fails there with 'clusterworkflowtemplates... not found' (IKEA prod, ZD-127901). Adds the typed v3 path, generated from the app's UI configmaps on a tenant with the uber app deployed (same pipeline as every other model/apps builder): - model/apps/csa_uber_asset_export_basic.py: CsaUberAssetExportBasicInputs + fluent CsaUberAssetExportBasic (export_via/export_scope/qn-prefix/ include-* toggles; S3/GCS/ADLS credential staging for CLOUD delivery) - generator MANIFEST entry so full regens include the app - deprecation pointer on the legacy packages class showing the migration Known v3 gap surfaced while validating (for CSA-454, not fixable in pyatlan): the v3 contract + objectstore credential forms carry NO equivalent of the legacy target_prefix — cloud delivery has bucket only.
…lder (AICHAT-1588) Every legacy model.packages class with a model.apps equivalent now carries _APPS_EQUIVALENT, and AbstractPackage's DeprecationWarning names it — 'Use pyatlan.model.apps.CsaUberAssetExportBasic instead.' — so a customer hitting the dead legacy template is one import away from the fix. 18 packages mapped (crawlers, miners, asset-export); the CSA utilities with no v3 builder yet keep the generic AppClient guidance.
…-1588) DIRECT / EMAIL / GCS-cloud flows with payload + credential-body assertions, kept outside tests/unit/apps/ so generator regens can't clear them.
…s hand-written tests (AICHAT-1588) - test_every_apps_equivalent_pointer_resolves: every migration pointer must name a real pyatlan.model.apps class — a typo here or a rename there now fails CI instead of shipping a dead pointer (18 mapped incl. the twin SQLServerCrawler modules) - generate_apps regen now clears only AUTO-GENERATED-bannered test files, so hand-written flow tests live alongside the generated ones in tests/unit/apps/ (moved test_asset_export_flows_handwritten.py in)
…app (AICHAT-1588)
Live testing showed the /v1/app inputs route 500s for CSA uber apps
('app validation error: 404 - No manifest available') — these apps have
no manifest for input-based creation. The UI instead posts an Argo-shaped
native document (legacy templateRef + kebab dag parameters, plus
execution_mode: native, metadata.app_service_url and metadata.entrypoint)
to POST /package-workflows?submit=..., which Heracles converts into the
AE DAG; runs are then managed via the /v1/app APIs.
- AppBuilder grows _SUBMIT_STYLE: 'app' (default, unchanged) vs
'package_workflow' — the latter assembles the native document (verified
field-for-field against a captured working UI submission) and posts it;
staged object-store credentials vault first and ride as credential-guid
- describe() supplies app_service_url when the token can read the
registry, with a generation-time fallback constant
- generator emits the style vars for csa-* apps; AppInfo gains
app_service_url; NativeWorkflowCreateResponse models the create result
- 2 new tests pin the document shape and the transport routing
…-workflows transport The server-side root cause is now fixed at the source (application-sdk #3090 + atlan-csa-uber-app#171: marketplace entry points resolve on the input-contract route and ship real AppInputContracts), validated live on a test tenant. With /v1/app working for CSA uber apps, the builder goes through the official typed API like every other app instead of mimicking the UI's internal package-workflows document — which was pinned to a captured wire format and would rot silently if the frontend reshaped it. Removes _SUBMIT_STYLE/_native_document/_create_native_package, the generator emission, the package-workflows create constant, and the transport tests; AppInfo.app_service_url stays (a real registry field). Requires the csa-uber release carrying the contracts on the target tenant; on older tenants the create fails with the pre-fix 500 (the package-workflows transport lives in git history if an interim escape hatch is ever needed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Tenants migrated to v3 native apps no longer carry the legacy Argo template
csa-asset-export-basic— so the long-documented path (pyatlan.model.packages.AssetExportBasic+client.workflow.run()) fails there with:Hit by a customer in production (ZD-127901 / AICHAT-1588, Sev1). There was no pyatlan code path to run Asset Export on the v3 app (
csa-uber-asset-export-basic).What this adds
1.
pyatlan.model.apps.CsaUberAssetExportBasic— generated from the app's live UI configmaps via the standardgenerate_appspipeline (nothing hand-tuned), mirroring the legacy builder's ergonomics:s3()/gcs()/adls()stage the object-store credential exactly like the legacy methods did (bucket in the credential'sextra). GeneratorMANIFESTgains the app so full regens include it.2. Migration pointers on all legacy packages. Every
model.packagesclass with a typed v3 builder (18 mapped: crawlers, miners, asset-export) now carries_APPS_EQUIVALENT, and the base deprecation warning names it:Packages without an equivalent yet (CSA utilities: asset-import, connection-delete, lineage-builder, …) keep the generic
AppClientguidance. The legacyAssetExportBasicdocstring additionally shows the replacement snippet inline.Known v3 gap surfaced during validation (not fixable in pyatlan)
The v3 contract + object-store credential forms have no equivalent of the legacy
target_prefix(destination path) — verified exhaustively against the live configmaps: 13 contract properties and S3/GCS/ADLS forms carry bucket + auth only. Customers usingobject_store(prefix=...)today lose that knob on v3 — flagged to CSA-454 for the uber app.How it submits — and the server-side fixes this rides on
The builder goes through the standard
POST /v1/apptyped-inputs route, like every other generated builder. Getting that to work for CSA uber apps required fixing the platform, not pyatlan:/v1/app500-ing (app validation error: 404 - No manifest available) for every csa-uber entry point: the app's@entrypointregistry only knows its internal DAG-node workflows, so the SDK's input-contract route 404'd on marketplace names and heracles' create flow died. Full RCA + live route differential: atlan-csa-uber-app#171.POST /package-workflowsdocument was built, validated live (runcsa-uber-asset-export-basic-1786352828-dAeCxBpS, native engine), and then removed in favor of the official API once the server-side fix was proven on the test tenant — mimicking an internal wire format would rot silently. It lives in git history if an escape hatch is ever needed.Merge train — this PR is #3 of 3
AppInputContractper entry point (validated live: all three contracts 200, were 404)Testing
_APPS_EQUIVALENTpointer resolves)./qa-checksclean (1,003 files)/v1/appcreate+run in progress — link will be posted hereCloses AICHAT-1588.
🤖 Generated with Claude Code