Skip to content

feat: v3 app builder for Asset Export (Basic) + migration pointers on all legacy packages (AICHAT-1588) - #1004

Merged
Aryamanz29 merged 8 commits into
mainfrom
aryaman/aichat-1588
Aug 13, 2026
Merged

feat: v3 app builder for Asset Export (Basic) + migration pointers on all legacy packages (AICHAT-1588)#1004
Aryamanz29 merged 8 commits into
mainfrom
aryaman/aichat-1588

Conversation

@Aryamanz29

@Aryamanz29 Aryamanz29 commented Aug 10, 2026

Copy link
Copy Markdown
Member

Problem

Tenants migrated to v3 native apps no longer carry the legacy Argo template csa-asset-export-basic — so the long-documented path (pyatlan.model.packages.AssetExportBasic + client.workflow.run()) fails there with:

clusterworkflowtemplates.argoproj.io "csa-asset-export-basic" not found

Hit by a customer in production (ZD-127901 / AICHAT-1588, Sev1). There was no pyatlan code path to run Asset Export on the v3 app (csa-uber-asset-export-basic).

What this adds

1. pyatlan.model.apps.CsaUberAssetExportBasic — generated from the app's live UI configmaps via the standard generate_apps pipeline (nothing hand-tuned), mirroring the legacy builder's ergonomics:

from pyatlan.model.apps import CsaUberAssetExportBasic

(CsaUberAssetExportBasic(client)
    .export_via("CLOUD")                                    # DIRECT | EMAIL | CLOUD
    .export_scope("ALL")                                    # + GLOSSARIES_ONLY / PRODUCTS_ONLY / ENRICHED_ONLY
    .qualified_name_prefix_for_assets("default/bigquery/123")
    .gcs(username="project", password="{svc-json}", gcs_bucket="exports")
    .run())

s3()/gcs()/adls() stage the object-store credential exactly like the legacy methods did (bucket in the credential's extra). Generator MANIFEST gains the app so full regens include it.

2. Migration pointers on all legacy packages. Every model.packages class with a typed v3 builder (18 mapped: crawlers, miners, asset-export) now carries _APPS_EQUIVALENT, and the base deprecation warning names it:

Atlan package workflows are deprecated… Use pyatlan.model.apps.CsaUberAssetExportBasic instead.

Packages without an equivalent yet (CSA utilities: asset-import, connection-delete, lineage-builder, …) keep the generic AppClient guidance. The legacy AssetExportBasic docstring additionally shows the replacement snippet inline.

Known v3 gap surfaced during validation (not fixable in pyatlan)

The v3 contract + object-store credential forms have no equivalent of the legacy target_prefix (destination path) — verified exhaustively against the live configmaps: 13 contract properties and S3/GCS/ADLS forms carry bucket + auth only. Customers using object_store(prefix=...) today lose that knob on v3 — flagged to CSA-454 for the uber app.

How it submits — and the server-side fixes this rides on

The builder goes through the standard POST /v1/app typed-inputs route, like every other generated builder. Getting that to work for CSA uber apps required fixing the platform, not pyatlan:

  • Live debugging on a test tenant showed /v1/app 500-ing (app validation error: 404 - No manifest available) for every csa-uber entry point: the app's @entrypoint registry only knows its internal DAG-node workflows, so the SDK's input-contract route 404'd on marketplace names and heracles' create flow died. Full RCA + live route differential: atlan-csa-uber-app#171.
  • An interim transport that mimicked the UI's POST /package-workflows document was built, validated live (run csa-uber-asset-export-basic-1786352828-dAeCxBpS, native engine), and then removed in favor of the official API once the server-side fix was proven on the test tenant — mimicking an internal wire format would rot silently. It lives in git history if an escape hatch is ever needed.

Merge train — this PR is #3 of 3

🚫 Functionally blocked on the csa-uber release: the /v1/app path only works on tenants running a csa-uber build with the marketplace input contracts. Merging earlier is safe code-wise but the builder returns the pre-fix 500 on unfixed tenants.

  1. application-sdk#3090 — input-contract route resolves marketplace (bundle) entry points
  2. atlan-csa-uber-app#171 — ships AppInputContract per entry point (validated live: all three contracts 200, were 404)
  3. This PR

Testing

  • 5 generator-emitted unit tests (defaults, payload, credential staging per store)
  • 3 hand-written flow tests (DIRECT / EMAIL / GCS-cloud payload + credential body, secret redaction)
  • deprecation-warning guard tests (mapped class names its equivalent; every _APPS_EQUIVALENT pointer resolves)
  • full unit suite 6,879 green · ./qa-checks clean (1,003 files)
  • Live on a test tenant running the fixed csa-uber build: all five app routes verified (see Dx 238 - Logging hygiene cleanup #171's results table); end-to-end /v1/app create+run in progress — link will be posted here

Closes AICHAT-1588.

🤖 Generated with Claude Code

…ic (AICHAT-1588)

Tenants migrated to v3 native apps no longer carry the legacy Argo
template csa-asset-export-basic, so pyatlan.model.packages.
AssetExportBasic + client.workflow.run() fails there with
'clusterworkflowtemplates... not found' (IKEA prod, ZD-127901).

Adds the typed v3 path, generated from the app's UI configmaps on a
tenant with the uber app deployed (same pipeline as every other
model/apps builder):

- model/apps/csa_uber_asset_export_basic.py: CsaUberAssetExportBasicInputs
  + fluent CsaUberAssetExportBasic (export_via/export_scope/qn-prefix/
  include-* toggles; S3/GCS/ADLS credential staging for CLOUD delivery)
- generator MANIFEST entry so full regens include the app
- deprecation pointer on the legacy packages class showing the migration

Known v3 gap surfaced while validating (for CSA-454, not fixable in
pyatlan): the v3 contract + objectstore credential forms carry NO
equivalent of the legacy target_prefix — cloud delivery has bucket only.
…lder (AICHAT-1588)

Every legacy model.packages class with a model.apps equivalent now carries
_APPS_EQUIVALENT, and AbstractPackage's DeprecationWarning names it —
'Use pyatlan.model.apps.CsaUberAssetExportBasic instead.' — so a customer
hitting the dead legacy template is one import away from the fix. 18
packages mapped (crawlers, miners, asset-export); the CSA utilities with
no v3 builder yet keep the generic AppClient guidance.
…-1588)

DIRECT / EMAIL / GCS-cloud flows with payload + credential-body assertions,
kept outside tests/unit/apps/ so generator regens can't clear them.
@linear

linear Bot commented Aug 10, 2026

Copy link
Copy Markdown

AICHAT-1588

…s hand-written tests (AICHAT-1588)

- test_every_apps_equivalent_pointer_resolves: every migration pointer must
  name a real pyatlan.model.apps class — a typo here or a rename there now
  fails CI instead of shipping a dead pointer (18 mapped incl. the twin
  SQLServerCrawler modules)
- generate_apps regen now clears only AUTO-GENERATED-bannered test files, so
  hand-written flow tests live alongside the generated ones in
  tests/unit/apps/ (moved test_asset_export_flows_handwritten.py in)
…app (AICHAT-1588)

Live testing showed the /v1/app inputs route 500s for CSA uber apps
('app validation error: 404 - No manifest available') — these apps have
no manifest for input-based creation. The UI instead posts an Argo-shaped
native document (legacy templateRef + kebab dag parameters, plus
execution_mode: native, metadata.app_service_url and metadata.entrypoint)
to POST /package-workflows?submit=..., which Heracles converts into the
AE DAG; runs are then managed via the /v1/app APIs.

- AppBuilder grows _SUBMIT_STYLE: 'app' (default, unchanged) vs
  'package_workflow' — the latter assembles the native document (verified
  field-for-field against a captured working UI submission) and posts it;
  staged object-store credentials vault first and ride as credential-guid
- describe() supplies app_service_url when the token can read the
  registry, with a generation-time fallback constant
- generator emits the style vars for csa-* apps; AppInfo gains
  app_service_url; NativeWorkflowCreateResponse models the create result
- 2 new tests pin the document shape and the transport routing
…-workflows transport

The server-side root cause is now fixed at the source (application-sdk
#3090 + atlan-csa-uber-app#171: marketplace entry points resolve on the
input-contract route and ship real AppInputContracts), validated live on
a test tenant. With /v1/app working for CSA uber apps, the builder goes
through the official typed API like every other app instead of mimicking
the UI's internal package-workflows document — which was pinned to a
captured wire format and would rot silently if the frontend reshaped it.

Removes _SUBMIT_STYLE/_native_document/_create_native_package, the
generator emission, the package-workflows create constant, and the
transport tests; AppInfo.app_service_url stays (a real registry field).

Requires the csa-uber release carrying the contracts on the target
tenant; on older tenants the create fails with the pre-fix 500 (the
package-workflows transport lives in git history if an interim escape
hatch is ever needed).
@Aryamanz29
Aryamanz29 merged commit 0fdd121 into main Aug 13, 2026
62 of 77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant