Fix snow stage copy --recursive for vstage (snow://) paths#2786
Open
sfc-gh-moczko wants to merge 1 commit intomainfrom
Open
Fix snow stage copy --recursive for vstage (snow://) paths#2786sfc-gh-moczko wants to merge 1 commit intomainfrom
snow stage copy --recursive for vstage (snow://) paths#2786sfc-gh-moczko wants to merge 1 commit intomainfrom
Conversation
Two bugs prevented recursive copy from working with vstage paths (e.g., `snow://project/.../deployments/DEPLOYMENT$1/`): 1. `iter_stage()` lost the `snow://` prefix when reconstructing file paths from `ls` output. Snowflake's `ls` returns plain relative paths (e.g., `deployments/DEPLOYMENT$1/manifest.yml`) without the `snow://` prefix. The existing code passed these to `build_path()`, which treated them as `@`-prefixed stage paths, producing invalid SQL like `get @deployments/DEPLOYMENT$1/manifest.yml ...` instead of `get 'snow://project/.../manifest.yml' ...`. Fix: Add a vstage-aware branch in `iter_stage()` that reconstructs full paths via `stage_path.root_path() / file["name"]` when the stage path starts with `snow://`. 2. `StagePath.relative_to()` used `PurePosixPath.relative_to()` which is case-sensitive. Snowflake normalizes unquoted identifiers to lowercase in `ls` output (e.g., `DEPLOYMENT$1` becomes `deployment$1`), causing `ValueError` when computing relative paths for local directory structure during recursive download. Fix: Replace with case-insensitive part-by-part comparison that preserves original casing in the returned relative path. Verified against a real DCM project on Snowflake -- 7 files downloaded successfully with correct directory structure preserved. .... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <[email protected]>
sfc-gh-moczko
force-pushed
the
fix/vstage-recursive-copy
branch
from
de5191c to
926d6a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two bugs that caused
snow stage copy --recursiveto fail when downloading from vstage (snow://) paths, particularly DCM deployment stages likesnow://project/.../deployments/DEPLOYMENT$1/.Bug 1 —
iter_stage()losessnow://prefix: Whenlsreturns plain relative paths (e.g.,deployments/DEPLOYMENT$1/manifest.yml),build_path()misinterprets them as@-prefixed stage paths, generating invalid SQL (get @deployments/...instead ofget 'snow://project/...'). Fixed by adding a vstage-aware branch that reconstructs full paths viastage_path.root_path() / file["name"].Bug 2 —
StagePath.relative_to()is case-sensitive: Snowflake normalizes unquoted identifiers to lowercase inlsoutput (DEPLOYMENT$1becomesdeployment$1), butPurePosixPath.relative_to()is case-sensitive, causingValueErrorwhen computing local directory structure. Fixed with case-insensitive part-by-part comparison.Files changed
src/snowflake/cli/_plugins/stage/manager.pyelifbranch initer_stage()forsnow://pathssrc/snowflake/cli/api/stage_path.pyrelative_to()with case-insensitive implementationtests/stage/test_stage.pytests/stage/test_stage_path.pyTest plan