v0.7.3 - Link Handling Modes

--link-handling Flag for MOVE Operations (#48)

This release adds link handling modes to control behavior when MOVE encounters symlinks/junctions that would create cycles with the destination.

The Problem

After using preserve MOVE ... -L junction to relocate subdirectories, users couldn't move parent directories because the nested junctions pointed back to the destination:

# Step 1: Move subdirectory with junction
preserve MOVE "C:\cache\huggingface" --dst "E:\cache\huggingface" -L junction

# Step 2: Later, try to consolidate parent - BLOCKED!
preserve MOVE "C:\cache" --dst "E:\cache"
# Error: Cycle detected - nested junction points to destination

The Solution

New --link-handling MODE flag:

Mode	Behavior
block	(default) Block operation if cycle-creating links found
skip	Skip links, move non-link content only
unlink	Remove links pointing to destination after successful move

# Skip the problematic links, move everything else
preserve MOVE "C:\cache" --dst "E:\cache" --link-handling skip

# Consolidation mode - remove links after move
preserve MOVE "C:\cache" --dst "E:\cache" --link-handling unlink

Added

• --link-handling CLI flag with block, skip, unlink modes
• LinkHandlingMode enum with from_string() parser
• LinkInfo dataclass for link analysis
• analyze_link() and decide_link_action() functions
• creates_cycle field in link report
• 19 new unit tests (284 total)

Phase 2 (Future)

• recreate mode: Copy links to destination with adjusted targets
• ask mode: Interactive prompt for each link

See #48 for progress on Phase 2.

Related

• #47 - Deep cycle detection (v0.7.2)
• #49 - Performance: In-situ detection for large trees

v0.7.2 - Deep Nested Cycle Detection

Deep Path Cycle Detection (#47)

This release adds critical safety improvements to prevent catastrophic data loss when moving directory structures containing nested symlinks or junctions that point back to the destination.

The Problem

When using preserve MOVE with the --create-link flag to relocate directories while maintaining compatibility via junctions/symlinks, a dangerous scenario could occur:

1. Move a subdirectory: preserve MOVE C:\base\path\to\sub --dst E:\backup\path\to\sub -L junction
2. Later move the parent: preserve MOVE C:\base\path --dst E:\backup\path -L junction

Step 2 would traverse the junction created in step 1, causing the MOVE to copy files from the destination onto themselves - resulting in data loss.

The Solution

Preserve now performs deep tree scanning before any MOVE operation:

• Deep Link Discovery: Walks the source tree with followlinks=False to find all symlinks and junctions
• Cycle Detection: Checks if any discovered link's target resolves to or inside the destination
• Early Blocking: Runs BEFORE file expansion to prevent even partial traversal of dangerous links
• Link Report: Shows all links found for visibility and debugging

Added

• detect_path_cycles_deep() function with comprehensive tree traversal
• Early cycle detection in MOVE handler before os.walk() expansion
• Link discovery report showing all symlinks/junctions in source tree
• 21 unit tests covering nested junction/symlink scenarios

Changed

• MOVE preflight now uses deep cycle detection (traverses entire source tree)
• COPY preflight uses simple cycle detection (top-level only, less critical since source files aren't deleted)

Technical Details

• Handles: junctions (Windows), symlinks (all platforms), circular links
• Uses inode tracking to prevent infinite loops from circular symlinks
• Cross-platform using os.path.samefile() for path comparison

v0.7.1 - Bug Fixes

Destination Awareness Release (0.7.x)

This release adds intelligent handling of pre-existing files at destinations, conflict resolution, and recovery tools for interrupted operations.

New Features (v0.7.0)

Destination Scanning & Conflict Resolution (#39)

• Pre-operation scanning: Analyze destination before COPY/MOVE operations
• Conflict resolution modes via --on-conflict:
  • skip - Keep destination, skip source
  • overwrite - Replace destination with source
  • newer - Keep whichever file has newer mtime
  • larger - Keep whichever file is larger
  • rename - Keep both, rename source with suffix
  • fail - Abort if any conflicts exist
  • ask - Interactive prompting (experimental)
• --incorporate-identical - Resume interrupted operations by marking matching files as incorporated
• --scan-only - Analyze destination without executing

CLEANUP Command (#43)

New command for recovering from interrupted MOVE operations:

• --mode status - Analyze partial move state
• --mode complete - Finish interrupted move
• --mode rollback - Undo partial move
• Supports --dry-run for safe previewing

Smart Path Mode Warnings (#42)

• Detects common mistakes with --abs and --rel path modes
• Shows visual preview of what would happen
• --no-path-warning to suppress warnings when intentional

Bug Fixes (v0.7.1)

• --scan-only manifest migration bug - Scan-only mode was incorrectly triggering manifest migration, violating the read-only contract
• Removed --trust-path-mode alias - Redundant alias for --no-path-warning removed to simplify CLI

Documentation

• New docs/cli-reference.md with v0.7.x feature documentation
• New ROADMAP.md outlining future development
• Updated CHANGELOG with v0.7.0 and v0.7.1 entries

Stats

• 244 tests passing (50 new tests added)
• 4000+ lines of new code

Related Issues

• Addresses #39 (Destination-Aware Operations) ~85%
• Addresses #42 (Smart Path Mode Detection) ~90%
• Addresses #43 (CLEANUP Command) ~90%
• Created #47 for symlink/junction cycle detection (safety enhancement)
• Previously completed: #40 (Disk Space Checking), #41 (Permission Errors)

What's Next (v0.8.x)

• File lineage tracking (#44)
• Symlink/junction cycle detection (#47)
• RESUME command for interrupted operations
• --newer-by configurable time comparison (#46)

v0.7.0 - Destination Awareness

Destination Awareness Release

This release adds intelligent handling of pre-existing files at destinations, conflict resolution, and recovery tools for interrupted operations.

New Features

Destination Scanning & Conflict Resolution (#39)

• Pre-operation scanning: Analyze destination before COPY/MOVE operations
• Conflict resolution modes via --on-conflict:
  • skip - Keep destination, skip source
  • overwrite - Replace destination with source
  • newer - Keep whichever file has newer mtime
  • larger - Keep whichever file is larger
  • rename - Keep both, rename source with suffix
  • fail - Abort if any conflicts exist
  • ask - Interactive prompting (experimental)
• --incorporate-identical - Resume interrupted operations by marking matching files as incorporated
• --scan-only - Analyze destination without executing

CLEANUP Command (#43)

New command for recovering from interrupted MOVE operations:

• --mode status - Analyze partial move state
• --mode complete - Finish interrupted move
• --mode rollback - Undo partial move
• Supports --dry-run for safe previewing

Smart Path Mode Warnings (#42)

• Detects common mistakes with --abs and --rel path modes
• Shows visual preview of what would happen
• --no-path-warning to suppress warnings when intentional

Documentation

• New docs/cli-reference.md with v0.7.x feature documentation
• New ROADMAP.md outlining future development

Stats

• 242 tests passing
• 82 new tests added
• 4000+ lines of new code

Related Issues

• Addresses #39 (Destination-Aware Operations) ~85%
• Addresses #42 (Smart Path Mode Detection) ~90%
• Addresses #43 (CLEANUP Command) ~90%
• Previously completed: #40 (Disk Space Checking), #41 (Permission Errors)

What's Next (v0.8.x)

• File lineage tracking (#44)
• RESUME command for interrupted operations
• --newer-by configurable time comparison (#46)

v0.6.2 - Smarter Safety Checks & Enhanced Messaging

What's New in v0.6.2

Smarter Disk Space Checking

• Fixed: Previous 10% margin could block valid transfers (91GB transfer with 100GB free would fail)
• New logic: recommended_free = max(1GB, transfer_size * 5%)
• Hard fail only when transfer literally won't fit
• Soft warning when remaining space below recommended minimum

--ignore Flag for Pre-flight Checks

Skip specific checks when you know what you're doing:

preserve MOVE source --dst dest --ignore space           # Skip low space warnings
preserve MOVE source --dst dest --ignore permissions     # Skip permission pre-checks
preserve MOVE source --dst dest --ignore space,permissions  # Skip both

Enhanced MOVE Operation Messaging

Clear feedback on what happened:

============================================================
MOVE Operation Summary
============================================================
  Files processed:  100
  Copy failed:      0
  Verified:         100

  Successfully MOVED (source deleted):     95
  COPIED ONLY (source file retained):      5

SUCCESS: All files moved. Source files have been deleted.
============================================================

When partial failure occurs:

• Explains two-phase process (COPY → VERIFY → DELETE)
• Shows exactly how many files moved vs retained
• Provides guidance for completing the move (--force)
• Explains that source files are SAFE

Interactive Prompting (MOVE)

When soft warnings detected, prompts: Continue anyway? [y/N]

---

0.6.x Series Summary

v0.6.0 - Link Creation

• --create-link / -L flag for MOVE operations
• Create junctions, symlinks, or hard links from source to destination
• Link-aware RESTORE operation
• New preservelib/links.py module

v0.6.1 - Pre-flight Safety Checks

• Disk space validation before operations
• Permission checking (read, write, delete)
• Windows path backslash detection
• Zero tolerance for destructive MOVE operations

v0.6.2 - This Release

• Smarter space check logic (1GB minimum + 5% of transfer)
• --ignore flag for bypassing checks
• Enhanced MOVE messaging for partial failures
• Interactive prompting for soft warnings

---

Install

pip install dazzle-preserve==0.6.2

Coming in 0.7.x

• CLEANUP command for partial MOVE recovery (#43)
• Destination-aware operations (#39)
• RESUME command for interrupted operations

Full Changelog: v0.6.1...v0.6.2

v0.6.1 - Pre-flight Safety Checks

What's New

Pre-flight Disk Space Checking (#40)

• Validates destination has sufficient space before MOVE/COPY operations
• Includes 10% safety margin by default
• Prevents partial transfers due to disk full errors
• Clear error messages with exact space requirements

Permission Error Handling (#41)

• Tests write permissions before destructive MOVE operations
• Checks read/delete permissions on source files for MOVE
• Clear error messages with remediation suggestions

Windows Path Backslash Detection

• Catches common --dst "E:\" quoting issue
• Backslash before closing quote escapes it, capturing subsequent arguments
• Helpful error message explains the problem and solution

Behavior

• MOVE operations: Fail completely if ANY pre-flight check fails (zero tolerance for destructive ops)
• COPY operations: Warn but continue for non-critical permission issues

Testing

• 27 unit tests for space and permission checking functions

Note

Core pre-flight safety checks are implemented. Additional features (opt-out flags, dry-run space display, Windows security descriptor testing) tracked in #40 and #41.

Install

pip install dazzle-preserve==0.6.1

Full Changelog: v0.6.0...v0.6.1

v0.6.0 - Link Creation for MOVE Operations

What's New

Link Creation on MOVE (--create-link / -L)

Create filesystem links from source to destination after move operations:

• Supported link types: junction (Windows NTFS), soft (symlink), hard, auto
• Enables moving large directories (e.g., model caches) while maintaining app compatibility
• Example: preserve MOVE "C:\cache\hub" -r --abs --dst "E:\cache" -L junction

Link-aware RESTORE

• Automatically detects links at restore destinations
• Tracks links created by preserve in manifest (link_result field)
• Safely removes links before restoring files
• Shows clear warnings for untracked links

New Module: preservelib/links.py

Cross-platform link detection, creation, and removal:

• create_link(), remove_link(), is_link(), detect_link_type(), verify_link()

Testing

• 20 unit tests covering junction, symlink, hard link operations
• CLI argument parsing tests

Changes

• Manifest schema extended with optional link_result field (backward compatible)
• MOVE handler now supports link creation after successful file transfer
• RESTORE handler checks for and handles links before restoration

Install

pip install dazzle-preserve==0.6.0

v0.5.2 - Unified Verbosity System & Critical Bug Fixes

v0.5.2 - First Public Release

Welcome to the first public release of Preserve - a handy file backup and preservation tool designed for easily mapping data back to its source (no more copying files and folders all over the place!) with verification at every step.

🎯 What is Preserve?

Preserve is a command-line tool that provides enterprise-grade file backup, verification, and restoration capabilities. Unlike simple copy utilities, Preserve creates detailed manifests of every operation, tracks file integrity with cryptographic hashes, and ensures your data can be verified and restored with complete confidence.

🚀 Key Features

Core Operations

• COPY - Create verified backups with manifest tracking
• MOVE - Relocate files with automatic verification
• VERIFY - Check file integrity using cryptographic hashes
• RESTORE - Return files to original or specified locations
• CONFIG - View and manage configuration settings

Path Preservation Modes

Choose how your directory structure is preserved:

• --abs - Preserve absolute paths (creates drive letter directories on Windows)
• --rel - Preserve relative paths from common parent
• --flat - Flatten all files into single directory
• --includeBase - Include the source directory name in destination structure

Advanced Features

• Manifest System - Every operation creates a detailed JSON manifest tracking:

  • Source and destination paths
  • File hashes (MD5, SHA1, SHA256, SHA512)
  • Timestamps and metadata
  • Operation parameters

• Numbered Manifests - Multiple operations to same destination create sequential manifests (_001, _002, etc.)

• Batch Operations - Process multiple files and directories:

  • --loadIncludes - Load file list from text file
  • --recursive - Process directories recursively
  • --exclude - Exclude files matching patterns
  • --max-depth - Limit directory traversal depth

• Time-Based Selection:

  • --newer-than "2 hours" - Files modified in last 2 hours
  • --newer-than "2025-01-15" - Files modified after specific date

• Verification Options:

  • Multiple hash algorithms (MD5, SHA1, SHA256, SHA512)
  • Three-way verification on restore
  • Detailed verification reports

🆕 New in v0.5.2

Bug Fixes

• Fixed subdirectory preservation with --includeBase flag (#32)
• Fixed RESTORE --dst functionality (#30)
• Fixed help display regression

New Features

• Unified Verbosity System (#9)
  • Progressive levels: quiet, normal, -v, -vv, -vvv
  • Color-coded output: ✅ success, ❌ errors, ⚠️ warnings, ⊖ skipped
  • Consistent flags across all operations

📋 Example Usage

Basic Backup Workflow

# 1. Create a backup with verification
preserve COPY C:\important\data --dst E:\backup --recursive --abs --hash SHA256

# 2. Verify the backup integrity
preserve VERIFY --src E:\backup

# 3. Restore if needed
preserve RESTORE --src E:\backup --dst C:\restored\data

Advanced Features

# Backup files from a list
preserve COPY --loadIncludes files-to-backup.txt --dst E:\backup --rel

# Backup only recent files
preserve COPY C:\projects --dst E:\backup --newer-than "7 days" --recursive

# Exclude certain patterns
preserve COPY C:\source --dst E:\backup --exclude "*.tmp" --exclude "*.log" --recursive

# List available restore points
preserve RESTORE --src E:\backup --list

# Restore from specific backup
preserve RESTORE --src E:\backup --number 2

Verbosity Control

# Quiet mode - minimal output
preserve COPY src dst --quiet

# Verbose - see file operations
preserve COPY src dst -v

# Very verbose - detailed operations
preserve COPY src dst -vv

# Debug mode - full diagnostic output
preserve COPY src dst -vvv

🔒 Data Integrity Features

• Cryptographic Hashing - Every file is hashed using your choice of algorithm
• Manifest Tracking - Complete record of every operation
• Three-Way Verification - Verify source, backup, and restored files match
• Atomic Operations - Operations complete fully or roll back
• Cross-Platform - Works on Windows, Linux, and macOS

📦 Installation

From Source

git clone https://github.com/djdarcy/preserve.git
cd preserve
pip install -e .

From GitHub Release

pip install git+https://github.com/djdarcy/preserve.git@v0.5.2

Requirements

• Python 3.8 or higher
• Windows, Linux, or macOS

📚 Documentation

• README - Overview, usage guide, and examples
• CHANGELOG - Detailed version history
• GitHub Issues - Report bugs or request features

🛡️ Why Use Preserve?

1. Verification at Every Step - Never wonder if your backup is corrupted
2. Detailed Manifests - Complete audit trail of all operations
3. Flexible Path Handling - Preserve structure exactly as needed
4. Cross-Platform - Same tool on all your systems
5. Open Source - Inspect the code, contribute improvements

🚦 Production Readiness

While this is labeled as v0.5.2, Preserve has been extensively tested with:

• Exhaustive test suite (100+ tests)
• Real-world usage on personal data
• Cross-platform validation
• Edge case handling

The tool is stable for production use with the understanding that:

• API may change before v1.0
• New features are being actively developed
• User feedback drives improvements

🔮 Coming Eventually

• UPDATE command to continuously push to a backup destination (#34)
• Binary distributions for easier installation
• PyPI package publication
• Progress bars for long operations (#29)
• Enhanced manifest validation
• GUI interface (planned)
• Cloud storage integration (planned)

🤝 Contributing

We welcome contributions! Please:

• Report bugs via GitHub Issues
• Suggest features and improvements
• Submit pull requests
• Share your use cases

📄 License

GPL3 License - See LICENSE file for details

🙏 Acknowledgments

Thank you to all early users who provided feedback and bug reports that shaped this release.

---

Get Started: Download and try Preserve today. Your data deserves verification at every step.

For complete version history, see CHANGELOG.md