Ignoring certain files when using the download_all_games.ps1 script

[pa-0]

Any PowerShell gurus who might have an easy solution to this? I love using the script provided by @habedi to ensure I have all game files backed up, but I wish there was a way to tell goggto ignore certain files. For example, CP_163.zip is an older version of cyberpunk (v1.63) that is 108GB in size and not necessary for my purposes.

How would I go about modifying the script to include maybe an array of files that gogg should skip when downloading all game fies?

[habedi]

@pa-0 What script do you exactly use for downloading/archiving game files?

Oh, just saw that you use download_all_games.ps1.

I think you can use this version of download_all_games.ps1, which includes a SKIP_FILES variable that is a list. Any file names in the list should be skipped (not downloaded).

# To run the script, open PowerShell and execute the following command:
# Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass; .\download_all_games.ps1

# Colors
$RED = "`e[31m"
$GREEN = "`e[32m"
$YELLOW = "`e[33m"
$NC = "`e[0m" # No Color

Write-Host "${GREEN}===================== Download All Games (Windows Powershell Version) ======================${NC}"
Write-Host "${GREEN}The code in this script downloads all games owned by the user on GOG.com with given options.${NC}"
Write-Host "${GREEN}============================================================================================${NC}"

$env:DEBUG_GOGG = 1 # Debug mode enabled
$GOGG = ".\bin/gogg" # Path to Gogg's executable file (for example, ".\bin\gogg")

# Download options
$LANG = "en" # Language English
$PLATFORM = "windows" # Platform Windows
$INCLUDE_DLC = 1 # Include DLCs
$INCLUDE_EXTRA_CONTENT = 1 # Include extra content
$RESUME_DOWNLOAD = 1 # Resume download
$NUM_THREADS = 4 # Number of worker threads for downloading
$FLATTEN = 1 # Flatten directory structure
$OUTPUT_DIR = "./games" # Output directory

# Skip-list: file names (exact names) to remove after download. You can add filenames like 'CP_163.zip'
# Supports exact filename matches. If you want wildcard matching, use patterns or extend the script.
$SKIP_FILES = @(
    'CP_163.zip'
)

# Dry-run: set to $true to only print what would be removed, without deleting files
$DRY_RUN = $false

# Function to clean up the CSV file
function Cleanup
{
  if ($latest_csv)
  {
    Remove-Item -Force $latest_csv
    if ($?)
    {
      Write-Host "${RED}Cleanup: removed $latest_csv${NC}"
    }
  }
}

# Update game catalogue and export it to a CSV file
& $GOGG catalogue refresh
& $GOGG catalogue export ./ --format=csv

# Find the newest catalogue file
$latest_csv = Get-ChildItem -Path . -Filter "gogg_catalogue_*.csv" | Sort-Object LastWriteTime -Descending | Select-Object -First 1

# Check if the catalogue file exists
if (-not $latest_csv)
{
  Write-Host "${RED}No CSV file found.${NC}"
  exit 1
}

Write-Host "${GREEN}Using catalogue file: $( $latest_csv.Name )${NC}"

# Download each game listed in catalogue file, skipping the first line
Get-Content $latest_csv.FullName | Select-Object -Skip 1 | ForEach-Object {
  $fields = $_ -split ","
  $game_id = $fields[0]
  $game_title = $fields[1]
  Write-Host "${YELLOW}Game ID: $game_id, Title: $game_title${NC}"
  & $GOGG download $game_id $OUTPUT_DIR --platform=$PLATFORM --lang=$LANG `
        --dlcs=$INCLUDE_DLC --extras=$INCLUDE_EXTRA_CONTENT --resume=$RESUME_DOWNLOAD --threads=$NUM_THREADS `
        --flatten=$FLATTEN

  # If skip-list is configured, try to find and remove matching files from the output directory
  if ($SKIP_FILES -and $SKIP_FILES.Count -gt 0)
  {
    Write-Host "${YELLOW}Checking for files to skip (dry-run = $DRY_RUN)...${NC}"
    # Find files in the output directory whose Name matches any entry in $SKIP_FILES
    $found = Get-ChildItem -Path $OUTPUT_DIR -Recurse -File -ErrorAction SilentlyContinue | Where-Object { $SKIP_FILES -contains $_.Name }

    if ($found -and $found.Count -gt 0)
    {
      foreach ($f in $found)
      {
        if ($DRY_RUN)
        {
          Write-Host "${YELLOW}Would remove: $($f.FullName)${NC}"
        }
        else
        {
          try
          {
            Remove-Item -Force $f.FullName -ErrorAction Stop
            Write-Host "${RED}Removed skipped file: $($f.FullName)${NC}"
          }
          catch
          {
            Write-Host "${RED}Failed to remove $($f.FullName): $($_.Exception.Message)${NC}"
          }
        }
      }
    }
    else
    {
      Write-Host "${GREEN}No skipped files found for this game.${NC}"
    }
  }

  Start-Sleep -Seconds 1
}

# Clean up
Cleanup

[pa-0]

Thank you!

[pa-0]

@habedi Using your script above:

[pa-0]

Oh nevermind, I see what the script does. It deletes the file after the fact.

[habedi]

I think the script I posted is buggy and won't work. I think we need to add an option to skip downloading files from a list or by matching a pattern in the CLI. I'll add it to the list of features to implement for the next release (if it's not already).

[pa-0]

Well, I don't think it's buggy. I think that it's just the best you can do with a PowerShell script rather than a new built-in method in the gogg tool.

Also another issue I find with the script is situations in which two GOG products might contain duplicate files. What if the user wants to skip those duplicates when downloading one of the GOG products, but retain at least one copy of those duplicates when downloading the other GOG product?

Example: Cyberpunk 2077 and Cyberpunk Digital Goodies are two separate games in the GOG Library. They share many of the same files, but also some unique ones. What if I want to skip all the duplicates when downloading Cyberpunk Digital Goodies but NOT skip those duplicates when downloading Cyberpunk 2077. Using your script, ALL files listed in SKIP_FILES array will be deleted.