Skip to content

Use environment variables for S3 bucket prefixes#395

Merged
nickclyde merged 6 commits intomainfrom
nickclyde/s3-prefixes
Mar 30, 2026
Merged

Use environment variables for S3 bucket prefixes#395
nickclyde merged 6 commits intomainfrom
nickclyde/s3-prefixes

Conversation

@nickclyde
Copy link
Copy Markdown
Member

Description

This PR standardizes the use of environment variables for S3 bucket and key prefixes, refactors S3 client usage for efficiency and testability, and updates tests and helper functions accordingly.

Lambda Handler (packages/lambda-handler/src/lambda_handler/lambda_handler.py):

  • Added optional s3_client parameter to get_file_content_from_s3() and put_file() for client reuse

TTC Lambda (packages/text-to-code-lambda/src/text_to_code_lambda/lambda_function.py):

  • Added S3_BUCKET env var (default: dibbs-text-to-code)
  • Updated TTC_INPUT_PREFIX default to TextToCodeValidateSubmissionV2/
  • Removed all .split("/")[0] bucket derivations. Now uses S3_BUCKET + prefix as object key
  • Threaded s3_client through helper functions

Augmentation Lambda (packages/augmentation-lambda/src/augmentation_lambda/lambda_function.py):

  • Added S3_BUCKET, AUGMENTED_EICR_PREFIX, AUGMENTATION_METADATA_PREFIX env vars
  • Added S3 client caching and S3 writes for augmented eICR and metadata after augmentation

Augmentation main.py (packages/augmentation/src/augmentation/main.py):

  • Replaced hardcoded bucket names with env var driven pattern

Terraform:

  • _variables.tf: Updated ttc_input_prefix default, added augmented_eicr_prefix and augmentation_metadata_prefix variables
  • main.tf: Renamed BUCKET_NAME → S3_BUCKET

Tests: Updated all test fixtures and assertions for the single-bucket + prefix pattern. All 27 tests pass across all three packages.

Related Issues

Closes #349

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 27, 2026

Codecov Report

❌ Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.73%. Comparing base (13c0bb0) to head (a8dae11).

Files with missing lines Patch % Lines
...-lambda/src/augmentation_lambda/lambda_function.py 93.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
- Coverage   93.75%   93.73%   -0.02%     
==========================================
  Files          42       42              
  Lines        2145     2156      +11     
==========================================
+ Hits         2011     2021      +10     
- Misses        134      135       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@robertandremitchell robertandremitchell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Copy link
Copy Markdown
Collaborator

@BradySkylight BradySkylight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work... some clean-up along the way with some good naming conventions and using env variables. 🚀

@nickclyde nickclyde merged commit c9a4ea4 into main Mar 30, 2026
5 checks passed
@nickclyde nickclyde deleted the nickclyde/s3-prefixes branch March 30, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update TTC Lambda input and output buckets

4 participants