-
Notifications
You must be signed in to change notification settings - Fork 9
dlt: Hello, World! #875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dlt: Hello, World! #875
Conversation
WalkthroughNew configuration, secrets, and ignore files have been added to the Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant PipelineScript as Python Script (basic.py/pokemon.py)
participant dlt
participant DataSource as Data Source (API/CSV/SQL)
participant CrateDB
User->>PipelineScript: Run script
PipelineScript->>dlt: Configure pipeline (destination, dataset)
PipelineScript->>DataSource: Fetch data (API/CSV/SQL)
DataSource-->>PipelineScript: Return data
PipelineScript->>dlt: Run pipeline with data
dlt->>CrateDB: Load data into destination
CrateDB-->>dlt: Confirm load
dlt-->>PipelineScript: Return load info
PipelineScript-->>User: Print load info
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (12)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (9)
🧰 Additional context used🪛 Pylint (3.3.7)framework/dlt/basic.py[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one (R1737) 🪛 GitHub Actions: dltframework/dlt/basic.py[error] 105-105: PipelineStepFailed: Pipeline execution failed at stage extract due to ResourceExtractionError caused by 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/dlt-hub/dlt/issues?state=open&per_page=100 🔇 Additional comments (8)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (5)
framework/dlt/.dlt/secrets.toml (1)
1-12: LGTM with suggestions for clarity.The credentials configuration is well-structured for example purposes. The different ports (5432 vs 4200) represent CrateDB's PostgreSQL wire protocol and HTTP interface respectively, but this could be confusing for users.
Consider adding comments to clarify the port usage:
[destination.cratedb.credentials] +# CrateDB PostgreSQL wire protocol port host = "localhost" port = 5432 username = "crate" password = "" [destination.sqlalchemy.credentials] +# CrateDB HTTP interface port drivername = "crate" host = "localhost" port = 4200 username = "crate" password = ""framework/dlt/pokemon.py (1)
37-43: Add basic error handling for production readiness.While the simplicity is perfect for a "Hello World" example, consider adding minimal error handling to make it more robust:
import dlt from dlt.sources.helpers import requests -dlt.run( - data=requests.get("https://pokeapi.co/api/v2/pokemon/").json()["results"], - destination="cratedb", - dataset_name="doc", - table_name="pokemon") +try: + response = requests.get("https://pokeapi.co/api/v2/pokemon/") + response.raise_for_status() + data = response.json()["results"] + + dlt.run( + data=data, + destination="cratedb", + dataset_name="doc", + table_name="pokemon") + print("Pipeline completed successfully!") +except Exception as e: + print(f"Pipeline failed: {e}")framework/dlt/basic.py (3)
18-19: Fix inconsistent comment.The comment mentions DuckDB but the code actually uses CrateDB as the destination.
- # Create a dlt pipeline that will load - # chess player data to the DuckDB destination + # Create a dlt pipeline that will load + # chess player data to the CrateDB destination
119-130: Add docstring and consider more descriptive function name.This function is missing a docstring and the name doesn't clearly indicate it loads GitHub data.
-def load_data_from_source(): +def load_github_data(): + """Load GitHub issues data using the github_api_source.""" pipeline = dlt.pipeline(
132-136: Consider adding the missing GitHub data loading function call.The main block calls three of the four data loading functions but omits
load_data_from_source(). If this function should be demonstrated alongside the others, consider adding it.if __name__ == "__main__": load_api_data() load_pandas_data() load_sql_data() + load_data_from_source() # or load_github_data() if renamed
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
framework/dlt/.dlt/config.toml(1 hunks)framework/dlt/.dlt/secrets.toml(1 hunks)framework/dlt/.gitignore(1 hunks)framework/dlt/basic.py(1 hunks)framework/dlt/pokemon.py(1 hunks)framework/dlt/requirements.txt(1 hunks)
🧰 Additional context used
🪛 Pylint (3.3.7)
framework/dlt/basic.py
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🔇 Additional comments (6)
framework/dlt/.dlt/config.toml (1)
1-12: LGTM! Well-configured for development examples.The configuration appropriately sets DEBUG logging for development and disables telemetry, which is ideal for example projects. The commented placeholders provide good guidance for users.
framework/dlt/.gitignore (1)
1-11: LGTM! Appropriate exclusions for example project.The gitignore correctly excludes development artifacts while allowing the example secrets.toml to be committed. The commented line
# secrets.tomlclearly indicates this is intentional for providing out-of-the-box examples.framework/dlt/pokemon.py (1)
1-34: Excellent documentation! Clear and comprehensive usage instructions.The docstring provides outstanding step-by-step guidance with configuration examples, multiple usage patterns, and helpful commands. This makes the example very accessible to new users.
framework/dlt/basic.py (3)
41-62: LGTM!Good implementation with proper URL handling and consistent pipeline structure.
64-93: Excellent implementation!Good use of connection context manager, streaming with
yield_per, and proper row conversion. This demonstrates best practices for handling large datasets from SQL databases.
95-117: Good implementation of paginated API resource.The pagination handling and optional authentication are well implemented. The imports inside the function are appropriate for optional dependencies.
Regarding the pylint hint about "yield from": this is likely a false positive since you're correctly yielding individual pages from the paginator, not yielding from a simple iterable.
🧰 Tools
🪛 Pylint (3.3.7)
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 5
♻️ Duplicate comments (1)
framework/dlt/basic.py (1)
11-11: Consider the previous feedback about hardcoded credentials.A previous review already suggested using environment variables for the database address instead of hardcoding credentials. This is still a valid security consideration.
🧹 Nitpick comments (2)
framework/dlt/README.md (1)
53-53: Fix grammatical issue in the sentence.The sentence is missing the article "a" before "development sandbox".
-Acquire `cratedb-example` repository, and set up development sandbox. +Acquire `cratedb-example` repository, and set up a development sandbox.🧰 Tools
🪛 LanguageTool
[uncategorized] ~53-~53: You might be missing the article “a” here.
Context: ...cratedb-example` repository, and set up development sandbox. ```shell git clone https://git...(AI_EN_LECTOR_MISSING_DETERMINER_A)
framework/dlt/basic.py (1)
104-110: Use 'yield from' for more concise iteration.The current code yields each page individually, but
yield fromwould be more idiomatic and concise.- for page in paginate( - url, - auth=auth, - paginator=HeaderLinkPaginator(), - params={"state": "open", "per_page": "100"}, - ): - yield page + yield from paginate( + url, + auth=auth, + paginator=HeaderLinkPaginator(), + params={"state": "open", "per_page": "100"}, + )🧰 Tools
🪛 Pylint (3.3.7)
[refactor] 104-110: Use 'yield from' directly instead of yielding each element one by one
(R1737)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
.github/dependabot.yml(1 hunks).github/workflows/framework-dlt.yml(1 hunks)framework/dlt/.dlt/config.toml(1 hunks)framework/dlt/.dlt/secrets.toml(1 hunks)framework/dlt/.gitignore(1 hunks)framework/dlt/README.md(1 hunks)framework/dlt/basic.py(1 hunks)framework/dlt/pokemon.py(1 hunks)framework/dlt/pyproject.toml(1 hunks)framework/dlt/requirements-dev.txt(1 hunks)framework/dlt/requirements.txt(1 hunks)framework/dlt/test.py(1 hunks)
✅ Files skipped from review due to trivial changes (3)
- .github/dependabot.yml
- framework/dlt/pyproject.toml
- framework/dlt/requirements-dev.txt
🚧 Files skipped from review as they are similar to previous changes (5)
- framework/dlt/requirements.txt
- framework/dlt/.dlt/config.toml
- framework/dlt/.gitignore
- framework/dlt/.dlt/secrets.toml
- framework/dlt/pokemon.py
🧰 Additional context used
🪛 GitHub Actions: dlt
framework/dlt/test.py
[error] 4-4: ImportError: cannot import name 'DatabaseAdapter' from 'cratedb_toolkit.util' when importing in test module test.py
framework/dlt/basic.py
[error] 4-4: ModuleNotFoundError: No module named 'pandas' when importing in test module basic.py
🪛 actionlint (1.7.7)
.github/workflows/framework-dlt.yml
5-5: character '~' is invalid for branch and tag names. ref name cannot contain spaces, ~, ^, :, [, ?, *. see man git-check-ref-format for more details. note that regular expression is unavailable. note: filter pattern syntax is explained at https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
(glob)
🪛 LanguageTool
framework/dlt/README.md
[uncategorized] ~53-~53: You might be missing the article “a” here.
Context: ...cratedb-example` repository, and set up development sandbox. ```shell git clone https://git...
(AI_EN_LECTOR_MISSING_DETERMINER_A)
🪛 Pylint (3.3.7)
framework/dlt/basic.py
[refactor] 104-110: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🔇 Additional comments (5)
framework/dlt/README.md (1)
1-71: Excellent documentation structure and comprehensive coverage!The README provides clear setup instructions, configuration examples for both local and cloud CrateDB instances, and helpful appendix sections. The documentation will help users get started quickly with the dlt framework integration.
🧰 Tools
🪛 LanguageTool
[uncategorized] ~53-~53: You might be missing the article “a” here.
Context: ...cratedb-example` repository, and set up development sandbox. ```shell git clone https://git...(AI_EN_LECTOR_MISSING_DETERMINER_A)
framework/dlt/test.py (2)
10-13: Database fixture setup looks appropriate.The session-scoped fixture using
DatabaseAdapter("crate://")follows good testing practices by reusing the database connection across tests.
15-76: Test coverage is comprehensive and well-structured.The test functions provide good coverage of the different pipeline types (API, pandas, SQL, pokemon) with appropriate assertions checking both record counts and specific data content.
.github/workflows/framework-dlt.yml (1)
36-75: Excellent workflow configuration with comprehensive testing setup.The matrix strategy covering multiple Python versions, proper service container setup for CrateDB, and comprehensive dependency caching provide a robust CI environment for testing the dlt framework integration.
framework/dlt/basic.py (1)
14-91: Excellent demonstration of dlt pipeline capabilities!The code provides comprehensive examples of loading data from various sources (API, CSV via pandas, SQL database, GitHub API) into CrateDB. The pipeline configurations are well-structured and the examples will be valuable for users learning the dlt framework.
Also applies to: 94-129, 131-135
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (2)
framework/dlt/basic.py (2)
4-5: Address the missing pandas dependency causing pipeline failure.The pandas import on line 5 is causing a
ModuleNotFoundErroras reported in the pipeline failures. This dependency must be added to the requirements file to make the pipeline functional.🧰 Tools
🪛 GitHub Actions: dlt
[error] 5-5: ModuleNotFoundError: No module named 'pandas' - The test module failed to import because the 'pandas' package is missing.
105-111: Optimize pagination with yield from syntax.Replace the manual loop with
yield fromfor cleaner and more Pythonic code, as suggested by pylint.- for page in paginate( - url, - auth=auth, - paginator=HeaderLinkPaginator(), - params={"state": "open", "per_page": "100"}, - ): - yield page + yield from paginate( + url, + auth=auth, + paginator=HeaderLinkPaginator(), + params={"state": "open", "per_page": "100"}, + )🧰 Tools
🪛 Pylint (3.3.7)
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🧹 Nitpick comments (5)
framework/dlt/basic.py (3)
15-39: Consider adding error handling for API requests.The Chess.com API calls lack comprehensive error handling beyond
raise_for_status(). Consider adding try-catch blocks to handle network timeouts, connection errors, and API rate limiting gracefully.def load_api_data() -> None: """Load data from the chess api, for more complex examples use our rest_api source""" # Create a dlt pipeline that will load # chess player data to the CrateDB destination pipeline = dlt.pipeline( pipeline_name="from_api", destination=dlt.destinations.cratedb(CRATEDB_ADDRESS), dataset_name="doc", ) # Grab some player data from Chess.com API data = [] for player in ["magnuscarlsen", "rpragchess"]: - response = requests.get(f"https://api.chess.com/pub/player/{player}") - response.raise_for_status() - data.append(response.json()) + try: + response = requests.get(f"https://api.chess.com/pub/player/{player}", timeout=30) + response.raise_for_status() + data.append(response.json()) + except requests.exceptions.RequestException as e: + print(f"Failed to fetch data for player {player}: {e}") # noqa: T201 + continue
69-69: Add dependency installation note to docstring.The comment mentions installing pymysql but this should be documented in the requirements file or handled more gracefully if the dependency is missing.
- # NOTE: you'll need to install pymysql with `pip install pymysql` + # NOTE: pymysql dependency should be included in requirements.txt for MySQL connectivity
137-137: Consider uncommenting GitHub data loading for complete demonstration.The
load_github_data()function is commented out in the main execution. Consider either uncommenting it or adding a comment explaining why it's disabled (e.g., requires API key setup).load_api_data() load_pandas_data() load_sql_data() - #load_github_data() + # load_github_data() # Uncomment after setting up GitHub API token in secrets.tomlframework/dlt/README.md (2)
53-53: Add missing article for better grammar.The phrase "set up development sandbox" should include the article "a" for proper English grammar.
-Acquire `cratedb-example` repository, and set up development sandbox. +Acquire `cratedb-example` repository, and set up a development sandbox.🧰 Tools
🪛 LanguageTool
[uncategorized] ~53-~53: You might be missing the article “a” here.
Context: ...cratedb-example` repository, and set up development sandbox. ```shell git clone https://git...(AI_EN_LECTOR_MISSING_DETERMINER_A)
65-65: Clarify the test command syntax.The command
ngr test framework/dltusesngrwhich may not be familiar to all users. Consider adding a brief explanation or alternative command.Invoke the integration test cases. ```shell -ngr test framework/dlt +# Using ngr (if available) or pytest directly +ngr test framework/dlt +# Alternative: pytest framework/dlt/test.py
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
.github/dependabot.yml(1 hunks).github/workflows/framework-dlt.yml(1 hunks)framework/dlt/.dlt/config.toml(1 hunks)framework/dlt/.dlt/secrets.toml(1 hunks)framework/dlt/.gitignore(1 hunks)framework/dlt/README.md(1 hunks)framework/dlt/basic.py(1 hunks)framework/dlt/pokemon.py(1 hunks)framework/dlt/pyproject.toml(1 hunks)framework/dlt/requirements-dev.txt(1 hunks)framework/dlt/requirements.txt(1 hunks)framework/dlt/test.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- .github/dependabot.yml
🚧 Files skipped from review as they are similar to previous changes (9)
- framework/dlt/.dlt/config.toml
- framework/dlt/.gitignore
- framework/dlt/.dlt/secrets.toml
- framework/dlt/pyproject.toml
- framework/dlt/requirements.txt
- framework/dlt/test.py
- framework/dlt/requirements-dev.txt
- .github/workflows/framework-dlt.yml
- framework/dlt/pokemon.py
🧰 Additional context used
🪛 Pylint (3.3.7)
framework/dlt/basic.py
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🪛 GitHub Actions: dlt
framework/dlt/basic.py
[error] 5-5: ModuleNotFoundError: No module named 'pandas' - The test module failed to import because the 'pandas' package is missing.
🪛 LanguageTool
framework/dlt/README.md
[uncategorized] ~53-~53: You might be missing the article “a” here.
Context: ...cratedb-example` repository, and set up development sandbox. ```shell git clone https://git...
(AI_EN_LECTOR_MISSING_DETERMINER_A)
🔇 Additional comments (4)
framework/dlt/basic.py (2)
12-12: Good security improvement with environment variable usage.The hardcoded database address has been properly replaced with environment variable fallback. This addresses the previous security concern effectively.
96-96:❓ Verification inconclusive
Verify secure API key handling.
Ensure that the
api_secret_keyparameter correctly retrieves secrets from dlt's configuration system and never exposes tokens in logs or error messages.
🏁 Script executed:
#!/bin/bash # Description: Check if there are any hardcoded API keys or tokens in the codebase # Expected: No hardcoded secrets should be found echo "Searching for potential hardcoded API keys or tokens..." rg -i "(?:api_key|token|secret).*=.*['\"][a-zA-Z0-9_-]{10,}['\"]" --type py echo "Searching for github token patterns..." rg "gh[ps]_[a-zA-Z0-9_]{36,255}" --type pyLength of output: 928
🏁 Script executed:
#!/bin/bash # Locate the secrets module secrets_file=$(fd -e py secrets.py | head -n1) echo "Secrets module path: $secrets_file" # Show where 'value' is defined or referenced grep -n "value" "$secrets_file" # Print top of the secrets module for context echo -e "\n----- Begin secrets.py (first 200 lines) -----" sed -n '1,200p' "$secrets_file" echo "----- End secrets.py snippet -----" # Inspect basic.py around the github_api_resource definition echo -e "\n----- Snippet from basic.py around line 96 -----" sed -n '85,120p' framework/dlt/basic.py # Check for any logging or printing that might expose api_secret_key echo -e "\n----- Searching for potential exposure of api_secret_key in basic.py -----" grep -nE "api_secret_key|print|logger|logging" framework/dlt/basic.py || echo "No logging or print statements found."Length of output: 2467
Secure API key handling verified—no hard-coded tokens or exposures found
- Ripgrep scan confirms there are no hard-coded API keys or GitHub tokens in the Python sources.
- The
github_api_resourcefunction does not log or print theapi_secret_keyanywhere.- The default value comes from
dlt.secrets.value, which is part of the external DLT package.Please ensure that
dlt.secrets.valueretrieves credentials from a secure store or environment variable and never emits them in logs or error messages.framework/dlt/README.md (2)
10-27: Excellent configuration documentation with clear examples.The configuration examples for both local CrateDB and CrateDB Cloud are well-structured and provide all necessary connection parameters. The TOML format is properly documented.
59-59:❌ Incorrect review comment
Verify requirements.txt path reference.
The command references
requirements.txtin the root directory, but based on the file structure, it should likely reference the local requirements file in the dlt directory.
🏁 Script executed:
#!/bin/bash # Description: Check the actual location of requirements.txt files # Expected: Find the correct requirements file path for the dlt framework echo "Checking for requirements.txt files in the repository..." fd "requirements.*\.txt" --type f echo "Checking the content of framework/dlt directory..." fd . framework/dlt --type f --max-depth 2Length of output: 2879
No change needed: Local requirements file is correct
The
framework/dlt/requirements.txtfile does exist, so running:pip install -r requirements.txtfrom within the
framework/dltdirectory correctly installs the intended dependencies. The reviewer’s concern about referencing the rootrequirements.txtcan be dismissed.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (2)
framework/dlt/basic.py (2)
4-6: Address the missing pandas dependency.The pandas import will cause a
ModuleNotFoundErroras pandas is not listed in the requirements files, which matches the pipeline failure logs.This issue was previously flagged and needs to be resolved by adding pandas to the requirements.txt file.
105-111: Use 'yield from' for cleaner iteration.The manual loop can be simplified using
yield fromas suggested by static analysis and previous review comments.-for page in paginate( - url, - auth=auth, - paginator=HeaderLinkPaginator(), - params={"state": "open", "per_page": "100"}, -): - yield page +yield from paginate( + url, + auth=auth, + paginator=HeaderLinkPaginator(), + params={"state": "open", "per_page": "100"}, +)🧰 Tools
🪛 Pylint (3.3.7)
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🧹 Nitpick comments (6)
framework/dlt/basic.py (6)
1-1: Consider adding more specific type hints for mypy.The broad mypy error code disabling reduces type safety benefits. Consider addressing specific type issues rather than broadly disabling error codes.
-# mypy: disable-error-code="no-untyped-def,arg-type" +# mypy: disable-error-code="no-untyped-def" # TODO: Add proper type hints
27-31: Add error handling for API requests.The Chess.com API requests lack comprehensive error handling beyond
raise_for_status(). Consider adding timeout and retry logic for production use.for player in ["magnuscarlsen", "rpragchess"]: - response = requests.get(f"https://api.chess.com/pub/player/{player}") + response = requests.get( + f"https://api.chess.com/pub/player/{player}", + timeout=30 + ) response.raise_for_status()
71-73: Document the external database dependency.The code connects to a public MySQL instance which may not always be available. Consider adding a comment about this external dependency and potential failure scenarios.
# Use any SQL database supported by SQLAlchemy, below we use a public # MySQL instance to get data. # NOTE: you'll need to install pymysql with `pip install pymysql` # NOTE: loading data from public mysql instance may take several seconds +# NOTE: this relies on external public database availability engine = sa.create_engine(
88-88: Improve lambda readability.The lambda function could be more readable using a comprehension or explicit function.
-data=map(lambda row: dict(row._mapping), rows), +data=(dict(row._mapping) for row in rows),
119-119: Add return type annotation.The function is missing a return type annotation for consistency with other functions.
-def load_github_data(): +def load_github_data() -> None:
133-137: Consider adding error handling for the main execution block.The sequential execution of all pipelines could benefit from error handling to ensure one failure doesn't prevent others from running.
if __name__ == "__main__": - load_api_data() - load_pandas_data() - load_sql_data() - load_github_data() + functions = [load_api_data, load_pandas_data, load_sql_data, load_github_data] + for func in functions: + try: + func() + except Exception as e: + print(f"Error in {func.__name__}: {e}") # noqa: T201
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
.github/dependabot.yml(1 hunks).github/workflows/framework-dlt.yml(1 hunks)framework/dlt/.dlt/config.toml(1 hunks)framework/dlt/.dlt/secrets.toml(1 hunks)framework/dlt/.gitignore(1 hunks)framework/dlt/README.md(1 hunks)framework/dlt/basic.py(1 hunks)framework/dlt/pokemon.py(1 hunks)framework/dlt/pyproject.toml(1 hunks)framework/dlt/requirements-dev.txt(1 hunks)framework/dlt/requirements.txt(1 hunks)framework/dlt/test.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- framework/dlt/.gitignore
🚧 Files skipped from review as they are similar to previous changes (10)
- .github/dependabot.yml
- framework/dlt/requirements-dev.txt
- framework/dlt/.dlt/config.toml
- framework/dlt/requirements.txt
- framework/dlt/.dlt/secrets.toml
- framework/dlt/pyproject.toml
- framework/dlt/pokemon.py
- framework/dlt/README.md
- framework/dlt/test.py
- .github/workflows/framework-dlt.yml
🧰 Additional context used
🪛 Pylint (3.3.7)
framework/dlt/basic.py
[refactor] 105-111: Use 'yield from' directly instead of yielding each element one by one
(R1737)
🔇 Additional comments (1)
framework/dlt/basic.py (1)
12-12: Environment variable usage looks good.The database address is properly configured using environment variables with a sensible default, addressing the previous security concern.
About
A bit of code spiked the other day.
Far from finished, probably.It works.
References