Fix documentation and the blocking bugs for local backend #249
Conversation
WalkthroughThe changes update several benchmark configurations and function interfaces. Four microbenchmark JSON files now include a new Changes
Sequence Diagram(s)sequenceDiagram
participant Caller
participant generate_input
Caller->>generate_input: Call with (data_dir, size, benchmarks_bucket, input_paths, output_paths, upload_func, nosql_func)
generate_input-->>Caller: Returns generated input dictionary
sequenceDiagram
participant PerfCost
participant DeploymentClient
PerfCost->>DeploymentClient: update_function(function, benchmark, False, '')
DeploymentClient-->>PerfCost: Returns update result
Assessment against linked issues
Suggested reviewers
Poem
Tip ⚡️ Faster reviews with caching
Enjoy the performance boost—your workflow just got faster. ✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
docs/usage.md (1)
180-180: Add language specification to code blockThe fenced code block at line 180 is missing a language specification, which is recommended for proper syntax highlighting.
-``` +```bash🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
180-180: Fenced code blocks should have a language specified
null(MD040, fenced-code-language)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
benchmarks/000.microbenchmarks/010.sleep/config.json(1 hunks)benchmarks/000.microbenchmarks/010.sleep/input.py(1 hunks)benchmarks/000.microbenchmarks/020.network-benchmark/config.json(1 hunks)benchmarks/000.microbenchmarks/020.network-benchmark/input.py(1 hunks)benchmarks/000.microbenchmarks/030.clock-synchronization/config.json(1 hunks)benchmarks/000.microbenchmarks/030.clock-synchronization/input.py(1 hunks)benchmarks/000.microbenchmarks/040.server-reply/config.json(1 hunks)benchmarks/000.microbenchmarks/040.server-reply/input.py(1 hunks)docs/usage.md(2 hunks)sebs/benchmark.py(1 hunks)sebs/config.py(0 hunks)sebs/experiments/perf_cost.py(1 hunks)
💤 Files with no reviewable changes (1)
- sebs/config.py
🧰 Additional context used
🧬 Code Definitions (4)
benchmarks/000.microbenchmarks/010.sleep/input.py (4)
benchmarks/000.microbenchmarks/030.clock-synchronization/input.py (1)
generate_input(6-7)benchmarks/000.microbenchmarks/020.network-benchmark/input.py (1)
generate_input(6-7)benchmarks/000.microbenchmarks/040.server-reply/input.py (1)
generate_input(11-12)sebs/benchmark.py (1)
generate_input(783-794)
sebs/experiments/perf_cost.py (1)
sebs/faas/system.py (1)
update_function(215-234)
benchmarks/000.microbenchmarks/020.network-benchmark/input.py (4)
benchmarks/000.microbenchmarks/030.clock-synchronization/input.py (1)
generate_input(6-7)benchmarks/000.microbenchmarks/040.server-reply/input.py (1)
generate_input(11-12)benchmarks/000.microbenchmarks/010.sleep/input.py (1)
generate_input(11-12)sebs/benchmark.py (1)
generate_input(783-794)
benchmarks/000.microbenchmarks/030.clock-synchronization/input.py (4)
benchmarks/000.microbenchmarks/020.network-benchmark/input.py (1)
generate_input(6-7)benchmarks/000.microbenchmarks/040.server-reply/input.py (1)
generate_input(11-12)benchmarks/000.microbenchmarks/010.sleep/input.py (1)
generate_input(11-12)sebs/benchmark.py (1)
generate_input(783-794)
🪛 markdownlint-cli2 (0.17.2)
docs/usage.md
180-180: Fenced code blocks should have a language specified
null
(MD040, fenced-code-language)
🔇 Additional comments (15)
benchmarks/000.microbenchmarks/040.server-reply/config.json (1)
4-5: Adds required "modules" field to the configuration structure.This change adds the previously missing "modules" field to the configuration, which is required by the BenchmarkConfig class as seen in benchmark.py. This fix addresses one of the PR objectives to ensure the local backend works correctly out-of-the-box.
benchmarks/000.microbenchmarks/010.sleep/config.json (1)
4-5: Adds required "modules" field to the configuration structure.This change adds the previously missing "modules" field to the configuration, which is required by the BenchmarkConfig class as seen in benchmark.py. This fix ensures proper functionality when deserializing the configuration.
benchmarks/000.microbenchmarks/030.clock-synchronization/config.json (1)
4-5: Adds required "modules" field to the configuration structure.The addition of the empty "modules" array ensures the configuration structure matches what's expected by the BenchmarkConfig deserializer. This consistency across benchmarks helps prevent runtime errors.
benchmarks/000.microbenchmarks/020.network-benchmark/config.json (1)
4-5: Adds required "modules" field to the configuration structure.Adding the empty "modules" array ensures this benchmark's configuration follows the same pattern as other benchmarks, providing consistency and preventing potential deserialization issues in the BenchmarkConfig class.
sebs/benchmark.py (1)
433-437: Simplifies Docker image naming by removing version component.This change removes the version segment from the Docker build image name, which streamlines the build process as mentioned in the PR objectives. By using a simpler naming convention, the system will likely be more maintainable and less prone to versioning-related issues.
benchmarks/000.microbenchmarks/010.sleep/input.py (1)
11-11: Function signature updated correctlyThe
generate_inputfunction signature has been updated to match the interface defined insebs/benchmark.py, which helps standardize the benchmark input generation across the codebase.benchmarks/000.microbenchmarks/040.server-reply/input.py (1)
11-11: Function signature updated correctlyThe
generate_inputfunction signature has been updated to match the interface defined insebs/benchmark.py, which helps standardize the benchmark input generation across the codebase.sebs/experiments/perf_cost.py (1)
86-86: Method signature update correctly implementedThe call to
update_functionhas been updated to match the new method signature by adding two additional parameters:Falseforcontainer_deploymentand an empty string forcontainer_uri. This change is consistent with the updated method signature in thesystem.pyfile and aligns with the PR objective to fix blocking bugs for local backend.docs/usage.md (7)
77-79: Storage start command updated correctlyThe command has been updated to use a configuration file instead of hardcoded port values, allowing for more flexible deployment. Using
allparameter suggests it now starts both object and NoSQL storage services as defined in the configuration.
87-89: Configuration update command enhancedThe command now correctly adds architecture information (
x64) to the configuration, which is a necessary parameter for local deployment. The output is also properly directed to a new configuration file.
94-133: Storage configuration structure properly updatedThe JSON structure now correctly includes both object storage (minio) and NoSQL storage (scylladb) sections, providing a comprehensive configuration template. This aligns with the updated command that uses a configuration file.
139-140: Local start command correctly updated with container removal optionThe command now includes the
--remove-containersoption, which is essential information for users who want to automatically clean up after testing. This change aligns with the note added at lines 185-186.
171-176: Function invocation command improved with dynamic valuesThe curl command now dynamically extracts the function URL and input data from the output JSON file using
jq, which is a significant improvement over hardcoded values. This makes the documentation more general and maintainable.
182-183: Storage stop command updated to match the start commandThe command now uses
allparameter to stop all storage services, which is consistent with the updated start command usingallparameter.
185-186: Helpful note about container removalThis addition provides important clarification about container lifecycle management, explaining that containers won't be automatically removed without the specific flag.
benchmarks/000.microbenchmarks/030.clock-synchronization/input.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Hi @mcopik,
Thanks for the review! I'm happy to have the opportunity to contribute to this repository.
I've replied to some of your comments regarding the further development plans. Once you've had a chance to review them, I'll start refining my PR.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
sebs/benchmark.py (1)
443-467: Added robust fallback mechanism for Docker images.Implemented a helper function
ensure_imageand a fallback mechanism that tries to pull the versioned image first and falls back to the unversioned image if necessary. This provides better resilience and backward compatibility.However, there's a minor issue with the exception handling that could be improved.
- except docker.errors.APIError: - raise RuntimeError("Docker pull of image {}:{} failed!".format(repo_name, name)) + except docker.errors.APIError as api_error: + raise RuntimeError("Docker pull of image {}:{} failed!".format(repo_name, name)) from api_error🧰 Tools
🪛 Ruff (0.8.2)
453-453: Within an
exceptclause, raise exceptions withraise ... from errorraise ... from Noneto distinguish them from errors in exception handling(B904)
docs/usage.md (3)
74-75: Grammar: missing articles in the sentence.To improve clarity, consider updating as follows:
- map the container's port to port defined in the configuration on host network, and write storage - instance configuration to file `out_storage.json` + map the container's port to the port defined in the configuration on the host network, and write a storage + instance configuration to the file `out_storage.json`🧰 Tools
🪛 LanguageTool
[uncategorized] ~74-~74: Possible missing article found.
Context: ... container, map the container's port to port defined in the configuration on host ne...(AI_HYDRA_LEO_MISSING_THE)
[uncategorized] ~74-~74: You might be missing the article “the” here.
Context: ...to port defined in the configuration on host network, and write storage instance co...(AI_EN_LECTOR_MISSING_DETERMINER_THE)
[uncategorized] ~75-~75: You might be missing the article “a” here.
Context: ...rite storage instance configuration to fileout_storage.json```bash ./sebs.py s...(AI_EN_LECTOR_MISSING_DETERMINER_A)
93-130: Ensure example JSON is syntactically valid.The provided JSON example now includes both object and NoSQL storage configurations, which reflects the PR changes. However, there is a trailing comma after the closing of the
localobject (Line 130), which is invalid in JSON. Please remove the comma to maintain a valid JSON example.
179-182: Fenced code block language identifier and storage stop command.
- Add
bashafter the opening triple backticks for syntax highlighting.- The new
./sebs.py storage stop all out_storage.jsoncommand correctly includes theallargument as per the PR.🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
179-179: Fenced code blocks should have a language specified
null(MD040, fenced-code-language)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
benchmarks/000.microbenchmarks/020.network-benchmark/input.py(1 hunks)benchmarks/000.microbenchmarks/020.network-benchmark/python/function.py(3 hunks)benchmarks/000.microbenchmarks/030.clock-synchronization/input.py(1 hunks)benchmarks/000.microbenchmarks/030.clock-synchronization/python/function.py(3 hunks)docs/usage.md(2 hunks)sebs/benchmark.py(8 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- benchmarks/000.microbenchmarks/030.clock-synchronization/input.py
🧰 Additional context used
🧬 Code Graph Analysis (1)
benchmarks/000.microbenchmarks/020.network-benchmark/input.py (2)
benchmarks/000.microbenchmarks/030.clock-synchronization/input.py (1)
generate_input(6-12)sebs/benchmark.py (1)
generate_input(800-811)
🪛 LanguageTool
docs/usage.md
[uncategorized] ~74-~74: Possible missing article found.
Context: ... container, map the container's port to port defined in the configuration on host ne...
(AI_HYDRA_LEO_MISSING_THE)
[uncategorized] ~74-~74: You might be missing the article “the” here.
Context: ...to port defined in the configuration on host network, and write storage instance co...
(AI_EN_LECTOR_MISSING_DETERMINER_THE)
[uncategorized] ~75-~75: You might be missing the article “a” here.
Context: ...rite storage instance configuration to file out_storage.json ```bash ./sebs.py s...
(AI_EN_LECTOR_MISSING_DETERMINER_A)
🪛 markdownlint-cli2 (0.17.2)
docs/usage.md
179-179: Fenced code blocks should have a language specified
null
(MD040, fenced-code-language)
🪛 Ruff (0.8.2)
sebs/benchmark.py
453-453: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (17)
benchmarks/000.microbenchmarks/030.clock-synchronization/python/function.py (3)
3-3: Good addition of os module import.The addition of the
osmodule import is necessary to support the use ofos.path.joinfor proper file path construction later in the code.
16-17: Refactored bucket handling for nested structure.The bucket information retrieval has been updated to use a nested structure with
bucket.get('bucket')andbucket.get('output'), improving consistency with the input generation interface.
69-70: Improved file path construction for uploads.Using
os.path.jointo combine the output prefix with the filename ensures that the uploaded file is stored in the correct subdirectory within the bucket, fixing potential path issues in the local backend.benchmarks/000.microbenchmarks/020.network-benchmark/python/function.py (3)
3-3: Good addition of os.path import.The import of
os.pathis necessary to support the proper construction of file paths usingos.path.join.
16-17: Refactored bucket handling for nested structure.The bucket information retrieval has been updated to use a nested structure with
bucket.get('bucket')andbucket.get('output'), improving consistency with the input generation interface.
55-56: Improved file path construction for uploads.Using
os.path.jointo combine the output prefix with the filename ensures that the uploaded file is stored in the correct subdirectory within the bucket, fixing potential path issues in the local backend.benchmarks/000.microbenchmarks/020.network-benchmark/input.py (2)
2-2: Clean code refactoring.Removed unnecessary parentheses around the returned tuple for cleaner code.
5-11: Successfully updated function signature and implementation.The
generate_inputfunction has been properly updated with the new parameter structure, and the return value now correctly uses the nested dictionary format expected by the handler functions. This change successfully addresses the issue mentioned in a previous review comment.The nested structure with
'bucket'and'output'keys properly supports the changes made in the function handler code.sebs/benchmark.py (4)
27-27: Good addition of modules parameter.Added the
modulesparameter to theBenchmarkConfig.__init__method, which aligns with the PR objective to add an empty "modules" entry to the configuration structure of microbenchmarks.
433-442: Improved Docker image naming and versioning.Refactored the Docker image naming logic to separate the base image name from the version suffix, which aligns with the PR objective to streamline the Docker build process.
625-627: Improved string formatting.Minor formatting improvement to the log message string.
800-811: Updated generate_input function signature.The function signature has been updated to match the implementation changes in the benchmark input modules. The addition of the
nosql_funcparameter and the restructured parameters for buckets and paths are consistent with the PR objective to supplement unused parameters in the Python input interface.docs/usage.md (5)
77-79: Storage start command updated correctly.The new
./sebs.py storage start allinvocation aligns with the PR objective, allowing ports to be configured via the JSON file.
86-88: JQ command for updating local storage config is accurate.The
jq '.deployment.local.storage = input'command correctly injects the storage details intoconfig/local_deployment.json.
137-139: Local start command updated correctly.The
--remove-containersand--architecture=x64flags have been added as per the PR objectives, and the example matches the updated CLI interface.
170-174: Dynamiccurlinvocation is clear and accurate.Replacing the hardcoded URL with a
jqextraction simplifies invocation and aligns with the newout_benchmark.jsonstructure.
184-185: Note on container removal is clear.The added note about the
--remove-containersflag accurately informs the user of its effect.
|
@mcopik Hi Marcin, this PR is ready for another review. Please feel free to provide any further feedback! |
|
@rabbull Thank you so much for your help! Merged :) |
This PR introduces serveral mini fixes to make the local backend runnable out-of-the-box again.
basic_imagefield in the configuration for the local backend.update_functionof theLocalsystem.Closes #248.
Summary by CodeRabbit
New Features
Documentation
Refactor