Include R errors in the log files. #18
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The API tests used to setup individual endpoints to run the tests. That is a bit verbose and duplicates a bunch of code from the `api.R` file. This will be made much worse when we introduce `url` parameters to all the endpoints, since all the tests will need to make a API call to fetch the repository first, which will lead to more boilerplate. Using the api object solves this issue.
In the previous design, outpack_server, the runner API and the workers all shared a single outpack directory, and a Git repository (mostly, the workers used a clone of the shared repository for the actual execution). This creates a very tight and brittle coupling between all the components. It makes it impossible to deploy the different components on separate machines. It requires careful reasoning about data races and conflicts between the different bits. It prevents us from sharing worker processes across multiple Packit instances, and it prevents us from using multiple Git repositories within a single instance. The new design completely splits up the storage. - The API server and each worker have their own local Git clones of the repositories, that are directly pulled from the upstream (eg. GitHub). - The API servers and workers store bare Git clones of the repositories, without any worktree. When running a report, workers create a new worktree in a temporary directory, run the report and delete the worktree. This ensures a completely clean slate every time. - The workers use their own outpack store, that is not shared with any other process. - The workers can pull and push packets using any protocol supported by orderly2. In practice, we will be using HTTP to interact with the outpack_server used by Packit. Currently, the workers create a new outpack store for each run, meaning they do not cache any of the packet dependencies and need to download them from the outpack_server from scratch every time. Given that, at least for now, workers and outpack_server will be operating on the same or nearby machines, this seems like a reasonable overhead. Ideally we would keep a per-worker cache, however we need to be careful not to mix packets between different instances. One possible approach may be to re-use the file store, but start from an empty metadata store everytime. This way large unnecessary file downloads are avoided, while preserving some degree of isolation between runs and instances.
Co-authored-by: Paul Liétar <[email protected]>
Co-authored-by: Paul Liétar <[email protected]>
Currently, when an R error is thrown by a report, it is caught by the rrq worker and is stored in Redis, but it is not exposed over the runner API anywhere. Rather than introduce yet another field in API, we print the error from the worker process, which will make it visible at the end of the tasks log file.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, when an R error is thrown by a report, it is caught by the rrq worker and is stored in Redis, but it is not exposed over the runner API anywhere.
Rather than introduce yet another field in API, we print the error from the worker process, which will make it visible at the end of the tasks log file.