Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postprocessing, DVC pipeline and Cml workflow #2

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

aldeav
Copy link

@aldeav aldeav commented Mar 3, 2023

Pull Request Template

Description

This PR adds 3 major components:

  1. Postprocessing in R
  2. DVC pipeline (including postprocessing and metrics)
  3. CML workflow

Modifications:

  1. Standalone_GCBM/gcbm_project/run_gcbm.bat - to include run path for windows runner in CML workflow
  2. Standalone_GCBM/tools/CompileGCBMResults/compileresults.py - to create compile_results.log file
  3. Standalone_GCBM/input_database/run_recliner2gcbm.bat - to create recliner_log.log file

Note: DVC pipeline uses batch files for all 6 stages.
Note 2: Added 2 kinds of CML workflow files:

  1. Triggered when the compiled db is pushed and hence only the postprocessing part is executed. The logs are here.
  2. Triggered when the input db is pushed and hence, it runs gcbm, then compiles the results and performs postprocessing. The 2 approaches used:
    a. Using the container rest_api_gcbm to run gcbm and compile results. Then switching to another job for the postprocessing. Logs here - container logs
    b. Using a windows runner to install deps using a requirements file (+ the GDAL whl file), run the gcbm and compile results. Next, postprocessing is performed in the next job which using a ubuntu runner. Logs here - windows logs

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

Refer to Note 2 in the description.

@Simpleshell3
Copy link

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants