Skip to content

New notebook with more details and simplistic example of using the Scheduler for bulk processing#171

Open
skarppinen wants to merge 1 commit intoecmwf-ifs:mainfrom
skarppinen:transformation-scheduler-docs
Open

New notebook with more details and simplistic example of using the Scheduler for bulk processing#171
skarppinen wants to merge 1 commit intoecmwf-ifs:mainfrom
skarppinen:transformation-scheduler-docs

Conversation

@skarppinen
Copy link
Contributor

Hello.

This PR suggests new documentation in the form of a Jupyter Notebook "06_transformations_and_the_scheduler.ipynb" which attempts to explain a bit about the inner workings of the Scheduler, and in particular, how the Scheduler passes data to the transformations. I believe there might be also other data-passing mechanisms that possibly could be covered here as well. This notebook grew out of my own misunderstandings of how Loki processes large source trees, so I believe it may be useful for other new users as well.

There are some blanks and probably even wrong information here that needs to be sorted out. In some bits I purposefully left things out because I was not sure what is the intended usage.

@FussyDuck
Copy link

FussyDuck commented Oct 13, 2023

CLA assistant check
All committers have signed the CLA.

@reuterbal
Copy link
Collaborator

Thanks Santeri, just to acknowledge that we've seen this and are keen to integrate it but it might take a little while to get around to it. Sorry about that!

@codecov
Copy link

codecov bot commented Nov 9, 2023

Codecov Report

Merging #171 (a7ce67b) into main (03324ef) will not change coverage.
Report is 13 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #171   +/-   ##
=======================================
  Coverage   92.13%   92.13%           
=======================================
  Files          90       90           
  Lines       16689    16689           
=======================================
  Hits        15377    15377           
  Misses       1312     1312           
Flag Coverage Δ
lint_rules 96.22% <ø> (ø)
loki 92.10% <ø> (ø)
transformations 91.40% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants