Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
226 commits
Select commit Hold shift + click to select a range
66bb5a9
filedicts: unit test example
pgierz Aug 4, 2022
5581aa7
ci: set up automatic testing for pyfakefs
pgierz Aug 4, 2022
7f66e9c
fix: setup.py for pyfakefs tests
pgierz Aug 4, 2022
9aa18e3
ci: prelim work for sprint (needs to be in main branch)
pgierz Aug 4, 2022
735926d
fix: load fake yaml into an actual dictionary
pgierz Aug 4, 2022
69ebf19
feat: example of base FileDicts class
pgierz Aug 8, 2022
b726305
ci: demo for fs creation
pgierz Aug 8, 2022
0fa2ab3
fix: suggestions from Miguel
pgierz Aug 8, 2022
8c26267
Merge pull request #763 from esm-tools/sprint/filedicts/unit_test_exa…
pgierz Aug 8, 2022
ec932db
Merge branch 'sprint/filedicts/main' into sprint/filedicts/base_class
pgierz Aug 8, 2022
1044f54
Merge pull request #766 from esm-tools/sprint/filedicts/base_class
pgierz Aug 8, 2022
37c3f3c
first cp unit test
mandresm Aug 8, 2022
7d028a7
first cp unit test
mandresm Aug 8, 2022
4d79195
use cp as a method of the class
mandresm Aug 8, 2022
7e7e77f
--wip--
pgierz Aug 9, 2022
712e6a6
feat: small init function
pgierz Aug 9, 2022
0f15905
feat: includes Miguel's idea for address in the main config
pgierz Aug 9, 2022
3c4759f
suggestion by Miguel
pgierz Aug 9, 2022
5c699ae
Merge pull request #769 from esm-tools/sprint/filedicts/init
pgierz Aug 9, 2022
68062a2
improvements in the cp unit tests and starting with cp function writting
mandresm Aug 9, 2022
27922c6
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 9, 2022
9e5ae96
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 9, 2022
1d4e073
draft of move function
pgierz Aug 9, 2022
35b0260
first approach to copying, still not working but close
mandresm Aug 9, 2022
85cbd6b
implements skeleton resolve_file_movements and adds unit test. First …
denizural Aug 9, 2022
f46ab12
fixes the cp problems (s missing in path.exists)
mandresm Aug 9, 2022
5a6adc1
seperates function for determining source and target path for reuse
pgierz Aug 9, 2022
5382aeb
removing the underscores from the functions that will be used somewhe…
mandresm Aug 9, 2022
f0e6ab9
moves the changes from filelists to filedicts
denizural Aug 9, 2022
f96b06e
fix copy folder
mandresm Aug 9, 2022
298e37a
adds changes requested from Paul
denizural Aug 9, 2022
a46874a
feat: final steps for mv functionality
pgierz Aug 9, 2022
0217529
more literal variable names as suggested by @denizural
mandresm Aug 9, 2022
240f4c4
test: fix the broken example test
pgierz Aug 9, 2022
c2f20ac
Update src/esm_runscripts/filedicts.py
pgierz Aug 9, 2022
b318daf
docstrings for path_type
mandresm Aug 9, 2022
44e0f14
--wip--
pgierz Aug 9, 2022
46b1d1f
raise Exception to failing copying, as @denizural suggested
mandresm Aug 9, 2022
b7f90a1
tests which magically work (????)
pgierz Aug 9, 2022
3a0ea10
docstrings for check_source_and_target and return of source_path_type…
mandresm Aug 9, 2022
987798c
Merge branch 'sprint/filedicts/main' into sprint/filedicts/allowed_mi…
pgierz Aug 10, 2022
cdc584d
Revert "implements skeleton resolve_file_movements and adds unit test…
denizural Aug 10, 2022
63fcfe4
suggestions from Deniz
pgierz Aug 10, 2022
63b0728
Merge pull request #770 from esm-tools/sprint/filedicts/mv
pgierz Aug 10, 2022
94e70c0
Merge pull request #772 from esm-tools/sprint/filedicts/fix_example_test
pgierz Aug 10, 2022
25b47d8
Merge branch 'sprint/filedicts/main' into sprint/filedicts/resolve_fi…
denizural Aug 10, 2022
85b811c
Merge pull request #771 from esm-tools/sprint/filedicts/resolve_file_…
denizural Aug 10, 2022
3882cad
Merge branch 'sprint/filedicts/main' into sprint/filedicts/allowed_mi…
pgierz Aug 10, 2022
b709990
restores missing test asserts after merge
pgierz Aug 10, 2022
9e311b9
fixing the tuple problem in the tests
mandresm Aug 10, 2022
5e2f9a9
Merge branch 'sprint/filedicts/main' into sprint/filedicts/copy
mandresm Aug 10, 2022
1d766a7
decorator works again merging this morning
pgierz Aug 10, 2022
51f0a26
Merge pull request #767 from esm-tools/sprint/filedicts/copy
mandresm Aug 10, 2022
6302229
tests fixed and path_in_computer implemented
mandresm Aug 10, 2022
084f861
documentation for the decorator
pgierz Aug 10, 2022
c790a1a
checks for absolute paths and unit test
mandresm Aug 10, 2022
4cdad88
--wip--: merge with main; still failing tests
pgierz Aug 10, 2022
d902869
adds filedicts.py to __init__.py
denizural Aug 10, 2022
accfc07
implements linking functionality. TDD rocks
denizural Aug 10, 2022
de92c93
black formatting
denizural Aug 10, 2022
d0689c7
suggestions by @denizural
mandresm Aug 11, 2022
57604aa
Merge pull request #776 from esm-tools/sprint/filedicts/path_in_computer
mandresm Aug 11, 2022
12eb4f4
--wip--
pgierz Aug 11, 2022
432a9b1
Merge pull request #783 from esm-tools/sprint/filedicts/cleanup_mv
pgierz Aug 11, 2022
2bb2411
cleaning of cp method
mandresm Aug 11, 2022
596f13d
Merge pull request #784 from esm-tools/sprint/filedicts/copy_unification
mandresm Aug 11, 2022
35993a8
[WIP] refactoring first steps
denizural Aug 11, 2022
e93b451
Merge branch 'sprint/filedicts/main' into sprint/filedicts/linking
denizural Aug 11, 2022
3115b57
merge back to main
pgierz Aug 11, 2022
0d5589a
ci: pytest workflow fix 001
pgierz Aug 11, 2022
37630f7
ci: test test test
pgierz Aug 11, 2022
6761696
skips doctests meant merely as examples
pgierz Aug 11, 2022
058fff8
ci fix also for no-docker runs
pgierz Aug 11, 2022
8ff05fd
Update filedicts.py
pgierz Aug 11, 2022
caff045
Merge branch 'release' into sprint/filedicts/test_ci
pgierz Aug 11, 2022
5f62fa0
resolve_paths added
mandresm Aug 11, 2022
14a2123
Merge pull request #785 from esm-tools/sprint/filedicts/test_ci
pgierz Aug 11, 2022
afd0def
merge to main up until refactoring of src/pathlib to just src
pgierz Aug 11, 2022
a1d0c74
unit tests updated
mandresm Aug 11, 2022
5d48807
unit test for resolve_paths
mandresm Aug 11, 2022
2e2374f
real config unit test for resolve_paths
mandresm Aug 11, 2022
f077075
completing variables during initialization
mandresm Aug 11, 2022
32d1c2c
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 11, 2022
4bf0d2c
Merge pull request #786 from esm-tools/sprint/filedicts/resolve_paths…
pgierz Aug 12, 2022
bf5c950
refactors file paths to file keys; named tuple, all tests are passing
denizural Aug 12, 2022
1323f7d
Merge branch 'sprint/filedicts/linking' of github.com:esm-tools/esm_t…
denizural Aug 12, 2022
33404cd
black and isort
denizural Aug 12, 2022
86f0e69
fixes the failing test for resolve_file_movements
denizural Aug 15, 2022
77f5f76
Merge pull request #779 from esm-tools/sprint/filedicts/linking
denizural Aug 15, 2022
1fdec05
Apply suggestions from code review
pgierz Aug 15, 2022
7fad3f7
merge main to allowed missing
pgierz Aug 15, 2022
5f9c39f
fixes for broken link tests
pgierz Aug 15, 2022
11dde8c
[WIP] first steps in refactoring and better documentation
denizural Aug 15, 2022
fd33832
skips doctests for now
pgierz Aug 15, 2022
45c9610
Merge pull request #773 from esm-tools/sprint/filedicts/allowed_missi…
pgierz Aug 15, 2022
806033c
Merge branch 'sprint/filedicts/main' into sprint/filedicts/unify_func…
denizural Aug 15, 2022
e19d138
test(test_filedicts): add not-implemented test stubs
pgierz Aug 15, 2022
d9d2948
[WIP] refactors _path_type() method
denizural Aug 15, 2022
9c1f35a
Merge remote-tracking branch 'origin/release' into sprint/filedicts/main
mandresm Aug 15, 2022
81e444e
unit test and first draft of init variables check
mandresm Aug 15, 2022
cf67ed4
[WIP] moves path checks from ln() to _check_source_and_targets(). Add…
denizural Aug 15, 2022
929fa21
adding check for types
mandresm Aug 15, 2022
9bbc18d
adds extra test check for incompatible types
denizural Aug 15, 2022
b02b8c6
refactor test
denizural Aug 15, 2022
3937b29
adds convert_to_path() method and unit tests
denizural Aug 15, 2022
23c327e
fixes minor leftover code
denizural Aug 15, 2022
e8af69d
black formatting
denizural Aug 15, 2022
0c40c48
tests and attributes for datestamp method
pgierz Aug 16, 2022
7b0f5f7
tests and attributes for datestamp format
pgierz Aug 16, 2022
1a1e18d
removes _convert_to_path and its test
denizural Aug 16, 2022
f64d183
brings back capturing output
denizural Aug 16, 2022
7a17c12
adds notes on failing tests
denizural Aug 16, 2022
4a13503
adding more syntax checks and unit tests
mandresm Aug 16, 2022
093f1b1
all_model_filetypes default not defined anymore
mandresm Aug 16, 2022
e95982f
docstrings and cosmetic changes
mandresm Aug 16, 2022
cb77df1
sugestion from @denizural
mandresm Aug 16, 2022
55f3a40
moves comments to docstrings
denizural Aug 17, 2022
1662399
Merge pull request #789 from esm-tools/sprint/filedicts/init_variable…
mandresm Aug 17, 2022
c0c9b4c
defines the default name for missing name variables and allows missin…
mandresm Aug 17, 2022
d75bd64
unit tests for globbing, the new tests fail the right way
mandresm Aug 17, 2022
43abb3b
unit tests for globbing + mv/ln
mandresm Aug 17, 2022
a832f9c
suggestion by @pgierz
mandresm Aug 17, 2022
36a0bc3
complete_file_names method added
mandresm Aug 17, 2022
8fca8dc
Merge branch 'sprint/filedicts/default_names_and_allow_missing_path_i…
mandresm Aug 17, 2022
9522f52
removes self pointing check, add non-existing target check
denizural Aug 17, 2022
9d6dcfa
datestamp feature
pgierz Aug 17, 2022
1b39d44
Merge branch 'sprint/filedicts/main' into sprint/filedicts/datestamp
pgierz Aug 17, 2022
4dd263a
isort and black
pgierz Aug 17, 2022
434f3b0
fixes failing tests from Miguels checks
pgierz Aug 17, 2022
3b183a9
refactors hard-coded string to enum type and adds separate unit tests
denizural Aug 17, 2022
9f95415
fixes bugs, all tests are GREEN
denizural Aug 17, 2022
80294d0
black and isort
denizural Aug 17, 2022
8f67d3a
Update src/esm_runscripts/filedicts.py
pgierz Aug 17, 2022
b9a008c
Merge pull request #791 from esm-tools/sprint/filedicts/default_names…
mandresm Aug 17, 2022
c825f29
Update src/esm_runscripts/filedicts.py
pgierz Aug 17, 2022
0138b52
Update tests/test_esm_runscripts/test_filedicts.py
pgierz Aug 17, 2022
c015625
Update tests/test_esm_runscripts/test_filedicts.py
pgierz Aug 17, 2022
d2864d6
fix for miguels suggestion of comment vs docstring
pgierz Aug 17, 2022
7e2c87f
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 17, 2022
f435ba3
adds some comments
denizural Aug 17, 2022
a7fbc03
wild_card_check functionality and update the unit tests
mandresm Aug 17, 2022
afe0914
globbing decorator working
mandresm Aug 17, 2022
39f0ed9
use the glob_file method, not the general file method, a better practice
mandresm Aug 17, 2022
449b986
fixes idiotic mistakes for allowed missing files
pgierz Aug 18, 2022
d7bfcaf
mini typo
pgierz Aug 18, 2022
c658d3f
finishing touches
pgierz Aug 18, 2022
53787fa
Merge pull request #793 from esm-tools/sprint/filedicts/fix_missing_f…
pgierz Aug 18, 2022
0289e44
wild_card_check rework
mandresm Aug 18, 2022
9cde86a
Update src/esm_runscripts/filedicts.py
pgierz Aug 18, 2022
223390a
Update src/esm_runscripts/filedicts.py
pgierz Aug 18, 2022
c48d41b
globbing files checker method
mandresm Aug 18, 2022
b57d327
docstrings and cleanup
mandresm Aug 18, 2022
a3ae750
mini formatting updates
pgierz Aug 18, 2022
6807bbc
separates the tests for check_source_and_targets()
denizural Aug 19, 2022
ae13ce3
fixes minor typo in the comment
denizural Aug 19, 2022
8e97835
Update src/esm_runscripts/filedicts.py
denizural Aug 19, 2022
32e965a
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 19, 2022
701415c
removes the TODO messages from test file
denizural Aug 19, 2022
bb86a0b
adds the suggestions from Miguel
denizural Aug 19, 2022
2151f81
Update src/esm_runscripts/filedicts.py
mandresm Aug 19, 2022
bbaec58
Update src/esm_runscripts/filedicts.py
mandresm Aug 19, 2022
1036dc0
Update src/esm_runscripts/filedicts.py
mandresm Aug 19, 2022
a26d5b4
Update src/esm_runscripts/filedicts.py
mandresm Aug 19, 2022
1c4002c
suggetion by @pgierz
mandresm Aug 19, 2022
602b59a
Update tests/test_esm_runscripts/test_filedicts.py
mandresm Aug 19, 2022
a3d2cad
undo the method return simplification for the globbing loop
mandresm Aug 19, 2022
42b849d
Merge pull request #792 from esm-tools/sprint/filedicts/globbing
mandresm Aug 19, 2022
6dfa480
unit tests
mandresm Aug 19, 2022
49db64e
merge main
pgierz Aug 19, 2022
df313da
merge main again
pgierz Aug 19, 2022
4327a19
Merge pull request #788 from esm-tools/sprint/filedicts/datestamp
pgierz Aug 19, 2022
57b7b9c
makedirs_in_name functionality
mandresm Aug 19, 2022
1adb4d1
adds logger. adds __str__
denizural Aug 19, 2022
ca9846b
fixes the missing quotes
denizural Aug 19, 2022
e1c3d2f
polishing and docstrings
mandresm Aug 19, 2022
ebf7fa7
removes separator
denizural Aug 19, 2022
0dcbdf0
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 19, 2022
87151ec
Merge pull request #796 from esm-tools/sprint/filedicts/name_subfolders
mandresm Aug 19, 2022
6e76a54
fixes most of the mypy warnings / errors
denizural Aug 19, 2022
3867c7f
Merge remote-tracking branch 'origin/sprint/filedicts/main' into spri…
mandresm Aug 19, 2022
051a262
move __str__ definition to the correct block
mandresm Aug 19, 2022
cf36bbd
Merge pull request #797 from esm-tools/sprint/filedicts/loguru
mandresm Aug 19, 2022
2e1098e
wip
pgierz Aug 19, 2022
df56473
removes copy_files() function
denizural Aug 19, 2022
fc39b82
black formatting
denizural Aug 19, 2022
f1989cc
adds flake8 suggestions
denizural Aug 19, 2022
393aad9
changes makedirs_in_name to a private method
mandresm Aug 19, 2022
b25ff7a
adds docstrings to undocumented methods
pgierz Aug 19, 2022
e9030bd
black formatting
denizural Aug 19, 2022
1bfee54
Update src/esm_runscripts/filedicts.py
pgierz Aug 19, 2022
307c780
Revert "black formatting" -> avoids massive merging problems
mandresm Aug 19, 2022
f93602c
Merge pull request #801 from esm-tools/sprint/filedicts/docstrings
mandresm Aug 19, 2022
7384c85
Revert "black formatting" -> avoids massive merging conflicts
mandresm Aug 19, 2022
bc56f37
blocks I hope
pgierz Aug 19, 2022
6e6940c
Merge pull request #799 from esm-tools/sprint/filedicts/remove_copy_f…
mandresm Aug 19, 2022
41e35f1
Merge pull request #800 from esm-tools/sprint/filedicts/cleanup_priva…
mandresm Aug 19, 2022
c083d49
Merge branch 'sprint/filedicts/main' into sprint/filedicts/mypy_fixes
mandresm Aug 19, 2022
99c0277
fix for private names
pgierz Aug 19, 2022
15fc2f1
Merge pull request #802 from esm-tools/sprint/filedicts/blocks
pgierz Aug 19, 2022
13476e5
Merge branch 'sprint/filedicts/main' into sprint/filedicts/mypy_fixes
mandresm Aug 19, 2022
82414c9
Merge pull request #798 from esm-tools/sprint/filedicts/mypy_fixes
mandresm Aug 19, 2022
e3579ce
black and isort
mandresm Aug 19, 2022
c0b9abd
Merge branch 'release' into sprint/filedicts/main
mandresm Mar 15, 2023
1c7d366
add SimulationFiles class placeholder
mandresm Mar 15, 2023
4a4d08e
correct typos in filedicts.py comments and docstrings
mandresm Mar 16, 2023
1b1e9c9
fix typo in the wild_card_check method that was breaking the pytest
mandresm Mar 17, 2023
6bffbd2
Merge pull request #919 from esm-tools/sprint/filedicts/wild_card_fix
mandresm Mar 17, 2023
49b5a07
docs: since I already had the file open, some comments for the calendar
Mar 20, 2023
ff87efd
add method to Date to find whether the date is in a given date interval
mandresm Mar 22, 2023
2388e71
Merge branch 'sprint/filedicts/main' into sprint/filedicts/calendar_d…
mandresm Mar 23, 2023
a2e398c
Merge pull request #921 from esm-tools/sprint/filedicts/calendar_docs…
mandresm Mar 23, 2023
bddd670
Update src/esm_calendar/esm_calendar.py
mandresm Mar 24, 2023
74a73cc
Merge branch 'release' into sprint/filedicts/class_constructor
pgierz Apr 24, 2023
3804549
Update src/esm_calendar/esm_calendar.py
mandresm Apr 24, 2023
66573ad
Update src/esm_calendar/esm_calendar.py
mandresm May 2, 2023
efaf14a
Update src/esm_calendar/esm_calendar.py
mandresm May 2, 2023
133a9df
Update src/esm_calendar/esm_calendar.py
mandresm May 2, 2023
e4bcba8
Update src/esm_calendar/esm_calendar.py
mandresm May 2, 2023
1dd7de1
Update tests/test_esm_calendar/test_esm_calendar.py
mandresm May 2, 2023
11d4812
add some changes for @pgierz suggestions
mandresm May 2, 2023
50d595d
Merge pull request #923 from esm-tools/sprint/filedicts/is_in_date_in…
pgierz May 2, 2023
685f389
feat(filedicts.py): includes new constructor to couple SimulationFile…
pgierz May 3, 2023
3af4693
Merge branch 'sprint/filedicts/main' into sprint/filedicts/class_cons…
pgierz May 22, 2023
86a92b4
Merge branch 'release' into sprint/filedicts/class_constructor
pgierz May 22, 2023
072bf91
Merge pull request #942 from esm-tools/sprint/filedicts/class_constru…
pgierz May 23, 2023
e0abb8f
paul's stuff
pgierz Jun 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/esm_tools_actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,14 @@ jobs:
run: |
./install.sh

- name: Run Pytest
- name: Install Test requirements
run: |
# pip install .[test] # <-- PG: seems not to work for reasons I do not really understand.
pip install pytest
pip install pyfakefs

- name: Run Pytest
run: |
pytest -v .
# For now, only test actual source code. Stuff like coupling scripts is ignored
pytest -v --doctest-modules src --ignore src/esm_runscripts/coupling --ignore-glob "*backup*"
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/esm_tools_actions_novm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,17 @@ jobs:
run: |
./install.sh

- name: Run Pytest
- name: Install Test requirements
run: |
# pip install .[test] # <-- PG: seems not to work for reasons I do not really understand.
pip install pytest
pip install pyfakefs

- name: Run Pytest
run: |
pytest -v .
# For now, only test actual source code. Stuff like coupling scripts is ignored
pytest -v --doctest-modules src --ignore src/esm_runscripts/coupling --ignore-glob "*backup*"

- name: run esm_master first time
run: |
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ universal = 1

[flake8]
exclude = docs
max-line-length = 88
max-line-length = 300 # This is dumb, but my editor keeps throwing errors at me :-(

[aliases]

Expand Down
8 changes: 7 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"PyGithub",
"colorama",
"coloredlogs",
"dpath==2.0.6",
"emoji",
"f90nml",
"gfw-creator",
Expand All @@ -39,7 +40,11 @@

setup_requirements = []

test_requirements = ["pyfakefs"]
test_requirements = ["pytest", "pyfakefs"]

extras = {
"test": test_requirements,
}

setup(
author="Dirk Barbi",
Expand Down Expand Up @@ -91,6 +96,7 @@
setup_requires=setup_requirements,
test_suite="tests",
tests_require=test_requirements,
extras_require=extras,
url="https://github.com/esm-tools/esm_tools",
version="6.21.11",
zip_safe=False,
Expand Down
Binary file added src/esm_calendar/.esm_calendar.py.swp
Binary file not shown.
24 changes: 24 additions & 0 deletions src/esm_calendar/esm_calendar.py
Original file line number Diff line number Diff line change
Expand Up @@ -895,3 +895,27 @@ def sub_tuple(self, to_sub):

new_date = self.from_list(result)
return new_date

def is_in_interval(self, interval: tuple) -> bool:
"""
Finds if a date falls in an open interval.


Parameters
----------
interval : tuple of Date
A date interval in which there are 2 Dates objects.

Returns
-------
True :
If the date of ``self`` is within the open interval defined by ``interval``
False :
If the date of ``self`` is out of the open interval defined by ``interval``
"""

if len(interval) != 2:
raise TypeError(f"You must have {interval} be a tuple of two Dates!")

# Check whether the ``self`` date falls within the interval
return interval[0] < self < interval[1]
1 change: 1 addition & 0 deletions src/esm_runscripts/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from .database import *
from .database_actions import *
from .dataprocess import *
from .filedicts import *
from .filelists import *
from .last_minute import *
from .namelists import *
Expand Down
Loading