feat: Parameters Converter (#718)

* test initializing the `ExtOldModel` * remove unused imported modules * remove unused imported modules * reformat the test_ext_old_to_new.py and test the printed messages * reformat the test_ext_old_to_new.py and test the printed messages * add `InitialCondInterpolationMethod` class and test * add `InitialCondFileType` class and test * fix initial condition tests * create `test_meteo_forcing_file_type` to test `MeteoForcingFileType` * change tests to check for the parameter values not the parameter name * test `MeteoInterpolationMethod` in the `test_meteo_interpolation_methods` * reformat and test the Meteo class * clean tests that were moved to the TestMeteo test class * move `test_missing_required_fields` to the separate `TestMeteo` class * solve the compare floating point values * rename the old `TestMeteo` to `TestExtModel` * reformat the `TestExtModel` * add the cases for time_series/boundary condition files as forcing files to the `TestMeteo` * correct field names in the `InitialCondition` class * test the `InitialConditions` class for all possible behaviours * fix issue in assigning mutable object (list) as a default * test `construct_filemodel_new_or_existing` function * reformat the test_ext_old_to_new.py file to use fixtures instead of the tests.utils module * import the used function directly at the top of the file do not use "module.function" * name test files by their respective modules not inner functions or classes * reformat test file * move check values to the conftest file * test the `_read_ext_old_data` function * test the `InitialConditionConverter` class * reformat test * move the initial condition fields tests from the test_ext.py to the test_inifield * remove duplicated initial condition fields classes * autoformat: isort & black * move the InitialCondition tests to the tests_inifield.py * fix use mutable as default value * create and test `InitialConditionConverter` * autoformat: isort & black * user snake case parameter names * integrate the Initial_Condition converter to the `ext_old_to_new` * autoformat: isort & black * delete test results * fix test error * clean * autoformat: isort & black * remove ignored tests * use default_factory=list instead of [] to prevent using mutable objects as default value * use fixtures instead of variables declared in the `tests.utils` module * convert relative imports to absolute imports * exclude python=3.12.5 bcause of black warning about memory safety issue. * create separate group for the docs dependencies * create `ExternalForcingConverter` class to include all converter functionality * create class method for the `ExternalForcingConverter` from the old external forcing `ExtOldModel` * autoformat: isort & black * convert the `ext_old_to_new` function into a method called `update` in the `ExternalForcingConverter` class * first step to move out the `construct_filemodel_new_or_existing` calling from the `update` method * create setter and getter properties for each model, and make abstract version of the `update` method, separate the `save` functionality * reformat the converter tests * separate tests for the update, save, and add default paths to the models in the constructor method * fix the error of using mutables as a default value. * create separate test class for the update method in the converter * test files for the update meteo test * rename the meteo_converter into converters and merge the boundary coverter to it * move the initial_condition_converter to the converters module * move the initial_condition_converter to the converters module * autoformat: isort & black * add the `BoundaryConditionConverter` to the converter_factory.py * create a `converters.BoundaryConditionConverter` class * fix using mutables as default value * fix using mutables as default value * use the getattr to fetch attributes from the object * create subfolders in the test directory * get rid of relative paths * add init file to the polyline test dir * add tests for different polyline cases (basic case, and with label) * add tests for different polyline cases (basic case, and with label) * split the test_ext.py to a separate testing modules * fix error in Boundary object instantiation in the `BoundaryConditionConverter` * test the `ext.models.boundary` with an existing polyline * test the `ExternalForcingConverter.update` with only boundary condition data * replace float point comparison with np.isclose * replace float point comparison with np.isclose * remove unused imported function * add a pull request template * correct test folder name from extold to ext * optimize the `ext.models.Boundary.forcing` property to have one return statement at the end of the property. * test the `ExtOldParametersQuantity` class * correct assert statement is `test_ext_old_initial_condition_quantity` test * move the TestModels.TestLateral from thr test_ext.py to a separate test file test_laterals.py * correct the test file description * reformat the test_boundary.py file * clean the test file from unused imported functions * remove unused class `ext.models.InitialConditions` * add reference to the initial condition documentation in the `InitialField` class * add `ParametersConverter` class and test * merge the `BaseConverter` class to the `converters` module * test `MeteoConverter` * merge `converter_factory` module to the `converters` module * merge the `enum_converters` module to `utils` * merge the `__contains__` to the `ConverterFactory` class * abstract duplicate lines in the parameter and initial condition converters * add try, except clause to avoid PermissionError in Windows * update MetroForcingFileType class with updates types * silence dimr and serializer tests * autoformat: isort & black * fix error in using mutables as default value * remove the `initialsalinitytopuse` quantity from the `ExtOldInitialConditionQuantity` class. * add missing initial condition quantities and add `__missing__` method to accept tracer quantities * adjust the `tracer` quantity to `initialtracer` * add list of old initial conditions quantities that comes with a suffix and test them * update docstring * remove duplicate `InitialConditions` class (duplicate if `InitialField`) * fix floating point comparison * fix floating point comparison * remove un-used parameter `postfix and fix sonar warnings` * remove the `locationtype` and convert the value of the extrapolation to yes/no from 1/0 * the `ExternalForcingConverter` class can be instantiated by `ExtOldModel` or a path to external forcing file and the `_read_old_file` not is used inside the constructor to read the external forcing file if path is given to the converter * rename boundary polylines files to `boundary-polyline-` prefix * update converter command line * update references to the `Inifield` class from the documentation * rename the `Advectiontype` to `Advectiontype` in the `ExtOldParametersQuantity` class * rename the `create_convert_inputs` to a descriptive name `create_initial_cond_and_parameter_input_dict` and add docstring * update the pull request template * correct the docstring * replace the string meteo interpolation method by the class method itself * Fix typo in converters.py * Fix docstring for ParameterField in converters.py * fix enum values in test asserts for meteo converter * check docstring * remove duplicated test file because of wrong renaming --------- Co-authored-by: MAfarrag <[email protected]> Co-authored-by: Arthur van Dam <[email protected]>
Deltares · Jan 7, 2025 · d9387e0 · d9387e0
1 parent b2e22aa
commit d9387e0
Show file tree

Hide file tree

Showing 20 changed files with 1,258 additions and 1,077 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -0,0 +1,40 @@
+# Description
+
+- Please include a summary of the change and which issue is fixed.
+- Please also include relevant motivation and context.
+- List any dependencies that are required for this change.
+
+
+- Fixes # (issue)
+## Type of change
+
+Check relevant points.
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] This change requires a documentation update
+
+# How Has This Been Tested?
+
+- Please describe the tests that you ran to verify your changes.
+- Provide instructions so we can reproduce.
+- Please also list any relevant details for your test configuration
+
+- [ ] Test A
+- [ ] Test B
+
+# Checklist:
+Prepare items below using:
+\[ :x: \] (markdown: `[ :x: ]`) for TODO items
+\[ :white_check_mark: \] (markdown: `[ :white_check_mark: ]`) for DONE items
+\[ N/A \] for items that are not applicable for this PR.
+
+
+- [ ] updated version number in setup.py/pyproject.toml/environment.yml.
+- [ ] updated the lock file.
+- [ ] added changes to History.rst.
+- [ ] updated the latest version in README file.
+- [ ] I have added tests that prove my fix is effective or that my feature works.
+- [ ] New and existing unit tests pass locally with my changes.
+- [ ] documentation are updated.
diff --git a/hydrolib/core/dflowfm/ext/models.py b/hydrolib/core/dflowfm/ext/models.py
@@ -103,20 +103,19 @@ def forcing(self) -> Union[ForcingBase, None]:
         Returns:
             ForcingBase: The corresponding forcing data. None when this boundary does not have a forcing file or when the data cannot be found.
         """
-
-        if self.forcingfile is None:
-            return None
-
-        for forcing in self.forcingfile.forcing:
-
-            if self.nodeid != forcing.name:
-                continue
-
-            for quantity in forcing.quantityunitpair:
-                if quantity.quantity.startswith(self.quantity):
-                    return forcing
-
-        return None
+        result = None
+        if self.forcingfile is not None:
+            for forcing in self.forcingfile.forcing:
+
+                if self.nodeid == forcing.name:
+                    if any(
+                        quantity.quantity.startswith(self.quantity)
+                        for quantity in forcing.quantityunitpair
+                    ):
+                        result = forcing
+                        break
+
+        return result
 
 
 class Lateral(INIBasedModel):

diff --git a/hydrolib/core/dflowfm/extold/models.py b/hydrolib/core/dflowfm/extold/models.py
@@ -164,6 +164,21 @@ class ExtOldBoundaryQuantity(StrEnum):
     """Discharge-water level dependency"""
 
 
+class ExtOldParametersQuantity(StrEnum):
+    """Enum class containing the valid values for the Spatial parameter category
+    of the external forcings.
+
+    for more details check D-Flow FM User Manual 1D2D, Chapter D.3.1, Table D.2
+    https://content.oss.deltares.nl/delft3d/D-Flow_FM_User_Manual_1D2D.pdf
+    """
+
+    FrictionCoefficient = "frictioncoefficient"
+    HorizontalEddyViscosityCoefficient = "horizontaleddyviscositycoefficient"
+    HorizontalEddyDiffusivityCoefficient = "horizontaleddydiffusivitycoefficient"
+    AdvectionType = "advectiontype"
+    InfiltrationCapacity = "infiltrationcapacity"
+
+
 class ExtOldMeteoQuantity(StrEnum):
 
     # Meteorological fields

diff --git a/hydrolib/core/dflowfm/inifield/models.py b/hydrolib/core/dflowfm/inifield/models.py
@@ -188,6 +188,9 @@ class InitialField(AbstractSpatialField):
     Initial condition field definition, represents an `[Initial]` block in
     an inifield file.
     Typically inside the definition list of a [FMModel][hydrolib.core.dflowfm.mdu.models.FMModel]`.geometry.inifieldfile.initial[..]`
+
+    All lowercased attributes match with the initial field input as described in
+    [UM Sec.D.2](https://content.oss.deltares.nl/delft3dfm1d2d/D-Flow_FM_User_Manual_1D2D.pdf#subsection.D.2).
     """
 
     _header: Literal["Initial"] = "Initial"

diff --git a/hydrolib/tools/ext_old_to_new/base_converter.py b/hydrolib/tools/ext_old_to_new/base_converter.py
diff --git a/hydrolib/tools/ext_old_to_new/converter_factory.py b/hydrolib/tools/ext_old_to_new/converter_factory.py