Refactor/td integration #1437 by Romankkl03 · Pull Request #1439 · aimclub/FEDOT

Romankkl03 · 2026-06-09T14:19:53Z

Summary

Context

сloses #1437

…lidation tests

…management

… parameters, sampling, and prediction intervals

…loading and error handling

…poser requirements

…ased loading and error handling

…on with schema validation and error handling

…ndling

… handling in PipelineRunConfig

github-actions · 2026-06-09T14:21:36Z

Code in this pull request still contains PEP8 errors, please write the /fix-pep8 command in the comments below to create commit with automatic fixes.

Comment last updated at Mon, 13 Jul 2026 21:41:17

Lopa10ko · 2026-06-11T10:06:28Z

+        self.tensor_data_config: Dict[str, Any] = resolve_tensor_data_config(
+            self.get('tensor_data_config'),
+            use_preprocessing_cache=self.get('use_preprocessing_cache', True),
+        )
+


why after the parameters: dict = self._params_repository.check_and_set_default_params(input_params) ?

deleted it in 58778cb

yes, but why

self.tensor_data_config: Dict[str, Any] = resolve_tensor_data_config( self.get('tensor_data_config'), use_preprocessing_cache=self.get('use_preprocessing_cache', True), )

after the

parameters: dict = self._params_repository.check_and_set_default_params(input_params) super().__init__(...)

Lopa10ko · 2026-06-11T10:07:13Z

+        if target is not None:
+            spec_kwargs['target'] = target
+
+        return TensorDataCreationRequest(backend_name=backend_name, spec_kwargs=spec_kwargs)


is the TensorDataCreationRequest necessary?

Now, it one of the way for correct aggregation TDCreator's params. But may be refactored later.

Lopa10ko · 2026-06-11T10:08:41Z

+_ALLOWED_KEYS: Set[str] = _USER_CONFIGURABLE_DATA_SPEC_KEYS | _CREATOR_ONLY_KEYS
+
+# TODO romankuklo: add "cuda:n" to the supported backends
+_SUPPORTED_BACKENDS = frozenset({'cpu', 'gpu'})


this should be initialized inside the Backend singleton

Added name normalization methods into Backend [66b1924]

Lopa10ko · 2026-06-11T10:09:49Z

+def validate_tensor_data_config(config: Optional[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
+    """
+    Validate user-provided ``tensor_data_config`` for :class:`~fedot.api.api_utils.params.ApiParams`.
+
+    The config is a flat dictionary of options forwarded to
+    :meth:`~fedot.core.data.tensor_data.tensor_data_creator.TensorDataCreator.create`
+    (as ``DataSpec`` kwargs) plus ``backend_name``. Runtime values such as ``task``,
+    ``state``, and ``target`` must not be set here — they are injected when data
+    is created during ``fit`` / ``predict``.
+
+    Args:
+        config: User config dictionary or ``None``.
+
+    Returns:
+        A shallow copy of the validated config, or ``None`` when *config* is ``None``.
+
+    Raises:
+        ValueError: If *config* is not a dict or contains unknown / forbidden keys.
+    """
+    if config is None:
+        return None
+    if not isinstance(config, dict):
+        raise ValueError('"tensor_data_config" must be a dictionary or None.')
+
+    unknown_keys = set(config) - _ALLOWED_KEYS
+    if unknown_keys:
+        raise ValueError(
+            f'Unknown keys in "tensor_data_config": {sorted(unknown_keys)}'
+        )
+
+    forbidden_keys = set(config) & _RUNTIME_KEYS
+    if forbidden_keys:
+        raise ValueError(
+            'Keys reserved for runtime injection must not appear in '
+            f'"tensor_data_config": {sorted(forbidden_keys)}'
+        )
+
+    # TODO romankuklo: should remove from here?
+    normalized = dict(config)
+    if 'backend_name' in normalized:
+        normalized['backend_name'] = _normalize_backend_name(normalized['backend_name'])
+
+    if 'use_cache' in normalized and not isinstance(normalized['use_cache'], bool):
+        raise ValueError('"tensor_data_config.use_cache" must be a boolean.')
+
+    return normalized


see #1438 for new validation logic

Lopa10ko · 2026-06-11T10:12:39Z

        )

+
+    def _prepare_fit_context_td(self,


no usage of this method

Lopa10ko · 2026-06-11T10:13:38Z

+    if n_rows <= 0:
+        raise ValueError("n_rows must be positive")
+
+    if max_ohe_width < 0:
+        raise ValueError("max_ohe_width must be non-negative")
+
+    if max_ohe_width_ratio < 0:
+        raise ValueError("max_ohe_width_ratio must be non-negative")


validation through marshmallow

Lopa10ko · 2026-06-11T10:18:15Z

use tuples within methods where applicable for (non-dirty) methods

Lopa10ko · 2026-06-11T10:20:50Z

+def test_api_params_builds_tensor_data_config_field_on_init():
+    params = ApiParams({}, problem='classification', timeout=1)
+
+    assert params.tensor_data_config == {
+        'backend_name': 'cpu',
+        'use_cache': True,
+    }
+    assert params['tensor_data_config'] == params.tensor_data_config


FAILED tests/api/api_utils/test_api_params.py::test_api_params_builds_tensor_data_config_field_on_init - KeyError: 'tensor_data_config'

…d ensemble config, and sampling configuration

…nd composer requirements

…module and updating imports

…data, and pipelines

…ed_cache_dir

…data tests for improved error handling

Lopa10ko · 2026-06-29T15:11:45Z

        self._log = default_log('InputAnalyzer')

-    def give_recommendations(self, input_data: Union[InputData, MultiModalData], input_params=None) \
+    def give_recommendations(self, input_data: Union[InputData, MultiModalData, TensorData], input_params=None) \


should we remove the InputData support in this PR, or have you decided to keep it and address it in the next one?

I'm going to keep InputData before test comparing with TensorData. Furthermore, the vast part of pipeline use ArrayType yet.

Lopa10ko · 2026-06-29T15:28:54Z

+    def fit_tensor_data(self,
+            features: FeaturesType,
+            target: TargetType = 'target',
+            predefined_model: Union[str, Pipeline] = None) -> Pipeline:
+
+        MemoryAnalytics.start()
+
+        initial_timeout = self.params.timeout


would it be better for fit_tensor_data method to work with TensorData not the features-target pair?

So you’re suggesting that the creation TensorData and preprocessing (obligatory) stages should be left to the user?

Lopa10ko · 2026-06-29T15:34:23Z

-        ...
-    """
+    DEFAULT_NAME = 'cpu'
+    GPU_ALIASES = frozenset({'gpu', 'cuda'})


gpus on m-series macs can also be used in mps, so it might be much better not to enforce cuda from gpu user-specs.
instead, it would be better to determine what GPU capabilities are available and assess which ones are compatible with FEDOT

Lopa10ko · 2026-06-29T15:36:21Z

+    # TODO romankuklo: make features as obligatory field
    features: Optional[torch.Tensor] = None


make features obligatory in this PR

Done: 4709069

Lopa10ko · 2026-06-29T15:39:28Z

+from fedot.core.operations.operation_parameters import OperationParameters
+
+
+class TorchLinearClassifier:


should be inherited or be a composite product of ModelImplementation, Operation, or Model - see which one suits better

Done: 4709069

Lopa10ko · 2026-06-29T15:43:19Z

+    def fit_tensordata(self,
+                       params: Optional[Union[OperationParameters, dict]],
+                       data: TensorData,
+                       predictions_cache: Optional[PredictionsCache] = None,
+                       fold_id: Optional[int] = None,
+                       descriptive_id: Optional[str] = None):


here, it's TensorData (see the comment in main.py regarding a similar consideration)

But after automatic TensorData creating

Lopa10ko · 2026-06-29T15:46:06Z

+    def fit_tensordata(self,
+                       tensor_data: TensorData,
+                       predictions_cache: Optional[PredictionsCache] = None,
+                       fold_id: Optional[int] = None) -> TensorData:
+        """Runs training process in the node on TensorData.
+
+        Slice 1 supports only primary nodes without parents (e.g. single predefined model).
+        """
+        self.log.debug(
+            f'Trying to fit pipeline node with operation: {self.operation} on TensorData')
+        tensor_data = self._get_tensor_data(
+            tensor_data=tensor_data, parent_operation='fit')
+
+        if self.fitted_operation is None:
+            with Timer() as t:
+                self.fitted_operation, operation_predict = self.operation.fit_tensordata(
+                    params=self._parameters,
+                    data=tensor_data,
+                    predictions_cache=predictions_cache,
+                    fold_id=fold_id,
+                    descriptive_id=self.descriptive_id,
+                )
+                self.fit_time_in_seconds = round(t.seconds_from_start, 3)
+        else:
+            operation_predict = self.operation.predict_tensordata(
+                fitted_operation=self.fitted_operation,
+                data=tensor_data,
+                params=self._parameters,
+                predictions_cache=predictions_cache,
+                fold_id=fold_id,
+                descriptive_id=self.descriptive_id,
+                is_fit_stage=True,
+            )
+
+        if should_update_node_parameters(self.operation.operation_type, self.operation.metadata.tags):
+            self.update_params()
+        return operation_predict
+
+    def predict_tensordata(self,
+                           tensor_data: TensorData,
+                           output_mode: str = 'default',
+                           predictions_cache: Optional[PredictionsCache] = None,
+                           fold_id: Optional[int] = None) -> TensorData:
+        """Runs prediction process in the node on TensorData."""
+        self.log.debug(
+            f'Trying to predict pipeline node with operation: {self.operation} on TensorData')
+        tensor_data = self._get_tensor_data(
+            tensor_data=tensor_data, parent_operation='predict')
+        with Timer() as t:
+            operation_predict = self.operation.predict_tensordata(
+                fitted_operation=self.fitted_operation,
+                data=tensor_data,
+                params=self._parameters,
+                output_mode=output_mode,
+                predictions_cache=predictions_cache,
+                fold_id=fold_id,
+                descriptive_id=self.descriptive_id,
+                is_fit_stage=False,
+            )
+            self.inference_time_in_seconds = round(t.seconds_from_start, 3)
+        return operation_predict


reuse some methods, clean them up, and replace the * methods with your *_for_tensordata versions

Lopa10ko · 2026-06-29T15:47:17Z

@@ -1,4 +1,8 @@
 {
+  "torch_linear": {
+    "learning_rate": 0.05,


thoughts on LR scheduler?

Done in b411d60

Lopa10ko · 2026-06-29T15:49:47Z

+from fedot.core.operations.evaluation.operation_implementations.models.torch import TorchLinearClassifier
+
+
+class TorchTabularClassificationStrategy(EvaluationStrategy):


why only tabular? can the Torch Linear model NOT be data-agnostic?

Lopa10ko · 2026-06-29T15:51:21Z

                    method=step.method.value if hasattr(step.method, "value") else str(step.method),
                    features_idx=step.features_idx,
                )
+        else:


shouldn't this be logged?

…refactor/td_integration

…ation

…ilding

Lopa10ko · 2026-07-10T12:31:28Z

+            logger.info(
+                f"Using CUDA stack with cudf/cupy and torch.device('{normalized_name}'). ",
+            )
+            return


nit: not urgent, but from a style point of view, rather difficult to read code. frequent checks, ornate code... try to simplify the logic of checks using strategy or factory patterns

maybe dict with strategies

Done, added dict with strategies: c44ce8c

Lopa10ko · 2026-07-10T12:33:46Z

+            )
+
+        normalized = value.strip().lower()
+        if normalized in ('cpu', 'gpu', 'cuda', 'mps'):


no magic set here - ('cpu', 'gpu', 'cuda', 'mps') should be initialized once as a backend layer constant SUPPORTED_BACKENDS or smth simillar

Done: c44ce8c

Lopa10ko · 2026-07-10T12:36:20Z

+from fedot.validation.context import ValidationContext
+
+BACKEND_CUDA_DEVICE_PATTERN = re.compile(r'^cuda:\d+$')
+BACKEND_SUPPORTED_NAMES_HINT = 'cpu, gpu, cuda, mps, cuda:<device_index>'


this should be built in-place (in 24th line) from SUPPORTED_BACKENDS constant set (see comment at 28th line)

Done: c44ce8c

Lopa10ko · 2026-07-10T12:38:03Z

@@ -391,7 +400,8 @@ def _parse_tasks(raw: Optional[str]) -> Optional[list[str]]:


 def main() -> None:
-    parser = argparse.ArgumentParser(description="Run OpenML benchmark with FEDOT TabICL and update foundational column.")
+    parser = argparse.ArgumentParser(
+        description="Run OpenML benchmark with FEDOT TabICL and update foundational column.")
    parser.add_argument("--classification-suite", type=int, default=DEFAULT_CLASSIFICATION_SUITE)


please configure your local linter to 120-symbols of code in one line, no need to format legacy code in that way

Lopa10ko · 2026-07-10T12:40:44Z



 def default_repository_name_for_data(data) -> str:
+    # TODO @romankuklo: delete multi_ts?


idk, could we support multi_ts through TensorData?
multivariate ts and multi_ts are not the same thing - seek guidance from @v1docq

Lopa10ko · 2026-07-10T12:47:32Z

        return valid_builders or [self.assumptions_generator.fallback_builder(self.ops_filter)]


+# TODO: refactor for TensorData if needed


still a question to research

Lopa10ko · 2026-07-10T12:50:16Z

-                                             operations_cache: Optional[OperationsCache] = None,
-                                             preprocessing_cache: Optional[PreprocessingCache] = None,
-                                             eval_n_jobs: int = -1) -> Pipeline:
+    def fit_assumption_and_check_correctness_with_tensordata(


why *_with_tensordata again? no point at providing 2 different signatures here and everywhere else, just refactor the old methods

Done: 879b3e8

Lopa10ko · 2026-07-10T12:52:03Z

            *initial_nodes, use_input_preprocessing=use_input_preprocessing)

+    @classmethod
+    def builder_for_tensordata(cls,


here also, just change logic inside actual builder_for_data, do not introduce a separate builder_for_tensordata - this becomes more difficult to handle in other PRs

maybe just comment the old methods out for now, and introduce the new logic, but not as a separate *tensordata methods

Done: 879b3e8

Lopa10ko · 2026-07-10T13:02:16Z

+def test_regression_metric_tensordata_implementation(metric_class, expected_value):
+    target = np.array([1.0, 2.0, 4.0, 8.0])
+    prediction = np.array([1.1, 1.8, 4.5, 7.5])
+    features = np.array([0.8, 1.2, 2.1, 4.1])
+    task = Task(TaskTypesEnum.regression)
+
+    tensor_reference, tensor_prediction = _build_tensor_metric_data(
+        target, prediction, task, features=features)
+
+    tensor_value = metric_class.metric(tensor_reference, tensor_prediction)
+
+    assert np.isclose(tensor_value, expected_value, rtol=1e-6, atol=1e-6)
+
+
+@pytest.mark.parametrize(
+    'metric_class, prediction, expected_value',
+    [
+        (Accuracy, np.array([0, 1, 1, 0, 1]), -0.6),
+        (F1, np.array([0, 1, 1, 0, 1]), -0.5),
+        (Precision, np.array([0, 1, 1, 0, 1]), -0.5),
+        (ROCAUC, np.array([0.1, 0.8, 0.7, 0.4, 0.9]), -0.6666666666666666),
+        (Logloss, np.array([0.1, 0.8, 0.7, 0.4, 0.9]), 0.6186249239125281),
+    ],


no point in writing tests inside test dir, since it tests legacy code, need rewriting in tests instead and deleting the old ones in test one by one

Will be done later

Lopa10ko · 2026-07-10T13:06:19Z

-                                                               fold_id=fold_id,
-                                                               descriptive_id=self.descriptive_id)
-
-        # Update parameters after operation fitting (they can be corrected)


do not delete previous comments and google-style docstrings for methods

Return comment: 4bc374d

Romankk03 and others added 16 commits June 9, 2026 13:30

refactor(api): added ApiParams for TensorData creating

f1e6394

chore: update requirements with marshmallow for validation

ac79558

feat: add validation error classes to the Fedot module

149b9d0

feat: replace KeyError with FedotInvalidKeysError in API parameter va…

f81d843

…lidation tests

feat: enhance API parameter validation with context support

30d2629

feat: implement validation framework with error handling and context …

35b3f7c

…management

feat: add validation schemas for various configurations including API…

815c953

… parameters, sampling, and prediction intervals

test: add unit tests for sampling configuration schema validation

67c3a47

feat: refactor sampling configuration validation to use schema-based …

9577d08

…loading and error handling

feat: integrate tabular file path validation into tensor data rules

74f1287

feat: implement validation for cross-validation folds in pipeline com…

4931d4a

…poser requirements

feat: enhance chunked ensemble configuration validation with schema-b…

3f2ba1e

…ased loading and error handling

feat: improve parameter validation in PredictionInterval initializati…

8ed1b9c

…on with schema validation and error handling

feat: add hyperparameter validation for external models with error ha…

cd4b62a

…ndling

feat: replace ValueError with FedotValidationError for improved error…

7b58b0c

… handling in PipelineRunConfig

feature(preprocessing): added auto building encoding strategy

3aa0fa3

Romankkl03 self-assigned this Jun 9, 2026

Automated autopep8 fixes

e563b01

Lopa10ko requested changes Jun 11, 2026

View reviewed changes

Romankk03 and others added 10 commits June 11, 2026 13:36

feature: added context for TensorData (optional preprocessing)

8faa489

refactore: refactored apply_default_params in class ApiParamsRepository

b90b723

style: deleted annotation in ApiParams

58778cb

feature: added name and device normilization in Backend

66b1924

ref: refactored preprocessing services for api

6f5e3e8

feature: added warning about strategy for large data

0003846

fix: added new category for label encoder for test data

03e90ca

feat: remove deprecated validation schemas for API parameters, chunke…

331ac1b

…d ensemble config, and sampling configuration

feat: remove deprecated validation schemas for prediction intervals a…

789dc80

…nd composer requirements

feat: refactor API parameter validation by introducing a new schemas …

84ace90

…module and updating imports

Lopa10ko and others added 10 commits June 17, 2026 22:26

test: add comprehensive validation schemas for API, sampling, tensor …

c84ed16

…data, and pipelines

refactor: update import statements in conftest.py files to use isolat…

ec6146d

…ed_cache_dir

test: replace ValueError with FedotValidationError in API and tensor …

331c6ab

…data tests for improved error handling

feat: added fitting pipline for TensorData

256fe5c

feat: added predict method for TensorData

96b94bb

feat: added simple linear torch model

770596b

refactor: changed TensorData creation in the API

bc0eee7

Automated autopep8 fixes

db2c01b

feat: added DataMerger for TensorData

570302c

fix: fixed predict in operations because of transformation operations

16b5aca

Lopa10ko requested changes Jun 29, 2026

View reviewed changes

Lopa10ko marked this pull request as ready for review June 29, 2026 15:53

Romankk03 added 13 commits July 2, 2026 10:23

ref: refactored backend and added mps

dc185d0

Merge remote-tracking branch 'origin/feature/validation-module' into …

67c6bec

…refactor/td_integration

feat: added validation-schemas for backend

104835e

fix: made features as obligatory field for TensorData

4709069

ref: deleted InputData operations in main and pipeline

e212781

ref: refactored data_merger, removed InputData from the node and oper…

b411d60

…ation

ref: refactored tune method for TensorData

7a5f826

ref: refactored metrics for TensorData

8b5d2b6

ref: refactored api get_metrics

d721207

ref: refactored api_composer and gp_composer

9b94d78

ref: refactored assumption handler

288492a

ref: refactored assumptions builder and added automatic assumption bu…

0bcc082

…ilding

feat: added validation schemas for refactored modules

bf59f83

Lopa10ko requested changes Jul 10, 2026

View reviewed changes

Romankk03 and others added 4 commits July 12, 2026 14:38

chore: renaming *_tensordata

879b3e8

chore: returned comment

4bc374d

ref: refactored backend

c44ce8c

chore: remove local deps directory

fdf4c05

		# TODO romankuklo: make features as obligatory field
		features: Optional[torch.Tensor] = None

		from fedot.core.operations.operation_parameters import OperationParameters


		class TorchLinearClassifier:

		from fedot.core.operations.evaluation.operation_implementations.models.torch import TorchLinearClassifier


		class TorchTabularClassificationStrategy(EvaluationStrategy):



		def default_repository_name_for_data(data) -> str:
		# TODO @romankuklo: delete multi_ts?

		return valid_builders or [self.assumptions_generator.fallback_builder(self.ops_filter)]


		# TODO: refactor for TensorData if needed

Uh oh!

Conversation

Romankkl03 commented Jun 9, 2026 • edited by Lopa10ko Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Uh oh!

github-actions Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at Mon, 13 Jul 2026 21:41:17

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Romankkl03 Jun 11, 2026 • edited by Lopa10ko Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Romankkl03 commented Jun 9, 2026 •

edited by Lopa10ko

Loading

github-actions Bot commented Jun 9, 2026 •

edited

Loading

Romankkl03 Jun 11, 2026 •

edited by Lopa10ko

Loading