Problem description
Currently, the Experiment class in Experiment.py has several issues leading to errors when running experiments:
- Order of operations —
check_minio_credentials() was called before env_setup(), causing environment variables for MinIO to not be set in time.
- Faulty condition — the code tried to log
self.auth.method even when self.auth was None, raising:
AttributeError: 'NoneType' object has no attribute 'method'
- Missing credentials — if MinIO credentials were only present in the
.cfg file but not exported in the environment, the process failed.
This behavior causes experiments to crash unexpectedly.
Steps to Reproduce
- Run:
mlops run scripts/train.py -c config/local_config.cfg --ignore_git_check
- Observe the crash in
check_minio_credentials().
Proposed Solution
Related Context
- Part of fixing an issue raised in MLOpsTutorial.
- The
.cfg updates to include MinIO credentials are tracked in the MLOpsTutorial repo, while the code fix is in MLOps.
Problem description
Currently, the
Experimentclass inExperiment.pyhas several issues leading to errors when running experiments:check_minio_credentials()was called beforeenv_setup(), causing environment variables for MinIO to not be set in time.self.auth.methodeven whenself.authwasNone, raising:.cfgfile but not exported in the environment, the process failed.This behavior causes experiments to crash unexpectedly.
Steps to Reproduce
check_minio_credentials().Proposed Solution
Change the initialization order:
self.config_setup().self.env_setup()to set environment variables.self.check_minio_credentials().Add setting of MinIO environment variables directly in
env_setup()to ensure they’re available before credentials check.Related Context
.cfgupdates to include MinIO credentials are tracked in the MLOpsTutorial repo, while the code fix is in MLOps.