feat(recording): introduce ingestion recording and replay functionality

[sgomezvillamor]

No description provided.

[aikido-pr-checks]

Use of known insecure function to create temp files - high severity
You're using a deprecated Python API tempfile.mktemp that does not guarantee atomicity during the creation and opening of this file. This can lead to unintended access of the file by different processes.

Show Remediation

Remediation - medium confidence
This patch mitigates the use of an insecure temporary file creation function in '_ensure_local_archive' by replacing 'tempfile.mktemp' with 'tempfile.NamedTemporaryFile' for safer temporary file handling.

Suggested change

	local_path = Path(tempfile.mktemp(suffix=".zip"))
	with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as tmpfile:
	local_path = Path(tmpfile.name)

^{View details in Aikido Security}

[aikido-pr-checks]

Potential file inclusion attack via reading file - medium severity
If an attacker can control the input leading into the open function, they might be able to read sensitive files and launch further attacks with that information.

Show Remediation

Remediation - high confidence
This patch mitigates potential file inclusion attacks by implementing path traversal checks on the parameters.

Suggested change

	with open(file_path, "rb") as f:
	if "../" in str(file_path) or "..\\" in str(file_path):
	raise Exception("Invalid file path")
	with open(file_path, "rb") as f:

^{View details in Aikido Security}

[aikido-pr-checks]

Exposed secret in metadata-ingestion/src/datahub/ingestion/recording/archive.py - critical severity
Identified a Private Key, which may compromise cryptographic security and sensitive data encryption.
^{View details in Aikido Security}

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
11137	2	11135	94

View the top 2 failed test(s) by shortest run time

tests.unit.recording.test_config.TestRecordingConfig::test_disabled_s3_requires_output_path

Stack Traces | 0.001s run time

self = <tests.unit.recording.test_config.TestRecordingConfig object at 0x7f056fda3d30>

    def test_disabled_s3_requires_output_path(self) -> None:
        """Test that output_path is required when s3_upload is disabled."""
        with pytest.raises(ValidationError) as exc_info:
>           RecordingConfig(
                enabled=True,
                password=SecretStr("mysecret"),
                s3_upload=False,
            )
E           Failed: DID NOT RAISE <class 'pydantic_core._pydantic_core.ValidationError'>

.../unit/recording/test_config.py:46: Failed

tests.unit.recording.test_config.TestRecordingConfig::test_enabled_with_password

Stack Traces | 0.001s run time

self = <tests.unit.recording.test_config.TestRecordingConfig object at 0x7f056fda3ca0>

    def test_enabled_with_password(self) -> None:
        """Test valid config with password."""
        config = RecordingConfig(
            enabled=True,
            password=SecretStr("mysecret"),
        )
        assert config.enabled is True
        assert config.password is not None
        assert config.password.get_secret_value() == "mysecret"
>       assert config.s3_upload is True
E       AssertionError: assert False is True
E        +  where False = RecordingConfig(enabled=True, password=SecretStr('**********'), s3_upload=False, output_path=None).s3_upload

.../unit/recording/test_config.py:41: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[aikido-pr-checks]

Use of known insecure function to create temp files - high severity
You're using a deprecated Python API tempfile.mktemp that does not guarantee atomicity during the creation and opening of this file. This can lead to unintended access of the file by different processes.

Show Remediation

Remediation - medium confidence
This patch mitigates the use of an insecure temporary file creation function in the '_create_archive' method of the 'recorder' module by replacing 'tempfile.mktemp()' with a more secure 'tempfile.NamedTemporaryFile()' approach.

Suggested change

	archive_path = Path(tempfile.mktemp(suffix=".zip"))
	with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as tmpfile:
	archive_path = Path(tmpfile.name)

^{View details in Aikido Security}

[aikido-pr-checks]

Potential file inclusion attack via reading file - low severity
If an attacker can control the input leading into the open function, they might be able to read sensitive files and launch further attacks with that information.

Remediation: Ignore this issue only after you've verified or sanitized the input going into this function.
^{View details in Aikido Security}

[aikido-pr-checks]

Use of known insecure function to create temp files - high severity
You're using a deprecated Python API tempfile.mktemp that does not guarantee atomicity during the creation and opening of this file. This can lead to unintended access of the file by different processes.

Show Remediation

Remediation - medium confidence
This patch mitigates the use of an insecure temporary file creation function in the '_download_from_s3' method of the replay module by replacing 'tempfile.mktemp' with 'tempfile.NamedTemporaryFile' for secure temporary file handling.

Suggested change

	local_path = Path(tempfile.mktemp(suffix=".zip"))
	with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as tmpfile:
	local_path = Path(tmpfile.name)

^{View details in Aikido Security}