Skip to content

Failing CLI tests are not verbose enough #1289

Description

@Flix6x

During development, I encountered a failing test that did not give a very informative error:

        result = runner.invoke(add_schedule_for_storage, cli_input)
    
>       assert result.exit_code == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = <Result KeyError('sensor')>.exit_code

flexmeasures/cli/tests/test_data_add.py:556: AssertionError

To fix the bug, I needed to find where the KeyError('sensor') was coming from. By adding catch_exceptions=False when invoking the CLI runner, I got the info I needed:

>           if result["sensor"].measures_power and result["sensor"].get_attribute(
                "consumption_is_positive", True
            ):
E           KeyError: 'sensor'

flexmeasures/data/services/scheduling.py:357: KeyError

Therefore, I would like us to stop catching exception when we invoke our runner = app.test_cli_runner() once and for all.

Maybe with this?

from flask.testing import FlaskCliRunner

class CustomFlaskCliRunner(FlaskCliRunner):
    def invoke(self, *args, **kwargs):
        kwargs.setdefault("catch_exceptions", False)  # Force catch_exceptions to False
        return super().invoke(*args, **kwargs)

app.test_cli_runner = lambda: CustomFlaskCliRunner(app)

I suggest adding an xfail test to make sure the CustomFlaskCliRunner does its job.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions