Optimize Upgrade path test E2E run

[mukundansundar]

Describe the proposal

Currently for running all the upgrade paths verification, it takes almost 25 minutes for the test suite to complete. (timeout 30mins)

The proposal here is that we only run the latest upgrade path (upgrade and downgrade) against each PR in CLI.
The full suite of upgrade paths will only be run against the master code in a scheduled run.

The issue here is, if there are changes in helm version that breaks the previous upgrade paths, that can only be caught after the code has been merged to master. But still that is a small risk to the benefit of having quicker PR check completion.

One other suggestion is make use of an /ok-to-test command to run the full suite of upgrade path tests as necessary in addition to the scheduled runs. That will also help mitigate this risk.

Thoughts on this? @artursouza @yaron2 @pravinpushkar @shubham1172 @ItalyPaleAle

Release Note

RELEASE NOTE:

[pravinpushkar]

I agree with - one upgrade + one downgrade test along with ability to trigger full suite with /ok-to-test. This can be used on need basis.

[shubham1172]

As a plus, post failures from scheduled upgrade path E2E tests on a Discord channel.

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[shubham1172]

👋🏻🤖