chore(ci): Upgrade performance test apps to React Native 0.85.1 - #6088
1 issue
code-review: Found 1 issue (1 medium)
Medium
startupTimeTest diffMin loosened to -3000 may mask regressions - `performance-tests/metrics-ios.yml:9`
The diffMin threshold for startupTimeTest was relaxed from -20 to -3000, a 150x increase in the allowed negative deviation. This effectively disables the lower-bound regression detection for startup time, since a -3000ms improvement is implausible and any real regression toward that magnitude would no longer be flagged. If the RN 0.85.1 upgrade legitimately changed baseline timings, the new bound should be tightened to a realistic value once the baseline stabilizes, otherwise future startup performance regressions in this performance-critical path will go undetected.
Duration: 22.3s · Tokens: 150.4k in / 575 out · Cost: $0.28
Annotations
Check warning on line 9 in performance-tests/metrics-ios.yml
sentry-warden / warden: code-review
startupTimeTest diffMin loosened to -3000 may mask regressions
The diffMin threshold for startupTimeTest was relaxed from -20 to -3000, a 150x increase in the allowed negative deviation. This effectively disables the lower-bound regression detection for startup time, since a -3000ms improvement is implausible and any real regression toward that magnitude would no longer be flagged. If the RN 0.85.1 upgrade legitimately changed baseline timings, the new bound should be tightened to a realistic value once the baseline stabilizes, otherwise future startup performance regressions in this performance-critical path will go undetected.