Skip to content

T1771: enable "reboot-on-upgrade-failure" for new VyOS installations #965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 20, 2025

Conversation

c-po
Copy link
Member

@c-po c-po commented May 17, 2025

Change summary

If any part of the system boot fails, we now set overall_status=1 in the vyos-router startup script. When an error during the image upgrade is detected, the system will automatically revert the default boot image to the previously used version.

The user is informed via console messages:

Booting failed, reverting to previous image
Automatic reboot in 5 minutes
Use "reboot cancel" to cancel

The user has 5 minutes to log in and run reboot cancel to remain in the faulty image for troubleshooting.

Once the system boots into the previous image, the MOTD will display a persistent warning message - cleared during next reboot.

WARNING: Image update to "VyOS 1.5.xxxx" failed
Please check the logs:
/usr/lib/live/mount/persistence/boot/NAME/rw/var/log
Message is cleared on next reboot!

Upgrade failure can be synthetically injected by booting with Kernel command line option: vyos-fail-migration

image

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes)
  • Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
  • Other (please describe):

Related Task(s)

Related PR(s)

How to test / Smoketest result

You can boot the new image using: vyos-fail-migration Kernel cmdline parameter to trigger this.

Checklist:

  • I have read the CONTRIBUTING document
  • I have linked this PR to one or more Phabricator Task(s)
  • I have run the components SMOKETESTS if applicable
  • My commit headlines contain a valid Task id
  • My change requires a change to the documentation
  • I have updated the documentation accordingly

Copy link

👍
No issues in PR Title / Commit Title

@c-po c-po added the bp/circinus Create automatic backport for circinus label May 17, 2025
@c-po c-po requested a review from sever-sever May 17, 2025 13:58
Copy link
Member

@sever-sever sever-sever left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Boot the previous image if an upgrade fails

@c-po c-po self-assigned this May 20, 2025
@c-po c-po added the current VyOS rolling release label May 20, 2025
Copy link
Member

@dmbaturin dmbaturin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout seems sensible to allow the user to examine the system and take action manually. I also agree with the idea to enable it for new installations via the default config but make it possible to disable by removing the option.

@dmbaturin dmbaturin merged commit 9f1bf2d into vyos:current May 20, 2025
2 of 6 checks passed
@github-actions github-actions bot added the mirror-initiated This PR initiated for mirror sync workflow label May 20, 2025
@vyosbot vyosbot added mirror-completed and removed mirror-initiated This PR initiated for mirror sync workflow labels May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bp/circinus Create automatic backport for circinus current VyOS rolling release mirror-completed
Development

Successfully merging this pull request may close these issues.

4 participants