Skip to content

[202511] Fix for issue #24892 Bug: [Smartswitch]: NPU critical services crash when restarting DPU database service #25740

Open
rameshraghupathy wants to merge 1 commit intosonic-net:202511from
rameshraghupathy:202511-fix-24892
Open

[202511] Fix for issue #24892 Bug: [Smartswitch]: NPU critical services crash when restarting DPU database service #25740
rameshraghupathy wants to merge 1 commit intosonic-net:202511from
rameshraghupathy:202511-fix-24892

Conversation

@rameshraghupathy
Copy link
Contributor

Description: This a double commit of #25178 to 202511 branch
Fixes #24892
NPU critical services crash when restarting DPU database service

Why I did it

NPU critical services crash when restarting DPU database service

Work item tracking
  • Microsoft ADO (number only):

How I did it

Gated multi-instance dependency rewriting in systemd-sonic-generator on num_asics > 1 and updated ssg tests so SmartSwitch NPU uses database.service instead of [email protected].

How to verify it

On the NPU, run systemctl restart database@dpu0. All ciritcal containers including swss should not restart.

Tested branch (Please provide the tested image version)

202506

Description for the changelog

Fix SmartSwitch NPU boot dependencies by avoiding unnecessary [email protected] requirements and only expanding multi-instance database units on true multi-ASIC platforms.

UT Results

root@sonic:/home/admin# for u in swss syncd bgp lldp gbsyncd macsec teamd bmp dash-ha; do
>     echo "===== $u.service ====="
>     systemctl show "$u.service" -p Requires -p After | tr ' ' '\n' | grep 'database' || echo "  no database deps?!"
> done
===== swss.service =====
database.service
database.service
===== syncd.service =====
database.service
database.service
===== bgp.service =====
Requires=database.service
database.service
===== lldp.service =====
database.service
database.service
===== gbsyncd.service =====
database.service
database.service
===== macsec.service =====
  no database deps?!
===== teamd.service =====
  no database deps?!
===== bmp.service =====
Requires=database.service
database.service
===== dash-ha.service =====
  no database deps?!
root@sonic:/home/admin# 
root@sonic:/home/admin# 
root@sonic:/home/admin# docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED       STATUS       PORTS     NAMES
5b284ac82553   docker-snmp:latest                   "/usr/bin/docker-snm…"   2 hours ago   Up 2 hours             snmp
0e133767f061   docker-platform-monitor:latest       "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours             pmon
e1a2441b88de   docker-sonic-mgmt-framework:latest   "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             mgmt-framework
8dcaa4c8d8c3   docker-lldp:latest                   "/usr/bin/docker-lld…"   2 hours ago   Up 2 hours             lldp
d7b433528a23   docker-sonic-gnmi:latest             "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             gnmi
37f3d6ce30fd   docker-dhcp-server:latest            "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours             dhcp_server
56967ca6c42f   docker-dhcp-relay:latest             "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours             dhcp_relay
1ea7469d25d9   docker-eventd:latest                 "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             eventd
4c38dc947c96   docker-fpm-frr:latest                "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours             bgp
c3a5f510f4e9   docker-router-advertiser:latest      "/usr/bin/docker-ini…"   2 hours ago   Up 2 hours             radv
b0aef8b799f7   docker-syncd-cisco:latest            "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             syncd
09e0bbbe14a0   docker-teamd:latest                  "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             teamd
f3423bf9b73e   docker-orchagent:latest              "/usr/bin/docker-ini…"   2 hours ago   Up 2 hours             swss
e813b1f9fd5a   docker-sysmgr:latest                 "/usr/local/bin/supe…"   2 hours ago   Up 2 hours             sysmgr
c26b92e8a630   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu6
9b24032b1d54   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu5
ce95f647d883   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu7
74c88bae31bf   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu1
a01b1e5c40e0   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu2
204b5c72ff9e   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu4
f53d01e86758   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu3
cd38e3b7eea4   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             databasedpu0
a20d1724b3e1   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             database
582d2c8c9393   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours             database-chassis
root@sonic:/home/admin# systemctl restart [email protected]

root@sonic:/home/admin# docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED       STATUS         PORTS     NAMES
5b284ac82553   docker-snmp:latest                   "/usr/bin/docker-snm…"   2 hours ago   Up 2 hours               snmp
0e133767f061   docker-platform-monitor:latest       "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours               pmon
e1a2441b88de   docker-sonic-mgmt-framework:latest   "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               mgmt-framework
8dcaa4c8d8c3   docker-lldp:latest                   "/usr/bin/docker-lld…"   2 hours ago   Up 2 hours               lldp
d7b433528a23   docker-sonic-gnmi:latest             "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               gnmi
37f3d6ce30fd   docker-dhcp-server:latest            "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours               dhcp_server
56967ca6c42f   docker-dhcp-relay:latest             "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours               dhcp_relay
1ea7469d25d9   docker-eventd:latest                 "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               eventd
4c38dc947c96   docker-fpm-frr:latest                "/usr/bin/docker_ini…"   2 hours ago   Up 2 hours               bgp
c3a5f510f4e9   docker-router-advertiser:latest      "/usr/bin/docker-ini…"   2 hours ago   Up 2 hours               radv
b0aef8b799f7   docker-syncd-cisco:latest            "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               syncd
09e0bbbe14a0   docker-teamd:latest                  "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               teamd
f3423bf9b73e   docker-orchagent:latest              "/usr/bin/docker-ini…"   2 hours ago   Up 2 hours               swss
e813b1f9fd5a   docker-sysmgr:latest                 "/usr/local/bin/supe…"   2 hours ago   Up 2 hours               sysmgr
c26b92e8a630   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu6
9b24032b1d54   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu5
ce95f647d883   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu7
74c88bae31bf   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu1
a01b1e5c40e0   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu2
204b5c72ff9e   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu4
f53d01e86758   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 7 seconds             databasedpu3
cd38e3b7eea4   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               databasedpu0
a20d1724b3e1   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               database
582d2c8c9393   docker-database:latest               "/usr/local/bin/dock…"   2 hours ago   Up 2 hours               database-chassis

…es crash when restarting DPU database service -- Double commit to 202511 branch

Signed-off-by: Ramesh Raghupathy <[email protected]>
@mssonicbld
Copy link
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@prsunny prsunny changed the title Fix for issue #24892 Bug: [Smartswitch]: NPU critical services crash when restarting DPU database service - Double commit to 202511 branch [202511] Fix for issue #24892 Bug: [Smartswitch]: NPU critical services crash when restarting DPU database service Feb 28, 2026
Copy link
Contributor

@prsunny prsunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vmittal-msft for viz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants