View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0013683 | CentOS-7 | systemd | public | 2017-08-17 19:46 | 2020-04-21 06:22 |
Reporter | morty | Assigned To | |||
Priority | high | Severity | crash | Reproducibility | random |
Status | new | Resolution | open | ||
Platform | PowerEdge C6220 II | OS | CentOS | OS Version | 7.3 |
Summary | 0013683: systemctl stops working after low memory conditions | ||||
Description | We're sometimes seeing cases where systemd stops working in all kinds of different ways after a system runs out of memory. When the OOM condition clears, systemd is still running but broken. systemctl doesn't work. It won't even harvest defunct PIDs with PPID==1. For example: [root@intprod21 ~]# systemctl Failed to list units: Activation of org.freedesktop.systemd1 timed out [root@intprod21 ~]# ps faxo user,pid,ppid,pgid,state,command|awk '$3==1 && $5=="Z"'|tail sshd 57937 1 57936 Z [sshd] <defunct> nrpe 57981 1 3153 Z [nrpe] <defunct> nrpe 58250 1 3153 Z [nrpe] <defunct> sshd 58298 1 58297 Z [sshd] <defunct> sshd 58381 1 58380 Z [sshd] <defunct> nrpe 58421 1 3153 Z [nrpe] <defunct> sshd 58663 1 58662 Z [sshd] <defunct> sshd 59028 1 59027 Z [sshd] <defunct> nrpe 59118 1 3153 Z [nrpe] <defunct> nrpe 59154 1 3153 Z [nrpe] <defunct> [root@intprod21 ~]# ps faxo user,pid,ppid,pgid,state,command|awk '$3==1 && $5=="Z"'|wc -l 16162 [root@intprod21 ~]# ps fp 1 PID TTY STAT TIME COMMAND 1 ? Ss 22:41 /usr/lib/systemd/systemd --switched-root --system -- [root@intprod21 ~]# This has happened on several of our systems. It always follows an "out of memory" condition. We get the system back for other purposes, but systemd doesn't recover. Rebooting clears it. Unfortunately, I don't (yet?) have a reliable way to reproduce this. | ||||
Tags | No tags attached. | ||||
abrt_hash | |||||
URL | |||||
Please, post the CentOS version and kernel you are using in your Servers: uname -r 3.10.0-693.11.6.el7.x86_64 cat /etc/centos-release CentOS Linux release 7.4.1708 (Core) |
|
any updates on this issue? | |
We are also facing this issue on multiple hosts.. It would be great to have a workaround/fix for this.. | |
Date Modified | Username | Field | Change |
---|---|---|---|
2017-08-17 19:46 | morty | New Issue | |
2018-01-25 15:36 | xavinux | Note Added: 0031038 | |
2020-03-06 10:42 | satty773 | Note Added: 0036469 | |
2020-04-21 06:22 | rajkumar.vinoo | Note Added: 0036741 |