View Issue Details

IDProjectCategoryView StatusLast Update
0017524CentOS-7systemdpublic2020-06-21 01:21
Reportersupreet.s8 
PrioritynormalSeverityminorReproducibilityN/A
Status newResolutionopen 
Product Version7.3.1611 
Target VersionFixed in Version 
Summary0017524: Dbus complaining about Systemd destination has a full message queue.
DescriptionJun 20 20:43:59 xxx0101 dbus[754]: [system] Rejected: destination has a full message queue, 0 matched rules; type="method_call", sender=":1.1" (uid=0 pid=750 comm="/usr/lib/systemd/systemd-logind ") interface="org.freedesktop.systemd1.Manager" member="StartTransientUnit" error name="(unset)" requested_reply="0" destination="org.freedesktop.systemd1" (uid=0 pid=1 comm="/usr/lib/systemd/systemd --switched-root --system ")
--------------------
Where;

sender=":1.1" (uid=0 pid=750 comm="/usr/lib/systemd/systemd-logind ")
root 750 1 0 2017 ? 3-21:46:08 /usr/lib/systemd/systemd-logind

destination="org.freedesktop.systemd1" (uid=0 pid=1 comm="/usr/lib/systemd/systemd --switched-root --system ")
root 1 0 1 2017 ? 13-15:49:41 /usr/lib/systemd/systemd --switched-root --system --deserialize 21
--------------------------------------

When investigated further, found that the 21 FD has a broken link to a non-existent socket process.

[root@xxx0101 ~]# ls -l /proc/1/fd/21
lrwx------. 1 root root 64 Apr 21 23:14 /proc/1/fd/20 -> socket:[8500]
[root@xxx0101 ~]#

socket[:8500] does not exist for any process.

[ Solution to get rid of the logging, apparently a temporary fix. ]

systemctl daemon-reexec
(root 1 0 1 2017 ? 13-15:51:11 /usr/lib/systemd/systemd --system --deserialize 20).

It simply creates a new process space for systemd with a new FD, which is still broken but it stops DBUS from emitting logs about destination message queue full. However, the broken links to the FD would again continue to fill up the message queue as it is not being consumed by any process. So, this is just a temporary solution (a time bomb).

There should be a kernel process/module/inode-monitor/redirection-to-null reading from the assigned FD in order for systemd message queue to stay healthy for receiving the events.






 



Steps To ReproduceShould get populated over time. The servers it has been observed at, are up since 1000+ days.
Not causing memory leak in systemd after a fix was issued in (https://access.redhat.com/errata/RHBA-2016:2216) however, this has probably become a side effect of the fix provided.
TagsD-Bus Error, systemd
abrt_hash
URL

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2020-06-21 01:20 supreet.s8 New Issue
2020-06-21 01:20 supreet.s8 Tag Attached: D-Bus Error
2020-06-21 01:20 supreet.s8 Tag Attached: systemd