0003942: logwatch fails to send designated report via email

(0010182)
range (administrator)
2009-10-28 00:31

Hmmm. I still seem to be getting logwatch mails from my systems after upgrading. Is there anything in you logfiles? And: What is it with qmail? We don't ship that.

(0010187)
ddelorey (reporter)
2009-10-28 02:19

I used "qmail" as an alternative MTA in an attempt to rule out "sendmail" as the source of the problem. Also tried changing the "logwatch" configuration with `mailer = "qmail"` and `mailer = "/var/qmail/bin/sendmail -t root"` in "/usr/share/logwatch/default.conf/logwatch.conf", both alternatives replacing `mailer = "sendmail -t"`. There are no "logwatch" entries in "/var/log/messages" or "/var/log/secure". All "/var/log/maillog" logs are empty. Both servers have been regularly updated via "yum update" at least once per week over the last year with no problem whatsoever. The CentOS 5.4 update had "logwatch-7.3-6.el5.noarch.rpm", "sendmail-8.13.8-2.el5.i386.rpm" and "sendmail-cf-8.13.8-2.el5.i386.rpm" as the "logwatch" and "sendmail" rpms. Any suggestions to identify what to do next would be appreciated.

(0010188)
SteveWalsh (reporter)
2009-10-28 04:01

well, as a start, are the emails showing up in /var/log/maillog? The default cron for logwatch is ~4am system time, are you seeing anything around then?

Do you have any sort of forward or alias in place to direct root's mail elsewhere, or is it staying on-box?

(0010191)
ddelorey (reporter)
2009-10-28 13:32

Thanks for the comments. I have not changed a forward or alias directly. Up until the CentOS 5.4 upgrade, all "logwatch" emails were sent with a time of "5:02 AM" as expected for my time zone. All "/var/log/maillog" logs are still empty on both servers. To obtain daily reports, I inserted after "# Save = /tmp/logwatch", "Save = /var/local/logwatch/logwatch.yesterday" in "/usr/share/logwatch/default.conf/logwatch.conf". This has made the daily reports available via SSH, a rather inconvenient alternative. This morning, I reinstalled "logwatch" and "sendmail" via "Yum reinstall ...". The "logwatch" configuration file "/usr/share/logwatch/default.conf/logwatch.conf" is now dated "05/24/2008". After the foregoing reinstall, my MTA settings are:

[root@server2 ~]# alternatives --config mta

There are 2 programs which provide 'mta'.

  Selection Command
-----------------------------------------------
*+ 1 /var/qmail/bin/sendmail
   2 /usr/sbin/sendmail.sendmail

Enter to keep the current selection[+], or type selection number:2
[root@server2 ~]#

I rebooted the server [hard reset], then executed the following:

[root@server2 ~]# alternatives --config mta

There are 2 programs which provide 'mta'.

  Selection Command
-----------------------------------------------
* 1 /var/qmail/bin/sendmail
+ 2 /usr/sbin/sendmail.sendmail

Enter to keep the current selection[+], or type selection number:2

[root@server2 ~]# ls -l /etc/alternatives/mta
lrwxrwxrwx 1 root root 27 Oct 28 08:20 /etc/alternatives/mta -> /usr/sbin/sendmail.sendmail
[root@air086 ~]# service sendmail restart
Shutting down sm-client: [ OK ]
Shutting down sendmail: [FAILED]
Starting sendmail: 554 5.0.0 No local mailer defined
554 5.0.0 QueueDirectory (Q) option must be set
                                                           [FAILED]
Starting sm-client: [ OK ]
[root@server2 ~]# service qmail restart
Stopping : Starting qmail: [ OK ]
[root@server2 ~]#

At this point all "/var/log/maillog" files and the "/etc/mail/sendmail.cf" file remains empty. I tried to generate the "/etc/mail/sendmail.cf" file as follows using "m4":

[root@air086 ~]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
/etc/mail/sendmail.mc:10: m4: cannot open `/usr/share/sendmail-cf/m4/cf.m4': No such file or directory

This appears to identify "logwatch" as the victim of "sendmail", rather than than the cause of the loss of server email functionality. As a result, I do not expect a "logwatch" email tomorrow at "5:02 AM". Any suggestions on what to do next to get "sendmail" to work under CentOS 5.4?

(0010227)
ddelorey (reporter)
2009-10-31 20:42

The "logwatch" email never arrived thus confirming that the CentOS 5.4 upgrade destroyed "sendmail" rather than "logwatch" not sending mail. It is unlikely a coincidence because both servers had "sendmail" workin just dandy before the upgrade, and were destroyed right after it. Reinstalling "sendmail" fails.

(0010235)
mulin (reporter)
2009-11-02 18:10

The "xen" kernel with Centos 5.4 may have bug. The system keeps adding load average (adding by 1 hourly), and when the number goes to 64, sendmail fails sending/receiving email.

What I did is: delete "mcelog.cron" from "/etc/cron.hourly" and rebooted my server . You may also load a kernel without "xen".

(0010246)
ddelorey (reporter)
2009-11-03 14:06

I have a pure CentOS 5.4 load managed by Plesk 8.6. Script "mcelog.cron" is not on either server. I've never modified, added or changed any configs, except php and http application level configs, to support my application requirements. The 5.4 update simply wiped out sendmail functionality on both servers. Each server was updated on a different day. So I know the problem is a function of the the upgrade via "YUM" rather than a quirky event on a particular day. Both servers are NOT 64 bit loads. It seems to me that if I remove and replace sendmail via "YUM", "YUM" should regenerate sendmail functionality. However, this fails to correct the problem. Therefore, the problem points to systemic bug in assembling the required packages to get the "YUM" udate right in terms of making sendmail functional, rather than my particular server load. It could also be that Plesk has failed to manage the upgrade via "YUM".

(0010247)
mulin (reporter)
2009-11-03 14:28

Is the load average number (you can use the command "uptime" or "top" to check) on your servers? When the load average number is bigger than 8 times of the number of cores, sendmail will stop working (default configure).

This was the problem on my server.

Good luck!

(0010248)
range (administrator)
2009-11-03 14:43

Ah, Plesk. If nobody can reproduce this problem on a clean CentOS install (I cannot, all machines send logwatch reports), I will close this report.

(0010255)
ddelorey (reporter)
2009-11-03 17:30

I agree with this approach if the problem does not manifest itself on pure CentOS 5.4 loads. However, my past experience with Plesk is that the entire company typically remains unresponsive to correcting their own errors unless you pay them to fix mistakes clearly in their domain. Is their a way for the CentOS team to port this problem to Plesk for resolution?

(0010256)
ddelorey (reporter)
2009-11-03 17:37

As a follow-up, I reinstalled "sendmail" again. Neither "/usr/share/sendmail-cf" or "/usr/lib/sendmail-cf" folders are present. Therefore, "m4 directory/m4/cf.m4 config.mc > sendmail.cf" will undoubtably fail and "sendmail" clearly will not function.

(0010269)
SteveWalsh (reporter)
2009-11-04 00:01

did you install sendmail-cf as well as sendmail?

What has probably happened is that plesk has borked your server (as they do), so you'd need to take this up with Plesk. If you want to prove this is a Plesk failure, build a clean 5.3 box, then upgrade it to 5.4. if that works, then it's not Centos.

(0010300)
ddelorey (reporter)
2009-11-06 00:44

The verdict is: "Borked by Plesk." The CentOS 5.4 upgrade from the Plesk mirror did not include "sendmail-cf".

The remedy is as follows:
1) "yum install sendmail-cf"
2) "m4 /usr/share/sendmail-cf/m4/cf.m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf"
3) "service sendmail restart"

I needed to do step #3 twice to get a claen start/stop sequence.

Thanks Steve very much for the post.

(0010301)
range (administrator)
2009-11-06 07:39

Okay, problem with Plesk, closing the ticket.

Notes
(0010182) range (administrator) 2009-10-28 00:31	Hmmm. I still seem to be getting logwatch mails from my systems after upgrading. Is there anything in you logfiles? And: What is it with qmail? We don't ship that.

(0010187) ddelorey (reporter) 2009-10-28 02:19	I used "qmail" as an alternative MTA in an attempt to rule out "sendmail" as the source of the problem. Also tried changing the "logwatch" configuration with `mailer = "qmail"` and `mailer = "/var/qmail/bin/sendmail -t root"` in "/usr/share/logwatch/default.conf/logwatch.conf", both alternatives replacing `mailer = "sendmail -t"`. There are no "logwatch" entries in "/var/log/messages" or "/var/log/secure". All "/var/log/maillog" logs are empty. Both servers have been regularly updated via "yum update" at least once per week over the last year with no problem whatsoever. The CentOS 5.4 update had "logwatch-7.3-6.el5.noarch.rpm", "sendmail-8.13.8-2.el5.i386.rpm" and "sendmail-cf-8.13.8-2.el5.i386.rpm" as the "logwatch" and "sendmail" rpms. Any suggestions to identify what to do next would be appreciated.

(0010188) SteveWalsh (reporter) 2009-10-28 04:01	well, as a start, are the emails showing up in /var/log/maillog? The default cron for logwatch is ~4am system time, are you seeing anything around then? Do you have any sort of forward or alias in place to direct root's mail elsewhere, or is it staying on-box?

(0010191) ddelorey (reporter) 2009-10-28 13:32	Thanks for the comments. I have not changed a forward or alias directly. Up until the CentOS 5.4 upgrade, all "logwatch" emails were sent with a time of "5:02 AM" as expected for my time zone. All "/var/log/maillog" logs are still empty on both servers. To obtain daily reports, I inserted after "# Save = /tmp/logwatch", "Save = /var/local/logwatch/logwatch.yesterday" in "/usr/share/logwatch/default.conf/logwatch.conf". This has made the daily reports available via SSH, a rather inconvenient alternative. This morning, I reinstalled "logwatch" and "sendmail" via "Yum reinstall ...". The "logwatch" configuration file "/usr/share/logwatch/default.conf/logwatch.conf" is now dated "05/24/2008". After the foregoing reinstall, my MTA settings are: [root@server2 ~]# alternatives --config mta There are 2 programs which provide 'mta'. Selection Command ----------------------------------------------- + 1 /var/qmail/bin/sendmail 2 /usr/sbin/sendmail.sendmail Enter to keep the current selection[+], or type selection number:2 [root@server2 ~]# I rebooted the server [hard reset], then executed the following: [root@server2 ~]# alternatives --config mta There are 2 programs which provide 'mta'. Selection Command ----------------------------------------------- 1 /var/qmail/bin/sendmail + 2 /usr/sbin/sendmail.sendmail Enter to keep the current selection[+], or type selection number:2 [root@server2 ~]# ls -l /etc/alternatives/mta lrwxrwxrwx 1 root root 27 Oct 28 08:20 /etc/alternatives/mta -> /usr/sbin/sendmail.sendmail [root@air086 ~]# service sendmail restart Shutting down sm-client: [ OK ] Shutting down sendmail: [FAILED] Starting sendmail: 554 5.0.0 No local mailer defined 554 5.0.0 QueueDirectory (Q) option must be set [FAILED] Starting sm-client: [ OK ] [root@server2 ~]# service qmail restart Stopping : Starting qmail: [ OK ] [root@server2 ~]# At this point all "/var/log/maillog" files and the "/etc/mail/sendmail.cf" file remains empty. I tried to generate the "/etc/mail/sendmail.cf" file as follows using "m4": [root@air086 ~]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf /etc/mail/sendmail.mc:10: m4: cannot open `/usr/share/sendmail-cf/m4/cf.m4': No such file or directory This appears to identify "logwatch" as the victim of "sendmail", rather than than the cause of the loss of server email functionality. As a result, I do not expect a "logwatch" email tomorrow at "5:02 AM". Any suggestions on what to do next to get "sendmail" to work under CentOS 5.4?

(0010227) ddelorey (reporter) 2009-10-31 20:42	The "logwatch" email never arrived thus confirming that the CentOS 5.4 upgrade destroyed "sendmail" rather than "logwatch" not sending mail. It is unlikely a coincidence because both servers had "sendmail" workin just dandy before the upgrade, and were destroyed right after it. Reinstalling "sendmail" fails.

(0010235) mulin (reporter) 2009-11-02 18:10	The "xen" kernel with Centos 5.4 may have bug. The system keeps adding load average (adding by 1 hourly), and when the number goes to 64, sendmail fails sending/receiving email. What I did is: delete "mcelog.cron" from "/etc/cron.hourly" and rebooted my server . You may also load a kernel without "xen".

(0010246) ddelorey (reporter) 2009-11-03 14:06	I have a pure CentOS 5.4 load managed by Plesk 8.6. Script "mcelog.cron" is not on either server. I've never modified, added or changed any configs, except php and http application level configs, to support my application requirements. The 5.4 update simply wiped out sendmail functionality on both servers. Each server was updated on a different day. So I know the problem is a function of the the upgrade via "YUM" rather than a quirky event on a particular day. Both servers are NOT 64 bit loads. It seems to me that if I remove and replace sendmail via "YUM", "YUM" should regenerate sendmail functionality. However, this fails to correct the problem. Therefore, the problem points to systemic bug in assembling the required packages to get the "YUM" udate right in terms of making sendmail functional, rather than my particular server load. It could also be that Plesk has failed to manage the upgrade via "YUM".

(0010247) mulin (reporter) 2009-11-03 14:28	Is the load average number (you can use the command "uptime" or "top" to check) on your servers? When the load average number is bigger than 8 times of the number of cores, sendmail will stop working (default configure). This was the problem on my server. Good luck!

(0010248) range (administrator) 2009-11-03 14:43	Ah, Plesk. If nobody can reproduce this problem on a clean CentOS install (I cannot, all machines send logwatch reports), I will close this report.

(0010255) ddelorey (reporter) 2009-11-03 17:30	I agree with this approach if the problem does not manifest itself on pure CentOS 5.4 loads. However, my past experience with Plesk is that the entire company typically remains unresponsive to correcting their own errors unless you pay them to fix mistakes clearly in their domain. Is their a way for the CentOS team to port this problem to Plesk for resolution?

(0010256) ddelorey (reporter) 2009-11-03 17:37	As a follow-up, I reinstalled "sendmail" again. Neither "/usr/share/sendmail-cf" or "/usr/lib/sendmail-cf" folders are present. Therefore, "m4 directory/m4/cf.m4 config.mc > sendmail.cf" will undoubtably fail and "sendmail" clearly will not function.

(0010269) SteveWalsh (reporter) 2009-11-04 00:01	did you install sendmail-cf as well as sendmail? What has probably happened is that plesk has borked your server (as they do), so you'd need to take this up with Plesk. If you want to prove this is a Plesk failure, build a clean 5.3 box, then upgrade it to 5.4. if that works, then it's not Centos.

(0010300) ddelorey (reporter) 2009-11-06 00:44	The verdict is: "Borked by Plesk." The CentOS 5.4 upgrade from the Plesk mirror did not include "sendmail-cf". The remedy is as follows: 1) "yum install sendmail-cf" 2) "m4 /usr/share/sendmail-cf/m4/cf.m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf" 3) "service sendmail restart" I needed to do step #3 twice to get a claen start/stop sequence. Thanks Steve very much for the post.

(0010301) range (administrator) 2009-11-06 07:39	Okay, problem with Plesk, closing the ticket.

Issue History
Date Modified	Username	Field	Change
2009-10-27 12:59	ddelorey	New Issue
2009-10-27 12:59	ddelorey	Assigned To	=> kbsingh@karan.org
2009-10-27 13:35	ddelorey	Issue Monitored: ddelorey
2009-10-28 00:31	range	Note Added: 0010182
2009-10-28 00:31	range	Assigned To	kbsingh@karan.org => range
2009-10-28 00:31	range	Status	new => feedback
2009-10-28 02:19	ddelorey	Note Added: 0010187
2009-10-28 04:01	SteveWalsh	Note Added: 0010188
2009-10-28 13:32	ddelorey	Note Added: 0010191
2009-10-31 20:42	ddelorey	Note Added: 0010227
2009-11-02 18:10	mulin	Note Added: 0010235
2009-11-03 14:06	ddelorey	Note Added: 0010246
2009-11-03 14:28	mulin	Note Added: 0010247
2009-11-03 14:43	range	Note Added: 0010248
2009-11-03 17:30	ddelorey	Note Added: 0010255
2009-11-03 17:37	ddelorey	Note Added: 0010256
2009-11-04 00:01	SteveWalsh	Note Added: 0010269
2009-11-06 00:44	ddelorey	Note Added: 0010300
2009-11-06 07:39	range	Note Added: 0010301
2009-11-06 07:39	range	Status	feedback => closed
2009-11-06 07:39	range	Resolution	open => no change required

Relationships