RSYSLOG / SYSLOG Check Failed

Just recently started to get a number of alerts send from my VPS running WHM/cPanel, letting me know that SYSLOG may not be running properly on it as it wasn’t able to find some code in the logs. Was quite puzzled about the messages as the rsyslog service seemed to run just fine. Initial attempt to fix it was to just restart the rsyslog service, which didn’t really resolve the issue. After rebooting the server the error messages disappeared. Thought that’d be all, but I was wrong there. Took a couple of days and the errors showed up again. Example error message below.

Time:  Sat Nov 27 21:50:49 2021 +0000
Error: Failed to detect code [lDd3EbWMH4pZmh1e0] in SYSLOG_LOG [/var/log/messages]

SYSLOG may not be running correctly on ****.***************.***

Got a message about every 10 minutes or so, and it got very annoying very soon. Was initially thinking of disabling logging in WHM, but that wasn’t really an option. So what to do? First thing was to check (again) if rsyslog is actually running.

[[email protected] ~]# /scripts/restartsrv_rsyslogd --check 

The 'rsyslog' service passed the check: rsyslog (/usr/sbin/rsyslogd -n) is running as root with PID 794 (systemd+/proc check method). 

[[email protected] ~]# ps auxf | grep rsyslog 

root 13446 0.0 0.0 9040 820 pts/0 S+ 21:50 0:00 \_ grep --color=auto rsyslog 
root 594 0.0 0.0 356936 2464 ? Ssl Nov27 0:05 /usr/sbin/rsyslogd -n 
[[email protected] ~]#

Everything looking fine here, rsyslogd is up and running. So, next thing is to manually force rsyslog into creating an entry and see if it can be found. 

[[email protected] ~]# logger -p auth.notice "checking..." 
[[email protected] ~]# grep "checking..." /var/log/messages 
[[email protected] ~]#

Great, nothing found. Seems like the checking… message hasn’t been logged at all. Time to look at the configuration of local logging now.#

[[email protected] ~]# grep -i "OmitLocalLogging\|ModLoad imjournal\|IMJournalStateFile" /etc/rsyslog.conf
$ModLoad imjournal # provides access to the systemd journal
$OmitLocalLogging on
$IMJournalStateFile imjournal.state
[[email protected] ~]# 

Based on that, and a bug in the rsyslog logging in CentOS here is how to correct the configuration. First and foremost though make a backup of you current configuration file. 

[[email protected] ~]# cp /etc/rsyslog.conf /etc/rsyslog.conf.bak
[[email protected] ~]# sed -i 's/OmitLocalLogging on/OmitLocalLogging off/' /etc/rsyslog.conf
[[email protected] ~]# sed -i 's/$IMJournalStateFile imjournal.state/#$IMJournalStateFile imjournal.state/'
/etc/rsyslog.conf
[[email protected] ~]# sed -i 's/$ModLoad imjournal/#$ModLoad imjournal/' /etc/rsyslog.conf
[[email protected] ~]# /scripts/restartsrv_rsyslog
Waiting for “rsyslog” to restart ………waiting for “rsyslog” to initialize ………finished.

Service Status
rsyslog (/usr/sbin/rsyslogd -n) is running as root with PID 32361 (systemd+/proc check method).

Startup Log
Nov 27 21:57:40 host systemd[1]: Starting System Logging Service...
Nov 27 21:57:40 host rsyslogd[32361]:
[origin software="rsyslogd" swVersion="8.24.0-34.el7" x-pid="32361" x-
info="http://www.rsyslog.com"] start
Noc 27 21:57:40 host systemd[1]: Started System Logging Service.

rsyslog restarted successfully.
[[email protected] ~]# systemctl restart systemd-journald
[[email protected] ~]#

After restarting rsyslog I did another test to see if a forced entry is now correctly logged.

[[email protected] ~]# logger -p auth.notice "checking" && grep checking /var/log/messages
Nov 27 21:58:08 myhost root: checking
[[email protected] ~]# 

For good measure I rebooted the server after testing and seeing that the entry was logged ok, and ever since there were no more errors along the lines of SYSLOG not finding any code in /var/log/messages.

Be the first to comment

Leave a Reply

Your email address will not be published.


*