[syslog-ng] syslog-ng no longer receiving kernel messages
Gabor Nagy (gnagy)
Gabor.Nagy at oneidentity.com
Mon Dec 13 15:07:42 UTC 2021
I think it's not syslog-ng, but logrotate that's having an issue.
Check your config files under /etc/logrotate.d, I think there are multiple configurations for the same file(s), like /var/log/messages.
You can disable the other configuration file by adding the .disabled suffix to the filename.
It's not just syslog-ng affected based on your logs, but xferlog too.
G?bor
________________________________
From: John Covici <covici at ccs.covici.com>
Sent: Monday, December 13, 2021 14:52
To: Gabor Nagy (gnagy) <Gabor.Nagy at oneidentity.com>
Cc: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] syslog-ng no longer receiving kernel messages
CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
Well here is the preprocessor output for source local
source {
systemd-journal();
}; # source
But what I did was to use the default conf file till early this
morning and then I restored my original configuration file and it
seems to be working again with the exception that every day, I always
get complaints from syslog-ng saying
error: syslog-ng:6 duplicate log entry for /var/log/messages
error: found error in file syslog-ng, skipping
error: xferlog:1 duplicate log entry for /var/log/xferlog
error: found error in file xferlog, skipping
Am I doing something wrong?
On Mon, 13 Dec 2021 06:36:08 -0500,
Gabor Nagy (gnagy) wrote:
>
> [1 <text/plain; us-ascii (quoted-printable)>]
> Hello!
>
> Do I understand it right, that while journalctl can read (kernel) messages, syslog-ng doesn't get them (at least not all of them)?
> As I understood the problem is more general than kernel messages as only a few messages reach /var/log/messages too.
>
> I didn't find any issues with the config.
> We can check a few things first, and see how should we proceed:
>
> * check the actual driver of system() source: system() source is a wrapper that expands to platform-specific drivers (e.g. it could systemd, or directly reading /dev/log).
> please run "syslog-ng --preprocess-into=/tmp/preprocessed.conf".
> In the generated preprocessed config, look for the
> "Start Block source generator system" string and check the underlying lines to see what is the actual driver.
> On Linux where systemd is available, it should be:
>
> #Start Block source generator system
> channel {
> source {
> systemd-journal();
>
> }; # source
> * check incoming message rate to see if any message is coming in:
> run "syslog-ng-ctl stats | grep local" # local is a source statement in your config
> * check internal logs of syslog-ng:
> you have internal() source in your config, so you can check syslog-ng's internal log.
> I would suggest to redirect it first to a separate file.
> * send a test message into your system log with logger:
> run "logger --rfc3164 test message" and see if you see it in both journalctl output and in syslog-ng's output file too.
>
> Regards,
> Gabor
> ________________________________
> From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of John Covici <covici at ccs.covici.com>
> Sent: Friday, December 10, 2021 18:35
> To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
> Subject: [syslog-ng] syslog-ng no longer receiving kernel messages
>
> CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
>
>
> Hi folks.
>
> I am having a problem that syslog-ng no longer gets any kernel
> messages. It seems to get other messages, but not kernel messages.
>
> Here is what happened.
>
> In an update from kernel 5.10.70 to 5.10.82, I enabled the kernel
> race condition sanitizer. I looked at its output and decided that it
> would not do me any good, so I took that out and recompiled the
> kernel. However after rebooting the system, syslog-ng only gives me a
> few messages from when the kernel sanitizer was active in my
> /var/log/messages and the same for /var/log/kernel.
>
> I am using systemd, so I have the jernal and it is OK, all messages
> are posted correctly there.
>
> Here is my /etc/syslog-ng/syslog-ng.conf .
>
> @version: 3.34
> # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/3.4/syslog-ng.conf.gentoo,v 1.2 2013/06/02 01:18:35 mr_bones_ Exp $
> #
> # Syslog-ng default configuration file for Gentoo Linux
>
> # https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.gentoo.org%2Fshow_bug.cgi%3Fid%3D426814&data=04%7C01%7CGabor.Nagy%40oneidentity.com%7Ca2e46b4838d44693b0a808d9be3fdc4d%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637750003764913996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=vOe8DZTB0H3Q3OG1U0WoCVcYI%2F%2F%2BrPK1E2qd8b3mvg4%3D&reserved=0
> @include "scl.conf"
>
>
> options { dir_perm(0755); perm(0644); chain_hostnames(no);
> threaded(yes);
> chain_hostnames(no);
>
> # The default action of syslog-ng is to log a STATS line
> # to the file every 10 minutes. That's pretty ugly after a while.
> # Change it to every 12 hours so you get a nice daily update of
> # how many messages syslog-ng missed (0).
> stats_freq(43200);
> # The default action of syslog-ng is to log a MARK line
> # to the file every 20 minutes. That's seems high for most
> # people so turn it down to once an hour. Set it to zero
> # if you don't want the functionality at all.
> mark_freq(0);
> keep_hostname(yes); };
>
> source local {
> system() ; internal();
> };
> # *.emerg *
>
> filter f_9 {
> level(emerg);
> };
>
> destination d_6 {
> usertty("*");
> };
>
> log { source(local); filter(f_9); destination(d_6); };
>
> # kern.=debug /var/log/kernel
>
> filter f_1 {
> facility(kern) and level(debug..emerg);
> };
>
> destination d_1 {
> file("/var/log/kernel" create_dirs(yes));
> };
>
> log { source(local); filter(f_1); destination(d_1); };
>
> # kern.err /dev/console
>
> filter f_2 {
> facility(kern) and level(err..emerg);
> };
>
> destination d_2 {
> file("/dev/console" create_dirs(yes));
> };
>
> log { source(local); filter(f_2); destination(d_2); };
>
> # *.debug;mail.none;news.none;authpriv.none /var/log/messages
>
> filter f_3 {
> level(debug..emerg);
> };
>
> filter f_4 {
> not facility(mail);
> };
>
> filter f_5 {
> not facility(news);
> };
>
> filter f_6 {
> not facility(authpriv);
> };
>
> destination d_3 {
> file("/var/log/messages" create_dirs(yes));
> };
>
> log { source(local); filter(f_3); filter(f_4); filter(f_5); filter(f_6); destination(d_3); flags(final flow-control); };
>
> # authpriv.* /var/log/secure
>
> filter f_7 {
> facility(authpriv) and level(debug..emerg);
> };
>
> destination d_4 {
> file("/var/log/secure" create_dirs(yes));
> };
>
> log { source(local); filter(f_7); destination(d_4); };
>
> # mail.* /var/log/maillog
>
> filter f_8 {
> facility(mail) and level(debug..emerg);
> };
>
> destination d_5 {
> file("/var/log/maillog" create_dirs(yes));
> };
>
> log { source(local); filter(f_8); destination(d_5); };
>
>
> # news.=crit /var/log/news/news.crit
>
> filter f_10 {
> facility(news) and level(crit..emerg);
> };
>
> destination d_7 {
> file("/var/log/news/news.crit" create_dirs(yes));
> };
>
> log { source(local); filter(f_10); destination(d_7); };
>
> # news.=err /var/log/news/news.err
>
> filter f_11 {
> facility(news) and level(err..emerg);
> };
>
> destination d_8 {
> file("/var/log/news/news.err" create_dirs(yes));
> };
>
> log { source(local); filter(f_11); destination(d_8); };
>
> # news.notice /var/log/news/news.notice
>
> filter f_12 {
> facility(news) and level(notice..emerg);
> };
>
> destination d_9 {
> file("/var/log/news/news.notice" create_dirs(yes));
> };
>
> log { source(local); filter(f_12); destination(d_9); };
>
> # local7.* /var/log/boot.log
>
> filter f_13 {
> facility(local7) and level(debug..emerg);
> };
>
> destination d_10 {
> file("/var/log/boot.log" create_dirs(yes));
> };
>
> log { source(local); filter(f_13); destination(d_10); };
>
> # local0.* /var/log/dhcpcd.log
>
> filter f_14 {
> facility(local0) and level(debug..emerg);
> };
>
>
> So, what have I been doing wrong, or is it something else?
>
> Thanks in advance for any suggestions.
>
>
> destination d_11 {
> file("/var/log/dhcpcd.log" create_dirs(yes));
> };
>
> log { source(local); filter(f_14); destination(d_11); };
>
>
> --
> Your life is like a penny. You're going to lose it. The question is:
> How do
> you spend it?
>
> John Covici wb2una
> covici at ccs.covici.com
> ______________________________________________________________________________
> Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7CGabor.Nagy%40oneidentity.com%7Ca2e46b4838d44693b0a808d9be3fdc4d%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637750003764913996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Fg0ux4K6lH9EW0EthCYVbxMUDnIEa4WGbyIHQuW3RG0%3D&reserved=0
> Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7CGabor.Nagy%40oneidentity.com%7Ca2e46b4838d44693b0a808d9be3fdc4d%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637750003764923990%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=LFF7%2FpjVIdzfDgfReFHsHG45sL%2BYkDZKYphvebKJvcM%3D&reserved=0
> FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7CGabor.Nagy%40oneidentity.com%7Ca2e46b4838d44693b0a808d9be3fdc4d%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637750003764923990%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=fE5zyc%2FP70So9Lh6%2FRRB%2FLlX0rC5ZvSX6DK%2BnKTXpzk%3D&reserved=0
>
> [2 <text/html; us-ascii (quoted-printable)>]
--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?
John Covici wb2una
covici at ccs.covici.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20211213/1b5fe9d1/attachment-0001.htm>
More information about the syslog-ng
mailing list