Re: [syslog-ng] syslog-ng no longer receiving kernel messages

13 Dec 2021

      Hello!

Do I understand it right, that while journalctl can read (kernel) messages, syslog-ng doesn't get them (at least not all of them)?
As I understood the problem is more general than kernel messages as only a few messages reach /var/log/messages too.

I didn't find any issues with the config.
We can check a few things first, and see how should we proceed:

  *   check the actual driver of system() source: system() source is a wrapper that expands to platform-specific drivers (e.g. it could systemd, or directly reading /dev/log).
please run "syslog-ng --preprocess-into=/tmp/preprocessed.conf".
In the generated preprocessed config, look for the
"Start Block source generator system" string and check the underlying lines to see what is the actual driver.
On Linux where systemd is available, it should be:

#Start Block source generator system
channel {
    source {
systemd-journal();

    }; # source
  *   check incoming message rate to see if any message is coming in:
run "syslog-ng-ctl stats | grep local" # local is a source statement in your config
  *   check internal logs of syslog-ng:
you have internal() source in your config, so you can check syslog-ng's internal log.
I would suggest to redirect it first to a separate file.
  *   send a test message into your system log with logger:
run "logger --rfc3164 test message" and see if you see it in both journalctl output and in syslog-ng's output file too.

Regards,
Gabor
________________________________
From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of John Covici <covici@ccs.covici.com>
Sent: Friday, December 10, 2021 18:35
To: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu>
Subject: [syslog-ng] syslog-ng no longer receiving kernel messages

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hi folks.

I am having a problem that syslog-ng no longer gets any kernel
messages.  It seems to get other messages, but not kernel messages.

Here is what happened.

In an update from kernel 5.10.70 to 5.10.82, I  enabled the kernel
race condition sanitizer.  I looked at its output and decided that it
would not do me any good, so I took that out and recompiled the
kernel.  However after rebooting the system, syslog-ng only gives me a
few messages from when the kernel sanitizer was active in my
/var/log/messages and the same for /var/log/kernel.

I am using systemd, so I have the jernal and it is OK, all messages
are posted correctly there.

Here is my /etc/syslog-ng/syslog-ng.conf .

@version: 3.34
# $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/3.4/syslog-ng.conf.gentoo,v 1.2 2013/06/02 01:18:35 mr_bones_ Exp $
#
# Syslog-ng default configuration file for Gentoo Linux

# https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.gentoo.org%2Fshow_bug.cgi%3Fid%3D426814&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C92a756514a99462dd0fc08d9bc03825f%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637747545552613402%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UmY4GoOlx9BBsEzar2GD0JPIgjDQwSQ5dCG6IgR2Yrg%3D&reserved=0
@include "scl.conf"

options { dir_perm(0755); perm(0644); chain_hostnames(no);
     threaded(yes);
     chain_hostnames(no);

     # The default action of syslog-ng is to log a STATS line
     # to the file every 10 minutes.  That's pretty ugly after a while.
     # Change it to every 12 hours so you get a nice daily update of
     # how many messages syslog-ng missed (0).
     stats_freq(43200);
     # The default action of syslog-ng is to log a MARK line
     # to the file every 20 minutes.  That's seems high for most
     # people so turn it down to once an hour.  Set it to zero
     # if you don't want the functionality at all.
     mark_freq(0);
          keep_hostname(yes); };

source local {
system() ; internal();
};
# *.emerg *

filter f_9 {
        level(emerg);
};

destination d_6 {
        usertty("*");
};

log { source(local); filter(f_9); destination(d_6); };

# kern.=debug                                           /var/log/kernel

filter f_1 {
        facility(kern) and level(debug..emerg);
};

destination d_1 {
        file("/var/log/kernel" create_dirs(yes));
};

log { source(local); filter(f_1); destination(d_1); };

# kern.err                                                      /dev/console

filter f_2 {
        facility(kern) and level(err..emerg);
};

destination d_2 {
        file("/dev/console" create_dirs(yes));
};

log { source(local); filter(f_2); destination(d_2); };

# *.debug;mail.none;news.none;authpriv.none        /var/log/messages

filter f_3 {
        level(debug..emerg);
};

filter f_4 {
        not facility(mail);
};

filter f_5 {
        not facility(news);
};

filter f_6 {
        not facility(authpriv);
};

destination d_3 {
         file("/var/log/messages"  create_dirs(yes));
};

log { source(local); filter(f_3); filter(f_4); filter(f_5); filter(f_6); destination(d_3); flags(final flow-control); };

# authpriv.*    /var/log/secure

filter f_7 {
        facility(authpriv) and level(debug..emerg);
};

destination d_4 {
        file("/var/log/secure" create_dirs(yes));
};

log { source(local); filter(f_7); destination(d_4); };

# mail.*        /var/log/maillog

filter f_8 {
        facility(mail) and level(debug..emerg);
};

destination d_5 {
        file("/var/log/maillog" create_dirs(yes));
};

log { source(local); filter(f_8); destination(d_5); };

# news.=crit     /var/log/news/news.crit

filter f_10 {
        facility(news) and level(crit..emerg);
};

destination d_7 {
        file("/var/log/news/news.crit" create_dirs(yes));
};

log { source(local); filter(f_10); destination(d_7); };

# news.=err              /var/log/news/news.err

filter f_11 {
        facility(news) and level(err..emerg);
};

destination d_8 {
        file("/var/log/news/news.err" create_dirs(yes));
};

log { source(local); filter(f_11); destination(d_8); };

# news.notice    /var/log/news/news.notice

filter f_12 {
        facility(news) and level(notice..emerg);
};

destination d_9 {
        file("/var/log/news/news.notice" create_dirs(yes));
};

log { source(local); filter(f_12); destination(d_9); };

# local7.*        /var/log/boot.log

filter f_13 {
        facility(local7) and level(debug..emerg);
};

destination d_10 {
        file("/var/log/boot.log" create_dirs(yes));
};

log { source(local); filter(f_13); destination(d_10); };

# local0.*     /var/log/dhcpcd.log

filter f_14 {
        facility(local0) and level(debug..emerg);
};

So, what have I been doing wrong, or is it something else?

Thanks in advance for any suggestions.

destination d_11 {
        file("/var/log/dhcpcd.log" create_dirs(yes));
};

log { source(local); filter(f_14); destination(d_11); };

--
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici wb2una
         covici@ccs.covici.com
______________________________________________________________________________
Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C92a756514a99462dd0fc08d9bc03825f%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637747545552613402%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=b3StS3zsS3GhfQY6NWRC5A1Eh5rYAK%2FvOcBu34Vt%2BS0%3D&reserved=0
Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C92a756514a99462dd0fc08d9bc03825f%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637747545552613402%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dg2vBO%2Fjr5NmRJiRAe4nPZKL9xN5UBccy0UP3n6icGI%3D&reserved=0
FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C92a756514a99462dd0fc08d9bc03825f%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637747545552613402%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zq4QgoK%2FT8%2FTKVCVMinHjy6hJu7D0%2BExhKDae6cN0%2Fs%3D&reserved=0

Re: [syslog-ng] syslog-ng no longer receiving kernel messages

Gabor Nagy (gnagy)