[syslog-ng] seems like program filter is broken
Scheidler, Balázs
balazs.scheidler at oneidentity.com
Fri Mar 22 04:26:00 UTC 2019
The problem seems to be that dovecot uses Rfc5424 formatted message on the
local log socket.
Syslog-ng is able to cope with this format, and the system () source has
recently been adapted to allow this.
If you are not using the system () source, just add flags(syslog-protocol)
to your unix-dgram() driver.
Bazsi
On Thu, Mar 21, 2019, 22:57 Stanislav <me at rooty.name wrote:
> I also did a test with following configuration:
>
> ===========
> @version: 3.20
>
> log {
> source { internal(); };
> if (program("syslog-ng")) {
> rewrite { set(":)" value(".FILTER")); };
> }
> else {
> rewrite { set(":(" value(".FILTER")); };
> };
>
> destination { file("/dev/stdout" template("${.FILTER} [${PROGRAM}]
> ${MESSAGE}\n")); };
> };
> ===========
>
> The result is pretty much the same, I can see this:
> # syslog-ng -F
> :) [syslog-ng] syslog-ng starting up; version='3.20.1'
>
> but that's pretty much it, when I restart dovecot or any other
> application I can't see new lines...
>
> My full syslog-ng configuration: http://rooty.name/syslog-ng.conf
>
>
>
> > Hey,
> >
> > There's a syntax error:
> > Error parsing log statement, syntax error, unexpected '(', expecting
> > ')' in /usr/local/etc/syslog-ng.conf:6:20-6:21
> >
> > ...assuming it should be like this:
> > =============
> > @version: 3.20
> >
> > log {
> > source { internal(); };
> > if {
> > filter{ program("syslog-ng"); };
> > rewrite { set(":)" value(".FILTER")); };
> > }
> > else {
> > rewrite { set(":(" value(".FILTER")); };
> > };
> >
> > destination { file("/dev/stdout" template("${.FILTER}\n")); };
> > };
> > =============
> > I'm getting following result:
> >
> > # syslog-ng -F
> > :)
> >
> >
> > Seems like everything should be fine */me confused*...
> >
> > =============
> >
> > ok, so what I did next is:
> > destination all { file("/var/log/all.log" template("DEBUG ${ISODATE}
> > >>>>${PROGRAM}<<<<< ${MESSAGE}\n")); };
> >
> > and I can see this:
> > DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
> > 2019-03-21T23:03:54.538134+02:00 rooty.name dovecot 62129 - - master:
> > Warning: Killed with signal 15 (by pid=62197 uid=0 code=kill)
> > DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
> > 2019-03-21T23:03:54.539049+02:00 rooty.name dovecot 62134 - -
> > imap(me at rooty.name)<62147><D8rkEaGEPHesOiU3>: Server shutting down.
> > in=27 out=775 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0
> > body_count=0 body_bytes=0
> > DEBUG 2019-03-21T23:03:56+02:00 >>>>1<<<<<
> > 2019-03-21T23:03:56.231605+02:00 rooty.name dovecot 62224 - - master:
> > Dovecot v2.3.5 (513208660) starting up for imap
> > DEBUG 2019-03-21T23:04:00+02:00 >>>>1<<<<<
> > 2019-03-21T23:04:00.003944+02:00 rooty.name /usr/sbin/cron 62249 - -
> > (root) CMD (/usr/home/stan/radio/frame/generate_me.sh)
> >
> > I'm getting the number "1" not just for "dovecot" app, but also for
> > crontab and pretty much for everything else... */me confused even
> > more*
> >
> >
> >
> >> Hello,
> >>
> >> Have you tried the configuration I provided ?
> >> My guess still that it is not an issue with the *program* filter,
> >> could you modify the file destination to also print the *${PROGRAM}*
> >> macro, to verify that it contains the value you expect ?
> >>
> >> --
> >> Kokan
> >>
> >> On Thu, Mar 21, 2019 at 8:57 PM Stanislav <me at rooty.name> wrote:
> >>
> >>> nah, I've just tried to replace that with "file( "/dev/klog"
> >>> owner(root)
> >>> group(wheel) perm(0666) );", didn't work.
> >>>
> >>> Also I'm getting logs to "/var/log/all.log" from dovecot without any
> >>>
> >>> issue, it just this filter, I feel something is not right there.
> >>>
> >>>> Hello,
> >>>>
> >>>> Is it possible that the *dovcot* application sends those logs via
> >>>> */dev/klog* ? Because in your configuration for that source the
> >>>> program is replaced with *kernel*.
> >>>>
> >>>> I tried the *program* filter with freebsd 12 + syslog-ng 3.20.1
> >>> with
> >>>> the following configuration:
> >>>>
> >>>> @version: 3.20
> >>>>
> >>>> log {
> >>>> source { internal(); };
> >>>> if {
> >>>> filter( program("syslog-ng"); };
> >>>> rewrite { set(":)" value(".FILTER")); };
> >>>> }
> >>>> else {
> >>>> rewrite { set(":(" value(".FILTER")); };
> >>>> }
> >>>>
> >>>> destination { file("/dev/stdout" template("${.FILTER}\n")); };
> >>>> };
> >>>>
> >>>> starting with syslog-ng -F
> >>>>
> >>>> The result seemed to be positive => :)
> >>>>
> >>>> --
> >>>> Kokan
> >>>>
> >>>> On Wed, Mar 20, 2019 at 4:41 AM Stanislav <me at rooty.name> wrote:
> >>>>
> >>>>> Greetings,
> >>>>>
> >>>>> I'm getting this issue after my last package upgrade
> >>>>>
> >>>>> ======================================
> >>>>> Name : syslog-ng
> >>>>> Version : 3.20.1
> >>>>> Installed on : Mon Mar 11 23:27:29 2019 EET
> >>>>> Origin : sysutils/syslog-ng
> >>>>> Architecture : FreeBSD:12:amd64
> >>>>> Prefix : /usr/local
> >>>>> Categories : sysutils
> >>>>> Licenses :
> >>>>> Maintainer : cy at FreeBSD.org
> >>>>> WWW : http://www.syslog-ng.org/
> >>>>> Comment : Powerful syslogd replacement
> >>>>> Options :
> >>>>> AMQP : off
> >>>>> CURL : off
> >>>>> DOCS : on
> >>>>> GEOIP2 : off
> >>>>> IPV6 : off
> >>>>> JAVA : off
> >>>>> JAVA_MOD : off
> >>>>> JSON : on
> >>>>> MONGO : off
> >>>>> PYTHON : off
> >>>>> REDIS : off
> >>>>> RIEMANN : off
> >>>>> SMTP : off
> >>>>> SPOOF : off
> >>>>> SQL : off
> >>>>> TCP_WRAPPERS : off
> >>>>> ======================================
> >>>>>
> >>>>> I have following configuration:
> >>>>>
> >>>>> options { chain_hostnames(off); flush_lines(0); threaded(yes);
> >>>>> create_dirs(yes); };
> >>>>> source local {
> >>>>> internal();
> >>>>> unix-dgram( "/var/run/log" owner(root) group(wheel)
> >>>>> perm(0666) );
> >>>>> unix-dgram( "/var/run/logpriv" owner(root)
> >>>>> group(wheel)
> >>>>> perm(0600) );
> >>>>> file( "/dev/klog" program_override("kernel") );
> >>>>> };
> >>>>> ...
> >>>>> destination all { file("/var/log/all.log"); };
> >>>>> destination maillog_mda { file("/var/log/maillog-mda"); };
> >>>>> ...
> >>>>> filter p_mail_imap { program("dovecot"); };
> >>>>> ...
> >>>>> log { source(local); destination(all); };
> >>>>> log { source(local); filter(p_mail_imap);
> >>> destination(maillog_mda);
> >>>>> };
> >>>>> ======================================
> >>>>> # ps auxww|grep dovecot
> >>>>> root 9648 0.0 0.1 13268 4196 - Is 00:46
> >>>>> 0:00.04
> >>>>> /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
> >>>>> dovecot 9651 0.0 0.0 12724 3784 - I 00:46
> >>>>> 0:00.01
> >>>>> anvil: [2 connections] (anvil)
> >>>>> root 15259 0.0 0.0 12796 4168 - I 01:42
> >>>>> 0:00.00
> >>>>> dovecot/log
> >>>>> root 16126 0.0 0.1 13744 5020 - I 01:52
> >>>>> 0:00.02
> >>>>> dovecot/config
> >>>>> dovecot 16127 0.0 0.0 12724 4180 - I 01:52
> >>>>> 0:00.01
> >>>>> stats: [3 connections] (stats)
> >>>>> dovecot 17328 0.0 0.1 21284 12276 - I 02:05
> >>>>> 0:00.01
> >>>>> auth: [0 wait, 0 passdb, 0 userdb] (auth)
> >>>>> ======================================
> >>>>> # syslog-ng -s
> >>>>> # echo $?
> >>>>> 0
> >>>>> ======================================
> >>>>>
> >>>>> I'm getting logs from dovecot program to /var/log/all.log but not
> >>>>> /var/log/maillog-mda . As I mentioned before it was working on
> >>>>> previous
> >>>>> version of syslog-ng .
> >>>>> Does anybody have this issue? Just me, lucky?
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> ______________________________________________________________________________
> >>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>>>> Documentation:
> >>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
> >>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>>
> >>>
> >>
> ______________________________________________________________________________
> >>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>>> Documentation:
> >>>> http://www.balabit.com/support/documentation/?product=syslog-ng
> >>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>
> >>
> ______________________________________________________________________________
> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>> Documentation:
> >>> http://www.balabit.com/support/documentation/?product=syslog-ng
> >>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>
> ______________________________________________________________________________
> >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >> Documentation:
> >> http://www.balabit.com/support/documentation/?product=syslog-ng
> >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >
> ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190322/508acdc7/attachment-0001.html>
More information about the syslog-ng
mailing list