[syslog-ng] seems like program filter is broken
Stanislav
me at rooty.name
Fri Mar 22 06:18:50 UTC 2019
Okay, just found this: https://reviews.freebsd.org/D14951
Thanks
> But this is not just dovecot. It's pretty much everything where
> "program" filter in use is broken including postfix, crontab, etc...
>
> Anyways flags(syslog-protocol) helped, works now
>
> DEBUG 2019-03-22T07:59:17+02:00 >>>>syslog-ng<<<<< syslog-ng shutting
> down; version='3.20.1'
> DEBUG 2019-03-22T07:59:19+02:00 >>>>syslog-ng<<<<< syslog-ng starting
> up; version='3.20.1'
> DEBUG 2019-03-22T07:59:30+02:00 >>>>dovecot<<<<< master: Warning:
> Killed with signal 15 (by pid=84861 uid=0 code=kill)
> DEBUG 2019-03-22T07:59:30+02:00 >>>>dovecot<<<<<
> imap(me at rooty.name)<84237><MdGdTKiEDuSsOiX2>: Server shutting down.
> in=27 out=928 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0
> body_count=0 body_bytes=0
> DEBUG 2019-03-22T07:59:31+02:00 >>>>postfix/smtpd<<<<< connect from
> unknown[178.62.196.23]
> DEBUG 2019-03-22T07:59:31+02:00 >>>>postfix/smtpd<<<<< disconnect from
> unknown[178.62.196.23] ehlo=1 auth=0/1 quit=1 commands=2/3
> DEBUG 2019-03-22T07:59:32+02:00 >>>>dovecot<<<<< master: Dovecot
> v2.3.5 (513208660) starting up for imap
>
> Thanks!
>
>
>
>> The problem seems to be that dovecot uses Rfc5424 formatted message on
>> the local log socket.
>>
>> Syslog-ng is able to cope with this format, and the system () source
>> has recently been adapted to allow this.
>>
>> If you are not using the system () source, just add
>> flags(syslog-protocol) to your unix-dgram() driver.
>>
>> Bazsi
>>
>> On Thu, Mar 21, 2019, 22:57 Stanislav <me at rooty.name wrote:
>>
>>> I also did a test with following configuration:
>>>
>>> ===========
>>> @version: 3.20
>>>
>>> log {
>>> source { internal(); };
>>> if (program("syslog-ng")) {
>>> rewrite { set(":)" value(".FILTER")); };
>>> }
>>> else {
>>> rewrite { set(":(" value(".FILTER")); };
>>> };
>>>
>>> destination { file("/dev/stdout" template("${.FILTER} [${PROGRAM}]
>>>
>>> ${MESSAGE}\n")); };
>>> };
>>> ===========
>>>
>>> The result is pretty much the same, I can see this:
>>> # syslog-ng -F
>>> :) [syslog-ng] syslog-ng starting up; version='3.20.1'
>>>
>>> but that's pretty much it, when I restart dovecot or any other
>>> application I can't see new lines...
>>>
>>> My full syslog-ng configuration: http://rooty.name/syslog-ng.conf
>>>
>>>> Hey,
>>>>
>>>> There's a syntax error:
>>>> Error parsing log statement, syntax error, unexpected '(',
>>> expecting
>>>> ')' in /usr/local/etc/syslog-ng.conf:6:20-6:21
>>>>
>>>> ...assuming it should be like this:
>>>> =============
>>>> @version: 3.20
>>>>
>>>> log {
>>>> source { internal(); };
>>>> if {
>>>> filter{ program("syslog-ng"); };
>>>> rewrite { set(":)" value(".FILTER")); };
>>>> }
>>>> else {
>>>> rewrite { set(":(" value(".FILTER")); };
>>>> };
>>>>
>>>> destination { file("/dev/stdout" template("${.FILTER}\n")); };
>>>> };
>>>> =============
>>>> I'm getting following result:
>>>>
>>>> # syslog-ng -F
>>>> :)
>>>>
>>>>
>>>> Seems like everything should be fine */me confused*...
>>>>
>>>> =============
>>>>
>>>> ok, so what I did next is:
>>>> destination all { file("/var/log/all.log" template("DEBUG
>>> ${ISODATE}
>>>>>>>> ${PROGRAM}<<<<< ${MESSAGE}\n")); };
>>>>
>>>> and I can see this:
>>>> DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
>>>> 2019-03-21T23:03:54.538134+02:00 rooty.name [1] dovecot 62129 - -
>>> master:
>>>> Warning: Killed with signal 15 (by pid=62197 uid=0 code=kill)
>>>> DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
>>>> 2019-03-21T23:03:54.539049+02:00 rooty.name [1] dovecot 62134 - -
>>>> imap(me at rooty.name)<62147><D8rkEaGEPHesOiU3>: Server shutting
>>> down.
>>>> in=27 out=775 deleted=0 expunged=0 trashed=0 hdr_count=0
>>> hdr_bytes=0
>>>> body_count=0 body_bytes=0
>>>> DEBUG 2019-03-21T23:03:56+02:00 >>>>1<<<<<
>>>> 2019-03-21T23:03:56.231605+02:00 rooty.name [1] dovecot 62224 - -
>>> master:
>>>> Dovecot v2.3.5 (513208660) starting up for imap
>>>> DEBUG 2019-03-21T23:04:00+02:00 >>>>1<<<<<
>>>> 2019-03-21T23:04:00.003944+02:00 rooty.name [1] /usr/sbin/cron
>>> 62249 - -
>>>> (root) CMD (/usr/home/stan/radio/frame/generate_me.sh)
>>>>
>>>> I'm getting the number "1" not just for "dovecot" app, but also
>>> for
>>>> crontab and pretty much for everything else... */me confused even
>>>> more*
>>>>
>>>>
>>>>
>>>>> Hello,
>>>>>
>>>>> Have you tried the configuration I provided ?
>>>>> My guess still that it is not an issue with the *program* filter,
>>>>> could you modify the file destination to also print the
>>> *${PROGRAM}*
>>>>> macro, to verify that it contains the value you expect ?
>>>>>
>>>>> --
>>>>> Kokan
>>>>>
>>>>> On Thu, Mar 21, 2019 at 8:57 PM Stanislav <me at rooty.name> wrote:
>>>>>
>>>>>> nah, I've just tried to replace that with "file( "/dev/klog"
>>>>>> owner(root)
>>>>>> group(wheel) perm(0666) );", didn't work.
>>>>>>
>>>>>> Also I'm getting logs to "/var/log/all.log" from dovecot without
>>> any
>>>>>>
>>>>>> issue, it just this filter, I feel something is not right there.
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Is it possible that the *dovcot* application sends those logs
>>> via
>>>>>>> */dev/klog* ? Because in your configuration for that source the
>>>>>>> program is replaced with *kernel*.
>>>>>>>
>>>>>>> I tried the *program* filter with freebsd 12 + syslog-ng
>>> 3.20.1
>>>>>> with
>>>>>>> the following configuration:
>>>>>>>
>>>>>>> @version: 3.20
>>>>>>>
>>>>>>> log {
>>>>>>> source { internal(); };
>>>>>>> if {
>>>>>>> filter( program("syslog-ng"); };
>>>>>>> rewrite { set(":)" value(".FILTER")); };
>>>>>>> }
>>>>>>> else {
>>>>>>> rewrite { set(":(" value(".FILTER")); };
>>>>>>> }
>>>>>>>
>>>>>>> destination { file("/dev/stdout" template("${.FILTER}\n")); };
>>>>>>> };
>>>>>>>
>>>>>>> starting with syslog-ng -F
>>>>>>>
>>>>>>> The result seemed to be positive => :)
>>>>>>>
>>>>>>> --
>>>>>>> Kokan
>>>>>>>
>>>>>>> On Wed, Mar 20, 2019 at 4:41 AM Stanislav <me at rooty.name>
>>> wrote:
>>>>>>>
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> I'm getting this issue after my last package upgrade
>>>>>>>>
>>>>>>>> ======================================
>>>>>>>> Name : syslog-ng
>>>>>>>> Version : 3.20.1
>>>>>>>> Installed on : Mon Mar 11 23:27:29 2019 EET
>>>>>>>> Origin : sysutils/syslog-ng
>>>>>>>> Architecture : FreeBSD:12:amd64
>>>>>>>> Prefix : /usr/local
>>>>>>>> Categories : sysutils
>>>>>>>> Licenses :
>>>>>>>> Maintainer : cy at FreeBSD.org
>>>>>>>> WWW : http://www.syslog-ng.org/
>>>>>>>> Comment : Powerful syslogd replacement
>>>>>>>> Options :
>>>>>>>> AMQP : off
>>>>>>>> CURL : off
>>>>>>>> DOCS : on
>>>>>>>> GEOIP2 : off
>>>>>>>> IPV6 : off
>>>>>>>> JAVA : off
>>>>>>>> JAVA_MOD : off
>>>>>>>> JSON : on
>>>>>>>> MONGO : off
>>>>>>>> PYTHON : off
>>>>>>>> REDIS : off
>>>>>>>> RIEMANN : off
>>>>>>>> SMTP : off
>>>>>>>> SPOOF : off
>>>>>>>> SQL : off
>>>>>>>> TCP_WRAPPERS : off
>>>>>>>> ======================================
>>>>>>>>
>>>>>>>> I have following configuration:
>>>>>>>>
>>>>>>>> options { chain_hostnames(off); flush_lines(0); threaded(yes);
>>>>>>>> create_dirs(yes); };
>>>>>>>> source local {
>>>>>>>> internal();
>>>>>>>> unix-dgram( "/var/run/log" owner(root) group(wheel)
>>>>>>>> perm(0666) );
>>>>>>>> unix-dgram( "/var/run/logpriv" owner(root)
>>>>>>>> group(wheel)
>>>>>>>> perm(0600) );
>>>>>>>> file( "/dev/klog" program_override("kernel") );
>>>>>>>> };
>>>>>>>> ...
>>>>>>>> destination all { file("/var/log/all.log"); };
>>>>>>>> destination maillog_mda { file("/var/log/maillog-mda"); };
>>>>>>>> ...
>>>>>>>> filter p_mail_imap { program("dovecot"); };
>>>>>>>> ...
>>>>>>>> log { source(local); destination(all); };
>>>>>>>> log { source(local); filter(p_mail_imap);
>>>>>> destination(maillog_mda);
>>>>>>>> };
>>>>>>>> ======================================
>>>>>>>> # ps auxww|grep dovecot
>>>>>>>> root 9648 0.0 0.1 13268 4196 - Is 00:46
>>>>>>>> 0:00.04
>>>>>>>> /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
>>>>>>>> dovecot 9651 0.0 0.0 12724 3784 - I 00:46
>>>>>>>> 0:00.01
>>>>>>>> anvil: [2 connections] (anvil)
>>>>>>>> root 15259 0.0 0.0 12796 4168 - I 01:42
>>>>>>>> 0:00.00
>>>>>>>> dovecot/log
>>>>>>>> root 16126 0.0 0.1 13744 5020 - I 01:52
>>>>>>>> 0:00.02
>>>>>>>> dovecot/config
>>>>>>>> dovecot 16127 0.0 0.0 12724 4180 - I 01:52
>>>>>>>> 0:00.01
>>>>>>>> stats: [3 connections] (stats)
>>>>>>>> dovecot 17328 0.0 0.1 21284 12276 - I 02:05
>>>>>>>> 0:00.01
>>>>>>>> auth: [0 wait, 0 passdb, 0 userdb] (auth)
>>>>>>>> ======================================
>>>>>>>> # syslog-ng -s
>>>>>>>> # echo $?
>>>>>>>> 0
>>>>>>>> ======================================
>>>>>>>>
>>>>>>>> I'm getting logs from dovecot program to /var/log/all.log but
>>> not
>>>>>>>> /var/log/maillog-mda . As I mentioned before it was working on
>>>>>>>> previous
>>>>>>>> version of syslog-ng .
>>>>>>>> Does anybody have this issue? Just me, lucky?
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>> ______________________________________________________________________________
>>>>>>>> Member info:
>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>>> Documentation:
>>>>>>>>
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>
>>>>>>
>>>>>
>>>
>> ______________________________________________________________________________
>>>>>>> Member info:
>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> Documentation:
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>
>>>>>
>>>
>> ______________________________________________________________________________
>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>> Documentation:
>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>
>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation:
>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>
>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>> Links:
>> ------
>> [1] http://rooty.name
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
More information about the syslog-ng
mailing list