[syslog-ng] seems like program filter is broken

Stanislav me at rooty.name
Fri Mar 22 06:07:59 UTC 2019


But this is not just dovecot. It's pretty much everything where 
"program" filter in use is broken including postfix, crontab, etc...

Anyways flags(syslog-protocol) helped, works now

DEBUG 2019-03-22T07:59:17+02:00 >>>>syslog-ng<<<<< syslog-ng shutting 
down; version='3.20.1'
DEBUG 2019-03-22T07:59:19+02:00 >>>>syslog-ng<<<<< syslog-ng starting 
up; version='3.20.1'
DEBUG 2019-03-22T07:59:30+02:00 >>>>dovecot<<<<< master: Warning: Killed 
with signal 15 (by pid=84861 uid=0 code=kill)
DEBUG 2019-03-22T07:59:30+02:00 >>>>dovecot<<<<< 
imap(me at rooty.name)<84237><MdGdTKiEDuSsOiX2>: Server shutting down. 
in=27 out=928 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 
body_count=0 body_bytes=0
DEBUG 2019-03-22T07:59:31+02:00 >>>>postfix/smtpd<<<<< connect from 
unknown[178.62.196.23]
DEBUG 2019-03-22T07:59:31+02:00 >>>>postfix/smtpd<<<<< disconnect from 
unknown[178.62.196.23] ehlo=1 auth=0/1 quit=1 commands=2/3
DEBUG 2019-03-22T07:59:32+02:00 >>>>dovecot<<<<< master: Dovecot v2.3.5 
(513208660) starting up for imap

Thanks!



> The problem seems to be that dovecot uses Rfc5424 formatted message on
> the local log socket.
> 
> Syslog-ng is able to cope with this format, and the system () source
> has recently been adapted to allow this.
> 
> If you are not using the system () source, just add
> flags(syslog-protocol) to your unix-dgram() driver.
> 
> Bazsi
> 
> On Thu, Mar 21, 2019, 22:57 Stanislav <me at rooty.name wrote:
> 
>> I also did a test with following configuration:
>> 
>> ===========
>> @version: 3.20
>> 
>> log {
>> source { internal(); };
>> if (program("syslog-ng")) {
>> rewrite { set(":)" value(".FILTER")); };
>> }
>> else {
>> rewrite { set(":(" value(".FILTER")); };
>> };
>> 
>> destination { file("/dev/stdout" template("${.FILTER} [${PROGRAM}]
>> 
>> ${MESSAGE}\n")); };
>> };
>> ===========
>> 
>> The result is pretty much the same, I can see this:
>> # syslog-ng -F
>> :) [syslog-ng] syslog-ng starting up; version='3.20.1'
>> 
>> but that's pretty much it, when I restart dovecot or any other
>> application I can't see new lines...
>> 
>> My full syslog-ng configuration: http://rooty.name/syslog-ng.conf
>> 
>>> Hey,
>>> 
>>> There's a syntax error:
>>> Error parsing log statement, syntax error, unexpected '(',
>> expecting
>>> ')' in /usr/local/etc/syslog-ng.conf:6:20-6:21
>>> 
>>> ...assuming it should be like this:
>>> =============
>>> @version: 3.20
>>> 
>>> log {
>>> source { internal(); };
>>> if {
>>> filter{ program("syslog-ng"); };
>>> rewrite { set(":)" value(".FILTER")); };
>>> }
>>> else {
>>> rewrite { set(":(" value(".FILTER")); };
>>> };
>>> 
>>> destination { file("/dev/stdout" template("${.FILTER}\n")); };
>>> };
>>> =============
>>> I'm getting following result:
>>> 
>>> # syslog-ng -F
>>> :)
>>> 
>>> 
>>> Seems like everything should be fine */me confused*...
>>> 
>>> =============
>>> 
>>> ok, so what I did next is:
>>> destination all { file("/var/log/all.log" template("DEBUG
>> ${ISODATE}
>>>>>>> ${PROGRAM}<<<<< ${MESSAGE}\n")); };
>>> 
>>> and I can see this:
>>> DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
>>> 2019-03-21T23:03:54.538134+02:00 rooty.name [1] dovecot 62129 - -
>> master:
>>> Warning: Killed with signal 15 (by pid=62197 uid=0 code=kill)
>>> DEBUG 2019-03-21T23:03:54+02:00 >>>>1<<<<<
>>> 2019-03-21T23:03:54.539049+02:00 rooty.name [1] dovecot 62134 - -
>>> imap(me at rooty.name)<62147><D8rkEaGEPHesOiU3>: Server shutting
>> down.
>>> in=27 out=775 deleted=0 expunged=0 trashed=0 hdr_count=0
>> hdr_bytes=0
>>> body_count=0 body_bytes=0
>>> DEBUG 2019-03-21T23:03:56+02:00 >>>>1<<<<<
>>> 2019-03-21T23:03:56.231605+02:00 rooty.name [1] dovecot 62224 - -
>> master:
>>> Dovecot v2.3.5 (513208660) starting up for imap
>>> DEBUG 2019-03-21T23:04:00+02:00 >>>>1<<<<<
>>> 2019-03-21T23:04:00.003944+02:00 rooty.name [1] /usr/sbin/cron
>> 62249 - -
>>> (root) CMD (/usr/home/stan/radio/frame/generate_me.sh)
>>> 
>>> I'm getting the number "1" not just for "dovecot" app, but also
>> for
>>> crontab and pretty much for everything else... */me confused even
>>> more*
>>> 
>>> 
>>> 
>>>> Hello,
>>>> 
>>>> Have you tried the configuration I provided ?
>>>> My guess still that it is not an issue with the *program* filter,
>>>> could you modify the file destination to also print the
>> *${PROGRAM}*
>>>> macro, to verify that it contains the value you expect ?
>>>> 
>>>> --
>>>> Kokan
>>>> 
>>>> On Thu, Mar 21, 2019 at 8:57 PM Stanislav <me at rooty.name> wrote:
>>>> 
>>>>> nah, I've just tried to replace that with "file( "/dev/klog"
>>>>> owner(root)
>>>>> group(wheel) perm(0666) );", didn't work.
>>>>> 
>>>>> Also I'm getting logs to "/var/log/all.log" from dovecot without
>> any
>>>>> 
>>>>> issue, it just this filter, I feel something is not right there.
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> Is it possible that the *dovcot* application sends those logs
>> via
>>>>>> */dev/klog* ? Because in your configuration for that source the
>>>>>> program is replaced with *kernel*.
>>>>>> 
>>>>>> I tried the *program* filter with freebsd 12  + syslog-ng
>> 3.20.1
>>>>> with
>>>>>> the following configuration:
>>>>>> 
>>>>>> @version: 3.20
>>>>>> 
>>>>>> log {
>>>>>> source { internal(); };
>>>>>> if {
>>>>>> filter( program("syslog-ng"); };
>>>>>> rewrite { set(":)" value(".FILTER")); };
>>>>>> }
>>>>>> else {
>>>>>> rewrite { set(":(" value(".FILTER")); };
>>>>>> }
>>>>>> 
>>>>>> destination { file("/dev/stdout" template("${.FILTER}\n")); };
>>>>>> };
>>>>>> 
>>>>>> starting with syslog-ng -F
>>>>>> 
>>>>>> The result seemed to be positive => :)
>>>>>> 
>>>>>> --
>>>>>> Kokan
>>>>>> 
>>>>>> On Wed, Mar 20, 2019 at 4:41 AM Stanislav <me at rooty.name>
>> wrote:
>>>>>> 
>>>>>>> Greetings,
>>>>>>> 
>>>>>>> I'm getting this issue after my last package upgrade
>>>>>>> 
>>>>>>> ======================================
>>>>>>> Name           : syslog-ng
>>>>>>> Version        : 3.20.1
>>>>>>> Installed on   : Mon Mar 11 23:27:29 2019 EET
>>>>>>> Origin         : sysutils/syslog-ng
>>>>>>> Architecture   : FreeBSD:12:amd64
>>>>>>> Prefix         : /usr/local
>>>>>>> Categories     : sysutils
>>>>>>> Licenses       :
>>>>>>> Maintainer     : cy at FreeBSD.org
>>>>>>> WWW            : http://www.syslog-ng.org/
>>>>>>> Comment        : Powerful syslogd replacement
>>>>>>> Options        :
>>>>>>> AMQP           : off
>>>>>>> CURL           : off
>>>>>>> DOCS           : on
>>>>>>> GEOIP2         : off
>>>>>>> IPV6           : off
>>>>>>> JAVA           : off
>>>>>>> JAVA_MOD       : off
>>>>>>> JSON           : on
>>>>>>> MONGO          : off
>>>>>>> PYTHON         : off
>>>>>>> REDIS          : off
>>>>>>> RIEMANN        : off
>>>>>>> SMTP           : off
>>>>>>> SPOOF          : off
>>>>>>> SQL            : off
>>>>>>> TCP_WRAPPERS   : off
>>>>>>> ======================================
>>>>>>> 
>>>>>>> I have following configuration:
>>>>>>> 
>>>>>>> options { chain_hostnames(off); flush_lines(0); threaded(yes);
>>>>>>> create_dirs(yes); };
>>>>>>> source local {
>>>>>>> internal();
>>>>>>> unix-dgram( "/var/run/log" owner(root) group(wheel)
>>>>>>> perm(0666) );
>>>>>>> unix-dgram( "/var/run/logpriv" owner(root)
>>>>>>> group(wheel)
>>>>>>> perm(0600) );
>>>>>>> file( "/dev/klog" program_override("kernel") );
>>>>>>> };
>>>>>>> ...
>>>>>>> destination all { file("/var/log/all.log"); };
>>>>>>> destination maillog_mda { file("/var/log/maillog-mda"); };
>>>>>>> ...
>>>>>>> filter p_mail_imap { program("dovecot"); };
>>>>>>> ...
>>>>>>> log { source(local); destination(all); };
>>>>>>> log { source(local); filter(p_mail_imap);
>>>>> destination(maillog_mda);
>>>>>>> };
>>>>>>> ======================================
>>>>>>> # ps auxww|grep dovecot
>>>>>>> root       9648   0.0  0.1   13268    4196  -  Is   00:46
>>>>>>> 0:00.04
>>>>>>> /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
>>>>>>> dovecot    9651   0.0  0.0   12724    3784  -  I    00:46
>>>>>>> 0:00.01
>>>>>>> anvil: [2 connections] (anvil)
>>>>>>> root      15259   0.0  0.0   12796    4168  -  I    01:42
>>>>>>> 0:00.00
>>>>>>> dovecot/log
>>>>>>> root      16126   0.0  0.1   13744    5020  -  I    01:52
>>>>>>> 0:00.02
>>>>>>> dovecot/config
>>>>>>> dovecot   16127   0.0  0.0   12724    4180  -  I    01:52
>>>>>>> 0:00.01
>>>>>>> stats: [3 connections] (stats)
>>>>>>> dovecot   17328   0.0  0.1   21284   12276  -  I    02:05
>>>>>>> 0:00.01
>>>>>>> auth: [0 wait, 0 passdb, 0 userdb] (auth)
>>>>>>> ======================================
>>>>>>> # syslog-ng -s
>>>>>>> # echo $?
>>>>>>> 0
>>>>>>> ======================================
>>>>>>> 
>>>>>>> I'm getting logs from dovecot program to /var/log/all.log but
>> not
>>>>>>> /var/log/maillog-mda . As I mentioned before it was working on
>>>>>>> previous
>>>>>>> version of syslog-ng .
>>>>>>> Does anybody have this issue? Just me, lucky?
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 
> ______________________________________________________________________________
>>>>>>> Member info:
>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> Documentation:
>>>>>>> 
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>> 
>>>>> 
>>>> 
>> 
> ______________________________________________________________________________
>>>>>> Member info:
>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>> Documentation:
>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>> 
>>>> 
>> 
> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation:
>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>> 
>> 
> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> 
>> 
> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>> 
> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
> 
> Links:
> ------
> [1] http://rooty.name
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: 
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq


More information about the syslog-ng mailing list