February 2018 - syslog-ng - lists.balabit.hu

enforcing file permissions
by Nik Ambrosch 20 Feb '18

20 Feb '18

i know it's possible to create a file with permissions using owner() and group() however if the file already exists with different permissions or the permissions are changed later syslog-ng does not change the permissions back. in a future release is it possible to change the behavior of owner() and group() to enforce permissions or possibly add a new option like enforce-permissions(true) with a default of false (to mimic the current behavior)? maybe something exists already and i'm just missing it :) thanks all.

2 1

syslog-ng 3.13 key-value parser crashes and aborts
by Don C 19 Feb '18

19 Feb '18

Hi, I was upgrading syslog-ng from 3.12 to 3.13 using the prebuilt unofficial RPMs. I was testing my config on 3.13 and ran into the following issue where the key-value parsing aborts.This appears to be a regression in 3.13, the abort never happened in 3.12. If I remove the kv_parser from my config, there is no issue. Here's the end of the debug and verbose output I get. Notice the garbage values in the key-value names parsed from the message. [2018-01-25T17:33:08.924895] Filter rule evaluation result; msg='0x7fc0a807b370', result='match', rule='f_compliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:66:27' [2018-01-25T17:33:08.924919] Setting value; msg='0x7fc0a80612f0', name='@\x15\x01pÀ', value='2017-11-21 19:11:24.817041' [2018-01-25T17:33:08.924930] Setting value; msg='0x7fc0a80612f0', name='', value='debug' [2018-01-25T17:33:08.924936] Setting value; msg='0x7fc0a80612f0', name='f_compliant_hosts', value='[robotnats kafka transport prepared]' [2018-01-25T17:33:08.924944] Setting value; msg='0x7fc0a80612f0', name='PK\x06¨À', value='ROBOT_serviceTracking_neo4j-topo-svc-2717019760-1sdn1' [2018-01-25T17:33:08.924952] Setting value; msg='0x7fc0a80612f0', name='', value='nats://nats:4222' [2018-01-25T17:33:08.924957] Setting value; msg='0x7fc0a80612f0', name='', value='[robot-kafka:9092]' [2018-01-25T17:33:08.924963] Setting value; msg='0x7fc0a80612f0', name='kv_kafka', value='robotnats_kafka' [2018-01-25T17:33:08.924969] Setting value; msg='0x7fc0a80612f0', name=' »\x08¬À', value='ROBOT_serviceTracking_neo4j-topo-svc-2717019760-1sdn1' [2018-01-25T17:33:08.924975] Message parsing complete; result='1', rule='p_kv', location='/etc/syslog-ng/syslog-ng.conf:75:5' [2018-01-25T17:33:08.924998] Incoming log entry; line='time="2017-11-21 19:11:24.817095" level=info msg="[skeleton core configuration parse stage complete]" config=/etc/robot/servicetracking.conf debug=true logfile= maxthreads=16 tag=ROBOT_serviceTracking_neo4j-topo-svc-2717019760-1sdn1 version=' [2018-01-25T17:33:08.925030] Setting value; msg='0x7fc0a807bd60', name='MESSAGE', value='time="2017-11-21 19:11:24.817095" level=info msg="[skeleton core configuration parse stage complete]" config=/etc/robot/servicetracking.conf debug=true logfile= maxthreads=16 tag=ROBOT_serviceTracking_neo4j-topo-svc-2717019760-1sdn1 version=' [2018-01-25T17:33:08.925040] Setting value; msg='0x7fc0a807bd60', name='HOST_FROM', value='syslog-ng-logging-1040168119-qsmrf' [2018-01-25T17:33:08.925046] Setting value; msg='0x7fc0a807bd60', name='HOST', value='syslog-ng-logging-1040168119-qsmrf/syslog-ng-logging-1040168119-qsmrf' [2018-01-25T17:33:08.925053] Setting value; msg='0x7fc0a807bd60', name='FILE_NAME', value='/mnt/logfs/neo4j-topo-svc/1/servicetracking_stdout.log' [2018-01-25T17:33:08.925057] Setting value; msg='0x7fc0a807bd60', name='SOURCE', value='s_file_stdout' [2018-01-25T17:33:08.925062] Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:81:5' [2018-01-25T17:33:08.824606] Incoming log entry; line='time="2017-11-21 19:10:56.399341" level=debug msg="[section not required]" section=etcd tag=ROBOT_serviceTracking_robot-topo-svc-2107321943-48rnw' [2018-01-25T17:33:09.022629] Filter rule evaluation begins; msg='0x7fc0a807bd60', rule='f_compliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:66:27' [2018-01-25T17:33:08.824700] Setting value; msg='0x7fc0a409b810', name='HOST_FROM', value='syslog-ng-logging-1040168119-qsmrf' # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # ... [2018-01-25T17:33:09.022911] Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:81:5' [2018-01-25T17:33:09.022916] Filter rule evaluation begins; msg='0x7fc08c0f5550', rule='f_compliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:66:27' [2018-01-25T17:33:09.022922] Filter node evaluation result; msg='0x7fc08c0f5550', result='not-match', type='==' [2018-01-25T17:33:09.022927] Filter rule evaluation result; msg='0x7fc08c0f5550', result='not-match', rule='f_compliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:66:27' [2018-01-25T17:33:09.022932] Filter rule evaluation begins; msg='0x7fc08c0f5550', rule='f_noncompliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:70:30' [2018-01-25T17:33:09.022937] Filter node evaluation result; msg='0x7fc08c0f5550', result='not-match', type='==' [2018-01-25T17:33:09.022942] Filter node evaluation result; msg='0x7fc08c0f5550', result='match', type='filter(f_compliant_hosts)' [2018-01-25T17:33:09.022947] Filter rule evaluation result; msg='0x7fc08c0f5550', result='match', rule='f_noncompliant_hosts', location='/etc/syslog-ng/syslog-ng.conf:70:30' Aborted Is this a known issue with the key-value parser in 3.13? Regards, Don

3 2

vCenter 6.5 RFC5425 prefacing messages with a space
by Evan Rempel 18 Feb '18

18 Feb '18

Our VMWare team has been busy upgrading out infrastructure to 6.5 which is great, but the syslogs started showing up as 2018-02-17T07:00:12 esx.host.name auth.info 1 2018-02-17T07:00:12.162028-08:00 esx.host.name sshd 4662 - - Did not receive identification string from 142.104.139.163 port 51088 which I recognise as RFC5425, So I added the flags(syslog-protocol) to my network source definition. Now the log message arrives much nicer as 2018-02-17T07:48:12 esx.host.name auth.info sshd[20835]: Did not receive identification string from 142.104.139.163 port 42060 but if you look very closely you will notice that there are 2 spaces in "sshd[20835]: Did" and in the original logged message there were 2 spaces preceding the word "Did". Has anyone seen this before (or currently with your own VMWare 6.5). I'm trying to figure out if this is a bug in syslog-ng or a bug in vCenter 6.5 or a config error on my systems. Thanks, Evan.

2 1

confgen not usable (by me) 3.13.2 on Redhat 7
by Evan Rempel 18 Feb '18

18 Feb '18

I am using the copr RPM build syslog-ng-3.13.2-3.el7.centos.x86_64 on a redhat 7 system. I am trying to use the confgen to create a value and use it in a rewrite. I realize that I can use $LOGHOST in this context but am using it as an example. @module confgen context(rewrite) name(z_myself) exec("/usr/bin/hostname") rewrite r_local { set( "z_myself()", value("HOST") ); }; log { source(local); rewrite(r_local); destination(d_var_syslog); }; d_var_syslog is any file destination to capture the log message. The above configuration actually sets the hostname in the log line to be "z_myself()". 2018-02-17T21:15:39.220-08:00 z_myself() solaris-cron.err erempel: just testing if I make the rewrite rewrite r_local { set( z_myself(), value("HOST") ); }; I get syntax errors. The top example works on redhat 6 with syslog-ng 3.9 I can't find any documentation on how to use a block (confgen) define in a rewrite statement. I have tried back ticks, $(z_myself()) macros "`z_myself()`" and any other combination but they all result in syslog-ng failing to start. What am I doing wrong? Evan.

2 1

Re: [syslog-ng] QuickQ syslog source drivers
by Balazs Scheidler 17 Feb '18

17 Feb '18

Yes, recent versions would accept normal bsd like messages. The key to watch out is the framing, as that needs to be correct. syslog(transport(tcp)) expects rfc5425 (or rfc5426, cant remember which) style framing network(transport(tcp) flags(syslog-protocol)) accepts both old and new formats without framing syslog(transport(udp)) has no framing, so again it accepts 5424 and 3164 and uses udp. Actually, we may want to imply flags(syslog-protocol) everywhere in the future, there's an easy way to autodetect the format. On Feb 16, 2018 21:31, "Scot" <scotrn(a)gmail.com> wrote: Will the syslog() source driver support both RFC5424 and RFC3164 from different hosts at the same time in flight? source s_syslog_tcp { syslog(port(514) transport("tcp")); }; source s_syslog_udp { syslog(port(514) transport("udp")); }; ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

QuickQ syslog source drivers
by Scot 16 Feb '18

16 Feb '18

Will the syslog() source driver support both RFC5424 and RFC3164 from different hosts at the same time in flight? source s_syslog_tcp { syslog(port(514) transport("tcp")); }; source s_syslog_udp { syslog(port(514) transport("udp")); };

1 0

syslog-ng destination to multiple mssql database
by Garry De Toffoli 16 Feb '18

16 Feb '18

Hello, I have configured syslog.ng OSE version 3.5.3 to log to a MSSQL destination and that all works well. However when I have specified 2 MSSQL destination, same SQL Server, same table on different database, syslog-ng have problem to insert on database, both on the first and on the second. Some time will insert, sometimes not. On /var/log/syslog I have found some lines reporting error CREATE TABLE on the second database, Restarting the service, sudo service syslog-ng restart, takes a long time. When I comment out the destination of the 2nd database, syslog-ng run very well. Is it a limit of the OSE version? Or may be an error on the configuration file? See attached my configuration file. Thank you for the patience. ----------------------- Garry De Toffoli -----------------------

2 1

tcp wrappers
by Czanik, Péter 15 Feb '18

15 Feb '18

Hi, I'm running a poll about tcp wrappers on twitter: https://twitter.com/PCzanik/status/961985140128911361 but feedback is also very welcome here: Is there anybody still using tcp wrappers support built into syslog-ng on Fedora / RHEL / CentOS? Fedora rawhide removed support for tcp wrappers, and if nobody opposes I'd remove it from the rest of my pacages as well to keep packaging simple. Some background info for those who don't even know what I'm talking about: https://fedoraproject.org/wiki/Changes/Deprecate_TCP_wrappers (In short: it's really archaic stuff I last time used in 1995... :) ) Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://syslog-ng.com/blog/author/peterczanik/ https://twitter.com/PCzanik

3 3

Umask funkiness
by Declan White 10 Feb '18

10 Feb '18

I have a directory owned by the syslog-ng user. Its group however belongs to a group of which the user is not a member. The directory is g+s, so that all files and dirs made within it inherit the group owner (and the g+s in the case of dirs). syslog-ng is running with a umask of 022 (interrogated running process to be sure). The file("/dir/${FOO}/${BAR}") destination driver has : create-dirs(yes) perm() dir-owner() dir-group() dir-perm() i.e. "don't change any perms" The aim of the game is to end up with files and dirs readable, but not writable, by the inherited group owner. I can't get it working. I am always ending up with drwx--S--- dirs and -rw------- files Solaris. syslog-ng-3.12.1 -- Declan White

4 7

Re: [syslog-ng] problem elasticsearch2 creating index per HOST
by Abe Lebo 08 Feb '18

08 Feb '18

ah! that must be it! I was not aware that upper case characters are not allowed in elasticsearch index names! $HOST = B8-27-AB-23-11-26 Thanks a million Fabien. Op 8-2-2018 om 10:00 schreef Fabien Wernli: > On Thu, Feb 08, 2018 at 09:42:38AM +0100, Abe Lebo wrote: >> i have templates, but not one specific for this index patern. >> >> i see no errors in the logs, i only see the indices being created if i do >> not add the $HOST, but only $YEAR.$MONTH.$DAY >> >> i'll see if i can set syslog-ng logging to debug > > yes, definitely try that: syslog-ng -Fdv > > One other thing, what does your $HOST resolve to? There are some > restrictions in ES index names, perhaps your macro contains upper case or > other illegal chars? > >

2 1