February 2013 - syslog-ng - lists.balabit.hu

need help with pattern, get .classifier.class=unknown
by Stefan Sabolowitsch 14 Feb '13

14 Feb '13

Hi all, i use the net screen pattern the ruleset: <ruleset name="NetScreen" id='1001'> <pattern>fw</pattern> <rules> <rule provider="ELSA" class='1001' id='1001'> <patterns> <pattern>NetScreen device_id=@ESTRING:s0: @@ESTRING:: @start_time="@ESTRING::"@ duration=@ESTRING:: @policy_id=@ESTRING:i0: @service=@ESTRING:s1: @proto=@ESTRING:: @src zone=@ESTRING:s2: @dst zone=@ESTRING:s3: @action=@ESTRING:s4: @sent=@ESTRING:: @rcvd=@ESTRING:: @src=@ESTRING:i1: @dst=@ESTRING:i2: @src_port=@ESTRING:i3: @dst_port=@ESTRING:i4: @session_id=@ESTRING:: @reason=Traffic Denied</pattern> </patterns> <examples> <example> <test_message program="NetScreen deny">NetScreen device_id=fw [Root]system-notification-00257(traffic): start_time="2012-10-02 09:46:20" duration=0 policy_id=10005 service=http proto=6 src zone=OUT dst zone=IN action=Deny sent=0 rcvd=40 src=192.168.0.1 dst=192.168.1.1 src_port=51271 dst_port=80 session_id=0 reason=Traffic Denied</test_message>  <test_value name="s0">fw</test_value>  <test_value name="i0">10005</test_value>  <test_value name="s1">http</test_value>  <test_value name="s2">OUT</test_value>  <test_value name="s3">IN</test_value>  <test_value name="s4">Deny</test_value>  <test_value name="i1">192.168.0.1</test_value>  <test_value name="i2">192.168.1.1</test_value>  <test_value name="i3">51271</test_value>  <test_value name="i4">80</test_value> </example> </examples> </rule> </rules> </ruleset> but i get this error : /usr/local/syslog-ng/bin/pdbtool match -c -D -v -p /usr/local/elsa/node/conf/patterndb.xml -P fw -M "NetScreen device_id=fw [Root]system-notification-00257(traffic): start_time="2012-10-02 09:46:20" duration=0 policy_id=10005 service=http proto=6 src zone=OUT dst zone=IN action=Deny sent=0 rcvd=40 src=192.168.0.1 dst=192.168.1.1 src_port=51271 dst_port=80 session_id=0 reason=Traffic Denied" Pattern matching part: NetScreen device_id=@ESTRING:s0=fw@@ESTRING:None=[Root]system-notification-00257(traffic):@start_time=2012-10-02 Matching part: NetScreen device_id=fw [Root]system-notification-00257(traffic): start_time= Values: MESSAGE=NetScreen device_id=fw [Root]system-notification-00257(traffic): start_time=2012-10-02 PROGRAM=fw .classifier.class=unknown Any help here thanks Stefan

2 2

[Bug 219] New: log_msg_is_write_protected when a source is used in multiple log statments
by bugzilla＠bugzilla.balabit.com 14 Feb '13

14 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=219 Summary: log_msg_is_write_protected when a source is used in multiple log statments Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=71) --> (https://bugzilla.balabit.com/attachment.cgi?id=71) minimal syslog-ng.conf It appears that if a source is used in two different log statements, all branches use a write protected instance of the log message. Module loaded and initialized successfully; module='syslogformat' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.4.1' Syslog connection accepted; fd='12', client='AF_INET(142.104.141.3:40576)', local='AF_INET(0.0.0.0:514)' Incoming log entry; line='<134>2013-02-04T15:28:46-08:00 pangolin.comp.uvic.ca/pangolin.comp.uvic.ca action-handler[24020]: starting' Initializing destination file writer; template='/var/syslog/archive.log', filename='/var/syslog/archive.log' Filter node evaluation result; result='not-match' Filter node evaluation result; result='not-match', type='filter(f_short_host_at)' ** ERROR:logmsg.c:535:log_msg_set_value_indirect: assertion failed: (!log_msg_is_write_protected(self)) Attached is a very simple config file that reproduces this issue. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

IPv6 rewrite rule PVCE
by strife＠riseup.net 14 Feb '13

14 Feb '13

Hi, I want to anonymize certain logs, ie. remove IP addresses. Thanks to the advice/experiments by Micah from Riseup, I have the following. It works fine for IPv4: rewrite r_ip { subst('([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}', "\[REDACTED\]", value("MESSAGE"), type("pcre"), flags("global")); }; I tried to extend this to IPv6 using the IPV6 PCRE expression found at http://home.deds.nl/~aeron/regex/ : subst('/(((?=.*(::))(?!.*\3.+\3))\3?|([\dA-F]{1,4}(\3|:\b|$)|\2))(?4){5}((?4){2}|(((2[0-4]|1\d|[1-9])?\d|25[0-5])\.?\b){4})\z/i', "\[REDACTED\]", value("MESSAGE"), type("pcre"), flags("global")); When I test the expression using pcretest, it successfully matches IPv6 addresses in text. In syslog-ng, it does not. Any ideas? Thanks! -- strife

3 6

syslog-ng Insider - February 2013
by Peter Czanik 14 Feb '13

14 Feb '13

Dear syslog-ng users, This is the 21th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS Video: Basic building blocks of syslog-ng.conf ---------------------------------------------- There is a video series in the work to introduce the basics of syslog-ng. The second video is online now, which explains the basic building blocks of syslog-ng.conf. You can watch it at http://www.youtube.com/watch?v=D3OjHQj42Jg New videos will be published on the same Youtube channel. You are very welcome to subscribe to receive notifications about further syslog-ng videos and other interesting content. Syslog-ng 3.4.1 is released --------------------------- Version 3.4.1was released at the end of January. Compared to the RC2release, it only has some minor bugfixes. This version brings many major new features to syslog-ng: junctions and channels to make syslog-ng configuration even more flexible, a community contributed AMQP destination, improved JSON support including a parser and many value pairs related improvements. For more details, check http://algernon.blogs.balabit.com/2013/01/syslog-ng-ose-3-4-1-released/ You cancheck the announcement athttps://lists.balabit.hu/pipermail/syslog-ng-announce/2013-January/000154.html For binary package availability check our 3^rd party binary page at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… At the time of this writing there are v3.4.1 packages for openSUSE and SLES, not only as 3^rd party packages but also as part of the openSUSE 12.3 release. It's also available for FreeBSD in the sysutils/syslog-ng-devel port, and already in Fedora Rawhide. Packages for various Debian and Ubuntu releases are available in algernon's repo. syslog-ng at conferences: FOSDEM and Brno ----------------------------------------- The syslog-ng team is actively participating in the life of the open source community. Last weekend CzP gave a talk at FOSDEM and participated a two day long brainstorming session about logging and monitoring in Antwerp. He met many interesting people from the syslog-ng community. You can read about his experiences at http://czanik.blogs.balabit.com/2013/02/czp-fosdem-2013/ We will also participate http://devconf.cz/ and the systemd hackfest the day before the conference. If you are also there and want to talk about syslog-ng, we are more than happy to meet you! SHORT NEWS: * To use cacti with syslog-ng, check configuration examples at https://czanik.blogs.balabit.com/2013/01/trying-cacti-with-syslog-ng/ * What's new presentation in Hungarian for syslog-ng 3.4 with configuration examples: http://asylum.madhouse-project.org/downloads/presentations/syslog-ng-ose-3.… and http://asylum.madhouse-project.org/downloads/presentations/syslog-ng-ose-3.… NEW RELEASES: * syslog-ng OSE 3.4.1:https://lists.balabit.hu/pipermail/syslog-ng-announce/2013-January/000154.html ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 172] New: tcp() destination driver ignores ts_format() parameter
by bugzilla＠bugzilla.balabit.com 13 Feb '13

13 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=172 Summary: tcp() destination driver ignores ts_format() parameter Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: chiestand(a)salk.edu Type of the Report: bug Estimated Hours: 0.0 The documentation (http://www.balabit.com/sites/default/files/syslog-ng-ose-v3.3-guide-admin-e…) states that you may use the ts_format() paramater to modify the timestamp format. However this parameter does not work, rfc3164 format seems to be used no matter what you specify in ts_format(). So either the documentation is wrong or there is a bug. I am running 3.3.4 on Debian Wheezy. I also had the problem running 3.1 in Debian Squeeze. my driver: destination My_Syslog { tcp("syslog.server.salk.edu" port(514) ts_format(iso) ); }; log { source(s_src); destination(My_Syslog); }; tcpdump: @.m..<hw<86>Apr 6 01:24:01 host CRON[1923]: pam_unix(cron:session): session closed for user root I emailed the mailing list on this issue, the thread can be viewed here: https://lists.balabit.hu/pipermail/syslog-ng/2012-April/018638.html -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 215] New: syslog-ng v3 - tcp() does not support no-multi-line as docs reference.
by bugzilla＠bugzilla.balabit.com 13 Feb '13

13 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=215 Summary: syslog-ng v3 - tcp() does not support no-multi-line as docs reference. Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: balabit(a)32ths.com Type of the Report: --- Estimated Hours: 0.0 So after doing some test cases, I've noticed that on no-multi-line is not stripping newlines on v3.3.7. I've not tested previous versions yet. Here is my syslog-ng config: source s_net_tcp { tcp(ip(0.0.0.0) port(514) flags(no-multi-line)); }; destination all { file("/var/log/all"); }; log { source(s_net_tcp); destination(all); flags(final); }; I've also attempted to do this with no-parse flags passed in to no avail. When using the udp() source, it works just fine. My test case: # cat blah line1 line2 line3 # cat blah | nc -r -n myhost 514 And on "myhost" Jan 2 12:25:16 192.2.0.1 line1 Jan 2 12:25:16 192.2.0.1 line2 Jan 2 12:25:16 192.2.0.1 line3 So I assume this has something to do with it: https://bugzilla.balabit.com/show_bug.cgi?id=74 Particularly about newlines being message terminators over tcp(), however, why do the docs even say tcp() supports the no-multi-line flag? Maybe there is a way to adjust the terminator if this is the case? I must admit, I was curious what syslog-ng would do if the message size exceeded the mtu of the network. With UDP, I assume the end of the packet is the end of the message so any messages large enough to span two udp packets would actually be two log entries. I assumed it would also be the case with TCP, even if it is stream based. So the potential problems I see are: 1. tcp() doesn't really support no-multi-line (bug) or 2. the docs should be updated to not say it supports no-multi-line (bug) or 3. I've overlooked the doc references or 4. I'm missing some option (terminator?) in order to get tcp() to work with no-multi-line. (env) Thanks as always, - Mike -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

Support for OpenBSD
by C. L. Martinez 12 Feb '13

12 Feb '13

Hi all, Is syslog-ng supported under openbsd (releses 5.x)?? Can I expect problems installing under OpenBSD?? Thanks.

2 2

Systemd troubles: incompatibility with journald ?
by Guillaume Rousse 08 Feb '13

08 Feb '13

Hello list. I'm unable to run syslog-ng, whatever the version (I tried 3.3.6 and 3.4.1), on a recent system using systemd (mageia cauldron 64 bits, systemd 195). As soon as I'm launching syslog-ng, systemd and systemd-journal cpu usages reach 80/95%, and systemd fill dmesg log with this kind of message: [ 5962.484250] systemd[1]: syslog-ng.service start request repeated too quickly, refusing to start I tried to disable journald, but i didn't suceed. I also found indication of using the journald socket ( /run/systemd/journal/syslog) instead of /dev/log in syslog-ng configuration, but it doesn't change much. Also, the fedora package for syslog-ng doesn't contains anything special, either in default configuration, or in systemd service unit (excepted a patch to force pid file location). So, I'm a bit lost here... There is a user bug report on this issue: https://bugs.mageia.org/show_bug.cgi?id=8944 -- BOFH excuse #223: The lines are all busy (busied out, that is -- why let them in to begin with?).

3 5

[Bug 214] New: Build / Install Issues on AIX
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=214 Summary: Build / Install Issues on AIX Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: AIX Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: blentz(a)cswg.com Type of the Report: --- Estimated Hours: 0.0 More fun for the AIX platform; Using version 3.3.7, GNU Make and GCC 4.2.0 However lib/syslog-ng/libafsocket.so is installed it is installed in such a way that the dependency for libsyslog-ng-crypto.a cannot be found. Seems like some kind of libtool problem. $ ldd /opt/local/syslogng/lib/syslog-ng/libafsocket.so /opt/local/syslogng/lib/syslog-ng/libafsocket.so needs: Cannot find libsyslog-ng-crypto.a(libsyslog-ng-crypto.so) /opt/local/syslogng/lib/libssl.so /opt/local/syslogng/lib/libcrypto.so /opt/local/syslogng/lib/libglib-2.0.a(libglib-2.0.so.0) /opt/local/syslogng/lib/libevtlog.a(libevtlog.so.0) /opt/local/syslogng/lib/libsyslog-ng.a(libsyslog-ng-3.3.7.so) /usr/lib/libc.a(shr.o) /usr/lib/librtl.a(shr.o) /opt/local/syslogng/lib/libgcc_s.a(shr.o) /usr/lib/libpthreads.a(shr_xpg5.o) /usr/lib/libiconv.a(shr4.o) /usr/lib/libpthread.a(shr_comm.o) /usr/lib/libpthread.a(shr_xpg5.o) /opt/local/syslogng/lib/libintl.a(libintl.so.8) /opt/local/syslogng/lib/libgmodule-2.0.a(libgmodule-2.0.so.0) /opt/local/syslogng/lib/libgthread-2.0.a(libgthread-2.0.so.0) /opt/local/syslogng/lib/libivykis.a(libivykis.so.0) /unix /usr/lib/libcrypt.a(shr.o) /usr/lib/libpthreads.a(shr_comm.o) /usr/lib/libthread.a(shr.o) Also, be aware that shared libraries on AIX use the .a suffix, not .so. Out of the box shared libraries are archived into .a files. $ ar tv /opt/local/syslogng/lib/syslog-ng/libafsocket.a rwxr-xr-x 0/0 572423 Dec 19 20:23 2012 libafsocket-tls.so The default "make install" process leaves nothing but *.a files in lib/syslog-ng and on startup the user is greeted with many errors about missing loadable modules. The RPM that Perzl makes gets around this by running ar -X32 -x'ing all of these .a files so the system is populated with both .a and .so versions of the files. ( cd ${RPM_BUILD_ROOT}%{_libdir} for f in *.a ; do /usr/bin/ar -X32 -x ${f} done cd ${RPM_BUILD_ROOT}%{_libdir}/%{name} for f in *.a ; do /usr/bin/ar -X32 -x ${f} done ) Can this be fixed? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 18

[Bug 121] New: After service syslog-ng reload, the first log don' t transfer to remote servicer via TLS
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=121 Summary: After service syslog-ng reload, the first log don't transfer to remote servicer via TLS Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: yong.zheng(a)nsn.com Type of the Report: bug Estimated Hours: 0.0 The version:syslog-ng 3.0.5 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq (3600); }; I will upload the logs to remote syslog-ng server via TLS, I encounter a issue, On client: after reload the configure via below command: #service syslog-ng reload. I write a log: #logger -p local1.info "this is test" but I don't see the log to transfer to server immediately, I wait many minutes, still don't see. I must write the second log to triger that client send the 2 logs to server #logger -p local1.info "this is test 2" the 2 logs have transfered to server immediately successfully, I can see the 2 logs on server. It is 100% reproduced. Is this a known issue, or not ? Thanks. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6