November 2011 - syslog-ng - lists.balabit.hu

[PATCH (3.3)] value-pairs: Fix a small memory leak in the commandline parser.
by Gergely Nagy 12 Nov '11

12 Nov '11

Whenever we parse a command-line, we construct a new argv array, to prepend a zeroth element for glibs option parser. We correctly free'd it on the error path, but never on the success path. This lead to a small memory leak every time value_pairs_new_from_cmdline() was called (that is, at the moment, each $(format-json) template function leaked a little bit). This patch fixes this issue by modifying the error handling a little: instead of freeing everything and returning NULL, we only free the value-pairs structure, and set it to NULL. The rest of the freeing is left up to the common parts afterwards, which now correctly frees the constructed argv array too. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/value-pairs.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/value-pairs.c b/lib/value-pairs.c index 803e14c..b5a95af 100644 --- a/lib/value-pairs.c +++ b/lib/value-pairs.c @@ -473,11 +473,10 @@ value_pairs_new_from_cmdline (GlobalConfig *cfg, if (!g_option_context_parse (ctx, &argc, &argv, error)) { value_pairs_free (vp); - g_option_context_free (ctx); - g_free (argv); - return NULL; + vp = NULL; } g_option_context_free (ctx); + g_free (argv); return vp; } -- 1.7.7.2

1 0

Loganalyzer
by Peter Czanik 11 Nov '11

11 Nov '11

Hello, Recently I found many e-mail and forum posts looking for a web GUI for syslog-ng. One of them, Loganalyzer is getting more popular, as a completely free web application. There is just one catch, it was not designed for syslog-ng originally. So I did some research and tests, and using my instructions quite easy to get started with it. Of course Loganalyzer has some rough edges, but it works for me on a number of different systems. For a step by step tutorial to get started, read my blog at http://czanik.blogs.balabit.com/2011/02/loganalyzer/ Have a nice weekend! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 3

Syslog_ng 3.0.7 issues running in Central server mode on AIX 7.1
by Rakesh Rajasekharan 11 Nov '11

11 Nov '11

Hi, I am facing some challenges configuring the syslog_ng( 3.0.7) for central logging on AIX. The subsystem becomes in-operative while trying to set it up for central logging. But otherwise while tryng to log the messages on the same machine it runs fine... Below are the logs 1. Logging on the same host. # /opt/soe/local/syslog_ng/etc/init.d/syslog-ng.d restart 0513-044 The syslog_ng Subsystem was requested to stop. 0513-059 The syslog_ng Subsystem has been started. Subsystem PID is 9043968. # ^C # lssrc -a |grep -i syslog syslog_ng local 9043968 active syslogd ras inoperative Now ,after I change the conf file to the central server one... # cp syslog-ng.conf.server-sample syslog-ng.conf # /opt/soe/local/syslog_ng/etc/init.d/syslog-ng.d restart 0513-044 The syslog_ng Subsystem was requested to stop. 0513-059 The syslog_ng Subsystem has been started. Subsystem PID is 3342368. # lssrc -a |grep -i syslog syslogd ras inoperative syslog_ng local inoperative Below is are the contents for myu conf file whiel trying to set up for central logging. # cat syslog-ng.conf.server-sample @version: 3.0 source s_remote { tcp(); }; destination d_clients { file("/var/adm/messages-ng.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; source s_local { internal();unix-dgram("/dev/log"); }; destination d_local { file("/var/log/syslog-ng.log" owner("root") group("adm") perm(0640)); }; log { source(s_local); destination(d_local); }; destination d_messages { file("/var/log/messages-ng" owner("root") group("adm") perm(0640)); }; log { source(s_local); destination(d_messages); }; Please let me know if somehting in he conf file does not look fine and might be causing the susbsytem to turn inoperative. Because If i revert back to my old config file the subsystem again becomes active. Regards, Rakesh

1 1

syslog-ng Insider - November 2011
by Peter Czanik 11 Nov '11

11 Nov '11

Dear syslog-ng users, This is the 8th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS syslog-ng 3.3.2 is about to be released! ---------------------------------------- A new version of syslog-ng is about to be released! There are no new features to announce, but all problems reported since 3.3.1 should be fixed by now! To make it the best syslog-ng ever, please test it to make sure, that all your problems are fixed. Sources are available in git or as a snapshot: * git://git.balabit.hu/bazsi/syslog-ng-3.3 * http://packages.madhouse-project.org/syslog-ng/3.3/3.3.2/syslog-ng-3.3.2-HE… Binary packages are available are available for several Linux distributions: * openSUSE: http://download.opensuse.org/repositories/home:/czanik:/syslog-ng33/ * Debian and Ubuntu: http://asylum.madhouse-project.org/projects/debian/ syslog-ng and CEE ----------------- The latest syslog-ng release, version 3.3 can be used to implement part of the “CEE over syslog” standard. BalaBit’s patterndb <http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…> technology was able to extract information from syslog messages already for a long time. With this release JSON output was added, meaning the extracted information can be output as JSON data. What it means in practice, that syslog-ng is able to parse log messages, and output the extracted fields in the form required by CEE. To see, how it works, check http://czanik.blogs.balabit.com/2011/10/cee-and-syslog-ng/ Development of syslog-ng 3.4 started ------------------------------------ While 3.3 was just released, development of 3.4 is already started. The first version of a JSON parser is already merged ( https://github.com/bazsi/syslog-ng-3.4/commit/e5569687bba2551c89a78faee55bc… ). There are some pending fixes and enhancements, which add boolean, array and nested JSON parsing ( https://github.com/algernon/syslog-ng/commits/feature/3.4/json/parser ). Value-pairs key rewrite is work in progress ( https://github.com/algernon/syslog-ng/commits/feature/3.4/value-pairs/rekey ) and nested JSON output is also planned. The above features among others help us to better support CEE. With key rewriting we could use a “.cee.” prefix in CEE related patterns and rewrite it later. It also makes parsing of messages possible. All the current code is available for testing in Algernon's 3.4 sandbox project: https://github.com/algernon/syslog-ng/tree/sandbox/3.4 To download it, use git: $ git clone -b sandbox/3.4 git@github.com:algernon/syslog-ng <mailto:git@github.com:algernon/syslog-ng> OTHER SHORT NEWS * An interesting article about centralized syslog server in Linux Journal: http://www.linuxjournal.com/content/creating-centralized-syslog-server NEW RELEASES * syslog-ng OSE 3.2.5: http://www.balabit.com/downloads/files/syslog-ng/sources/3.2.5 WHITE PAPERS A longer paper about the “Future of logging tools”, which also provides some background information about HSRL, as used in syslog-ng: http://andrea.blogs.balabit.com/files/2011/10/HSRL_backgrounder_english_fin… ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

3.3.x conditional rewrite bug
by Martin Holste 11 Nov '11

11 Nov '11

The following will not work in the 3.3.x branch: filter f_rewrite_cisco_program { match('^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}): ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); }; It will process a few lines and then hang indefinitely. It will process ok if the condition is taken off of the filter, so I am assuming this is a PCRE issue, though I found it made no difference to remove the PCRE requirement and use a simpler matching expression. All works perfectly in 3.2.4.

3 4

Quick filter question
by Lay, James 10 Nov '11

10 Nov '11

Hey all! Real quick...trying to filter OUT firewall hits that have say...169.254. Will this do the trick? filter f_firewall { not program (firewall flags(ignore-case)); and not message("169\.254\.[0-9]+\.[0-9]+"); }; Thanks all. James

4 10

Syslog-ng under cygwin.
by andrea.meconi＠libero.it 10 Nov '11

10 Nov '11

Hello. Does anyone know the step by step to compile syslog-ng under Cygwin? I'm running Cygwin on Windows 2008 R2 64 bit and I need option spoof-source enabled. I'm trying this guide, http://mathieu.carbou.free.fr/wiki/index.php?title=How_to_compile_Libnet_un… without success. Thanks. Regards. Andrea

1 0

[PULL REQ]: scratch-buffers
by Gergely Nagy 09 Nov '11

09 Nov '11

I've added the finishing touches to the scratch buffer patch, and I believe it's ready for a merge. Scratch buffers are a way to store reusable GStrings on a per-thread basis. They're allocated on demand, and once released, they're not free'd, but returned to the pool, so the next time the code requests a scratch buffer, it will reuse an existing one, instead of creating a new one. This reduces memory fragmentation, memory use, among other things. It's also very trivial. I have prepared a branch for both syslog-ng 3.3 and 3.4 too. The 3.3 branch also updates the value-pairs implementation to use scratch buffers (and thus, become thread-safe). The 3.4 branch does this too, and it also updates the json parser similarly. The branches are available as feature/3.3/scratch-buffers[0] and feature/3.4/scratch-buffers[1] in my github repo[2]. [0]: https://github.com/algernon/syslog-ng/commits/feature/3.3/scratch-buffers [1]: https://github.com/algernon/syslog-ng/commits/feature/3.4/scratch-buffers [2]: git://github.com/algernon/syslog-ng.git I'm not posting the patches here, as they'd be pretty much the same, and I've already posted a previous version (and the difference is about 3 lines). Though, if it's easier to review if I post it again, let me know, it's a git send-email away. -- |8]

2 1

Re: [syslog-ng] Quick filter question
by Lay, James 08 Nov '11

08 Nov '11

Excellent...thanks so much Patrick. This is working nicely. At some point in time I may want to add in the firewalls "Deny" statement, but for now this will work. Thanks again. James From: Patrick H. [mailto:syslogng@feystorm.net] Sent: Tuesday, November 08, 2011 11:57 AM To: Lay, James Cc: Syslog-ng users' and developers' mailing list Subject: [Spam] Re: [syslog-ng] Quick filter question Importance: Low That might work, but what the order of operations is on boolean operations (and/or) is, is unclear. In situations like that its always best to explicitly force the order yourself. filter f_firewall { not ( program ("firewall" flags(ignore-case)) and ( message("192\.168\.") or message("169\.254\.") ) ); }; Sent: Tue Nov 08 2011 11:46:35 GMT-0700 (MST) From: Lay, James <james.lay(a)wincofoods.com> <mailto:james.lay@wincofoods.com> To: Patrick H. <syslogng(a)feystorm.net> <mailto:syslogng@feystorm.net> , Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> <mailto:syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Quick filter question Ah...so I need or then yes? filter f_firewall { not ( program ("firewall" flags(ignore-case)) and message("192\.168\.") or message("169\.254\.") ); }; How's that look? James P.S. And thank you :) James From: Patrick H. [mailto:syslogng@feystorm.net] Sent: Tuesday, November 08, 2011 11:40 AM To: Syslog-ng users' and developers' mailing list Cc: Lay, James Subject: Re: [syslog-ng] Quick filter question I'm not sure if semicolons are valid in filter rules, but technically valid or not, they shouldnt be there so try removing them. The filter should look like filter f_firewall { not ( program ("firewall" flags(ignore-case)) and message("192\.168\.") and message("169\.254\.") ); }; Note though, that filter will only trigger if both 192.168. and 169.254. are in the same log entry. Unless that IP address you masked out with "x"s is 169.254 it wont trigger. Sent: Tue Nov 08 2011 11:21:11 GMT-0700 (MST) From: Lay, James <james.lay(a)wincofoods.com> <mailto:james.lay@wincofoods.com> To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> <mailto:syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Quick filter question Hey again all. So...I'm still having issue with this..not sure why. Here's the raw log: Nov 8 11:13:38 x.x.x.x firewall: Deny tcp 20 125 x.x.x.x 192.168.0.15 9517 17777 offset 7 S 3371425811 win 64 And from my syslog-ng.conf filter f_firewall { not ( program ("firewall" flags(ignore-case)); and message("192\.168\."); and message("169\.254\."); ) }; log { source(s_local); filter(f_dumb); filter(f_firewall); destination(d_file); destination(other); }; Any hints as to why these aren't matching? Should I not be \ing the periods? Thanks all. James From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Frank Collette Sent: Tuesday, November 08, 2011 8:36 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Quick filter question filter f_firewall { not ( program("firewall" flags(ignore-case)) and message("169\.254\.[0-9]+\.[0-9]+" value("MESSAGE")); ) }; Thanks, Frank E. Collette IV Technical Services Systems Administrator II Trustmark National Bank Office: 601-208-7517 Fax: 601-208-6105 fcollette(a)trustmark.com From: "Lay, James" <james.lay(a)wincofoods.com> <mailto:james.lay@wincofoods.com> To: <syslog-ng(a)lists.balabit.hu> <mailto:syslog-ng@lists.balabit.hu> Date: 11/08/2011 09:14 AM Subject: [syslog-ng] Quick filter question Sent by: syslog-ng-bounces(a)lists.balabit.hu ________________________________________ Hey all! Real quick...trying to filter OUT firewall hits that have say...169.254. Will this do the trick? filter f_firewall { not program (firewall flags(ignore-case)); and not message("169\.254\.[0-9]+\.[0-9]+"); }; Thanks all. James___________________________________________________________________ ___________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Documentation errors
by VÁMOS Balázs 08 Nov '11

08 Nov '11

Hi, I have found some mistakes on the documentation page. 1. This link does not work: http://www.balabit.com/support/documentation/syslog-ng-windows-agent-v3.1-g… 2. This link is for version 3.1 but the header says it is 3.2 http://www.balabit.com/sites/default/files/documents/syslog-ng-windows-agen… -- VÁMOS Balázs LOGalyze.com

2 2