November 2010 - syslog-ng - lists.balabit.hu

syslog-ng 3.1.3 has been released
by devel＠balabit.hu 30 Nov '10

30 Nov '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.1.3 SUMMARY : new stable release DATE : Nov 26, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.1.3) has been released. For latest fixes in the 3.1.x feature branch you are recommended to upgrade to this version. CHANGES: 3.1.3 Thu, 18 Nov 2010 07:50:42 +0100 Bugfixes: * csv-parser() defaults to "escape-none" mode even if it is not specified. * Fixed a possible CPU spinning in case mark_freq() is changed from a non-zero value to zero and syslog-ng is reloaded. * Fixed a flow control problem for the internal() source possibly causing an assertion to fail, e.g. syslog-ng to crash voluntarily. * Make it possible to use a literal '$' character by using two of them. * Make it possible to use binary characters within strings in configuration files by using a C-like \xHH, \oOOO syntax. * Attempt to flush the messages sitting in output queues a bit harder at shutdown. * Handle ESTALE errors in the file source for NFS file systems, which might cause a spin on the CPU otherwise. * Fixed a possible use-after-free problem potentially causing crashes or misbehaviour. No concrete errors were reported though. * The SQL driver didn't honour the port() option, this was fixed. * Fixed program_override() option, previously it overrode the hostname instead because of a typo. * Accept 3.1 as configuration version, previously it only accepted 3.0. * Fixed internal() message loop detection in case an error happened on the log writer flush path. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

3 3

chained hostnames format broken in v3.1.2?
by Paul Krizak 30 Nov '10

30 Nov '10

Hi, all. Hopefully this is a quick and easy one -- I am working on upgrading our grid from syslog-ng 2.1.4 to 3.1.2. But it looks like chained hostnames are broken now. Here's what a typical message looked like in 2.1.4: Nov 19 12:24:58 s_internal@svvspd02 syslog-ng[4083]: syslog-ng starting up; version='2.1.4' And in v3, it now looks like this: Nov 22 09:17:39 s_self@svvspd02@F<D5> ^G syslog-ng[2527]: syslog-ng starting up; version='3.1.2' Notice the stray characters after the hostname (this is pasted from 'less' so the nonprintable chars have been expanded). This looks suspiciously like a strcpy() gone awry -- like a NULL didn't get tacked onto the end of the hostname when the chained hostname string was built up. Is this a known bug? If so, is there a patch posted in git that I can pull down and rebuild? Thanks! -- Paul Krizak 7171 Southwest Pkwy MS B200.3A MTS Systems Engineer Austin, TX 78735 Advanced Micro Devices Desk: (512) 602-8775 Linux/Unix Systems Engineering Cell: (512) 791-0686 Global IT Infrastructure Fax: (512) 602-0468

3 7

Bazsi's blog: syslog-ng 3.2.1 released
by Balazs Scheidler 30 Nov '10

30 Nov '10

syslog-ng 3.2.1 released Hi, After 2 alpha and a beta release I’ve decided to declare that syslog-ng OSE 3.2 is now stable, and thus I’ve released 3.2.1, the first in the 3.2.x series. This version has the largest list of features even since the syslog-ng project was born, so make sure you check out all the goodies. The key features were already covered on this blog in earlier posts, the most important ones in my opinion are: * licensing change to LGPL/GPL combo, without the need to sign a contributory agreement * plugins * log message correllation support in db-parser * automatic pattern generation for existing log files with pdbtool patternize * template functions Read the changelog and/or the documentation for more information. The diffstat between 3.1 and 3.2 are quite big: 407 files changed, 50305 insertions(+), 36487 deletions(-) Considering that OSE 3.2 has 70k lines (more specifically 70163) in its git repository, this is quite a lot. Source code: https://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.2.1… The list of changes are summarised in the changelog here: https://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.2.1… The updated documentation is also available here: https://www.balabit.com/support/documentation/documents/syslog-ng-ose-v3.2-… Happy logging!

2 2

unable to setup syslog-ng for log monitoring
by Supratik Goswami 30 Nov '10

30 Nov '10

Hi I am using syslog-ng 3.0 and I am trying to monitor log files. If there is word "attackalert" found in the log file syslog-ng should invoke a script. I created "/root/workspace/test" a bash script only with a simple echo statement to check if it is working correctly or not. The problem is whenever I start syslog-ng the script starts executing even if the word "attackalert" is not recorded. The following is my syslog-ng.conf configuration. Is there anything I am missing here ? Please advice. options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_file { file("/root/workspace/app_desktop/sup.log.php");}; destination d_prog { program("/root/workspace/test"); }; filter f_attack_alert { match("attackalert" value("MESSAGE")); }; log { source(s_file); filter(f_attack_alert); destination(d_prog); }; -- Warm Regards Supratik

3 3

configuring a redis syslog-ng destination
by Jacob Vorreuter 27 Nov '10

27 Nov '10

I'd like to configure syslog-ng to send logs to redis via a unix socket. This would require implementing the redis protocol as a template or rewrite rule. The only trick is getting the size in bytes of the message being processed. Is there a macro for this or some other way of determining message size? The protocol would look like this (pretending there existed a macro, MSG_SIZE): *3 (number of arguments) $5 (size in bytes of argument 1) RPUSH (operation) $4 (size in bytes of argument 2) logs (key) $${MSG_SIZE} (size in bytes of argument 3) $MSG (value) or as a string: *3\r\n$5\r\nRPUSH\r\n$4\r\nlogs\r\n$${MSG_SIZE}\r\n$MSG\r\n and finally as a destination with template: destination d_redis { unix-stream("/tmp/redis.sock") template("*3\r\n$5\r\nRPUSH\r\n$4\r\nlogs\r\n$${MSG_SIZE}\r\n$MSG\r\n"); }; Does anyone have suggestions of how to accomplish this? Thanks, Jake

3 2

please help spread the word about syslog-ng
by Balazs Scheidler 26 Nov '10

26 Nov '10

Dear syslog-ng users, You probably know that the motivation behind free software development is in most cases the joy of seeing the use of our work by others. The feedback you can give us and all the information you share about your affair with syslog-ng. I'm watching google alerts, reading blog posts where syslog-ng is mentioned, because that's the fuel that keeps me going. You probably know that running a commercial company and still keeping an open source project alive has a lot of conflicts. It is a tricky business to balance the needs of the community and the interests of an enterprise. We may have made our mistakes in the past but we are really trying to learn. (see my blog posts [1] and LWN articles [2] on the topic). One thing is sure: syslog-ng wouldn't be the thing it is today without your support, ideas and your _use_ of syslog-ng. In this spirit, I'd like to ask your involvement: 1) provide feedback, either positive/negative, on this list, or on my blog or even in private email. 2) please help spread the word on syslog-ng, tell the world how and why you are using it. If I get asked what my profession is, I always get into trouble. Some times I'm a developer, sometimes I'm a manager here at BalaBit, other times I'm just a system administrator. But I'm not really good at words and I definitely have less than enough time to advocate syslog-ng. So please help me in this regard. The easiest way is to put the "I love syslog-ng" badge on your blog or website. [3] Somewhat more time is needed to produce a blog post or a FAQ entry. If you ever spared a cent on using syslog-ng instead of something else, please contribute back. Thanks. It is appreciated. [1] http://bazsi.blogs.balabit.com/2010/08/syslog_ng_and_open_core_why_we_think… [2] http://lwn.net/Articles/402298/ [3] http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… -- Bazsi

1 0

3.2 beta 1 configure script help typo
by Matthew Hall 26 Nov '10

26 Nov '10

The following text appears in the ./configure --help output: --with-pidfile-dir=path Use path as the directory for storing pidfiles --with-module-dir=path Use path as the directory for storing pidfiles Having both of these seems either redundant, or incorrect. Or they are different but it's not clearly explained. Either way, probably worth fixing before final 3.2 is released. Matthew.

2 1

syslog-ng 3.0.9 has been released
by devel＠balabit.hu 26 Nov '10

26 Nov '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.9 SUMMARY : new stable release DATE : Nov 26, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.9) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.9 Thu, 18 Nov 2010 06:43:49 +0100 Bugfixes: * csv-parser() defaults to "escape-none" mode even if it is not specified. * Fixed a possible CPU spinning in case mark_freq() is changed from a non-zero value to zero and syslog-ng is reloaded. * Fixed a flow control problem for the internal() source possibly causing an assertion to fail, e.g. syslog-ng to crash voluntarily. * Make it possible to use a literal '$' character by using two of them. * Make it possible to use binary characters within strings in configuration files by using a C-like \xHH, \oOOO syntax. * Attempt to flush the messages sitting in output queues a bit harder at shutdown. * Handle ESTALE errors in the file source for NFS file systems, which might cause a spin on the CPU otherwise. * Fixed a possible use-after-free problem potentially causing crashes or misbehaviour. No concrete errors were reported though. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

non syslog files
by maillists0＠gmail.com 22 Nov '10

22 Nov '10

I have some homegrown application logs that are not in syslog format. I want to have syslog-ng tail those files and forward them to a syslog server (well, really a Splunk server.) In this case, does syslog-ng care what format the file is in? If so, is there a way around it?

2 1

use_fqdn()
by Brian A. Seklecki 22 Nov '10

22 Nov '10

Quick question: How does use_fqdn() interact with the various HOST, HOST_FROM, FULLHOST, FULLHOST_FROM? My understanding is that $FULLHOST and $FULLHOST_FROM are the only ones that will include the FQDN. Does use_fqdn() override that? Is the ideal config to have use_fqdn(yes) + use_dns(yes) + keep_hostname(no) and set $FULLHOST_FROM or $FULLHOST everywhere you want a FQDN so long as DNS is there? That doesnt seem to be working for me, use_fqdn(no) just kills all FQDNs everywhere regardless of $HOST* variable. I'm running the latest 2.x because I suck so much; maybe this is fixed in 3.x Thanks, ~BAS

2 3