- syslog-ng - lists.balabit.hu

syslog-ng performance tuning
by Zoltán Pallagi 07 Feb '11

07 Feb '11

Hi all, I've just measured the performance of syslog-ng PE and written a blog about syslog-ng performance tuning. Perhaps, it will be useful for OSE, too: http://pzolee.blogs.balabit.com/en/2011/02/syslog-ng-performance-tuning -- pzolee

1 0

syslog-ng af_sql driver experiencing segfault
by Matthew Hall 07 Feb '11

07 Feb '11

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf7b12b90 (LWP 23608)] 0x00c5c7dc in memcpy () from /lib/libc.so.6 (gdb) bt #0 0x00c5c7dc in memcpy () from /lib/libc.so.6 #1 0xf7e21fed in g_string_insert_len () from /home/y/lib/libglib-2.0.so.0 #2 0xf7e223e8 in g_string_append_len () from /home/y/lib/libglib-2.0.so.0 #3 0xf7edc1d6 in result_append (result=0x8cc1780, sstr=0x910ffaf "", len=3160627, escape=0) at templates.c:213 #4 0xf7ede535 in log_template_append_format_with_context (self=0x8ba0098, messages=0xf7b12204, num_messages=1, opts=0x8bb8aec, tz=1, seq_num=945, result=0x8cc1780) at templates.c:1058 #5 0xf7ede83f in log_template_append_format (self=0x8ba0098, lm=0x90d93b0, opts=0x8bb8aec, tz=1, seq_num=945, result=0x8cc1780) at templates.c:1109 #6 0xf7ede898 in log_template_format (self=0x8ba0098, lm=0x90d93b0, opts=0x8bb8aec, tz=1, seq_num=945, result=0x8cc1780) at templates.c:1116 #7 0xf7b459e6 in afsql_dd_insert_db (self=0x8bb8a58) at afsql.c:681 #8 0xf7b45fd9 in afsql_dd_database_thread (arg=0x8bb8a58) at afsql.c:823 #9 0xf7ed1405 in worker_thread_func (st=0x8ba4238) at misc.c:593 #10 0xf7e28c51 in ?? () from /home/y/lib/libglib-2.0.so.0 #11 0x08ba4238 in ?? () #12 0x08ba4248 in ?? () #13 0x00000005 in ?? () #14 0xf7e28b06 in ?? () from /home/y/lib/libglib-2.0.so.0 #15 0x00d352c6 in ?? () from /lib/libpthread.so.0 #16 0x00000000 in ?? () (gdb) Looks like I am running into problems with the database code. I'll investigate but it's going to take a while so I hope somebody else might have an idea. This is 3.2 verion 2010-01-31 git. Thanks, Matthew.

2 2

How to stop repeated messages in Syslog-ng?
by Wagner Pereira 06 Feb '11

06 Feb '11

Hi, folks. I am using Syslog-ng as a Cacti's plugin. There are some messages which had been logged since last months. I'd like to ignore these specific messages, I mean, stop to login them. I tried to turn ON two variables in the php.ini... ignore_repeated_source = On ignore_repeated_errors = On ...but unsuccesfully (at least, I changed them and didn't restart any service...is it required?) Is that possible to stop them? Att, -- Wagner Pereira PoP-SP/RNP - Ponto de Presença da RNP em São Paulo CCE/USP - Centro de Computação Eletrônica da Universidade de São Paulo http://www.pop-sp.rnp.br Tel. +55 11 3091-8901 fone@RNP: 10158902

3 2

Testing structured data syslog messages
by Lance Laursen 06 Feb '11

06 Feb '11

Hello list, I am currently trying to test structured data log messages with syslog-ng. I'm running syslog-ng-3.1.2-1.rhel5. Setup is very simple, with a udp source and a destination to /var/log/messages. Here's the line i'm trying to use to test with, which according to the RFC, should be fine: echo -n '<165>1 2011-02-03T22:14:15.003Z superhostomg process - ID47 [exampleSDID@32473 iut="9" eventSource="rawr" eventID="69"] Message portion. Test log with structured data.' | nc -w 1 -u localhost 514 Unfortunately it is showing up in logs as this: 2011-02-03T22:45:30+00:00 localhost 1 2011-02-03T22:14:15.003Z superhostomg process - ID47 [exampleSDID@32473 iut="9" eventSource="rawr" eventID="69"] Message portion. Test log with structured data. So all of the metadata is being printed to $MSG. What am I doing wrong? Thanks,

4 4

Enhancement Request For program(...) Destination
by Matthew Hall 06 Feb '11

06 Feb '11

Could the program dest driver be enhanced to allow specifying the working directory? This would simplify the process of making movable software.

2 1

[PATCH] support for older PCRE
by Matthew Hall 06 Feb '11

06 Feb '11

Can we get this one added for older PCRE support? Bazsi made the old version, I tested and I updated it to apply cleanly on 3.2 master. Signed-off-by: Matthew Hall <mhall(a)mhcomputing.net> --- diff --git a/configure.in b/configure.in index 0c6cef2..f574f48 100644 --- a/configure.in +++ b/configure.in @@ -25,6 +25,8 @@ EVTLOG_MIN_VERSION="0.2" OPENSSL_MIN_VERSION="0.9.8" LIBDBI_MIN_VERSION="0.8.0" PCRE_MIN_VERSION="7.3" +dnl mhall allow older pcre +PCRE_MIN_VERSION="6.1" dnl *************************************************************************** dnl Initial setup diff --git a/lib/logmatcher.c b/lib/logmatcher.c index f0a822c..d003c58 100644 --- a/lib/logmatcher.c +++ b/lib/logmatcher.c @@ -520,8 +520,13 @@ log_matcher_pcre_re_compile(LogMatcher *s, const gchar *re) if (self->super.flags & LMF_ICASE) flags |= PCRE_CASELESS; + +/* mhall allow older pcre */ +#ifdef PCRE_NEWLINE_ANYCRLF if (self->super.flags & LMF_NEWLINE) flags |= PCRE_NEWLINE_ANYCRLF; +#endif + if (self->super.flags & LMF_UTF8) { gint support;

2 1

Re: [syslog-ng] Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

This upgrade went fine with some minor issues. Thanks for all the great assistance from the folks on this list. James From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Lay, James Sent: Friday, January 28, 2011 3:26 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Upgrade gotchas? Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

[PATCH] Extend the parsing of NUMBER and FLOAT type in patterndb
by SZALAY Attila 06 Feb '11

06 Feb '11

Hi! At the weekend I tried to create patterndb rules to spamassassin message and realised that it fail in to way. First when spamassasin add negative spam score result. This negative number is not recognised. Second issue is when I use bayes. Sometimes bayes give a very small probabaility number (ie 10^-13 as the order of magnitude) In this situation the exponent is appear in the log (ie.: bayes=1.11022302462516e-16) which not handled by patterndb very good either. SZALAY Attila (1): Added two feature into NUMBER and FLOAT parsing src/radix.c | 37 ++++++++++++++++++++++++++++++------- tests/unit/test_radix.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 7 deletions(-)

2 2

Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

Impossibility to use multiple occurences of the same parser
by Guillaume Rousse 06 Feb '11

06 Feb '11

Hello list. My syslog-ng configuration is modular, so as to be easily deployed on all our hosts. There is a main configuration files, and one additional included file by service. Our LDAP and Kerberos servers are hosted on the same host (LDAP is our Kerberos backend). Main file has: parser p_db { db-parser(); }; filter f_drop { tags("dropthis"); }; destination d_drop { }; LDAP file has: log { source(s_sys); filter(f_ldap); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_ldap); flags(final); }; }; Kerberos file has: log { source(s_sys); filter(f_kerberos); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_kerberos); flags(final); }; }; This is incorrect, because parser(p_db) is used in two different log statements, and syslog-ng doesn't start with a cryptic message: Error initializing message pipeline; If I try to use the parser only once, in the main file, tagging doesn't occur. The only solution I found was to declare two different parsers, for each of the two files. As they both use a different pattern db fragment, it reduce the interest of merging them into a single pattern db file with update-patterndb. -- BOFH excuse #179: multicasts on broken packets

3 2