January 2011 - syslog-ng - lists.balabit.hu

using correlation to filter out some messages ?
by Guillaume Rousse 22 Feb '11

22 Feb '11

Hello. I've had a look at message correlation in the 3.2 manual, but it's still a bit obscure for me to understand if it could allow me to achieve filtering of pairs of log messages, as described here: https://lists.balabit.hu/pipermail/syslog-ng/2010-January/013857.html -- BOFH excuse #360: Your parity check is overdrawn and you're out of cache.

5 19

abandon libdbi and write a new sql driver?
by Patrick H. 22 Feb '11

22 Feb '11

Reading through the mailing list lately I've seen a fair amount of problems reported with libdbi. Given that it also doesnt support prepared statements which is a very important feature, I'm wondering if libdbi should just be dropped entirely. Libdbi seems to be of use to applications that want just very basic sql support, and not for anything involving high performance or reliability. I know its no small undertaking to write a new DBI for all the common databases out there, but if syslog-ng is supposed to be high-performing, libdbi does not fit in with that. I dont know if it'd be best to start a new project to replace libdbi but accomplish the same thing of providing a common API for all databases, or to just write separate modules for syslog-ng 3.2+ for each database out there. Writing modules would certainly be easier as they could be written independently by people who best know each database API. This is all just my opinion, but I for one am all for it. In my production environment, we do logging to a database, and libdbi just didnt have the flexibility or performance we needed, so I ended up writing a separate program which connects to the database to perform the inserts. Would be nice to avoid having to do that. I know I've also heard many recommendations of piping out to a perl script to do the inserts. A perl script should never do its job faster than C code. -Patrick

6 9

[Bug 109] New: building on Cygwin fails in modules/dbparser
by bugzilla＠bugzilla.balabit.com 22 Feb '11

22 Feb '11

https://bugzilla.balabit.com/show_bug.cgi?id=109 Summary: building on Cygwin fails in modules/dbparser Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Other Status: NEW Severity: blocker Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: vinschen(a)redhat.com Type of the Report: regression Estimated Hours: 0.0 Created an attachment (id=35) --> (https://bugzilla.balabit.com/attachment.cgi?id=35) Patch to fix build problem in modules/dbparser I just tested the new implementation from git, and gcc fails to build libsyslog-ng-patterndb.dll.a since a lot of function calls are missing: libtool: link: gcc -std=gnu99 -shared .libs/radix.o .libs/patterndb.o .libs/t imerwheel.o .libs/patternize.o -lssl -lcrypto -lz -lrt /usr/lib/libgmodule-2.0.dll.a -L/usr/lib /usr/lib/libglib-2.0.dll.a /usr/lib/libintl.dll.a /usr/lib/libiconv.dll.a /usr/lib/libevtlog.dll.a -lresolv /usr/lib/libpcre.dll.a -O2 -Wl,--export-all-symbols -o .libs/cygsyslog-ng-patterndb-0.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libsyslog-ng-patterndb.dll.a Creating library file: .libs/libsyslog-ng-patterndb.dll.a .libs/radix.o: In function `r_new_pnode': modules/dbparser/radix.c:374: undefined reference to `_log_msg_get_value_handl e' modules/dbparser/radix.c:618: undefined reference to `_msg_limit_internal_mess age' modules/dbparser/radix.c:618: undefined reference to `_log_msg_get_value_name' modules/dbparser/radix.c:618: undefined reference to `_msg_event_create' modules/dbparser/radix.c:629: undefined reference to `_msg_limit_internal_mess age' modules/dbparser/radix.c:567: undefined reference to `_msg_event_create' modules/dbparser/radix.c:674: undefined reference to `_msg_limit_internal_mess age' modules/dbparser/radix.c:674: undefined reference to `_msg_event_create' etc. The reason is that the dependency to libsyslog is missing. Would you mind to apply the attached patch? It fixes the build problem and the resulting pdbtool appears to work fine. Thanks, Corinna -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

filter/parse based on certificate attributes?
by Christian Tramnitz 22 Feb '11

22 Feb '11

With syslog 3.0 and ssl support using mutual certificate authentication is it possible to parse/filter/tag based on attributes in the certificate? I'd like to have a common listener and then relay or append messages based on client-cert attribute values... (Sorry if this message is arriving more than once, I seem to have problems with posting through Gmane.) Regards, Christian

4 4

Syslog-ng Windows Agent & WIN2008 Event Forwarding Subscription
by Szilárd Szabó 08 Feb '11

08 Feb '11

Üdv mindenkinek, Van egy kis problémám. Adott egy Windows Server 2008 melyen Event Forwarding Subscription van beállítva a következőek szerint: http://blogs.technet.com/b/wincat/archive/2008/08/11/quick-and-dirty-large-… Az hozzáadtam pár klienst. Az események megérkeznek a Windows Server 2008-ra hiba nélkül a ForwardedEvents -be. A Problémám a következő: A Windows Server 2008-ra telepítettem egy Syslog-ng Windows Agent 3.2.1 verziót, és beállítottam a log továbbítást egy Syslog-ng PE-felé. Az események megérkeznek, de felettéb érdekesen :) Jan 20 16:06:34 COMPUTER1 NT: AUTHORITY\ANONYMOUS LOGON: ForwardedEvents Security: [] (EventID 538) Jan 20 16:06:34 COMPUTER2 NT: AUTHORITY\ANONYMOUS LOGON: ForwardedEvents Security: [] (EventID 538) Jan 20 16:06:34 COMPUTER3 NT: AUTHORITY\ANONYMOUS LOGON: ForwardedEvents Security: [] (EventID 538) A Windows 2008 továbbá 64bites, tehát AD-ból van menedzselve. Ez most Agent probléma lehet, vagy a Event Forwarding Subscription-nal van a baj? Üdv Szilárd

5 11

[PATCH] Extend the parsing of NUMBER and FLOAT type in patterndb
by SZALAY Attila 06 Feb '11

06 Feb '11

Hi! At the weekend I tried to create patterndb rules to spamassassin message and realised that it fail in to way. First when spamassasin add negative spam score result. This negative number is not recognised. Second issue is when I use bayes. Sometimes bayes give a very small probabaility number (ie 10^-13 as the order of magnitude) In this situation the exponent is appear in the log (ie.: bayes=1.11022302462516e-16) which not handled by patterndb very good either. SZALAY Attila (1): Added two feature into NUMBER and FLOAT parsing src/radix.c | 37 ++++++++++++++++++++++++++++++------- tests/unit/test_radix.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 7 deletions(-)

2 2

Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

[PATCH][ivykis] configure.gnu: Quote $@, so that parameters are passed correctly.
by Gergely Nagy 05 Feb '11

05 Feb '11

When a parent configure calls ivykis' configure.gnu, some parameters might contain spaces (for example, CFLAGS="-ggdb3 -O0"), and without quoting $@, these will end up as separate options passed down to the real configure. With quoting, however, the parameters will be passed along as-is, without breaking them up. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.gnu | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure.gnu b/configure.gnu index 99dd8a7..4fb578f 100644 --- a/configure.gnu +++ b/configure.gnu @@ -4,4 +4,4 @@ echo $0 configure="`dirname $0`/`basename $0 .gnu`" echo "Running: " $configure $@ --disable-shared --enable-static --with-pic -$SHELL $configure $@ --disable-shared --enable-static --with-pic +$SHELL $configure "$@" --disable-shared --enable-static --with-pic -- 1.7.0.4

2 1

syslog-ng deadlock if /dev/console locks?
by Paul Krizak 05 Feb '11

05 Feb '11

Hi, we're using syslog-ng 3.1.2 and have run into what appears to be a bug, but I'd like to get the community's opinion before we dig further into it. We have a bunch of HP servers with iLO2 and iLO3 devices, configured with their virtual serial ports on COM1 (ttyS0). We subsequently have the OS (RHEL4, RHEL5) configured to use COM1 as its console (e.g. /dev/console). This is a very standard configuration that allows us to get remote access to the machines without having to purchase the iLO Advanced KVM feature. It also lets us use the Magic SysRq keys to probe dead systems and stuff, so in general it's not something we're keen to change. What we have found, however, is that there are some cases where the iLO will freeze and requires a reboot. When the iLO reboots, however, the kernel's connection to /dev/console (through the virtual serial port) hangs and blocks. Any traffic to /dev/console just sits in the kernel's buffer and is never delivered. Once the buffer is full, the kernel simply blocks on any write to /dev/console. Now this is a Bad Thing in general, and we're working with HP to try and remedy this bug. However, what concerns me is that syslog-ng, when faced with this behavior, also blocks, even for log messages not bound for /dev/console. What we have observed is that a system with syslog-ng will keep delivering the occasional console message to /dev/console (ex. *.emerg messages) and meanwhile the file-based log paths keep working. But once /dev/console blocks, the next time a console message is delivered, *all* of syslog-ng blocks waiting for that message to be delivered, and all of the file-based paths block as well. The result is that pretty much everything on the system stops working. For example, you can't log in, even as root, because the login process blocks on the syslog command that writes to /var/log/secure. Anything that uses syslog suddenly blocks. Is this expected behavior? I would think that syslog-ng would be able to continue accepting and delivering messages, even if one of the log paths is stalled on a blocked write. -- Paul Krizak 7171 Southwest Pkwy MS B200.3A MTS Systems Engineer Austin, TX 78735 Advanced Micro Devices Desk: (512) 602-8775 Linux/Unix Systems Engineering Cell: (512) 791-0686 Global IT Infrastructure Fax: (512) 602-0468

6 15

Heavy memory leak using correlation or patterndb
by Guillaume Rousse 05 Feb '11

05 Feb '11

Hello. Since I installed the rules dedicated to dropping LDAP probe connection traces, the memory usage of my ldap server exploded. top confirms than syslog-ng is the culprit, using 60% of the available memory. Here is a weekly munin graph presenting the issue: http://www.zarb.org/~guillomovitch/memory-week.png The rules have been installed on 19th, and I restarted syslog-ng yesterday. I'm using syslog-ng 3.2.1, on mandriva 2010.0, 64 bits. Here are the exact rules used, if that matters (I couldn't try latest Balazs suggestions yet). The context-timeout for the first rule is quite low, because the probe checks every two seconds. By any chance, doesn't the second timeout, set to 0, set an illimited session expiration time ? If not the case, I'd gladly try any needed experiment, such as running syslog-ng under valgrind, to identify the leak source. -- BOFH excuse #400: We are Microsoft. What you are experiencing is not a problem; it is an undocumented feature.

2 3