February 2011 - syslog-ng - lists.balabit.hu

Re: [syslog-ng] Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

This upgrade went fine with some minor issues. Thanks for all the great assistance from the folks on this list. James From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Lay, James Sent: Friday, January 28, 2011 3:26 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Upgrade gotchas? Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

[PATCH] Extend the parsing of NUMBER and FLOAT type in patterndb
by SZALAY Attila 06 Feb '11

06 Feb '11

Hi! At the weekend I tried to create patterndb rules to spamassassin message and realised that it fail in to way. First when spamassasin add negative spam score result. This negative number is not recognised. Second issue is when I use bayes. Sometimes bayes give a very small probabaility number (ie 10^-13 as the order of magnitude) In this situation the exponent is appear in the log (ie.: bayes=1.11022302462516e-16) which not handled by patterndb very good either. SZALAY Attila (1): Added two feature into NUMBER and FLOAT parsing src/radix.c | 37 ++++++++++++++++++++++++++++++------- tests/unit/test_radix.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 7 deletions(-)

2 2

Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

Impossibility to use multiple occurences of the same parser
by Guillaume Rousse 06 Feb '11

06 Feb '11

Hello list. My syslog-ng configuration is modular, so as to be easily deployed on all our hosts. There is a main configuration files, and one additional included file by service. Our LDAP and Kerberos servers are hosted on the same host (LDAP is our Kerberos backend). Main file has: parser p_db { db-parser(); }; filter f_drop { tags("dropthis"); }; destination d_drop { }; LDAP file has: log { source(s_sys); filter(f_ldap); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_ldap); flags(final); }; }; Kerberos file has: log { source(s_sys); filter(f_kerberos); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_kerberos); flags(final); }; }; This is incorrect, because parser(p_db) is used in two different log statements, and syslog-ng doesn't start with a cryptic message: Error initializing message pipeline; If I try to use the parser only once, in the main file, tagging doesn't occur. The only solution I found was to declare two different parsers, for each of the two files. As they both use a different pattern db fragment, it reduce the interest of merging them into a single pattern db file with update-patterndb. -- BOFH excuse #179: multicasts on broken packets

3 2

[fyi] ivykis reference handling
by Balazs Scheidler 06 Feb '11

06 Feb '11

Hi, Those experimenting with the 3.3 tree may have noticed that syslog-ng is using ivykis, a main loop implementation by Lennert Buytenhek. Since this library is not in wide-spread use in distributions, I've decided to embed it into the syslog-ng source code / git repository. Source code releases include a lib/ivykis directory, that is automatically compiled when syslog-ng is compiled except if --with-ivykis=system is specified. The git repository contains a submodule reference to the ivykis repo, which points to my ivykis repository on git.balabit.hu. The way it is implemented in git, is that it contains a sha1 reference to the exact imported commit. Since I don't intend to fork ivykis, this ivykis repo is rebased from time-to-time to its upstream. Rebasing causes commits to be changed (e.g. their sha1 changes), thus the submodule reference from the syslog-ng repo could break. To avoid this, I'm going to tag each and every "import" into the syslog-ng repository with a tag in the form "syslog-ng-3.3-refXXX", where XXX is a kind of version number, increased by one every time. By putting this tag in the ivykis repository, I can ensure that any import I make will still be available. I intend to do the same with libmongo-client, used by the mongodb driver. -- Bazsi

1 0

[PATCH][ivykis] configure.gnu: Quote $@, so that parameters are passed correctly.
by Gergely Nagy 05 Feb '11

05 Feb '11

When a parent configure calls ivykis' configure.gnu, some parameters might contain spaces (for example, CFLAGS="-ggdb3 -O0"), and without quoting $@, these will end up as separate options passed down to the real configure. With quoting, however, the parameters will be passed along as-is, without breaking them up. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.gnu | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure.gnu b/configure.gnu index 99dd8a7..4fb578f 100644 --- a/configure.gnu +++ b/configure.gnu @@ -4,4 +4,4 @@ echo $0 configure="`dirname $0`/`basename $0 .gnu`" echo "Running: " $configure $@ --disable-shared --enable-static --with-pic -$SHELL $configure $@ --disable-shared --enable-static --with-pic +$SHELL $configure "$@" --disable-shared --enable-static --with-pic -- 1.7.0.4

2 1

syslog-ng deadlock if /dev/console locks?
by Paul Krizak 05 Feb '11

05 Feb '11

Hi, we're using syslog-ng 3.1.2 and have run into what appears to be a bug, but I'd like to get the community's opinion before we dig further into it. We have a bunch of HP servers with iLO2 and iLO3 devices, configured with their virtual serial ports on COM1 (ttyS0). We subsequently have the OS (RHEL4, RHEL5) configured to use COM1 as its console (e.g. /dev/console). This is a very standard configuration that allows us to get remote access to the machines without having to purchase the iLO Advanced KVM feature. It also lets us use the Magic SysRq keys to probe dead systems and stuff, so in general it's not something we're keen to change. What we have found, however, is that there are some cases where the iLO will freeze and requires a reboot. When the iLO reboots, however, the kernel's connection to /dev/console (through the virtual serial port) hangs and blocks. Any traffic to /dev/console just sits in the kernel's buffer and is never delivered. Once the buffer is full, the kernel simply blocks on any write to /dev/console. Now this is a Bad Thing in general, and we're working with HP to try and remedy this bug. However, what concerns me is that syslog-ng, when faced with this behavior, also blocks, even for log messages not bound for /dev/console. What we have observed is that a system with syslog-ng will keep delivering the occasional console message to /dev/console (ex. *.emerg messages) and meanwhile the file-based log paths keep working. But once /dev/console blocks, the next time a console message is delivered, *all* of syslog-ng blocks waiting for that message to be delivered, and all of the file-based paths block as well. The result is that pretty much everything on the system stops working. For example, you can't log in, even as root, because the login process blocks on the syslog command that writes to /var/log/secure. Anything that uses syslog suddenly blocks. Is this expected behavior? I would think that syslog-ng would be able to continue accepting and delivering messages, even if one of the log paths is stalled on a blocked write. -- Paul Krizak 7171 Southwest Pkwy MS B200.3A MTS Systems Engineer Austin, TX 78735 Advanced Micro Devices Desk: (512) 602-8775 Linux/Unix Systems Engineering Cell: (512) 791-0686 Global IT Infrastructure Fax: (512) 602-0468

6 15

Heavy memory leak using correlation or patterndb
by Guillaume Rousse 05 Feb '11

05 Feb '11

Hello. Since I installed the rules dedicated to dropping LDAP probe connection traces, the memory usage of my ldap server exploded. top confirms than syslog-ng is the culprit, using 60% of the available memory. Here is a weekly munin graph presenting the issue: http://www.zarb.org/~guillomovitch/memory-week.png The rules have been installed on 19th, and I restarted syslog-ng yesterday. I'm using syslog-ng 3.2.1, on mandriva 2010.0, 64 bits. Here are the exact rules used, if that matters (I couldn't try latest Balazs suggestions yet). The context-timeout for the first rule is quite low, because the probe checks every two seconds. By any chance, doesn't the second timeout, set to 0, set an illimited session expiration time ? If not the case, I'd gladly try any needed experiment, such as running syslog-ng under valgrind, to identify the leak source. -- BOFH excuse #400: We are Microsoft. What you are experiencing is not a problem; it is an undocumented feature.

2 3

[Bug 110] New: Advancing the system date and time beyond 2038 causes flood of messages
by bugzilla＠bugzilla.balabit.com 05 Feb '11

05 Feb '11

https://bugzilla.balabit.com/show_bug.cgi?id=110 Summary: Advancing the system date and time beyond 2038 causes flood of messages Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: gaudet(a)verizon.net Type of the Report: bug Estimated Hours: 0.0 On a 64-bit Linux system (SLES 11 SP1), if the date and time are advanced beyond January 19, 2038 @ 03:14:07, syslog begins flooding the log with MARK messages. It is possible that this is due to the so-called Year 2038 or Unix Millenium bug (see http://www.deepsky.com/~merovech/2038.html) The test: - Set the syslog mark frequency to 2 seconds and restart the syslog service - Tail the log - From another window, set the date to January 19, 2038 @ 03:14:00 - Observe the mark messages are correct until the clock moves beyond 03:14:07 # Display the starting date/time plus seconds since epoch $ date ; date +%s Tue Jan 25 02:10:00 UTC 2011 1295921228 # Set the date and time to January 19, 2038 @ 03:14:00 $ date 011903142038 Tue Jan 19 03:14:00 UTC 2038 # Display the new date/time and note that seconds since epoch have advanced beyond the 32-bit unsigned integer maximum $ date ; date +%s Tue Jan 19 03:14:10 UTC 2038 2147483650 $ tail -f /var/log/messages Jan 25 02:10:00 localhost -- MARK -- Jan 25 02:10:02 localhost -- MARK -- Jan 25 02:10:04 localhost -- MARK -- Jan 25 02:10:06 localhost -- MARK -- Jan 25 02:10:08 localhost -- MARK -- Jan 25 02:10:10 localhost -- MARK -- Jan 25 02:10:12 localhost -- MARK -- Jan 25 02:10:14 localhost -- MARK -- Jan 25 02:10:16 localhost -- MARK -- Jan 25 02:10:18 localhost -- MARK -- Jan 25 02:10:20 localhost -- MARK -- Jan 19 03:14:00 localhost -- MARK -- Jan 19 03:14:00 localhost syslog-ng[4961]: Log statistics; dropped='pipe(/dev/xconsole)=0', dropped='pipe(/dev/tty10)=0', processed='center(queued)=71', processed='center(received)=65', processed='destination(messages)=65', processed='destination(mailinfo)=0', processed='destination(mailwarn)=0', processed='destination(localmessages)=0', processed='destination(newserr)=0', processed='destination(mailerr)=0', processed='destination(netmgm)=0', processed='destination(warn)=2', processed='destination(console)=2', processed='destination(null)=0', processed='destination(mail)=0', processed='destination(xconsole)=2', processed='destination(firewall)=0', processed='destination(acpid)=0', processed='destination(newscrit)=0', processed='destination(newsnotice)=0', processed='source(src)=65' Jan 19 03:14:02 localhost -- MARK -- Jan 19 03:14:04 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- In a very short period of time, there are thousands of the "Jan 19 03:14:06..." messages in the log. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

tfjson: JSON-generating template function
by Gergely Nagy 05 Feb '11

05 Feb '11

Hi! Looks like the list didn't like me sending the whole patch, so I'll have to omit the patch part this time, and hope that the wall of text below won't hit the size limit. Based on the work of Balint Kovacs <blint(a)balabit.hu>, a $(format_json) template function was implemented: with this function, one can hammer log messages into JSON format. While that is not immediately useful, we have a few things planned that will - with a bit of luck - be able to use this function. The function itself, without arguments, will reformat the log message into JSON. For example, a loggen-generated message will look like this (formatting added by me): { "HOST": "localhost", "HOST_FROM": "localhost", "MESSAGE": "seq: 0000000000, thread: 0000, runid: 1295625842, stamp: 2011-01-21T17:04:02 PADD...", "PROGRAM": "prg00000", "PID": "1234", "SOURCE": "s_tcp", "LEGACY_MSGHDR": "prg00000[1234]: ", ".classifier.class": "unknown" } While this is all cool, there's more! The function accepts parameters: '--select <GLOB>' and '--exclude <GLOB>', where <GLOB> is a shell-like glob pattern. One can have any number of selects or globs. The way this works, is that if a key doesn't match any of the select globs, it will not be included. If it does, but matches any of the exclude globs, it will be dropped. Otherwise it's included. The default select glob is "*", which is why everything is included by default. The globs are matched against the keys: macro names without the dollar sign, patterndb keys, and so on. With this knowledge, we can tweak the template like this: $(format_json --select .classifier.* --select usracct.* --exclude *.*id) Then, given the following message: <86>Jan 21 17:10:21 moria sshd[31930]: pam_unix(sshd:session): session closed for user algernon We'll end up with this JSON: { ".classifier.class": "system", "usracct.username": "algernon", "usracct.type": "logout", "usracct.application": "sshd" } Now, that's not very useful, as we're lacking the HOST and a whole lot of other things. Adding all that as selects would be a bit excessive, therefore one can list macro names (without the dollar sign) along with the select/exclude options, and they'll be included too. Furthermore, one can list key=value pairs, for example, using the log message quite like the one above, and the following template: $(format_json -s .classifier.* -s usracct.* -x *.*id HOST MESSAGE prg='$PROGRAM[$PID]') We'll end up with: { ".classifier.class": "system", "usracct.username": "algernon", "usracct.type": "logout", "usracct.application": "sshd", "MESSAGE": "pam_unix(sshd:session): session closed for user algernon", "prg": "sshd[32216]", "HOST": "localhost" } At the moment, only strings are supported, and there is no way to structure the JSON properly yet. We'll figure out a way to do that too, eventually. There's a few more things in the pipeline for tfjson, so treat this version as a beta. The code is available on the integration/tfjson/base branch of my repository at git://git.madhouse-project.org/syslog-ng/syslog-ng-3.3.git Browsable online at: http://git.madhouse-project.org/syslog-ng/syslog-ng-3.3/tree/modules/tfjson… The bulk of the work was done by Balint, I just happened to accidentally refactor it a tiny bit while reviewing his code. -- |8]

3 4