- syslog-ng - lists.balabit.hu

[PATCH] support for older PCRE
by Matthew Hall 06 Feb '11

06 Feb '11

Can we get this one added for older PCRE support? Bazsi made the old version, I tested and I updated it to apply cleanly on 3.2 master. Signed-off-by: Matthew Hall <mhall(a)mhcomputing.net> --- diff --git a/configure.in b/configure.in index 0c6cef2..f574f48 100644 --- a/configure.in +++ b/configure.in @@ -25,6 +25,8 @@ EVTLOG_MIN_VERSION="0.2" OPENSSL_MIN_VERSION="0.9.8" LIBDBI_MIN_VERSION="0.8.0" PCRE_MIN_VERSION="7.3" +dnl mhall allow older pcre +PCRE_MIN_VERSION="6.1" dnl *************************************************************************** dnl Initial setup diff --git a/lib/logmatcher.c b/lib/logmatcher.c index f0a822c..d003c58 100644 --- a/lib/logmatcher.c +++ b/lib/logmatcher.c @@ -520,8 +520,13 @@ log_matcher_pcre_re_compile(LogMatcher *s, const gchar *re) if (self->super.flags & LMF_ICASE) flags |= PCRE_CASELESS; + +/* mhall allow older pcre */ +#ifdef PCRE_NEWLINE_ANYCRLF if (self->super.flags & LMF_NEWLINE) flags |= PCRE_NEWLINE_ANYCRLF; +#endif + if (self->super.flags & LMF_UTF8) { gint support;

2 1

Re: [syslog-ng] Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

This upgrade went fine with some minor issues. Thanks for all the great assistance from the folks on this list. James From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Lay, James Sent: Friday, January 28, 2011 3:26 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Upgrade gotchas? Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

[PATCH] Extend the parsing of NUMBER and FLOAT type in patterndb
by SZALAY Attila 06 Feb '11

06 Feb '11

Hi! At the weekend I tried to create patterndb rules to spamassassin message and realised that it fail in to way. First when spamassasin add negative spam score result. This negative number is not recognised. Second issue is when I use bayes. Sometimes bayes give a very small probabaility number (ie 10^-13 as the order of magnitude) In this situation the exponent is appear in the log (ie.: bayes=1.11022302462516e-16) which not handled by patterndb very good either. SZALAY Attila (1): Added two feature into NUMBER and FLOAT parsing src/radix.c | 37 ++++++++++++++++++++++++++++++------- tests/unit/test_radix.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 7 deletions(-)

2 2

Upgrade gotchas?
by Lay, James 06 Feb '11

06 Feb '11

Hey All! I'm about to do an upgrade from 3.0.1, to 3.2.2. Any surprises or gotchas I should be aware of? I've done some searching online and didn't see too much in the way of what I needed to do, if anything, besides ./configure make and make install of eventlog and syslog-ng. My config below is what I am workin with: @version:3.0 options { use_dns(no); }; source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(50000)); file("/proc/kmsg"); }; destination d_file { file("/syslogs/messages"); }; log { source(s_local); destination(d_file); }; Anything besides changing the version number? Thanks all. James

2 1

Impossibility to use multiple occurences of the same parser
by Guillaume Rousse 06 Feb '11

06 Feb '11

Hello list. My syslog-ng configuration is modular, so as to be easily deployed on all our hosts. There is a main configuration files, and one additional included file by service. Our LDAP and Kerberos servers are hosted on the same host (LDAP is our Kerberos backend). Main file has: parser p_db { db-parser(); }; filter f_drop { tags("dropthis"); }; destination d_drop { }; LDAP file has: log { source(s_sys); filter(f_ldap); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_ldap); flags(final); }; }; Kerberos file has: log { source(s_sys); filter(f_kerberos); parser(p_db); log { filter(f_drop); destination(d_drop); flags(final); }; log { destination(d_kerberos); flags(final); }; }; This is incorrect, because parser(p_db) is used in two different log statements, and syslog-ng doesn't start with a cryptic message: Error initializing message pipeline; If I try to use the parser only once, in the main file, tagging doesn't occur. The only solution I found was to declare two different parsers, for each of the two files. As they both use a different pattern db fragment, it reduce the interest of merging them into a single pattern db file with update-patterndb. -- BOFH excuse #179: multicasts on broken packets

3 2

[fyi] ivykis reference handling
by Balazs Scheidler 06 Feb '11

06 Feb '11

Hi, Those experimenting with the 3.3 tree may have noticed that syslog-ng is using ivykis, a main loop implementation by Lennert Buytenhek. Since this library is not in wide-spread use in distributions, I've decided to embed it into the syslog-ng source code / git repository. Source code releases include a lib/ivykis directory, that is automatically compiled when syslog-ng is compiled except if --with-ivykis=system is specified. The git repository contains a submodule reference to the ivykis repo, which points to my ivykis repository on git.balabit.hu. The way it is implemented in git, is that it contains a sha1 reference to the exact imported commit. Since I don't intend to fork ivykis, this ivykis repo is rebased from time-to-time to its upstream. Rebasing causes commits to be changed (e.g. their sha1 changes), thus the submodule reference from the syslog-ng repo could break. To avoid this, I'm going to tag each and every "import" into the syslog-ng repository with a tag in the form "syslog-ng-3.3-refXXX", where XXX is a kind of version number, increased by one every time. By putting this tag in the ivykis repository, I can ensure that any import I make will still be available. I intend to do the same with libmongo-client, used by the mongodb driver. -- Bazsi

1 0

[PATCH][ivykis] configure.gnu: Quote $@, so that parameters are passed correctly.
by Gergely Nagy 05 Feb '11

05 Feb '11

When a parent configure calls ivykis' configure.gnu, some parameters might contain spaces (for example, CFLAGS="-ggdb3 -O0"), and without quoting $@, these will end up as separate options passed down to the real configure. With quoting, however, the parameters will be passed along as-is, without breaking them up. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.gnu | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure.gnu b/configure.gnu index 99dd8a7..4fb578f 100644 --- a/configure.gnu +++ b/configure.gnu @@ -4,4 +4,4 @@ echo $0 configure="`dirname $0`/`basename $0 .gnu`" echo "Running: " $configure $@ --disable-shared --enable-static --with-pic -$SHELL $configure $@ --disable-shared --enable-static --with-pic +$SHELL $configure "$@" --disable-shared --enable-static --with-pic -- 1.7.0.4

2 1

syslog-ng deadlock if /dev/console locks?
by Paul Krizak 05 Feb '11

05 Feb '11

Hi, we're using syslog-ng 3.1.2 and have run into what appears to be a bug, but I'd like to get the community's opinion before we dig further into it. We have a bunch of HP servers with iLO2 and iLO3 devices, configured with their virtual serial ports on COM1 (ttyS0). We subsequently have the OS (RHEL4, RHEL5) configured to use COM1 as its console (e.g. /dev/console). This is a very standard configuration that allows us to get remote access to the machines without having to purchase the iLO Advanced KVM feature. It also lets us use the Magic SysRq keys to probe dead systems and stuff, so in general it's not something we're keen to change. What we have found, however, is that there are some cases where the iLO will freeze and requires a reboot. When the iLO reboots, however, the kernel's connection to /dev/console (through the virtual serial port) hangs and blocks. Any traffic to /dev/console just sits in the kernel's buffer and is never delivered. Once the buffer is full, the kernel simply blocks on any write to /dev/console. Now this is a Bad Thing in general, and we're working with HP to try and remedy this bug. However, what concerns me is that syslog-ng, when faced with this behavior, also blocks, even for log messages not bound for /dev/console. What we have observed is that a system with syslog-ng will keep delivering the occasional console message to /dev/console (ex. *.emerg messages) and meanwhile the file-based log paths keep working. But once /dev/console blocks, the next time a console message is delivered, *all* of syslog-ng blocks waiting for that message to be delivered, and all of the file-based paths block as well. The result is that pretty much everything on the system stops working. For example, you can't log in, even as root, because the login process blocks on the syslog command that writes to /var/log/secure. Anything that uses syslog suddenly blocks. Is this expected behavior? I would think that syslog-ng would be able to continue accepting and delivering messages, even if one of the log paths is stalled on a blocked write. -- Paul Krizak 7171 Southwest Pkwy MS B200.3A MTS Systems Engineer Austin, TX 78735 Advanced Micro Devices Desk: (512) 602-8775 Linux/Unix Systems Engineering Cell: (512) 791-0686 Global IT Infrastructure Fax: (512) 602-0468

6 15

Heavy memory leak using correlation or patterndb
by Guillaume Rousse 05 Feb '11

05 Feb '11

Hello. Since I installed the rules dedicated to dropping LDAP probe connection traces, the memory usage of my ldap server exploded. top confirms than syslog-ng is the culprit, using 60% of the available memory. Here is a weekly munin graph presenting the issue: http://www.zarb.org/~guillomovitch/memory-week.png The rules have been installed on 19th, and I restarted syslog-ng yesterday. I'm using syslog-ng 3.2.1, on mandriva 2010.0, 64 bits. Here are the exact rules used, if that matters (I couldn't try latest Balazs suggestions yet). The context-timeout for the first rule is quite low, because the probe checks every two seconds. By any chance, doesn't the second timeout, set to 0, set an illimited session expiration time ? If not the case, I'd gladly try any needed experiment, such as running syslog-ng under valgrind, to identify the leak source. -- BOFH excuse #400: We are Microsoft. What you are experiencing is not a problem; it is an undocumented feature.

2 3

[Bug 110] New: Advancing the system date and time beyond 2038 causes flood of messages
by bugzilla＠bugzilla.balabit.com 05 Feb '11

05 Feb '11

https://bugzilla.balabit.com/show_bug.cgi?id=110 Summary: Advancing the system date and time beyond 2038 causes flood of messages Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: gaudet(a)verizon.net Type of the Report: bug Estimated Hours: 0.0 On a 64-bit Linux system (SLES 11 SP1), if the date and time are advanced beyond January 19, 2038 @ 03:14:07, syslog begins flooding the log with MARK messages. It is possible that this is due to the so-called Year 2038 or Unix Millenium bug (see http://www.deepsky.com/~merovech/2038.html) The test: - Set the syslog mark frequency to 2 seconds and restart the syslog service - Tail the log - From another window, set the date to January 19, 2038 @ 03:14:00 - Observe the mark messages are correct until the clock moves beyond 03:14:07 # Display the starting date/time plus seconds since epoch $ date ; date +%s Tue Jan 25 02:10:00 UTC 2011 1295921228 # Set the date and time to January 19, 2038 @ 03:14:00 $ date 011903142038 Tue Jan 19 03:14:00 UTC 2038 # Display the new date/time and note that seconds since epoch have advanced beyond the 32-bit unsigned integer maximum $ date ; date +%s Tue Jan 19 03:14:10 UTC 2038 2147483650 $ tail -f /var/log/messages Jan 25 02:10:00 localhost -- MARK -- Jan 25 02:10:02 localhost -- MARK -- Jan 25 02:10:04 localhost -- MARK -- Jan 25 02:10:06 localhost -- MARK -- Jan 25 02:10:08 localhost -- MARK -- Jan 25 02:10:10 localhost -- MARK -- Jan 25 02:10:12 localhost -- MARK -- Jan 25 02:10:14 localhost -- MARK -- Jan 25 02:10:16 localhost -- MARK -- Jan 25 02:10:18 localhost -- MARK -- Jan 25 02:10:20 localhost -- MARK -- Jan 19 03:14:00 localhost -- MARK -- Jan 19 03:14:00 localhost syslog-ng[4961]: Log statistics; dropped='pipe(/dev/xconsole)=0', dropped='pipe(/dev/tty10)=0', processed='center(queued)=71', processed='center(received)=65', processed='destination(messages)=65', processed='destination(mailinfo)=0', processed='destination(mailwarn)=0', processed='destination(localmessages)=0', processed='destination(newserr)=0', processed='destination(mailerr)=0', processed='destination(netmgm)=0', processed='destination(warn)=2', processed='destination(console)=2', processed='destination(null)=0', processed='destination(mail)=0', processed='destination(xconsole)=2', processed='destination(firewall)=0', processed='destination(acpid)=0', processed='destination(newscrit)=0', processed='destination(newsnotice)=0', processed='source(src)=65' Jan 19 03:14:02 localhost -- MARK -- Jan 19 03:14:04 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- In a very short period of time, there are thousands of the "Jan 19 03:14:06..." messages in the log. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3