- syslog-ng - lists.balabit.hu

[fyi] ivykis reference handling
by Balazs Scheidler 06 Feb '11

06 Feb '11

Hi, Those experimenting with the 3.3 tree may have noticed that syslog-ng is using ivykis, a main loop implementation by Lennert Buytenhek. Since this library is not in wide-spread use in distributions, I've decided to embed it into the syslog-ng source code / git repository. Source code releases include a lib/ivykis directory, that is automatically compiled when syslog-ng is compiled except if --with-ivykis=system is specified. The git repository contains a submodule reference to the ivykis repo, which points to my ivykis repository on git.balabit.hu. The way it is implemented in git, is that it contains a sha1 reference to the exact imported commit. Since I don't intend to fork ivykis, this ivykis repo is rebased from time-to-time to its upstream. Rebasing causes commits to be changed (e.g. their sha1 changes), thus the submodule reference from the syslog-ng repo could break. To avoid this, I'm going to tag each and every "import" into the syslog-ng repository with a tag in the form "syslog-ng-3.3-refXXX", where XXX is a kind of version number, increased by one every time. By putting this tag in the ivykis repository, I can ensure that any import I make will still be available. I intend to do the same with libmongo-client, used by the mongodb driver. -- Bazsi

1 0

[PATCH][ivykis] configure.gnu: Quote $@, so that parameters are passed correctly.
by Gergely Nagy 05 Feb '11

05 Feb '11

When a parent configure calls ivykis' configure.gnu, some parameters might contain spaces (for example, CFLAGS="-ggdb3 -O0"), and without quoting $@, these will end up as separate options passed down to the real configure. With quoting, however, the parameters will be passed along as-is, without breaking them up. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.gnu | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure.gnu b/configure.gnu index 99dd8a7..4fb578f 100644 --- a/configure.gnu +++ b/configure.gnu @@ -4,4 +4,4 @@ echo $0 configure="`dirname $0`/`basename $0 .gnu`" echo "Running: " $configure $@ --disable-shared --enable-static --with-pic -$SHELL $configure $@ --disable-shared --enable-static --with-pic +$SHELL $configure "$@" --disable-shared --enable-static --with-pic -- 1.7.0.4

2 1

syslog-ng deadlock if /dev/console locks?
by Paul Krizak 05 Feb '11

05 Feb '11

Hi, we're using syslog-ng 3.1.2 and have run into what appears to be a bug, but I'd like to get the community's opinion before we dig further into it. We have a bunch of HP servers with iLO2 and iLO3 devices, configured with their virtual serial ports on COM1 (ttyS0). We subsequently have the OS (RHEL4, RHEL5) configured to use COM1 as its console (e.g. /dev/console). This is a very standard configuration that allows us to get remote access to the machines without having to purchase the iLO Advanced KVM feature. It also lets us use the Magic SysRq keys to probe dead systems and stuff, so in general it's not something we're keen to change. What we have found, however, is that there are some cases where the iLO will freeze and requires a reboot. When the iLO reboots, however, the kernel's connection to /dev/console (through the virtual serial port) hangs and blocks. Any traffic to /dev/console just sits in the kernel's buffer and is never delivered. Once the buffer is full, the kernel simply blocks on any write to /dev/console. Now this is a Bad Thing in general, and we're working with HP to try and remedy this bug. However, what concerns me is that syslog-ng, when faced with this behavior, also blocks, even for log messages not bound for /dev/console. What we have observed is that a system with syslog-ng will keep delivering the occasional console message to /dev/console (ex. *.emerg messages) and meanwhile the file-based log paths keep working. But once /dev/console blocks, the next time a console message is delivered, *all* of syslog-ng blocks waiting for that message to be delivered, and all of the file-based paths block as well. The result is that pretty much everything on the system stops working. For example, you can't log in, even as root, because the login process blocks on the syslog command that writes to /var/log/secure. Anything that uses syslog suddenly blocks. Is this expected behavior? I would think that syslog-ng would be able to continue accepting and delivering messages, even if one of the log paths is stalled on a blocked write. -- Paul Krizak 7171 Southwest Pkwy MS B200.3A MTS Systems Engineer Austin, TX 78735 Advanced Micro Devices Desk: (512) 602-8775 Linux/Unix Systems Engineering Cell: (512) 791-0686 Global IT Infrastructure Fax: (512) 602-0468

6 15

Heavy memory leak using correlation or patterndb
by Guillaume Rousse 05 Feb '11

05 Feb '11

Hello. Since I installed the rules dedicated to dropping LDAP probe connection traces, the memory usage of my ldap server exploded. top confirms than syslog-ng is the culprit, using 60% of the available memory. Here is a weekly munin graph presenting the issue: http://www.zarb.org/~guillomovitch/memory-week.png The rules have been installed on 19th, and I restarted syslog-ng yesterday. I'm using syslog-ng 3.2.1, on mandriva 2010.0, 64 bits. Here are the exact rules used, if that matters (I couldn't try latest Balazs suggestions yet). The context-timeout for the first rule is quite low, because the probe checks every two seconds. By any chance, doesn't the second timeout, set to 0, set an illimited session expiration time ? If not the case, I'd gladly try any needed experiment, such as running syslog-ng under valgrind, to identify the leak source. -- BOFH excuse #400: We are Microsoft. What you are experiencing is not a problem; it is an undocumented feature.

2 3

[Bug 110] New: Advancing the system date and time beyond 2038 causes flood of messages
by bugzilla＠bugzilla.balabit.com 05 Feb '11

05 Feb '11

https://bugzilla.balabit.com/show_bug.cgi?id=110 Summary: Advancing the system date and time beyond 2038 causes flood of messages Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: gaudet(a)verizon.net Type of the Report: bug Estimated Hours: 0.0 On a 64-bit Linux system (SLES 11 SP1), if the date and time are advanced beyond January 19, 2038 @ 03:14:07, syslog begins flooding the log with MARK messages. It is possible that this is due to the so-called Year 2038 or Unix Millenium bug (see http://www.deepsky.com/~merovech/2038.html) The test: - Set the syslog mark frequency to 2 seconds and restart the syslog service - Tail the log - From another window, set the date to January 19, 2038 @ 03:14:00 - Observe the mark messages are correct until the clock moves beyond 03:14:07 # Display the starting date/time plus seconds since epoch $ date ; date +%s Tue Jan 25 02:10:00 UTC 2011 1295921228 # Set the date and time to January 19, 2038 @ 03:14:00 $ date 011903142038 Tue Jan 19 03:14:00 UTC 2038 # Display the new date/time and note that seconds since epoch have advanced beyond the 32-bit unsigned integer maximum $ date ; date +%s Tue Jan 19 03:14:10 UTC 2038 2147483650 $ tail -f /var/log/messages Jan 25 02:10:00 localhost -- MARK -- Jan 25 02:10:02 localhost -- MARK -- Jan 25 02:10:04 localhost -- MARK -- Jan 25 02:10:06 localhost -- MARK -- Jan 25 02:10:08 localhost -- MARK -- Jan 25 02:10:10 localhost -- MARK -- Jan 25 02:10:12 localhost -- MARK -- Jan 25 02:10:14 localhost -- MARK -- Jan 25 02:10:16 localhost -- MARK -- Jan 25 02:10:18 localhost -- MARK -- Jan 25 02:10:20 localhost -- MARK -- Jan 19 03:14:00 localhost -- MARK -- Jan 19 03:14:00 localhost syslog-ng[4961]: Log statistics; dropped='pipe(/dev/xconsole)=0', dropped='pipe(/dev/tty10)=0', processed='center(queued)=71', processed='center(received)=65', processed='destination(messages)=65', processed='destination(mailinfo)=0', processed='destination(mailwarn)=0', processed='destination(localmessages)=0', processed='destination(newserr)=0', processed='destination(mailerr)=0', processed='destination(netmgm)=0', processed='destination(warn)=2', processed='destination(console)=2', processed='destination(null)=0', processed='destination(mail)=0', processed='destination(xconsole)=2', processed='destination(firewall)=0', processed='destination(acpid)=0', processed='destination(newscrit)=0', processed='destination(newsnotice)=0', processed='source(src)=65' Jan 19 03:14:02 localhost -- MARK -- Jan 19 03:14:04 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- Jan 19 03:14:06 localhost -- MARK -- In a very short period of time, there are thousands of the "Jan 19 03:14:06..." messages in the log. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

tfjson: JSON-generating template function
by Gergely Nagy 05 Feb '11

05 Feb '11

Hi! Looks like the list didn't like me sending the whole patch, so I'll have to omit the patch part this time, and hope that the wall of text below won't hit the size limit. Based on the work of Balint Kovacs <blint(a)balabit.hu>, a $(format_json) template function was implemented: with this function, one can hammer log messages into JSON format. While that is not immediately useful, we have a few things planned that will - with a bit of luck - be able to use this function. The function itself, without arguments, will reformat the log message into JSON. For example, a loggen-generated message will look like this (formatting added by me): { "HOST": "localhost", "HOST_FROM": "localhost", "MESSAGE": "seq: 0000000000, thread: 0000, runid: 1295625842, stamp: 2011-01-21T17:04:02 PADD...", "PROGRAM": "prg00000", "PID": "1234", "SOURCE": "s_tcp", "LEGACY_MSGHDR": "prg00000[1234]: ", ".classifier.class": "unknown" } While this is all cool, there's more! The function accepts parameters: '--select <GLOB>' and '--exclude <GLOB>', where <GLOB> is a shell-like glob pattern. One can have any number of selects or globs. The way this works, is that if a key doesn't match any of the select globs, it will not be included. If it does, but matches any of the exclude globs, it will be dropped. Otherwise it's included. The default select glob is "*", which is why everything is included by default. The globs are matched against the keys: macro names without the dollar sign, patterndb keys, and so on. With this knowledge, we can tweak the template like this: $(format_json --select .classifier.* --select usracct.* --exclude *.*id) Then, given the following message: <86>Jan 21 17:10:21 moria sshd[31930]: pam_unix(sshd:session): session closed for user algernon We'll end up with this JSON: { ".classifier.class": "system", "usracct.username": "algernon", "usracct.type": "logout", "usracct.application": "sshd" } Now, that's not very useful, as we're lacking the HOST and a whole lot of other things. Adding all that as selects would be a bit excessive, therefore one can list macro names (without the dollar sign) along with the select/exclude options, and they'll be included too. Furthermore, one can list key=value pairs, for example, using the log message quite like the one above, and the following template: $(format_json -s .classifier.* -s usracct.* -x *.*id HOST MESSAGE prg='$PROGRAM[$PID]') We'll end up with: { ".classifier.class": "system", "usracct.username": "algernon", "usracct.type": "logout", "usracct.application": "sshd", "MESSAGE": "pam_unix(sshd:session): session closed for user algernon", "prg": "sshd[32216]", "HOST": "localhost" } At the moment, only strings are supported, and there is no way to structure the JSON properly yet. We'll figure out a way to do that too, eventually. There's a few more things in the pipeline for tfjson, so treat this version as a beta. The code is available on the integration/tfjson/base branch of my repository at git://git.madhouse-project.org/syslog-ng/syslog-ng-3.3.git Browsable online at: http://git.madhouse-project.org/syslog-ng/syslog-ng-3.3/tree/modules/tfjson… The bulk of the work was done by Balint, I just happened to accidentally refactor it a tiny bit while reviewing his code. -- |8]

3 4

connecting to mongodb
by Peter Czanik 03 Feb '11

03 Feb '11

Hello, When I try to use mongodb, I receive the following error message: Feb 2 15:33:54 linux-8k4b syslog-ng[9403]: syslog-ng starting up; version='3.3.0alpha0' Feb 2 15:33:54 linux-8k4b syslog-ng[9403]: Error connecting to MongoDB; Feb 2 15:33:54 linux-8k4b syslog-ng[9403]: Error connecting to MongoDB; Feb 2 15:33:54 linux-8k4b syslog-ng[9403]: Network error while inserting into MongoDB; time_reopen='60' The relevant part of my syslog-ng.conf is (taken from an earlier e-mail on the list): destination d_mongodb { mongodb( host("localhost") port(27017) database("syslog-ng") collection("logs") keys("date", "facility", "level", "host", "program", "pid", "message") values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "$FACILITY", "$LEVEL", "$HOST","$PROGRAM", "$PID", "$MSGONLY") ); }; log { source(s_local); destination(d_mongodb); }; Version (latest git) compiled on openSUSE factory: linux-8k4b:/usr/local/etc # /usr/local/sbin/syslog-ng -V syslog-ng 3.3.0alpha0 Installer-Version: 3.3.0alpha0 Revision: Compile-Date: Feb 2 2011 14:27:08 Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off linux-8k4b:/usr/local/etc # Using tcpdump I can't even see a connection attempt. Connecton using mongodb tools work fine. linux-8k4b:/usr/local/etc # mongo --version MongoDB shell version: 1.6.6-pre- But as packets don't even reach the mongodb server, it's most likely not a relevant information... Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 4

[RFC]: value-pairs()
by Gergely Nagy 02 Feb '11

02 Feb '11

Hi! The idea of a generic framework to set up value pairs came up first during the afmongodb discussion (see Bazsi's ideas at http://article.gmane.org/gmane.comp.syslog-ng/10432) and while working on the tfjson driver, it came up again. I've learnt a lot since the dynamic-variables implementation in afmongodb, and following Bazsi's comments on the list earlier, we managed to talk a little about the value-pairs() feature too. That, and another quick chat with Balint, followed by working on tfjson helped form the proposal below. I'll divide this into two sections: the first part, mostly aimed at people using syslog-ng in production would be a request for comments regarding the syntax of the feature. The second part will be a few notes on the planned implementation. The Syntax ========== value-pairs ( glob-select ("usracct.*") glob-exclude ("*.*id") builtins (no) $HOST $MESSAGE "program_n_pid" = "$PROGRAM[$PID]" ) What we have here, is basically three possible ways to define - or control - value pairs: * Using a function, like glob-select and glob-exclude or builtins (more about these later) * Macro names, with the '$' sign. * "key" = "value" pairs. The way this is supposed to work is that the base set is all the various key=value pairs syslog-ng collected about a particular log message: the built-in macros ($HOST, $MESSAGE, etc), sdata, patterndb - you name it. This is the set we can select or exclude from, the set we can add new pairs to (different values for each value-pairs() instance, of course). With glob-select(), we can mark a subset of the orignal set for inclusion. Subsequent glob-exclude()s can override this, and remove parts of it from the set. Specifying builtins(no) removes every built-in key from the set. Listing known keys, unquoted, '$' prefixed, will add that value to the set, with the key set to the macro name without the dollar sign. For example, $HOST is exactly the same as "HOST" = "$HOST". Finally, "key" = "value" pairs do the obvious thing: add custom pairs to our set. The order of functions does matter, so glob-select("*") glob-exclude("*id") glob-select("usracct.id") will select usracct.id, but no other key ending with "id". This is partly due to me finding this logical, and because if/when we add other functions later on (say, ltrim() or add-prefix() or similar that modify the keys), the order becomes essential. But, back to the purpose of this all: this would be templating on steroids, so to say. Drivers should be able to easily implement support for value-pairs(), which would make system administrators to tweak the way syslog-ng discovered data will appear in the output. Particularly, this functionality is what will make the afmongodb driver shine. This is a much more advanced version of the dynamic-variables() functionality. This will also come in handy for the JSON template function. I leave the rest up to your imagination. Feedback on the syntax would be most appreciated. The Design ========== Value pairs would be stored in a struct like the following: typedef struct { GPatternSpec *pattern; gboolean exclude; } ValuePairPattern; typedef struct { ValuePairPattern **patterns; gboolean builtins; NVRegistry extras_registry; NVTable extras; } ValuePairs; ValuePairs->patterns would be a NULL terminated array (we can allow for a lot of reallocs during init, imo - otherwise we can make it a GPtrArray or something similar to save time on that), where each pattern is either an exclude or a select pattern. The ->builtins boolean would flag whether we should include the builtin variables ($HOST, etc), and would default to TRUE. And ->extras are the extra keys specified explicitly: both in $MACRO and "key"="value" form (the former translated to "MACRO"="$MACRO" form). We'd have the following support API: ValuePairs *value_pairs_new (); void value_pairs_add_pattern (ValuePairs *vp, const gchar *pattern, gboolean exclude); void value_pairs_set_builtins (ValuePairs *vp, gboolean state); void value_pairs_add_extra (ValuePairs *vp, const gchar *key, const gchar *value); void value_pairs_foreach (ValuePairs *vp, VPForeachFunc *func, gpointer user_data); typedef gboolean (*VPForeachFunc) (const gchar *name, const gchar *value, gssize value_len, gpointer user_data); We'd also have a VALUE_PAIRS keyword in the parser (if it is possible, which I think it is), which drivers could use. The basic VALUE_PAIRS stuff would set up a ValuePairs structure, and store it in a global variable, which the driver could then either copy, or just assign it to it's own (since the main parser won't free or touch it anymore anyway). Then, all the drivers have to do, is call value_pairs_foreach(), which would firs iterate over logmsg_registry's keys, taking the specified patterns into account. Then it would iterate over it's own extras, and pass the results to the foreach function one by one, similar to how nv_table_foreach does. That's about it. It's a bit vague, mostly because I did not research if this is possible the way I imagine it yet. Hopefully this all made at least some sense. -- |8]

5 12

fosdem 2011
by Peter Czanik 02 Feb '11

02 Feb '11

Hello, FOSDEM, the largest European free software conference is coming this weekend: http://fosdem.org/2011/ I'll be there wearing Balabit and Syslog-ng T-shirts :-) I'll be mostly at the "distribution miniconf" and join talks about upstream -- downstream relations. If you want to talk abut syslog-ng (or Zorp) or just see somebody from the mailing list in person, I'm happy to meet you there! I'll have my smartbook with me and will try to read my e-mail regularly. Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Re: [syslog-ng] Systemd support
by Gergely Nagy 02 Feb '11

02 Feb '11

Tom's message probably didn't make it to the list, due to the size of the attachments. But since I suppose more people are interested in systemd support, I'm CCing the list anyway. On Wed, 2011-02-02 at 02:15 +0100, Tom Gundersen wrote: > On Tue, Feb 1, 2011 at 10:19 PM, Balazs Scheidler <bazsi(a)balabit.hu> wrote: > > the suse maintainer has created a patch, and that branch is my preliminary > > integration, but it has changed somewhat since. it was sent in a private > > email, and unfortunately I simply had no chance to work on that. > > > > algernon(a)balabit.hu may have more information. > > It was not clear to me if the patch in your tree is the most > up-to-date one, but it appears to work fine for me. The patches you attached are not enough for proper systemd integration: if you have any unix-dgram() sockets that are not managed by systemd, syslog-ng will break. So will syslog-ng-ctl in certain cases. My tree has a fix for those issues, and to the best of my knowledge, works both on systemd-enabled systems, and non-systemd systems aswell. It's available (for 3.2) at git://git.madhouse-project.org/syslog-ng/syslog-ng-3.2.git On the 'systemd' branch. I'll prepare integration branches for pulling this all into the official tree during the next few days (both for 3.2 and 3.3). > Together, all of this works perfectly on my machine. Let me know if > you want help with testing of other patches (in case what I found was > not up-to-date). Looking forward to seeing systemd support in a > release :-) There's an extra few patches in my tree, on top of Marius Tomaschewski's systemd integration work, that wrap up the systemd support. More testing with those is always welcomed! (Especially since the testing I was able to conduct myself were pretty basic so far) -- |8]

1 0