- syslog-ng - lists.balabit.hu

Interesting issue with syslog-ng 3.3
by Gergely Nagy 03 Sep '11

03 Sep '11

Hi! While this mail might sound a bit vague, it will - if nothing else - serve as a reminder for me to investigate the issue furhter. On one of my servers (PowerPC, running Debian Squeeze), I have a syslog-ng 3.3 running, a reasonably recent (2-3 day old) git snapshot. It works quite well, except that I was able to trace back my server's recent hangs to syslog-ng: The server had a ~120 day uptime when I upgraded from 3.1 to 3.3, and since that time, it had to be rebooted two times already, just in two weeks time. Last time, I didn't have any open connections to it, so couldn't investigate, but tonight, I had an ssh session open with a screen session inside. So I tried to look around: first, I wanted to check the logs, but knew I wouldn't find anything, as it stopped sending the logs to my other server about two hours before I noticed the problem. Even worse, when I tried to sudo, that hung, indefinitely. Weird. There was nothing in dmesg, and nothing interesting in the logs it did send before becoming unresponsive. HTTP still worked too, as did a few other services. I could do nearly anything as a user. So I tried stracing crontab, and it hung when it tried to send logs to /dev/log. Interesting! I tried logger, same happens. I suspect that for one reason or the other, /dev/log got overwhelmed, and even worse, syslog-ng ended up trying to log something aswell, which made it hang too. And thus, the queue remained full, and everything that tried to log, got stuck. HTTP continued to work, since my httpd isn't using syslog for its logs. I could poke around in my shell, since that wasn't logging, either. This never happened with 3.1, and the only thing I changed in the config is the @version, pretty much. Thus, I suspect, there's some very nasty bug in 3.3beta2 that I haven't found yet. I'm leaving a root shell open this time, so that I can poke around further next time (along with a syslog-ng compiled with debug symbols). In the meantime, I thought I'll drop a note, hoping that perhaps Bazsi or someone from the syslog-ng devel team would have an idea where to look, and what to check next time this happens. -- |8]

5 6

Redirecting logs and UDP issues
by Aldrich, Jamie S 02 Sep '11

02 Sep '11

All, I am trying to redirect a log file from an NFS mount (AIX server = source and Redhat server = target). The redirect is going to a LogLogic device over UDP, and I am seeing quite a few dropped packets in /var/log/messages. Samples below: Aug 31 07:34:16 lxfwossecp3 syslog-ng[30574]: syslog-ng starting up; version='2.1.4' Aug 31 07:44:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=15', processed='center(received)=15', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=1', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=1', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=4', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 07:54:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=17', processed='center(received)=17', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=2', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=2', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=6', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:04:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=20', processed='center(received)=20', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=4', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=3', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=9', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:14:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=22', processed='center(received)=22', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=5', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=4', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=11', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:24:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=28', processed='center(received)=28', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=6', processed='destination(d_cron)=6', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=5', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=17', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Here is what my syslog-ng.conf file looks like: # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { time_reopen (10); flush_lines (2000); flush_timeout (20000); log_fifo_size (30000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: # Setup for PeopleSoft logs to transmit to Loglogic source s_file { file("/tmp/APPSRV_current.LOG" flags(no-parse)); }; destination d_messages{ udp("10.13.33.11" port(514) log_fifo_size(900000000)); }; log { source(s_file); destination(d_messages); }; Does anyone see anything wrong with the setup? Could the NFS link be an issue? Any help would be greatly appreciated! Thanks, Jamie

2 3

[PATCH] afsocket: Compile against libsystemd-daemon, if present.
by Gergely Nagy 02 Sep '11

02 Sep '11

systemd v31+ comes with a libsystemd-daemon shared library. If systemd is enabled, try to detect this library, and use it if present. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 2 ++ modules/afsocket/Makefile.am | 12 ++++++++---- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/configure.in b/configure.in index 2685568..b6c95cf 100644 --- a/configure.in +++ b/configure.in @@ -895,6 +895,7 @@ if test "x$enable_systemd" = "xauto"; then fi if test "x$enable_systemd" = "xyes"; then + PKG_CHECK_MODULES(libsystemd_daemon, libsystemd-daemon >= 31,with_libsystemd="yes",with_libsystemd="no") if test "x$with_systemdsystemunitdir" = "xyes"; then # no arguments, just --with-systemdsystemunitdir systemdsystemunitdir=$($PKG_CONFIG --variable=systemdsystemunitdir systemd); @@ -1064,6 +1065,7 @@ AM_CONDITIONAL(ENABLE_SUN_STREAMS, [test "$enable_sun_streams" = "yes"]) AM_CONDITIONAL(ENABLE_PACCT, [test "$enable_pacct" = "yes"]) AM_CONDITIONAL(ENABLE_MONGODB, [test "$enable_mongodb" = "yes"]) AM_CONDITIONAL(ENABLE_JSON, [test "$enable_json" = "yes"]) +AM_CONDITIONAL(WITH_LIBSYSTEMD, [test "$with_libsystemd" = "yes"]) # substitution into manual pages expanded_sysconfdir=[`patheval $sysconfdir | sed -e 's/-/\\\\-/g'`] diff --git a/modules/afsocket/Makefile.am b/modules/afsocket/Makefile.am index 08ffc19..79f9c99 100644 --- a/modules/afsocket/Makefile.am +++ b/modules/afsocket/Makefile.am @@ -3,7 +3,11 @@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib export top_srcdir if ENABLE_SYSTEMD +if WITH_LIBSYSTEMD +SYSTEMD_SOURCES = +else SYSTEMD_SOURCES = sd-daemon.c sd-daemon.h +endif else SYSTEMD_SOURCES = endif @@ -14,8 +18,8 @@ libafsocket_notls_la_SOURCES = \ afsocket.c afsocket.h afunix.c afunix.h afinet.c afinet.h \ afsocket-grammar.y afsocket-parser.c afsocket-parser.h afsocket-plugin.c \ $(SYSTEMD_SOURCES) -libafsocket_notls_la_CPPFLAGS = $(AM_CPPFLAGS) -libafsocket_notls_la_LIBADD = $(MODULE_DEPS_LIBS) $(LIBNET_LIBS) $(LIBWRAP_LIBS) +libafsocket_notls_la_CPPFLAGS = $(AM_CPPFLAGS) $(libsystemd_daemon_CFLAGS) +libafsocket_notls_la_LIBADD = $(MODULE_DEPS_LIBS) $(LIBNET_LIBS) $(LIBWRAP_LIBS) $(libsystemd_daemon_LIBS) libafsocket_notls_la_LDFLAGS = $(MODULE_LDFLAGS) if ENABLE_SSL @@ -24,8 +28,8 @@ libafsocket_tls_la_SOURCES = \ afsocket.c afsocket.h afunix.c afunix.h afinet.c afinet.h \ afsocket-grammar.y afsocket-parser.c afsocket-parser.h afsocket-plugin.c \ $(SYSTEMD_SOURCES) -libafsocket_tls_la_CPPFLAGS = $(AM_CPPFLAGS) -DENABLE_SSL=1 -libafsocket_tls_la_LIBADD = $(MODULE_DEPS_LIBS) ../../lib/libsyslog-ng-crypto.la $(OPENSSL_LIBS) $(ZLIB_LIBS) $(LIBNET_LIBS) $(LIBWRAP_LIBS) +libafsocket_tls_la_CPPFLAGS = $(AM_CPPFLAGS) $(libsystemd_daemon_CFLAGS) -DENABLE_SSL=1 +libafsocket_tls_la_LIBADD = $(MODULE_DEPS_LIBS) ../../lib/libsyslog-ng-crypto.la $(OPENSSL_LIBS) $(ZLIB_LIBS) $(LIBNET_LIBS) $(LIBWRAP_LIBS) $(libsystemd_daemon_LIBS) libafsocket_tls_la_LDFLAGS = $(MODULE_LDFLAGS) endif -- 1.7.5.4

2 1

[PATCH 0/2] Various stuff for easier packaging
by Gergely Nagy 02 Sep '11

02 Sep '11

Two patches will follow, that make it slightly easier to package syslog-ng and to build external modules against it. The first will change the SONAME of libsyslog-ng, from libsyslog-ng.so.0 to libsyslog-ng-${VERSION} (as of this writing, this would be libsyslog-ng-3.3.0beta2.so). The rationale behind this, is that at the moment, there are no guarantees that either ABI or API will be preserved between versions, so it's easier and more straightforward to tie the library to a particular syslog-ng version instead. The other patch adds moduledir to the pkg-config file, so that external modules can easily figure out where to install themselves, without resorting to guessing and parsing syslog-ng --help output (where syslog-ng might not even be available in the build chroot..).

2 3

[Bug 132] New: Timestamp issue with negative offset
by bugzilla＠bugzilla.balabit.com 02 Sep '11

02 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=132 Summary: Timestamp issue with negative offset Product: syslog-ng Version: 3.2.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mjrmedina(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Hi: I have syslog-ng 3.2.4 installed in Red Hat EL 6. My timezone is GMT-04:30 (Caracas,Venezuela) and I'm using ISO8601 timestamp format in syslog-ng. The timestamp format of the syslog-ng using my timezone is like follow: 2011-08-31T12:32:25-04:-30 2011-08-31T12:32:31-04:-30 See that before the "30" there is a second "-", because the first "-" is before "04". This issue is presented with all the negative time zones that has "30" in the minute offset (I test with Newfoundland,Canada too). Too I have tested this syslog-ng version in RHEL5 and the behaviour is the same. I think that this should be a kind of BUG. Rigth? Any workaround to handle this? Thanks in advanced... Mario -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 4

[PATCH] afmongodb: Fix threaded operation.
by Gergely Nagy 02 Sep '11

02 Sep '11

Ported a patch from afsql to afmongodb, that makes the writer thread wake up only when it's synced. This makes threaded(yes) work more reliably. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 18 ++++++++++++++++-- 1 files changed, 16 insertions(+), 2 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 9e155ed..0f01f05 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -497,6 +497,17 @@ afmongodb_dd_free(LogPipe *d) } static void +afmongodb_dd_queue_notify(gpointer user_data) +{ + MongoDBDestDriver *self = (MongoDBDestDriver *)user_data; + + g_mutex_lock(self->queue_mutex); + g_cond_signal(self->writer_thread_wakeup_cond); + log_queue_reset_parallel_push(self->queue); + g_mutex_unlock(self->queue_mutex); +} + +static void afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_options, gpointer user_data) { MongoDBDestDriver *self = (MongoDBDestDriver *)s; @@ -509,12 +520,15 @@ afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_optio g_mutex_lock(self->queue_mutex); self->last_msg_stamp = cached_g_current_time_sec (); queue_was_empty = log_queue_get_length(self->queue) == 0; - log_queue_push_tail(self->queue, msg, path_options); g_mutex_unlock(self->queue_mutex); + + log_queue_push_tail(self->queue, msg, path_options); g_mutex_lock(self->suspend_mutex); if (queue_was_empty && !self->writer_thread_suspended) - g_cond_signal(self->writer_thread_wakeup_cond); + { + log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); + } g_mutex_unlock(self->suspend_mutex); } -- 1.7.0.4

2 1

log for patterns
by Peter Czanik 02 Sep '11

02 Sep '11

Hello, While my previous post was about Windows, we did not forget about our UNIX/Linux users :) I'd like to extend our pattern database with new patterns. So I have some related questions: - Until now we dealt mostly with login/logut events of different applications, as this is one of the most interesting information on servers. Is it OK with you, or we should cover also other events? (Which?) - Which applications are you mostly interested in? - Creating logs in a "lab" environment just for pattern creation is very time consuming. It would be very helpful for us, if you could send logs in exchange for patterns. Just make sure, that there is no sensitive data left in the logs, as the result will be published to make it available for the whole syslog-ng community. I published a blog ( http://czanik.blogs.balabit.com/2010/11/log-sample-collecting-project/ ), how I collect logs for pattern creation, but of course, any logs are welcome! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

4 3

License for syslog-ng OSE v 3.2.x
by Yu Watanabe 01 Sep '11

01 Sep '11

Hello all. I would like to ask question for the license of syslog-ng. syslog-ng is licensed under the combination of the GPL and LGPL licenses. The syslog-ng core (src/ subdirectory) is licensed under the Lesser General Public License as described in the file LGPL.txt The syslog-ng modules (src/xxxx/ subdirectories) are licensed under the General Public License as described in the file GPL.txt Would it be correct to understand as below? LGPL = LGPLv2.1 GPL = GNU General Public License, version 1 I couldn't find the both text in the tarball so asked this question. Thanks! Yu

3 3

[PATCH] [pdbtool] debug-id option
by Balint Kovacs 31 Aug '11

31 Aug '11

Hi, While working with `pdbtool test`, I found that it's quite uncomfortable to find problems with non-matching messages, as it only displays the ID of the erroneous pattern. Right now you would need to copy the example message and program name, and pass it to `pdbtool match` as arguments to find out what's the exact issue and get a nice colorized output pointing to the problematic part of the pattern. To make that easier, I've done a small enhancement to pdbtool, an option to do a full, colorized debug output on a given rule by only supplying its ID. This would look something like (coloring lost in email): blint@lyra:~/blah/syslog-ng-ose-mainline-3.4$ ./bin/pdbtool debug-id -p /var/tmp/patterndb/system-bind.xml -r "b57a384f-c8be-41e9-bc10-735695dc63e7" Pattern matching part: unexpected RCODE (REFUSED) resolving @QSTRING:.dict.arpa=hushmail.com/AAAA/IN@:@QSTRING:.dict.src=203.197.12.30@deliberately freaked up test message 53 Matching part: unexpected RCODE (REFUSED) resolving 'hushmail.com/AAAA/IN': 203.197.12.30# Values: MESSAGE=unexpected RCODE (REFUSED) resolving 'hushmail.com/AAAA/IN': 203.197.12.30#deliberately freaked up test message 53 PROGRAM=named .classifier.class=unknown The patch is against 3.4, but should apply to 3.3 as well. Balint commit 93ca04700f8706643fedea51936af02daa314766 Author: Balint Kovacs<blint(a)balabit.hu> Date: Mon Aug 15 11:07:50 2011 +0200 [pdbtool] Implemented debug-id option Added a new option to pdbtool to test a specific rule against its example message with colorizing debug turned on. Signed-off-by: Balint Kovacs<blint(a)balabit.hu> diff --git a/modules/dbparser/pdbtool.c b/modules/dbparser/pdbtool.c index 64ccd17..f64012d 100644 --- a/modules/dbparser/pdbtool.c +++ b/modules/dbparser/pdbtool.c @@ -640,6 +640,75 @@ static GOptionEntry match_options[] = { NULL, 0, 0, G_OPTION_ARG_NONE, NULL, NULL } }; +static gchar *rule_id = NULL; + +static gboolean +pdbtool_debug_id(int argc, char *argv[]) +{ + PatternDB *patterndb; + PDBExample *example; + GList *examples = NULL; + gboolean id_is_found = FALSE; + + debug_pattern = TRUE; + debug_pattern_parse = FALSE; + color_out = TRUE; + colors = full_colors; + + if (!rule_id) + { + printf("Please specify a rule ID to be tested against its example message!\n"); + return FALSE; + } + + patterndb = pattern_db_new(); + if (!pdb_rule_set_load(patterndb->ruleset, configuration, patterndb_file,&examples)) + { + pattern_db_free(patterndb); + return FALSE; + } + + while (examples) + { + example = examples->data; + + if (strcmp(example->rule->rule_id, rule_id) != 0) + { + examples = g_list_delete_link(examples, examples); + continue; + } + + id_is_found = TRUE; + + if (example->message&& example->program) + { + match_message = example->message; + match_program = example-> program; + pdbtool_match(argc, argv); + } + examples = g_list_delete_link(examples, examples); + } + + pattern_db_free(patterndb); + + if (!id_is_found) + { + printf("Could not find a corresponding ID in the patterndb file or the rule does not have an example message.\n"); + return FALSE; + } + + return TRUE; +} + +static GOptionEntry debug_id_options[] = +{ + { "pdb", 'p', 0, G_OPTION_ARG_STRING,&patterndb_file, + "Name of the patterndb file", "<patterndb_file>" }, + { "ruleid", 'r', 0, G_OPTION_ARG_STRING,&rule_id, + "ID of the patterndb rule to debug", "<rule_id>" }, + { NULL, 0, 0, G_OPTION_ARG_NONE, NULL, NULL } +}; + static gboolean test_validate = FALSE; static gboolean @@ -934,6 +1003,7 @@ static struct { "dump", dump_options, "Dump pattern datebase tree", pdbtool_dump }, { "merge", merge_options, "Merge pattern databases", pdbtool_merge }, { "test", test_options, "Test pattern databases", pdbtool_test }, + { "debug-id", debug_id_options, "Test pattern databases", pdbtool_debug_id }, { "patternize", patternize_options, "Create a pattern database from logs", pdbtool_patternize }, { NULL, NULL }, };

2 7

Duplicate Hostnames entries in syslog-ng
by Swati4 M/TVM/TCS 30 Aug '11

30 Aug '11

Hi, My client machine is running syslog-ng 3.0 version and remote server is running syslog. On logging messages from client to server,I am encountering duplicate entries for hostnames in the remote syslog messages and an extra colon. In local log files running syslog-ng,there are no duplicate entries. Have tried the following options,but no luck:- keep_hostname(yes); chain_hostnames(no); long_hostnames(no); For "extra colon" seen in remote messages,I have tried attaching some templates to filter my messages. Since,I guess in syslog-ng 3.0 the macro $MSG has been changed to $MSGHDR. Remote Output :- Aug 12 03:42:49 base0-0-0-1-0-11-1 base0-0-0-1-0-11-1 kernel: : device eth0 left promiscuous mode Expected Output :- Aug 12 03:42:49 base0-0-0-1-0-11-1 kernel:device eth0 left promiscuous mode Can anybody please help me with the issue ? Thanks in advance. Regards This is a snippet of my syslog-ng.conf file :- options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

1 0