- syslog-ng - lists.balabit.hu

Syslog-ng 2.1 OSE rpm on Suse 10
by Raffaele 13 Jun '08

13 Jun '08

Hi, I'm trying to install syslog-ng 2.1 OSE from balabit precompiled packages and I get this output: #~/syslog-ng # rpm -ivh syslog-ng-2.1beta1-1.i586.rpm error: Failed dependencies: libdbi8.so.0 is needed by syslog-ng-2.1beta1-1.i586 #~/syslog-ng # cat /etc/issue Welcome to SUSE Linux Enterprise Server 10 (i586) - Kernel \r (\l). #~/syslog-ng # uname -a Linux 2.6.16.27-0.9-smp #1 SMP Tue Feb 13 09:35:18 UTC 2007 i686 athlon i386 GNU/Linux Searching on docs seems that libdbi8 is required for PE and not for OSE, am I wrong? Best regards Raffaele

2 3

Ronald Jones is out of the office.
by Ronald Jones 13 Jun '08

13 Jun '08

I will be out of the office starting 06/12/2008 and will not return until 06/16/2008. I will respond to your message when I return. ----------------------------------------- ________________________________________________________ DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

1 0

[Bug 22] New: Incorrect PID macro content
by bugzilla＠bugzilla.balabit.com 10 Jun '08

10 Jun '08

https://bugzilla.balabit.com/show_bug.cgi?id=22 Summary: Incorrect PID macro content Product: syslog-ng Version: unspecified Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: kgushtin(a)n-ix.com.ua Type of the Report: --- Estimated Hours: 0.0 I'm working for op5 and we have sometimes this very important to us problem. So, on boot we expirienced incorrect content of $PID macro, that can't be converted to integer. For example PID contains - PCI0, LNKA, LNKD, ... I think the problem is in parser, which parse Log Message that contains brackets [], which shouldn't do this, it needs to parse only Log Header for []. Some more description in my latter on postlist - "Problem with PID macro" (2008-06-10). Example of inccorect message : Feb 25 18:23:34 test-system kernel: ACPI: PCI Root Bridge [PCI0] (0000:00) Some additional info: We saw this in syslog-ng premium edition 2.1.9a, in our build of open source syslog-ng 2.2.0 and in old syslog-ng-2.0.5-op5.1_RHEL5 . I run all the tests on VMWare machine. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

unusual syntax error while parsing .conf file
by Andrej Lobovski 10 Jun '08

10 Jun '08

I upgraded from 1.6.9 ro 2.0.8 but am using same .conf file, which on 1.6.9. works perfectly. The only line that gives me error (but syslog-ng 2.0.8 then works normally if i comment it out) in 2.0.8 is destination d_root { usertty("root" template("$MSGONLY\n")); }; Has this changed? I get syslog-ngsyntax error 273 if I start it. (273 is line in .conf file which i mention in this email). Thank you in advance _________________________________________________________________ Explore the seven wonders of the world http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE

3 3

Problem with PID macro
by Kostyantyn Gushtin 10 Jun '08

10 Jun '08

2 1

windows syslog-mng agent
by James B Horwath 10 Jun '08

10 Jun '08

Does anyone have any experience with the windows syslog-ng agent? If so could you contact me off-line so I could here about your experience. TIA. Jim ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

file() source, facility, priority and new events only,
by Hari Sekhon 09 Jun '08

09 Jun '08

Hi, I'm using the file() source to include logs from an application that doesn't do syslog (heresy! :-) ). It looks like it's logging the thing under the kern facility. I've been through the admin manual v2 and cannot see any way to change the facility and priority that it marks the source as. My current source looks like this: source s_app_name{ file( "/path/to/app.log" log_prefix("App Name: ") flags(no-parse) ) ; }; Also, the app.log is quite big and syslog-ng has gone and taken the whole thing and inserted it into syslog with the current timestamp. I'd like to know if there is a tail -n 0 type switch to tell it to take only things that have been appended since it has started watching the file. Can this be done? I wrote a script that basically tails to logger which does all these things but I'd rather use the file() source if I can get these niggles working the way I want. I'm running syslog-ng 2.0.6. -h -- Hari Sekhon

2 1

How to skip parsing?
by Clay Loveless 09 Jun '08

09 Jun '08

Hi, I'm relatively new to syslog-ng, but have dug in enough to see that there's an option in 2.0.9 (possibly earlier as well) to skip message parsing that throws off the "unparseable log message" errors. In src/logmsg.c, there's apparently a flag that can be set to skip parsing. (See line 140). I have one particular message source that I'd like to skip parsing on. What I haven't been able to determine is how to configure syslog- ng.conf with the flag that would skip the parsing of the log message. Can anyone help out on this? Thanks, Clay -- Killersoft.com

2 3

Re: [syslog-ng] Filtering duplicate messages
by chris packham 09 Jun '08

09 Jun '08

Updated patch - back to using suppress, fix potential segfault. Implement "Last message repeated N times" functionality. This behaviour can be enabled by adding the new "suppress(<num>)" option to an output configuration e.g. destination tologfile { file("/var/log/messages" template(t) suppress(30)); }; As a log message is added to the queue it is remembered as the last message seen. When a new message is added to the queue it is checked against the last message. If its contents are the same the message is dropped and a counter incremented. A message summary indicating the value of the last message counter and a snippet of the message will be inserted into the log queue if a new message that differs is seen or if the configurable timeout period expires. --- src/cfg-grammar.y | 3 +- src/cfg-lex.l | 3 +- src/logmsg.c | 19 ++++++++++ src/logmsg.h | 2 +- src/logwriter.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++- src/logwriter.h | 5 +++ src/messages.c | 4 +-- 7 files changed, 130 insertions(+), 7 deletions(-) diff --git a/src/cfg-grammar.y b/src/cfg-grammar.y index 39ff172..734daf1 100644 --- a/src/cfg-grammar.y +++ b/src/cfg-grammar.y @@ -62,7 +62,7 @@ gint last_addr_family = AF_INET; %token KW_USER KW_DOOR KW_SUN_STREAMS KW_PROGRAM /* option items */ -%token KW_FSYNC KW_MARK_FREQ KW_STATS_FREQ KW_FLUSH_LINES KW_FLUSH_TIMEOUT KW_LOG_MSG_SIZE KW_FILE_TEMPLATE KW_PROTO_TEMPLATE +%token KW_FSYNC KW_MARK_FREQ KW_STATS_FREQ KW_FLUSH_LINES KW_SUPPRESS KW_FLUSH_TIMEOUT KW_LOG_MSG_SIZE KW_FILE_TEMPLATE KW_PROTO_TEMPLATE %token KW_CHAIN_HOSTNAMES KW_NORMALIZE_HOSTNAMES KW_KEEP_HOSTNAME KW_CHECK_HOSTNAME KW_BAD_HOSTNAME %token KW_KEEP_TIMESTAMP @@ -671,6 +671,7 @@ dest_writer_option : KW_FLAGS '(' dest_writer_options_flags ')' { last_writer_options->options = $3; } | KW_LOG_FIFO_SIZE '(' NUMBER ')' { last_writer_options->fifo_size = $3; } | KW_FLUSH_LINES '(' NUMBER ')' { last_writer_options->flush_lines = $3; } + | KW_SUPPRESS '(' NUMBER ')' { last_writer_options->suppress= $3; } | KW_FLUSH_TIMEOUT '(' NUMBER ')' { last_writer_options->flush_timeout = $3; } | KW_TEMPLATE '(' string ')' { last_writer_options->template = cfg_lookup_template(configuration, $3); diff --git a/src/cfg-lex.l b/src/cfg-lex.l index 5c5f54f..0190278 100644 --- a/src/cfg-lex.l +++ b/src/cfg-lex.l @@ -6,7 +6,7 @@ /*************************************************************************** * - * Copyright (c) 1999 Balzs Scheidler + * Copyright (c) 1999 Balázs Scheidler * Copyright (c) 1999-2007 BalaBit IT Ltd. * * This program is free software; you can redistribute it and/or modify @@ -86,6 +86,7 @@ static struct keyword keywords[] = { { "flush_timeout", KW_FLUSH_TIMEOUT }, { "sync_freq", KW_FLUSH_LINES }, /* obsolete */ { "sync", KW_FLUSH_LINES }, /* obsolete */ + { "suppress", KW_SUPPRESS }, { "fsync", KW_FSYNC }, { "long_hostnames", KW_CHAIN_HOSTNAMES }, { "chain_hostnames", KW_CHAIN_HOSTNAMES }, diff --git a/src/logmsg.c b/src/logmsg.c index 05354b5..9219a01 100644 --- a/src/logmsg.c +++ b/src/logmsg.c @@ -653,6 +653,25 @@ log_msg_new(gchar *msg, gint length, GSockAddr *saddr, guint flags, regex_t *bad } /** + * log_msg_new_internal: + * @prio: message priority (LOG_*) + * @msg: message text + * @flags: parse flags (LP_*) + * + * This function creates a new log message for messages originating + * internally to syslog-ng + **/ +LogMessage * +log_msg_new_internal(gint prio, gchar *msg, guint flags) +{ + gchar *buf = g_strdup_printf("<%d> syslog-ng[%d]: %s\n", prio, getpid(), msg); + LogMessage *self = log_msg_new(buf, strlen(buf), NULL, flags, NULL); + g_free(buf); + + return self; +} + +/** * log_msg_new_mark: * * This function returns a new MARK message. MARK messages have the LF_MARK diff --git a/src/logmsg.h b/src/logmsg.h index 420a711..d780282 100644 --- a/src/logmsg.h +++ b/src/logmsg.h @@ -100,7 +100,7 @@ void log_msg_unref(LogMessage *m); LogMessage *log_msg_new(gchar *msg, gint length, GSockAddr *saddr, guint flags, regex_t *bad_hostname); LogMessage *log_msg_new_mark(void); - +LogMessage *log_msg_new_internal(gint prio, gchar *msg, guint flags); void log_msg_ack_block_inc(LogMessage *m); void log_msg_ack_block_start(LogMessage *m, LMAckFunc func, gpointer user_data); void log_msg_ack_block_end(LogMessage *m); diff --git a/src/logwriter.c b/src/logwriter.c index bb82b43..f831d93 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -179,10 +179,104 @@ log_writer_watch_new(LogWriter *writer, FDWrite *fd) } static void +log_writer_last_msg_release(LogWriter *self) +{ + if(self->last_msg_timerid) + g_source_remove(self->last_msg_timerid); + + if(self->last_msg) + log_msg_unref(self->last_msg); + + self->last_msg = NULL; + self->last_msg_count = 0; + self->last_msg_timerid = 0; +} + +static void +log_writer_last_msg_flush(LogWriter *self) +{ + gchar *msg; + LogMessage *m; + + msg = g_strdup_printf("Last message '%.20s' repeated %d times\n", + self->last_msg->msg.str, + self->last_msg_count); + + m = log_msg_new_internal(self->last_msg->pri, msg, 0); + + g_string_assign(&m->host,self->last_msg->host.str ); + g_string_assign(&m->host_from,self->last_msg->host_from.str ); + g_queue_push_tail(self->queue, m); + g_queue_push_tail(self->queue, GUINT_TO_POINTER(0x80000000)); + g_free(msg); + + log_writer_last_msg_release(self); +} + +static gboolean +last_msg_timer(gpointer pt) +{ + LogWriter *self = (LogWriter *)pt; + + if(self->last_msg_count) + log_writer_last_msg_flush(self); + else + log_writer_last_msg_release(self); + + return FALSE; +} + +static void +log_writer_last_msg_record(LogWriter *self, LogMessage *lm) +{ + if(self->last_msg) + log_msg_unref(self->last_msg); + + log_msg_ref(lm); + self->last_msg = lm; + self->last_msg_count = 0; + self->last_msg_timerid = g_timeout_add(self->options->suppress * 1000, last_msg_timer, self); +} + +static gboolean +log_writer_last_msg_check(LogWriter *self, LogMessage *lm, gint path_flags) +{ + if (self->options->suppress <= 0) + return FALSE; + + if(self->last_msg) + { + if(strcmp(self->last_msg->msg.str, lm->msg.str) == 0 && + strcmp(self->last_msg->host.str, lm->host.str) == 0) + { + if (self->dropped_messages) + (*self->dropped_messages)++; + msg_debug("Dropping duplicate message", + NULL); + self->last_msg_count++; + log_msg_drop(lm, path_flags); + return TRUE; + } + + if (self->last_msg_count) + log_writer_last_msg_flush(self); + else + log_writer_last_msg_release(self); + } + + log_writer_last_msg_record(self, lm); + + return FALSE; +} + +static void log_writer_queue(LogPipe *s, LogMessage *lm, gint path_flags) { LogWriter *self = (LogWriter *) s; - + + if(log_writer_last_msg_check(self, lm, path_flags)) + return; + if ((self->queue->length / 2) == self->options->fifo_size) { /* drop incoming message, we must ack here, otherwise the sender might @@ -415,6 +509,7 @@ log_writer_free(LogPipe *s) log_msg_unref(lm); } g_queue_free(self->queue); + log_writer_last_msg_release(self); g_free(self); } @@ -462,6 +557,9 @@ log_writer_new(guint32 flags, LogPipe *control, LogWriterOptions *options) self->queue = g_queue_new(); self->flags = flags; self->control = control; + self->last_msg = NULL; + self->last_msg_count = 0; + self->last_msg_timerid = 0; return &self->super; } @@ -476,6 +574,7 @@ log_writer_options_defaults(LogWriterOptions *options) options->ts_format = -1; options->zone_offset = -1; options->frac_digits = -1; + options->suppress = 0; } void diff --git a/src/logwriter.h b/src/logwriter.h index 284899b..cf6074b 100644 --- a/src/logwriter.h +++ b/src/logwriter.h @@ -66,6 +66,7 @@ typedef struct _LogWriterOptions gshort ts_format; glong zone_offset; gshort frac_digits; + gint suppress; } LogWriterOptions; typedef struct _LogWriter @@ -79,6 +80,10 @@ typedef struct _LogWriter gint partial_pos; LogPipe *control; LogWriterOptions *options; + LogMessage *last_msg; + guint32 last_msg_count; + guint last_msg_timerid; + } LogWriter; void log_writer_set_options(LogWriter *self, LogPipe *control, LogWriterOptions *options); diff --git a/src/messages.c b/src/messages.c index 6c69f48..206206a 100644 --- a/src/messages.c +++ b/src/messages.c @@ -56,10 +56,8 @@ msg_send_internal_message(int prio, const char *msg) if (G_LIKELY(internal_msg_queue)) { - buf = g_strdup_printf("<%d> syslog-ng[%d]: %s\n", prio, getpid(), msg); - m = log_msg_new(buf, strlen(buf), NULL, LP_INTERNAL | LP_LOCAL, NULL); + m = log_msg_new_internal(prio, msg, LP_INTERNAL | LP_LOCAL); g_queue_push_tail(internal_msg_queue, m); - g_free(buf); } } } -- 1.5.4.5

2 1

syslog_ng and NFS
by Chris Wagner 09 Jun '08

09 Jun '08

We run syslog-ng (both 1.x and now 2.0.9) on FreeBSD writing log files to a NetApp NFS server. At a very low load we start dropping inbound packets (udp, large socket buffers). A quick tcpdump showed a pathological interaction with NFS revolving around how NFS (at least on BSD) deals with APPEND mode. The upshot is that for each write, it flushes the attributes and actually re-reads the partial file block back from the server! A simple change to the source to remove O_APPEND had great results we can handle 10x the number of inbound messages. Our .conf file has every logging host going to a separate file, and with a quick scan of the code it sure looked like syslog-ng won¹t open the same file more than once anyway (I could be wrong here). So, a couple questions (I scanned the last few months of archives but didn¹t see any discussion about this) - what breaks if anything by turning off APPEND, is this interesting to others/how might I get this into future releases ( I can see this being a per-destination option). thanks. -- It is the mark of an instructed mind to rest satisfied with the degree of precision which the nature of a subject admits, and not to seek exactness when only an approximation of the truth is possible. Aristotle

2 3

Re: [syslog-ng] Collecting logs from a chrooted appli
by Gault Stephane 06 Jun '08

06 Jun '08

Hi, > Well, /dev/log shouldn't be a directory. It should be a socket which programs could use by calling openlog/sendlog libc functions. syslog-ng will create the socket for you, you shouldn't do it yourself. I setup syslog-ng.conf with this setings to let create the stream: source src_chroot_appli { unix-stream("/var/appli/dev/log"); }; It works and create the socket. then i config the application log stream path to /dev/log (as the application is chrooted, the root directory is already /var/appli, so logging to /dev/log is ok ) then the configuration to log it in the appli.log file in the /var/log/ outside the chroot : destination appli_log { file("/var/log/appli.log"); }; then log { source(src_chroot_appli); destination(appli_log); }; But nothing is created in the /var/log directory (outside chroot). Btw: syslog-ng version is 1.6.9 Best regards, S.Gautch _________________________________________________________________ Caroline vient de mettre à jour son profil Messenger ! Connectez-vous ! http://login.live.com/login.srf?wa=wsignin1.0&rpsnv=10&ct=1198837564&rver=4…

2 4

Collecting logs from a chrooted appli
by Gault Stephane 05 Jun '08

05 Jun '08

Hi, i'm curretly trying to get the logs of a chrooted application wich is under (/var/appli/) and logging in a file called appli.log (/var/appli/dev/log/appli.log). I tryed to create a /dev/log directory (and a /dev/log stream too, when the file didn't work) where the appli is logging. The application logging is ok, the file or stream work, but syslog-ng can't duplicate and create the appli.log in /var/log/appli.log or send tcp to the remote syslog server as well. In the syslog-ng.conf i Tryed to add a local source : source src_applilog { file("/var/appli/dev/log/appli.log"); }; (or for the stream) source src_applilog { unix-stream("/dev/log"); }; then duplicating the log : destination appli { file("/var/log/appli.log"); }; then statement : log { source(src_applilog); destination(appli); }; I don't want to filter anything in this log, but even if a tryed tu setup a unix-stream in /var/appli/dev/log with the application logging in this stream, nothing get the job done. I must duplicate the log outside the chroot (local then remote with a tcp syslog server listening). Any clue of what am i doing wrong ? Best regards. S.gautch _________________________________________________________________ Votre contact a choisi Hotmail, l'e-mail ultra sécurisé. Créez un compte gratuitement ! http://www.windowslive.fr/hotmail/default.asp

2 1

How many host by relay?
by olivier "madmax"rolland 05 Jun '08

05 Jun '08

Hello, I would like to know how many hosts can be relayed by a syslog-ng relay with the 1.6.8 version of syslog-ng. For the moment I use a relay for 200 hosts but at least I will collect logs of 3000 hosts. How many relay do you think I will need? Thank for your help. __________________________________________________ Do You Yahoo!? En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités http://mail.yahoo.fr Yahoo! Mail

2 1

log route problem
by Marcin Orlowski 05 Jun '08

05 Jun '08

Hi, I setup client with mostly default settings. To log to remote server I added (remote server listens on 514 as by default): log { destination(d_remote); flags(catchall); }; this used to work pretty ok. So now I wanted to log xferlog too. Set the source: source s_ftp_xfer { file("/var/log/proftpd/xferlog" follow_freq(2) flags(no-parse)); }; destination d_remote_515 { tcp('x.x.x.x' port(515)); }; log { source(s_ftp_xfer); destination(d_remote_515); }; Port 515 is intentional here for other reasons. On the server: source s_xferlog {tcp( ip(0.0.0.0) port(515)); }; template t_ftp { template("$MSG\n"); template_escape(no); }; destination df_ftp_xfer { ("/<PATH>/xferlog" template(t_ftp)); }; filter f_proftpd { program('^proftpd$'); }; log { source(s_xferlog); filter( f_proftpd ); destination(df_ftp_xfer); flags(final); }; My problem is that everytime xferlog entry arrives to the server it's being added to xferlog as expected, but it also goes into syslog and console. I am stuck to find out why. There's no other rule that touches s_xferlog on the server. Any hints appreciated. Marcin

2 1

Linking syslog-ng statically error
by Andrej Lobovski 05 Jun '08

05 Jun '08

Hello. While dynamic linking works, for some reason I need static version. Well when I try it I get output like this: checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for gcc option to accept ANSI C... none needed checking for bison... bison -y checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking whether make sets $(MAKE)... (cached) yes checking for ranlib... ranlib checking how to enable static linking for certain libraries... GNU or Solaris checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking dmalloc.h usability... no checking dmalloc.h presence... no checking for dmalloc.h... no checking for strings.h... (cached) yes checking openssl/ssl.h usability... no checking openssl/ssl.h presence... no checking for openssl/ssl.h... no checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking stropts.h usability... yes checking stropts.h presence... yes checking for stropts.h... yes checking sys/strlog.h usability... no checking sys/strlog.h presence... no checking for sys/strlog.h... no checking door.h usability... no checking door.h presence... no checking for door.h... no checking tcpd.h usability... no checking tcpd.h presence... no checking for tcpd.h... no checking whether struct tm is in sys/time.h or time.h... time.h checking for struct tm.tm_gmtoff... yes checking for I_CONSLOG... no checking for O_LARGEFILE... yes checking for struct sockaddr_storage... yes checking for struct sockaddr_in6... yes checking for door_create in -ldoor... no checking for socket in -lsocket... no checking for nanosleep in -lrt... yes checking for gethostbyname in -lnsl... yes checking for regexec in -lregex... no checking for res_init in -lresolv... no checking for strdup... yes checking for strtol... yes checking for strtoll... yes checking for strtoimax... yes checking for inet_aton... yes checking for inet_ntoa... yes checking for getopt_long... yes checking for getaddrinfo... yes checking for getutent... yes checking for TCP wrapper library... checking for LIBNET... yes checking whether to enable Sun STREAMS support... no checking whether to enable Sun door support... no checking whether to enable TCP wrapper support... no checking whether to enable spoof source support... yes checking whether to enable IPv6 support... yes checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for GLIB... yes checking for EVTLOG... yes checking for static GLib libraries... yes checking sanity checking Glib headers... yes configure: creating ./config.status config.status: creating dist.conf config.status: creating Makefile config.status: creating syslog-ng.spec.bb config.status: creating src/Makefile config.status: creating doc/Makefile config.status: creating doc/docvars.xml config.status: creating contrib/Makefile config.status: creating tests/Makefile config.status: creating solbuild/Makefile config.status: creating tgzbuild/Makefile config.status: creating tests/unit/Makefile config.status: creating tests/functional/Makefile config.status: creating tests/loggen/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands and the make command gives me macros.c: In function ÔÇśresult_appendÔÇÖ: macros.c:136: warning: pointer targets in passing argument 2 of ÔÇśg_string_append_lenÔÇÖ differ in signedness macros.c: In function ÔÇślog_macro_expandÔÇÖ: macros.c:148: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:212: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:232: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:240: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:258: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:403: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:422: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness macros.c:440: warning: pointer targets in passing argument 2 of ÔÇśresult_appendÔÇÖ differ in signedness logmsg.c: In function ÔÇślog_msg_parseÔÇÖ: logmsg.c:151: warning: pointer targets in assignment differ in signedness logmsg.c:216: warning: pointer targets in passing argument 2 of ÔÇśg_string_assign_lenÔÇÖ differ in signedness logmsg.c:224: warning: pointer targets in assignment differ in signedness logmsg.c:242: warning: pointer targets in passing argument 1 of ÔÇśstrlenÔÇÖ differ in signedness logmsg.c:283: warning: pointer targets in passing argument 2 of ÔÇśg_string_assign_lenÔÇÖ differ in signedness logmsg.c:309: warning: pointer targets in passing argument 2 of ÔÇśg_string_assign_lenÔÇÖ differ in signedness logmsg.c:335: warning: pointer targets in passing argument 2 of ÔÇśg_string_assign_lenÔÇÖ differ in signedness logmsg.c:394: warning: pointer targets in assignment differ in signedness logmsg.c:398: warning: pointer targets in assignment differ in signedness logmsg.c:429: warning: pointer targets in assignment differ in signedness logmsg.c:463: warning: pointer targets in assignment differ in signedness logmsg.c:474: warning: pointer targets in assignment differ in signedness logmsg.c:486: warning: pointer targets in assignment differ in signedness logmsg.c:499: warning: pointer targets in assignment differ in signedness logmsg.c:520: warning: pointer targets in assignment differ in signedness logmsg.c:525: warning: pointer targets in assignment differ in signedness logmsg.c:530: warning: pointer targets in passing argument 2 of ÔÇśg_string_assign_lenÔÇÖ differ in signedness main.o: In function `setup_creds': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/main.c:462: warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o): In function `resolve_group': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/misc.c:274: warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o): In function `resolve_user': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/misc.c:254: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gutils.o): In function `g_get_any_init_do': (.text+0x141e): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gutils.o): In function `g_get_any_init_do': (.text+0x1411): warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gutils.o): In function `g_get_any_init_do': (.text+0x1426): warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gutils.o): In function `g_get_any_init_do': (.text+0x115a): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gutils.o): In function `g_get_any_init_do': (.text+0x1198): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(afinet.o): In function `afinet_resolve_name': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/afinet.c:103: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o): In function `resolve_hostname': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/misc.c:165: warning: Using 'gethostbyaddr' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o): In function `getlonghostname': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/misc.c:70: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libnet.a(libnet_resolve.o): In function `libnet_name2addr6': (.text+0x453): warning: Using 'gethostbyname2' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(afinet.o): In function `afinet_set_port': /root/syslog-za-oddat/MYKGXX90/rpmbuild/SOURCES/syslog-ng-2.0.8/src/afinet.c:58: warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o): In function `g_timer_elapsed': (.text+0x220): undefined reference to `clock_gettime' /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o): In function `g_timer_continue': (.text+0x2a4): undefined reference to `clock_gettime' /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o): In function `g_timer_reset': (.text+0x34a): undefined reference to `clock_gettime' /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o): In function `g_timer_stop': (.text+0x391): undefined reference to `clock_gettime' /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o): In function `g_timer_start': (.text+0x3de): undefined reference to `clock_gettime' /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libglib-2.0.a(gtimer.o):(.text+0x49f): more undefined references to `clock_gettime' follow collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 I googled for this problem and found nothing. :( Please help! _________________________________________________________________ Connect to the next generation of MSN Messenger http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=…

2 1

Flow-control behaviour and Cached lines timestamping, plus a suspend error in 2.1beta1
by Behal, Pavel 05 Jun '08

05 Jun '08

Dear Bazsi, I have found two "features" and one bug I would like to discuss a little: 1) We use the syslog-ng 2.1alpha1 as a root system syslog. The problem is, we have had two destinations setup. First destination was the traditional write of system messages to the file like "/var/log/messages" etc. The second destination path was to the TCP stream to the central syslog server. And the problem is, we have used the flow-control flag on the central TCP path. But, when the central TCP syslog-ng becomes unavailable, after all the buffers filled up, the local system syslog-ng stopped processing any system messages. It did not write anything through the local file destinations until the TCP communication was re-established. Is this a correct behaviour? We have found, that to allow smooth processing of local system sources, we can not use flow-control flag on any destination paths it used.. 2) When messages are in syslog-ng buffer, they do not contain any timestamp. We have realized, that the timestamp is added at the time the syslog-ng flushes all the buffers to the disk. This is not very good in case you need to debug the messages ages and all of them have same and non-related timestamp. Is it possible to modify the logic of syslog-ng to timestamp the messages as soon as possible - even before the buffer, at the collection time? We have found it important for log integrity reasons. 3) Directly in the 2.1beta1 there has been introduced some error with the "suspend writing a destination file when an I/O error ..." patch. During our tests we have found it continously thinks the destination is unavailable and writes strange messages into to the internal log: May 29 15:00:02 europa syslog-ng[16798]: I/O error occurred while writing; fd='7', error='Connection timed out (110)' May 29 15:00:02 europa syslog-ng[16798]: Connection broken; time_reopen='60' May 29 15:00:02 europa syslog-ng[16798]: Suspending write operation because of an I/O error; fd='7', time_reopen='60' the lsof -p 16798: COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME syslog-ng 16798 root cwd DIR 253,0 4096 2 / syslog-ng 16798 root rtd DIR 253,0 4096 2 / syslog-ng 16798 root txt REG 253,0 114584 671841 /sbin/syslog-ng syslog-ng 16798 root mem REG 253,0 106397 820372 /lib/ld-2.3.4.so syslog-ng 16798 root mem REG 253,0 1454546 820373 /lib/tls/libc-2.3.4.so syslog-ng 16798 root mem REG 253,0 28504 2919694 /usr/lib/libwrap.so.0.7.6 syslog-ng 16798 root mem REG 253,0 93985 820376 /lib/tls/libpthread-2.3.4.so syslog-ng 16798 root mem REG 253,0 47671 820386 /lib/tls/librt-2.3.4.so syslog-ng 16798 root mem REG 253,0 505200 2930064 /usr/lib/libglib-2.0.so.0.400.7 syslog-ng 16798 root mem REG 253,0 95148 820384 /lib/libnsl-2.3.4.so syslog-ng 16798 root mem REG 253,0 36639 2920956 /usr/local/lib/libevtlog.so.0.0.0 syslog-ng 16798 root 0r CHR 1,3 1619 /dev/null syslog-ng 16798 root 1w CHR 1,3 1619 /dev/null syslog-ng 16798 root 2w CHR 1,3 1619 /dev/null syslog-ng 16798 root 3r REG 0,3 0 4026531850 /proc/kmsg syslog-ng 16798 root 4u unix 0xc1be7ba0 351802 /dev/log syslog-ng 16798 root 5u IPv4 351804 TCP europa.training.hp.com:37205->europa.training.hp.com:5140 (ESTABLISHED) syslog-ng 16798 root 6w REG 253,0 4802 1655155 /var/log/messages syslog-ng 16798 root 8w REG 253,0 525 1655159 /var/log/boot.log This happens during standard system syslog processing on RH ELinux 4 and Solaris 9. You have the syslog-ng confs attached. Could you pleae check it? Best Regards, Pavel

3 3

Re: [syslog-ng] file() sourcing application logs
by Alan Edmonds 04 Jun '08

04 Jun '08

We are doing something similar; we use tomcat and apache. For apache, something like this (we are using 2.0): CustomLog "|/usr/bin/logger -p local1.info -t apache-access " access-format Then on syslog-ng I use the "tag" to point it to the right log file. For tomcat, that is tougher; stack dumps are a curse to syslog. There is a hook in the Catalina startup scripts to create a named pipe for gc.out and tomcat.out. The it spawns a logger -p local2.info < gc.out-named-pipe thingy to send to syslog-ng. I can send more details if you need them. Alan T-Mobile International UK Limited Company Registered Number: 3951860 Registered Office Address: Hatfield Business Park, Hatfield, Hertfordshire, AL10 9BW Registered in England and Wales NOTICE AND DISCLAIMER This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose

2 1

file() sourcing application logs
by Chris Pratt 04 Jun '08

04 Jun '08

Hi syslog-ng, Been all over google and still not found what I need so I'm hoping to get some help here... I have syslog-ng capturing and duplicating all of my *normal* /var/logs. I still need to capture some application specific logs that are not using syslog at all currently but dumping to text in application log directories. I wanted to use the file() source but I am not getting the results I wanted. I have tried both of these options to get these files logged: # See http://www.syslog.org/syslog-ng/v2/#id2535001 # tail -f [file log] | logger -p local#.info # # See http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch09s01.html # source s_tail { file("/var/log/apache/access.log" follow_freq(1) flags(no-parse)); }; but I am running into two issues: 1.) Either above way of sourcing the files results in each line in my application log showing up as a separate syslog event even though it is a single application event that wraps lines in its own application log. For example: {Heap before GC invocations=1087 (full 0): par new generation total 307200K, used 219907K [0x5af10000, 0x73f10000, 0x73f10000) eden space 204800K, 100% used [0x5af10000, 0x67710000, 0x67710000) from space 102400K, 14% used [0x6db10000, 0x6e9d0d38, 0x73f10000) to space 102400K, 0% used [0x67710000, 0x67710000, 0x6db10000) concurrent mark-sweep generation total 991232K, used 116200K [0x73f10000, 0xb0710000, 0xb0710000) concurrent-mark-sweep perm gen total 71680K, used 46017K [0xb0710000, 0xb4d10000, 0xb4d10000) 27877.615: [GC 27877.615: [ParNew Desired survivor size 94371840 bytes, new threshold 15 (max 15) - age 1: 32802608 bytes, 32802608 total - age 2: 604752 bytes, 33407360 total - age 3: 40968 bytes, 33448328 total - age 4: 50728 bytes, 33499056 total - age 5: 54952 bytes, 33554008 total - age 6: 36096 bytes, 33590104 total - age 7: 47128 bytes, 33637232 total - age 8: 44048 bytes, 33681280 total - age 9: 40976 bytes, 33722256 total - age 10: 53912 bytes, 33776168 total - age 11: 63680 bytes, 33839848 total - age 12: 90944 bytes, 33930792 total - age 13: 33560 bytes, 33964352 total - age 14: 60552 bytes, 34024904 total - age 15: 59328 bytes, 34084232 total : 219907K->33808K(307200K), 0.0730690 secs] 336108K->150064K(1298432K), 0.0732270 secs] Heap after GC invocations=1088 (full 0): par new generation total 307200K, used 33808K [0x5af10000, 0x73f10000, 0x73f10000) eden space 204800K, 0% used [0x5af10000, 0x5af10000, 0x67710000) from space 102400K, 33% used [0x67710000, 0x69814218, 0x6db10000) to space 102400K, 0% used [0x6db10000, 0x6db10000, 0x73f10000) concurrent mark-sweep generation total 991232K, used 116256K [0x73f10000, 0xb0710000, 0xb0710000) concurrent-mark-sweep perm gen total 71680K, used 46017K [0xb0710000, 0xb4d10000, 0xb4d10000) } becomes something like... 2008-06-03T14:27:40-04:00 s_catalinag@tomcat {Heap before GC invocations=1087 (full 0): 2008-06-03T14:27:40-04:00 s_catalinag@tomcat par new generation total 307200K, used 219907K [0x5af10000, 0x73f10000, 0x73f10000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat eden space 204800K, 100% used [0x5af10000, 0x67710000, 0x67710000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat from space 102400K, 14% used [0x6db10000, 0x6e9d0d38, 0x73f10000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat to space 102400K, 0% used [0x67710000, 0x67710000, 0x6db10000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat concurrent mark-sweep generation total 991232K, used 116200K [0x73f10000, 0xb0710000, 0xb0710000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat concurrent-mark-sweep perm gen total 71680K, used 46017K [0xb0710000, 0xb4d10000, 0xb4d10000) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat 27877.615: [GC 27877.615: [ParNew 2008-06-03T14:27:40-04:00 s_catalinag@tomcat Desired survivor size 94371840 bytes, new threshold 15 (max 15) 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 1: 32802608 bytes, 32802608 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 2: 604752 bytes, 33407360 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 3: 40968 bytes, 33448328 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 4: 50728 bytes, 33499056 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 5: 54952 bytes, 33554008 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 6: 36096 bytes, 33590104 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 7: 47128 bytes, 33637232 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 8: 44048 bytes, 33681280 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 9: 40976 bytes, 33722256 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 10: 53912 bytes, 33776168 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 11: 63680 bytes, 33839848 total 2008-06-03T14:27:40-04:00 s_catalinag@tomcat - age 12: 90944 bytes, 33930792 total 2008-06-03T14:27:41-04:00 s_catalinag@tomcat - age 13: 33560 bytes, 33964352 total 2008-06-03T14:27:41-04:00 s_catalinag@tomcat - age 14: 60552 bytes, 34024904 total 2008-06-03T14:27:41-04:00 s_catalinag@tomcat - age 15: 59328 bytes, 34084232 total 2008-06-03T14:27:41-04:00 s_catalinag@tomcat : 219907K->33808K(307200K), 0.0730690 secs] 336108K->150064K(1298432K), 0.0732270 secs] 2008-06-03T14:27:41-04:00 s_catalinag@tomcat Heap after GC invocations=1088 (full 0): 2008-06-03T14:27:41-04:00 s_catalinag@tomcat par new generation total 307200K, used 33808K [0x5af10000, 0x73f10000, 0x73f10000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat eden space 204800K, 0% used [0x5af10000, 0x5af10000, 0x67710000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat from space 102400K, 33% used [0x67710000, 0x69814218, 0x6db10000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat to space 102400K, 0% used [0x6db10000, 0x6db10000, 0x73f10000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat concurrent mark-sweep generation total 991232K, used 116256K [0x73f10000, 0xb0710000, 0xb0710000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat concurrent-mark-sweep perm gen total 71680K, used 46017K [0xb0710000, 0xb4d10000, 0xb4d10000) 2008-06-03T14:27:41-04:00 s_catalinag@tomcat } 2.) I also have a couple of log files that use the date in their name but from what I can tell macros only work with destinations, not sources? I'd like to do something like: #source s_tomcat { file("/usr/local/server/apache-tomcat-5.5.20/logs/tomcat.${YEAR}-${MONTH}-${DAY}.log"); }; to accommodate this. Anyone have any similar app log setup and a solution to these issues? TIA, Chris

1 0

Source optional() option causes syntax error
by Clay Loveless 04 Jun '08

04 Jun '08

Reposting in its own thread: I'm using a pipe source, and would like to make it optional, however the optional(yes) parameter seems to be considered a syntax error. Here's the entry: source test { pipe("/tmp/test_fifo" optional(yes) flags(no-parse) log_msg_size(1024)); }; Am I doing this incorrectly? This line is considered a syntax error by syslog-ng 2.0.9. -Clay -- Killersoft.com

1 0

Re: [syslog-ng] (no subject)
by Ivey, Chris 04 Jun '08

04 Jun '08

I ran into a similar issue with Netcool from syslog-ng, thinking the microsecond portion of the message was the hostname. What we found out is that there is a bug in the WCs. If you upgrade your WCs, you will see the hostnames coming in with your syslog messages (which you don't currently get, hence why your system is not working as expected). Hope that helps!! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "I have not failed, I have simply found 10,000 ways which do not work!" -- Thomas Edison "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of chris packham Sent: Tuesday, June 03, 2008 4:59 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] (no subject) What version of syslog-ng does your distro use? (run syslog-ng --version to find out) >From looking at the source code history support for this Cisco extension was added in v2.0.5. I haven't got a Cisco device handy so I can't confirm that it is working but there is code to deal with the fraction of a second scenario. -----Original Message----- From: Andy Kanyer <AKanyer(a)directs.com> Reply-To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] (no subject) Date: Tue, 3 Jun 2008 09:16:30 -0500 Hello everyone, I am currently workin on setting up a debian box as a central syslog server. One goal of this server is to filter syslog messages into different folders based on what server they were sent by. This works as expected with all devices EXCEPT for my cisco wireless controllers. When they log: [CODE] Jun 02 20:52:29.063 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f Jun 02 20:52:09.663 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:13:e8:d9:9d:eb Jun 02 20:50:49.064 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f [/CODE] My syslog-ng, rather than filtering these into a folder given by the wireless controllers ip/hostname, creates folders named .063 and .064. It seems like syslog-ng is reading the microsecond portion of the timestamp as the hostname! After sniffing some other syslog messages, I noticed that ONLY these cisco devices have timestamps that include microseconds. Does anyone have any idea how to work around this and filter the cisco messages by hostname? Someway to truncate the timestamp or force it to look further to find the actual hostname? Just FYI I will post the relevant portion of my syslog-ng.conf: [CODE] source remote_src { udp(); tcp(); }; destination remote_syslog { file( "/usr/local/syslog/$HOST/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; ### added 4-16-08 all three wireless controllers destinations ### manually creating directory names destination wir-c-syd-3-00 { file( "/usr/local/syslog/wir-c-syd-3-00/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; filter wir-c-syd-3-00 { netmask(172.25.198.10/32); }; log { source(remote_src); filter(wir-c-syd-3-00); destination(wir-c-syd-3-00); }; [/CODE] ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

local6 doesn't seem to work...
by ruud.feringa＠mondrian.nl 04 Jun '08

04 Jun '08

Hi, When I have the following in syslog-ng.conf: filter f_cyrus { level(warn, err, crit) and facility(local6); }; destination cyrus { file("/var/log/cyrus.log"); }; log { source(src); filter(f_cyrus); destination(cyrus); }; nothing is being logged in /var/log/cyrus.log however when I use: filter f_cyrus { level(warn, err, crit) and match("imap"); }; it works fine, why is local6 not working?? (cyrus-imap is default compiled with local6 enabled) Running on Suse 10.0 ( Cannot upgrade yet :-( ) with syslog-ng version 1.6.8 Greetings, Ruud Feringa.

3 3

syslog-ng to trigger rancid
by Mladen Tsvetkov 04 Jun '08

04 Jun '08

Hi, Guys, I am new to syslog-ng. I have successfully installed it and integrated with mysql and php. My question is how can I make syslog-ng to start rancid when a syslog configuration message is received from a cisco device: 112578: 1564643: May 15 09:05:19.141 EEST: %SYS-5-CONFIG_I: Configured from console by ******** on vty0 (10.10.10.10). So far I know that I have to use the program() directive. But how can I pass the hostname and other parameters to rancid? Thanks in advance, Longman

3 2

Re: [syslog-ng] (no subject)
by chris packham 03 Jun '08

03 Jun '08

What version of syslog-ng does your distro use? (run syslog-ng --version to find out) >From looking at the source code history support for this Cisco extension was added in v2.0.5. I haven't got a Cisco device handy so I can't confirm that it is working but there is code to deal with the fraction of a second scenario. -----Original Message----- From: Andy Kanyer <AKanyer(a)directs.com> Reply-To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] (no subject) Date: Tue, 3 Jun 2008 09:16:30 -0500 Hello everyone, I am currently workin on setting up a debian box as a central syslog server. One goal of this server is to filter syslog messages into different folders based on what server they were sent by. This works as expected with all devices EXCEPT for my cisco wireless controllers. When they log: [CODE] Jun 02 20:52:29.063 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f Jun 02 20:52:09.663 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:13:e8:d9:9d:eb Jun 02 20:50:49.064 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f [/CODE] My syslog-ng, rather than filtering these into a folder given by the wireless controllers ip/hostname, creates folders named .063 and .064. It seems like syslog-ng is reading the microsecond portion of the timestamp as the hostname! After sniffing some other syslog messages, I noticed that ONLY these cisco devices have timestamps that include microseconds. Does anyone have any idea how to work around this and filter the cisco messages by hostname? Someway to truncate the timestamp or force it to look further to find the actual hostname? Just FYI I will post the relevant portion of my syslog-ng.conf: [CODE] source remote_src { udp(); tcp(); }; destination remote_syslog { file( "/usr/local/syslog/$HOST/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; ### added 4-16-08 all three wireless controllers destinations ### manually creating directory names destination wir-c-syd-3-00 { file( "/usr/local/syslog/wir-c-syd-3-00/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; filter wir-c-syd-3-00 { netmask(172.25.198.10/32); }; log { source(remote_src); filter(wir-c-syd-3-00); destination(wir-c-syd-3-00); }; [/CODE] ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

(no subject)
by Andy Kanyer 03 Jun '08

03 Jun '08

Hello everyone, I am currently workin on setting up a debian box as a central syslog server. One goal of this server is to filter syslog messages into different folders based on what server they were sent by. This works as expected with all devices EXCEPT for my cisco wireless controllers. When they log: [CODE] Jun 02 20:52:29.063 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f Jun 02 20:52:09.663 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:13:e8:d9:9d:eb Jun 02 20:50:49.064 1x_auth_pae.c:2488 DOT1X-3-MAX_EAP_RETRIES: Max EAP identity request retries (21) exceeded for client 00:19:d2:78:ee:8f [/CODE] My syslog-ng, rather than filtering these into a folder given by the wireless controllers ip/hostname, creates folders named .063 and .064. It seems like syslog-ng is reading the microsecond portion of the timestamp as the hostname! After sniffing some other syslog messages, I noticed that ONLY these cisco devices have timestamps that include microseconds. Does anyone have any idea how to work around this and filter the cisco messages by hostname? Someway to truncate the timestamp or force it to look further to find the actual hostname? Just FYI I will post the relevant portion of my syslog-ng.conf: [CODE] source remote_src { udp(); tcp(); }; destination remote_syslog { file( "/usr/local/syslog/$HOST/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; ### added 4-16-08 all three wireless controllers destinations ### manually creating directory names destination wir-c-syd-3-00 { file( "/usr/local/syslog/wir-c-syd-3-00/$YEAR/$MONTH/$YEAR-$MONTH-$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); }; filter wir-c-syd-3-00 { netmask(172.25.198.10/32); }; log { source(remote_src); filter(wir-c-syd-3-00); destination(wir-c-syd-3-00); }; [/CODE]

1 0

Re: [syslog-ng] Filtering duplicate messages
by chris packham 03 Jun '08

03 Jun '08

Hi Bazsi, Thanks for the feedback. I'll re-submit a new patch shortly. Is the organisational agreement Anthony mentioned good enough (and current?) for you. On Thu, 2008-05-29 at 22:13 +0200, Balazs Scheidler wrote: > Hi Chris, > > Thanks for your contribution. Please find my comments in-line. > > Please note that in order to get this integrated into syslog-ng, you'll > need to sign a "Contributory License Agreement" which is available on > our website here: > > http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… > > Thanks. > > So after the legalese, let's see your notes and patch below. > > On Thu, 2008-05-29 at 12:57 +1200, chris packham wrote: > > Here's a rough cut of my patch. It still needs a bit of cleanup and I > > have yet to implement the time-based behaviour. I just wanted to get > > some > > feedback. > > > > One particular thing I wanted to clarify is where in the log message > > pipeline would best to attach this behaviour to? I've currently > > attached it to the LogWriter. The disadvantage of that is that the new > > message I generate does not get passed through the filters or templates > > (and we've just done a bunch of wasted processing on a message that > > gets dropped) > > There are basically three options: > 1) sources > 2) as a processing element between sources/destinations (for example > filters, LogProcess in syslog-ng 2.2) > 3) in the LogWriter > > All three has pros and cons: > 1) suppress on input: > pros: > * drop messages as early as possible > * every destination would receive the same suppressions from the same > host > cons: > * would need a host-name indexed hashtable to store host specific > suppression info, this needs memory and CPU > > 2) suppress as a processing element: > pros: > * probably the most flexible, the user could choose which log pipes > have suppressions and which don't > cons: > * it requires the same amount of state as suppressing on input > * it is more complex than any other alternative and the flexibility > it provides is probably not needed > > 3) suppress in the LogWriter > pros: > * the easiest to implement > * no need for per-host state, it uses a per-destination state but > that's already available (every destination has a separate writer) > cons: > * different destinations will have a different set of suppressions > depending on the set of sources that feed the given destination > > I'd say that doing it in LogWriter is a sane choice. We could do better > but that'd cost us memory and CPU for an added flexibility which is > probably not needed. > > > > > > The behaviour implemented is as follows. > > > > As a log message is output to a destination it is remembered > > as the last message seen. > > > > When a new message is to be output it is checked against the > > last message seen. If its contents (message text, including > > hostname and pid if present) are the same the message is dropped > > and a couter incremented. If the message is different to the > > last message seen a new log message indicating the value of the > > last message counter will be inserted into the queue and the new > > message will be processed. > > > > TODO: Implement a time based flush of last seen messages > > > > --- > > src/cfg-grammar.y | 3 +- > > src/cfg-lex.l | 3 +- > > src/logmsg.c | 26 +++++++++++++++++++++++ > > src/logmsg.h | 2 + > > src/logwriter.c | 59 > > ++++++++++++++++++++++++++++++++++++++++++++++++++++- > > src/logwriter.h | 9 ++++++++ > > 6 files changed, 99 insertions(+), 3 deletions(-) > > > > diff --git a/src/cfg-grammar.y b/src/cfg-grammar.y > > index 39ff172..386c231 100644 > > --- a/src/cfg-grammar.y > > +++ b/src/cfg-grammar.y > > @@ -62,7 +62,7 @@ gint last_addr_family = AF_INET; > > %token KW_USER KW_DOOR KW_SUN_STREAMS KW_PROGRAM > > > > /* option items */ > > -%token KW_FSYNC KW_MARK_FREQ K> > +%token KW_FSYNC KW_MARK_FREQ KW_STATS_FREQ KW_FLUSH_LINES KW_SUPPRESS > > KW_FLUSH_TIMEOUT KW_LOG_MSG_SIZE KW_FILE_TEMPLATE KW_PROTO_TEMPLATE > > > > %token KW_CHAIN_HOSTNAMES KW_NORMALIZE_HOSTNAMES KW_KEEP_HOSTNAME > > KW_CHECK_HOSTNAME KW_BAD_HOSTNAME > > %token KW_KEEP_TIMESTAMP > > @@ -671,6 +671,7 @@ dest_writer_option > > : KW_FLAGS '(' dest_writer_options_flags ')' { > > last_writer_options->options = $3; } > > | KW_LOG_FIFO_SIZE '(' NUMBER ')' { > > last_writer_options->fifo_size = $3; } > > | KW_FLUSH_LINES '(' NUMBER ')' { > > last_writer_options->flush_lines = $3; } > > + | KW_SUPPRESS '(' yesno ')' { > > last_writer_options->suppress= $3; } > > | KW_FLUSH_TIMEOUT '(' NUMBER ')' { > > last_writer_options->flush_timeout = $3; } > > | KW_TEMPLATE '(' string ')' { > > > > last_writer_options->template = cfg_lookup_template(configuration, $3); > > ok. > > > diff --git a/src/cfg-lex.l b/src/cfg-lex.l > > index 5c5f54f..4eb0c63 100644 > > --- a/src/cfg-lex.l > > +++ b/src/cfg-lex.l > > @@ -6,7 +6,7 @@ > > > > > > /*************************************************************************** > > * > > - * Copyright (c) 1999 Balzs Scheidler > > + * Copyright (c) 1999 Bal�zs Scheidler > > * Copyright (c) 1999-2007 BalaBit IT Ltd. > > * > > * This program is free software; you can redistribute it and/or modify > > @@ -86,6 +86,7 @@ static struct keyword keywords[] = { > > { "flush_timeout", KW_FLUSH_TIMEOUT }, > > { "sync_freq", KW_FLUSH_LINES }, /* obsolete */ > > { "sync", KW_FLUSH_LINES }, /* obsolete */ > > + { "suppress", KW_SUPPRESS }, > > { "fsync", KW_FSYNC }, > > { "long_hostnames", KW_CHAIN_HOSTNAMES }, > > { "chain_hostnames", KW_CHAIN_HOSTNAMES }, > > ok. > > > diff --git a/src/logmsg.c b/src/logmsg.c > > index 05354b5..4da3af5 100644 > > --- a/src/logmsg.c > > +++ b/src/logmsg.c > > @@ -766,3 +766,29 @@ log_msg_drop(LogMessage *m, guint path_flags) > > log_msg_ack(m); > > log_msg_unref(m); > > } > > + > > +/** > > + * log_msg_dup: > > + * @m: LogMessage instance > > + * > > + * This function duplicates an existing log message. The new message is > > not parsed. > > + */ > > +LogMessage * > > +log_msg_dup(LogMessage *m) > > +{ > > + LogMessage *nm = g_new0(LogMessage, 1); > > + > > + log_msg_init(nm, m->saddr); > > + nm->flags > > = m->flags; > > + nm->pri = m->pri; > > + g_string_assign_len(&nm->date, m->date.str, m->date.len); > > + g_string_assign_len(&nm->host, m->host.str, m->host.len); > > + g_string_assign_len(&nm->host_from, m->host_from.str, > > m->host_from.len); > > + g_string_assign_len(&nm->program, m->program.str, m->program.len); > > + g_string_assign_len(&nm->msg, m->msg.str, m->msg.len); > > + memcpy(&nm->stamp, &m->stamp, sizeof(LogStamp)); > > + memcpy(&nm->recvd, &m->recvd, sizeof(LogStamp)); > > + > > + return nm; > > +} > > I don't think this is needed, but see below. > > > + > > diff --git a/src/logmsg.h b/src/logmsg.h > > index 420a711..9dbb886 100644 > > --- a/src/logmsg.h > > +++ b/src/logmsg.h > > @@ -107,6 +107,8 @@ void log_msg_ack_block_end(LogMessage *m); > > void log_msg_ack(LogMessage *msg); > > void log_msg_drop(LogMessage *msg, guint path_flags); > > void log_msg_clear_matches(LogMessage *self); > > +LogMessage *log_msg_dup(LogMessage *m); > > + > > > > gchar *log_msg_find_cr_or_lf(gchar *s, gsize n); > > > > diff --git a/src/logwriter.c b/src/logwriter.c > > index bb82b43..4843100 100644 > > --- a/src/logwriter.c > > +++ b/src/logwriter.c > > @@ -178,11 +178,63 @@ log_writer_watch_new(LogWriter *writer, FDWrite > > *fd) > > return &self->super; > > } > > > > +static gboolean > > +log_writer_supress_duplicate(LogWriter *self, LogMessage *lm, gint > > path_flags) > > +{ > > + if (!self->options->suppress) > > + return FALSE; > > + > > + if(self->suppress.hash) > > + { > > + guint hash = g_string_hash(&lm->msg); > > + if(self->suppress.hash == hash) > > + { > > + msg_debug("Dropping duplicate message", > > + NULL); > > + self->suppress.count ++; > > + log_msg_drop(lm, path_flags); > > + return TRUE; > > + } > > I don't think g_string_hash buys us anything, in fact it might well be > slower than doing a direct string comparison (strcmp is performed in > chunks of sizeof(long) bytes, g_string_hash iterates over characters > individually). > > You mentioned that you are also checking the hostname, which is not the > case, lm->msg contains the application name and pid only. > > I think we should also check that. > > > + > > + if (self->suppress.count) > > + { > > + gchar *buf; > > + LogMessage *m; > > + GString *line = g_string_sized_new(128); > > + > > + buf = g_strdup_printf("<%d> syslog-ng[%d]: Last message > > '%.20s' repeated %d times\n", > > + self->suppress.msg->pri, > > + getpid(), > > + self->suppress.msg->msg.str, > > + self->suppress.count); > > we'd probably need a log_msg_new_internal() function, that would produce > the same message without having to parse it. This and the code in > messages.c would be a direct user of that function. > > > + m = log_msg_new(buf, strlen(buf), self->suppress.msg->saddr, > > 0, NULL); > > + g_queue_push_tail(self->queue, m); > > + g_queue_push_tail(self->queue, GUINT_TO_POINTER(0x80000000)); > > + g_free(buf); > > + } > > + log_msg_drop(self->suppress.msg, 0); > > + self->suppress.msg = log_msg_dup(lm); > > As a LogMessage is read-only (well most of, and I'm directing syslog-ng > 2.2 so that LogMessage is going to be read only), I think it is enough > to store a reference and no need to copy the complete message. This is > too expensive. > > > + self->suppress.hash = g_string_hash(&lm->msg); > > + self->suppress.count = 0; > > + } > > + else > > + { > > + self->suppress.msg = log_msg_dup(lm); > > + self->suppress.hash = g_string_hash(&lm->msg); > > + self->suppress.count = 0; > > + } > > + > > + return FALSE; > > +} > > + > > static void > > log_writer_queue(LogPipe *s, LogMessage *lm, gint path_flags) > > { > > LogWriter *self = (LogWriter *) s; > > - > > + > > + if(log_writer_supress_duplicate(self, lm, path_flags)) > > + return; > > + > > if ((self->queue->length / 2) == self->options->fifo_size) > > { > > /* drop incoming message, we must ack here, otherwise the sender > > might > > @@ -462,6 +514,9 @@ log_writer_new(guint32 flags, LogPipe *control, > > LogWriterOptions *options) > > self->queue = g_queue_new(); > > self->flags = flags; > > self->control = control; > > + self->suppress.msg = NULL; > > + self->suppress.hash = 0; > > + self->suppress.count = 0; > > return &self->super; > > } > > > > @@ -476,6 +531,8 @@ log_writer_options_defaults(LogWriterOptions > > *options) > > options->ts_format = -1; > > options->zone_offset = -1; > > options->frac_digits = -1; > > + options->suppress = FALSE; > > + > > } > > > > void > > diff --git a/src/logwriter.h b/src/logwriter.h > > index 284899b..d3a136f 100644 > > --- a/src/logwriter.h > > +++ b/src/logwriter.h > > @@ -41,6 +41,13 @@ > > /* several writers use the same counter */ > > #define LWOF_SHARE_STATS 0x0002 > > > > +typedef struct _LogSuppress > > +{ > > + LogMessage *msg; > > + guint hash; > > + guint32 count; > > +} LogSuppress; > > As I've removed hash from the picture, I don't think we need a separate > structure for this. Ju> > } LogWriterOptions; > > > > typedef struct _LogWriter > > @@ > > -7 9,6 +87,7 @@ typedef struct _LogWriter > > gint partial_pos; > > LogPipe *control; > > LogWriterOptions *options; > > + LogSuppress suppress; > > } LogWriter; > > > > void log_writer_set_options(LogWriter *self, LogPipe *control, > > LogWriterOptions *options); > > -- > > 1.5.4.5 > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.campin.net/syslog-ng/faq.html > >

2 2