October 2005 - syslog-ng - lists.balabit.hu

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

ignore src{}; contents at startup?
by Brian A. Seklecki 25 Oct '06

25 Oct '06

All: On FreeBSD, /dev/klog provides for output from the kernel (normally redirected to /dev/console before syslog starts). At startup, /etc/rc.d/dmesg copies the kernel boot hardware paramters to /var/run/dmesg.boot. If I have my src(); set to: source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ...at starup, syslog-ng drains the contents of /dev/klog. By default it tsends to them to user.notice. I like to log{} these to an SMS pager (so I know if a RAID is failing or other hardware problem). However, I'd like to avoid duplication of the efforts of /etc/rc.d/dmesg on behalf of syslog-ng(8) ...especially since I route user.notice to my pager, and kernel boot dmesg(8) can grow over 50 lines. /dev/klog really isn't a socket. It's a special char device. Is there anyway to tell syslog-ng(8) to ignore it's contents at startup? If it was tail(1), i'd say "tail -0 -f /file". Perhaps clear out the contents of /dev/klog somehow? TIA, ~BAS

2 2

Pix problem
by Damien Michau 28 Dec '05

28 Dec '05

Hi All ! I have some probleme ta log my pix's log into my syslog-ng server . i have mounted a Syslog-ng server to store my pix's log . But there is nothing in my pix.log i'have put this lines in my syslog-ng.conf source pix { udp(ip(10.60.10.111) port());}; destination pix { file("/var/log/pix.log"); }; log { source(pix);destination(pix); }; Have you one idea ? Thx Damien Michau Paris

4 5

TZ DST offset problem in 1.6.8 syslog-ng using $TZOFFSET or $ISODATE....
by Peter Valdemar Mørch 04 Nov '05

04 Nov '05

Hi, I've seen the "ISODATE offset?" thread, and I seem to be seeing the exact same thing: https://lists.balabit.hu/pipermail/syslog-ng/2004-November/006633.html The problem appears only during Daylight Savings Time ("summer time"), not during standard/non-DST/"Winter" time. We just entered non-DST time this weekend (and hence the bug "stopped"). During DST the TZ offset does not include the offset due to DST. So TZ is off-by-one during DST. About this section #if HAVE_GLOBAL_TIMEZONE length = snprintf(dest, left - 1, "%c%02ld%02ld", timezone > 0 ? '-' : '+', (timezone < 0 ? -timezone : timezone) / 3600, (timezone % 3600) / 60); #else Bazsi writes in the thread: > Thanks. Either the value in timezone is not set properly, or it does not > contain the daylight-saving offset, or my code has an error. > > I'll check (I've checked DST before, and timezone should reflect it, so > it _might_ be a problem in libc) I'll see what I can do. Has there been any progress on this? After typing up this report I notice the 1.9.6 NEWS and see: "Fixes in local timezone detection, it should properly detect DSTs on Linux, Solaris and probably BSDs". Is this the same bug? Any chance of this fix being back-ported into a 1.6 maintenance release to show up automagically in debian? (There are no TZ environment variables set on this machine. Also what should it be set to? Following GNUs docs doesn't help. See below.) ************ environment ************* > cat /etc/timezone Europe/Copenhagen > ls -l /etc/localtime lrwxrwxrwx 1 root root 37 2005-10-25 07:13 /etc/localtime -> /usr/share/zoneinfo/Europe/Copenhagen syslog-ng 1.6.8-1 > set | grep -i TZ (no output - se below under "Tried setting TZ env") # I'm using 1.6.8-1 on debian (==newest in testing+unstable), # and the poster in the thread from Nov 2004 was using 1.6.4-1. > dpkg-query -W 'syslog-ng' > dpkg-query -W libc6 libc6 2.3.5-6 ************ syslog-ng.conf snippet ************* destination d_template { file("/tmp/syslog-template.log" template("\"DATE'$ISODATE'TZOFFSET'$TZOFFSET'FULLDATE'$FULLDATE'MSG'$MSG\n") template_escape(yes)); }; log { source(s_all); destination(d_template); }; ************ non-DST example - works ************ # date, perl and syslog-ng all agree with the truth that # we're +0100 during "wintertime" - non-DST. > date --rfc-2822 Mon, 31 Oct 2005 00:00:06 +0100 > perl -le 'use POSIX; print strftime ("%a, %d %b %Y %H:%M:%S %z", localtime);' Mon, 31 Oct 2005 00:00:08 +0100 > logger 'A message' > sudo tail /tmp/syslog-template.log | grep 'A message' "DATE'2005-10-31T00:00:11+0100'TZOFFSET'+0100'FULLDATE'2005 Oct 31 00:00:11'MSG'pvm: A message ************ DST example - fails ************ # Date and perl agree with the truth that # we're +0200 during "summertime" - dst. > date --rfc-2822 Tue, 25 Oct 2005 00:14:13 +0200 > perl -le 'use POSIX; print strftime ("%a, %d %b %Y %H:%M:%S %z", localtime);' Tue, 25 Oct 2005 00:14:15 +0200 # syslog-ng seems to think we're +0100 > logger 'A message' > sudo tail /tmp/syslog-template.log | grep 'A message' "DATE'2005-10-25T00:14:17+0100'TZOFFSET'+0100'FULLDATE'2005 Oct 25 00:14:17'MSG'pvm: A message *********** Tried setting TZ env **************** I tried setting the TZ environment variable as pr. http://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html And notice that perl (or rather the POSIX implementation of strftime()) adheres to it: > TZ=:/usr/share/zoneinfo/Asia/Tokyo perl -le 'use POSIX; print strftime ("%a, %d %b %Y %H:%M:%S %z", localtime);' Tue, 25 Oct 2005 07:03:32 +0900 > TZ=:/usr/share/zoneinfo/Europe/Copenhagen perl -le 'use POSIX; print strftime ("%a, %d %b %Y %H:%M:%S %z", localtime);' Tue, 25 Oct 2005 00:03:39 +0200 Using Asia/Tokyo or Europe/Copenhagen does affect syslog-ng, but it still has a offset-by-one error during DST! # Not my real timezone # export TZ=:/usr/share/zoneinfo/Asia/Tokyo # /etc/init.d/syslog-ng restart # logger 'A message' # tail /tmp/syslog-template.log | grep 'A message' "DATE'2005-10-25T07:06:14+0900'TZOFFSET'+0900'FULLDATE'2005 Oct 25 07:06:14'MSG'pvm: A message # My real timezone - should be +0200 on oct 25 in # Copenhagen as strftime also knows. # # export TZ=:/usr/share/zoneinfo/Europe/Copenhagen # /etc/init.d/syslog-ng restart # logger 'A message' # tail /tmp/syslog-template.log | grep 'A message' "DATE'2005-10-25T00:06:57+0100'TZOFFSET'+0100'FULLDATE'2005 Oct 25 00:06:57'MSG'pvm: A message -- Peter Valdemar Mørch http://www.morch.com

3 2

syslog-ng-1.9.6 template bug patch
by Peter Nahas 03 Nov '05

03 Nov '05

I have stumbled upon a bug in syslog-ng-1.9.6 caused by an un-cleared buffer in logwriter.c which causes templated messages to be output improperly. Up until a certain point (not certain why it stops, perhaps due to a buffer size?), for each message output, all previous messages are output. For example, if the messages "One" "Two" and "Three" are sent to syslog-ng, the destination will receive "One" "One" "Two" "One" "Two" "Three". I have included a patch which solves the issue, but there may be a better solution. If you want a syslog-ng.conf which will produce this issue, let me know. Thanks, Peter Nahas Software Engineer MRV Communications - InReach Division --- 1.9.6/logwriter.c 2005-10-26 09:59:09.000000000 -0400 +++ mine/logwriter.c 2005-10-26 09:59:58.000000000 -0400 @@ -160,7 +160,8 @@ { LogTemplate *template = NULL; - + g_string_assign(result, ""); + if (self->options->template) { template = self->options->template;

2 1

program() destination in 1.9.6: respawning not automatic?
by Paul Krizak 28 Oct '05

28 Oct '05

I built the latest stable copy of 1.9 (1.9.6) a few days ago and I'm seeing some behavior that doesn't seem to jive with the documentation. The docs say that the program() destination in 2.0 (which is presumably what 1.9.X is preparing for) should respawn the program if it dies. However, this doesn't seem to be the case; you must signal syslog-ng with a HUP before it'll respawn the program. Is this expected behavior? And if it is, is there some way to make syslog-ng respawn the program immediately instead of waiting for a HUP signal? Thanks! -- Paul Krizak 5900 E. Ben White Blvd. MS 625 Advanced Micro Devices Austin, Tx 78741 Linux/Unix Systems Engineering Phone: (512) 602-8775 Microprocessor Solutions Sector

2 1

syslog-ng.conf : option group(<gid>); does not work ( stable version )
by Yan M. 28 Oct '05

28 Oct '05

Hi, in syslog-ng.conf man page it is said that the option group(gid); is used to set group on created logfiles. Im trying to use that and it does not work it is setting the group, not to the group I specify in that group() directive, but to the group syslog-ng runs as. Also, the man page specifies you have to enter the gid in the group() option but the syslog-ng daemon wont start when the gid is specified in its numeral form. It should be group(groupname); instead of group(gid); in the man page. Does the chgrp works only when syslog-ng is running as root ? If so, it is not a documented feature .. --------------------------------- Yahoo! FareChase - Search multiple travel sites in one click.

3 4

filter help
by Greg Rodenhiser 27 Oct '05

27 Oct '05

I have several remote servers that are logging via regular syslog to my central syslog-ng server. This is working fine. Now, I have several distinct messages from different servers that I want to filter to a single log file (without having anything else in this file). I have tried setting it up but I'm having problems... 1.) I have made six filter lines in syslog-ng.conf that contain the host/message combination I want to put in my special log. I have named these six filters the same name (f_interest). I then have a log line that contains the name of the filer (the filter name is used for six filter declarations), and the destination for the file I want to log to. My problem/question is, How does syslog-ng handle multiple filters that have the same name? Each of these 6 filters contain a host and word match I want to have interpreted. My problem is nothing is getting logged to my log. Does syslog-ng somehow combine these 6 filters because they have the same name? I want each one to be interpreted separately from the others with the same name. I only named them the same to prevent having to make 6 different log directives to the same file. 2.) How do I do a multiple not match. For instance I want to log messages from serverA that do NOT contain the text, message1, or message2, or message3, but log everything else from serverA. Right now I do... filter f_interest { not match("(message1|message2|message3)") and host(serverA); }; Does that look right? Thanks in advance!!

1 0

high performance server
by JR Mayberry 27 Oct '05

27 Oct '05

I've googled this topic and read the mailing list and haven't had much luck improving my situation. I have a very high volume syslog-ng server. I currently have logs that are being received across the network but not being written to disk. This could be as much as 25% of the logs being dropped. The STAT to syslog always says 0 drops. It's a Netro 1405 w/ two CPUs and 1 gig of memory connected to a SAN. syslog-ng is always at 49% CPU and never goes beyond that. What settings can I use to stop dropping logs? This is an output of iostate with an interval of 1. This will give you an idea of performance. As you can see there are no disk waits. Thanks. device r/s w/s Mr/s Mw/s wait actv svc_t %w %b md89 2.0 86.0 0.0 0.4 0.0 0.2 1.9 0 5 md89 0.0 3.0 0.0 0.0 0.0 0.0 0.9 0 0 md89 0.0 1.0 0.0 0.0 0.0 0.0 2.5 0 0 md89 0.0 1.0 0.0 0.0 0.0 0.0 1.2 0 0 md89 0.0 2.0 0.0 0.0 0.0 0.0 1.2 0 0 md89 0.0 61.0 0.0 0.3 0.0 0.1 1.3 0 3 md89 0.0 1.0 0.0 0.0 0.0 0.0 1.5 0 0 md89 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 md89 0.0 2.0 0.0 0.0 0.0 0.0 0.9 0 0 md89 0.0 2.0 0.0 0.0 0.0 0.0 0.7 0 0 md89 0.0 83.0 0.0 0.4 0.0 0.2 2.2 0 3 md89 2.0 4.0 0.0 0.0 0.0 0.0 3.0 0 2 md89 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 md89 0.0 1.0 0.0 0.0 0.0 0.0 0.8 0 0 md89 0.0 1.0 0.0 0.0 0.0 0.0 0.8 0 0 md89 1.0 86.0 0.0 0.4 0.0 0.1 1.2 0 4 md89 0.0 2.0 0.0 0.0 0.0 0.0 0.9 0 0 md89 0.0 5.0 0.0 0.0 0.0 0.0 0.9 0 0 md89 0.0 1.0 0.0 0.0 0.0 0.0 1.0 0 0 md89 0.0 2.0 0.0 0.0 0.0 0.0 0.8 0 0 md89 2.0 751.9 0.0 5.6 0.0 6.5 8.6 0 25

3 5

1.6.5 performance
by Scott C 24 Oct '05

24 Oct '05

I am currently experiencing an odd performance issue with syslog-ng 1.6.5. I use several load-balanced relays which receive syslog traffic via both udp and tcp. They forward all traffic to the back-end collection hosts exclusively via tcp. Overall, this environment collects around 60GB of log data per day. All of the relays have identical Solaris 8 builds, as well as identical syslog-ng configurations. Here is the pertinent snippet from a configuration file: options { stats(300); sync(0); time_reopen(10); log_fifo_size(3000000); gc_busy_threshold(200000); check_hostname(no); keep_hostname(yes); }; As you can see, my fifo queue is enormous. Despite that fact, syslog-ng on any given relay will begin dropping messages at an alarming rate (600-700 per second) when combined inbound and outbound tcp traffic increases beyond a certain threshold. Said threshold seems to be a bit low to me, though. We're talking somewhere in the neighborhood of 700 tcp segments per second. The corresponding inbound UDP dgrams are usually between 200 and 400 per second. It should be noted that the IP stack is handling the load just fine. Also, CPU utilization rarely exceeds 60 or 70 percent. The collection hosts are fairly beefy systems that are barely breathing even during peak load. I've watched both the IP stack and syslog-ng on these systems, and they're keeping up just fine. I know the FIFO queue size probably seems unreasonable here, but the lower I make it, the worse the drops. Of course, the higher I make it, the more memory it eats up. And in some cases, I've seen syslog-ng eventually eat it all up and the scanner kicks in. But what's really most peculiar in this scenario is the fact that the numbers simply don't add up. Why does syslog-ng appear (on the surface) to be dropping a very large percentage of the messages that it receives? I realize that it's not, but the numbers tell a different story. And how could it possibly drop so many messages when the FIFO queue is configured to buffer three million lines? How preposterous! Am I way off base here? --------------------------------- Yahoo! FareChase - Search multiple travel sites in one click.

4 13