December 2006 - syslog-ng - lists.balabit.hu

AIX startup
by Brian Loe 27 Mar '08

27 Mar '08

Okay, my syslog-ng instance died at some point last night and because I don't have it setup properly for respawning... Can anyone running syslog-ng on AIX 5.* give me some tips for this? I'm feeding pipes so i have to start instances of SEC as soon as I start syslog-ng - but the SEC scripts appear to "close" so if they're configured to respawn I'll have 2000 instances of them in an hour.

7 21

incorrect firewall logs with syslogng-2.0.0 [security error?]
by Jacek Kalinski 12 Mar '08

12 Mar '08

Hello, After upgradeing from syslogng-1.6.11 to 2.0.0 I've got a strange messages in /var/log/messages file: Nov 29 08:14:05 denise WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 11:10:19 denise PT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 16:42:16 denise W=0 RES=0x00 RST URGP=0 Nov 29 21:37:15 denise 116 ID=34901 DF PROTO=TCP SPT=3584 DPT=80 WINDOW=65535 RES=0x00 ACK FIN URGP=0 These are truncated iptables logs. Because in firewall.log file are simmilar entries to truncated one, I think full file should be: Nov 29 11:10:19 denise firewallp=INVALID:1 a=DROP IN=eth1 OUT= MAC=00:30:4f:36:2b:dc:00:04:9a:2c:7f:20:08:00 SRC=193.41.230.81 DST=xxx.xxx.xxx.x LEN=576 TOS=0x00 PREC=0x00 TTL=117 ID=3495 PROTO=TCP SPT=443 DPT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 (xxx.xxx.xxx.x is a real server IP address - one 'x' is one digit) Config file with interesting lines: options { dir_perm(0700); perm(0600); chain_hostnames(no); stats(1200); keep_hostname(yes); dir_owner(root); dir_group(root); owner(root); group(root); }; source local { unix-stream("/dev/log" max_connections(300)); file("/proc/kmsg"); # udp(ip(0.0.0.0) port(514)); internal(); }; # ..... # authpriv.none;cron.none;mail.none;news.none -/var/log/messages filter f_2 { not facility(news); }; filter f_3 { not facility(local5); }; filter f_4 { not facility(local6); }; filter f_5 { not facility(authpriv); }; filter f_6 { not facility(cron); }; filter f_7 { not facility(mail); }; filter f_notlocal7 { not facility(local7); }; filter f_notfirewall { not match("firewallp="); }; destination d_2 { file("/var/log/messages" create_dirs(yes)); }; log { source(local); filter(f_2); filter(f_4); filter(f_5); filter(f_6); filter(f_7); filter(f_notimapd); filter(f_notpopa3d); filter(f_notnamed); filter(f_notfirewall); destination(d_2); }; #..... # firewallp.* -/var/log/firewall.log filter f_16 { level(warning) and match("firewallp="); }; destination d_12 { file("/var/log/firewall.log" create_dirs(yes)); }; log { source(local); filter(f_16); destination(d_12); }; #..... Do you know where could be an error? It is a production server so I can't debug syslog-ng on this machine (sorry). NOTE: the only truncated lines I can find are iptables (AFAIR they are logged by /proc/kmesg) entries. All other lines are complete. And it is probably a critical error in security. PS: simple statistics line from syslog-ng: Nov 30 00:00:06 denise syslog-ng[32682]: Log statistics; dropped='pipe(/dev/tty12)=0', processed='center(queued)=758289', processed='center(received)=369020', processed='destination(d_maillogimapd)=4245', processed='destination(d_5)=1', processed='destination(d_4)=0', processed='destination(d_3)=14259', processed='destination(d_2)=14259', processed='destination(d_13)=0', processed='destination(d_1)=6000', processed='destination(d_12)=6000', processed='destination(d_11)=369020', processed='destination(d_10)=36', processed='destination(d_8)=0', processed='destination(d_maillogpopa3d)=103224', processed='destination(d_7)=241245', processed='destination(d_6)=0', processed='source(local)=369020' I think you see that it could be a little hard to debug (with so many messages parsed by syslog-ng) ;) Jacek

3 6

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

Analysis of logs and alerting
by Muath Al Khalaf 11 Feb '07

11 Feb '07

Hi, I am implementing a syslog-ng centralized correlation server. The server receives the log entries from some servers, apps and devices. Is there any good product (commercial or open-source) with GUI or web interface to analyze these logs and give real time alerts with strong data mining tools? I need a tool which could be used by both help desk non-technical staff and technical engineers. It should give basic alerting on applications' or devices' failures first for help desk, and then engineers could dig deeper into these alerts to understand the problem. Kind regards

4 3

syslog-ng cvs / svn
by Brian A. Seklecki 24 Jan '07

24 Jan '07

Where is the repo? I want to submit a patch to a bad example in the docs. References to cvs.balabit.hu are clearly invalid. l8* -lava (Brian A. Seklecki - Pittsburgh, PA, USA) http://www.spiritual-machines.org/ "...from back in the heady days when "helpdesk" meant nothing, "diskquota" meant everything, and lives could be bought and sold for a couple of pages of laser printout - and frequently were."

3 4

Can't install syslog-ng 2 on fedora 6
by Martin H 29 Dec '06

29 Dec '06

Hi, I am trying to install syslog-ng 2 on a Fedora Core 6 computer. I have the glib2-2.12.3-2.fc6 rpm package installed, but when I do a ./configure of syslog-ng, I get the following: checking pkg-config is at least version 0.9.0... yes checking for GLIB... configure: error: Package requirements (glib-2.0) were not met: No package 'glib-2.0' found I even tried installing glib2 from source, but I get the same thing. Specifically, I installed glib-2.12.0.tar.bz2 from source with default path (did not change prefix or any other install dir). I tried pointing the GLIB_LIBS variable to /usr/local/lib/ but I still get the same problem. Does anyone have any ideas why this is? Am I using the wrong path? Or did I install the wrong package? If I choose to do a ./configure --enable-dynamic-linking will syslog-ng start properly, or will I simply get a glib2 error when I try to start syslog-ng? Thanks for your help. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

4 3

Issues getting data into a MYSQL Database
by Chris Killian 27 Dec '06

27 Dec '06

All, I have without any issues setup syslog-ng with Mysql using the script that I found called syslog2mysql.sh which is shown below. #!/bin/bash if [ ! -e /var/log/mysql.pipe ] then mkfifo /var/log/mysql.pipe fi while [ -e /var/log/mysql.pipe ] do mysql -u syslogfeeder --password=ergosum syslog < /var/log/mysql.pipe >/dev/null done This works fine with most devices however I have added a new one and no messages are being recieved, this is what happens when I strace the process. viper:/home/chrisk# strace -p 13232 Process 13232 attached - interrupt to quit read(0, "INSERT INTO logs\n(host, facility"..., 4096) = 275 read(0, "INSERT INTO logs\n(host, facility"..., 4096) = 275 read(0, "INSERT INTO logs\n(host, facility"..., 4096) = 275 write(2, "ERROR", 5) = 5 write(2, " at line 13", 11) = 11 write(2, ": Unknown command \'\\\'\'.\n", 24) = 24 fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0 read(3, 0x8060318, 16384) = -1 EAGAIN (Resource temporarily unavailable) fcntl64(3, F_SETFL, O_RDWR) = 0 write(3, "\1\0\0\0\1", 5) = 5 shutdown(3, 2 /* send and receive */) = 0 close(3) = 0 exit_group(1) = ? Process 13232 detached viper:/home/chrisk# The process dies and it restarted with nothing being put into the database. Anyone got any ideas at all?? Regards Chris K

1 0

Double hostname in log excerpt
by Steve Finkelstein 23 Dec '06

23 Dec '06

Hi all, I'm sending messages over a udp() stream to a centralized logging host using syslog-ng. The centralized server is displaying output like this: Dec 22 11:36:31 cc15-5 cc15-5 syslog-ng[4748]: syslog-ng version 1.6.8 starting Is there a global option to disable that second c15-5? I'm not sure why it's being displayed twice. Generally syslogd would only incorporate this once. I have the following global options set: options { stats(43200); chain_hostnames(off); sync(0); }; Thanks, Steve !DSPAM:3343,458c14af306121939910367!

2 1

Syntax error when using the tcp-keep-alive() option
by Jonathan Turner 22 Dec '06

22 Dec '06

Hi there, I'm working on using syslog-ng within our environment mainly due to the ability to use syslog with tcp. I will be traversing firewalls and therefore the tcp-keep-alive() option is vital for me. However when I try to turn this on in my tcp source it gives me a syntax error. I have pasted in the relevant part of the config below. I'm pretty sure I have it right. The strange thing is that when I search the source code for the option I can't find anything as if it has been removed from version 2.0 (the version I'm using). Does anyone know what may have happened to the tcp-keep-alive option in version 2.0 and, if I'm right and it isn't there, what are the options in syslog-ng that I could use to prevent the sessions timing out on firewalls? source testsrc { tcp( ip(0.0.0.0) port(5514) max-connections(300) keep-alive(yes) tcp-keep-alive(yes) ); }; Thanks.

2 2

UDP Socket Dying
by William Bell 22 Dec '06

22 Dec '06

We have recently switched to the latest version of syslog-ng/ eventlog and we are experiencing a problem where syslog-ng is losing its udp listener after about 20 minutes. We are receiving about 400 msgs/s to this port which can easily be handled by syslog-ng. Does anyone have any ideas as to the cause of this? We are experiencing it on multiple builds of linux.( 2 LFS Builds, 1 CentOS build). Load on the box never budges during this time period. I have not attached any debugging as of yet. Version: Snapshot 122106 of both eventlog and syslog-ng. William L. Bell II, CISSP, SSCP

2 1