October 2002 - syslog-ng - lists.balabit.hu

missing 33% of syslog messages
by Bob Kupiec 02 Oct '02

02 Oct '02

Drew, I tried tweaking sync() to 25 and to 100. No real change. I still average capturing about 80% to 85% of the syslog data. See my previous message for my configs and version. All the packets are hitting the machine, verfied by tcpdump, but are getting lost after that. The machine has a hardware RAID 5 array and it's only running syslog-ng, it's not a hardware issue. Tuning syslog-ng.conf doesn't do anything noticeable. How do I go about fixing/debugging syslog-ng further? >From Andrew.Hamilton(a)afccc.af.mil Thu, 19 Sep 2002 12:47:25 -0400 >Date: Thu, 19 Sep 2002 12:47:25 -0400 >From: Hamilton, Andrew Andrew.Hamilton(a)afccc.af.mil >Subject: [syslog-ng]missing 33% of syslog messages > >Bob, > >Have you checked to see that the messages are getting to the loghost? On a >busy network it is possible to lose a number of packets. With UDP you never >know. You could use a network sniffer to find out. Something else you >might try is raising sync some. If you are getting a ton of messages you >might be getting lost in I/O somewhere. sync(0) writes immediately so you >might crank it up a little to get some buffering going. You might have to >play around with some of the options to get it to work with your >configuration. I believe that stats are where you have the internal >source going. It is just a line in the file I think. > -- Bob Kupiec Security/Network Administrator Email: kupiec(a)ias.edu Institute for Advanced Study Phone: 609-734-8179 Einstein Drive (A208) Fax: 609-951-4418 Princeton, NJ 08540-4907

1 0

lost messages when logging to a database
by harald schmidt 02 Oct '02

02 Oct '02

hy, I am using syslog-ng on a loghost with a MySQL-database. Everything seems to work fine. But when I compare the number of messages which are logged to the SQL-database and which are logged to a file, I recognize that some messages are lost in the MySQL-database. Could it be that sys-ng is to fast for the pipe which goes to MySQL? I'm using syslog-ng v1.5.21 on a PIII-1000,512MB,SCSI-Raid1,SuSE 7.3, my config is like that: source s_kernel { pipe("/proc/kmsg"); internal(); }; source s_stream { unix-stream("/dev/log"); internal(); }; source sn_udp { udp(); }; destination df_all_messages { file("/var/log/all_messages-$YEAR$MONTH$DAY"); }; destination dq_mysql_all_messages { pipe("/var/pipes/mysql.fifo" template("INSERT INTO all_messages_$YEAR$MONTH (host, facility, priority, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(s_stream); source(s_kernel); source(sn_udp); destination(df_all_messages); };log { source(s_stream); source(s_kernel); source(sn_udp); destination(dq_mysql_all_messages); }; the script that reads from the pipe is like that: #!/bin/sh datum=$(date +%Y%m) if [ -e /var/pipes/mysql.fifo ]; then while [ -e /var/pipes/mysql.fifo ] do mysql syslog_$datum < /var/pipes/mysql.fifo done else mkfifo /var/pipes/mysql.fifo fi I thought my problem with the lost messages would be fixed after the update from 1.5.19 to 1.5.21. In one second there were 43 messages logged to the file, but only 24 to mysql. Could it be that the pipe is too "thin" to fit 43 messages per second? thanks for any help in advance. Harald -- paybox macht Ihr Handy zur Geldbörse! http://bonus.gmx.net/?gid=46842444&prid=540

1 0

localport binding...
by Robert D. Beal 01 Oct '02

01 Oct '02

I'm trying to get syslog-ng (1.5.20) to use 514 as its source port when using the udp destination driver as follows: <<Following taken from my syslog-ng.conf>> source s_sys { unix-stream("/dev/log"); internal(); }; destination d_syslog { udp("192.168.0.2" destport(514) localport(514)); }; log { source(s_src); destination(d_syslog); }; but it always seems to send the syslog udp datagrams using the next available non-privileged port as its source (>1024). I'm running as root (on RedHat 7.3) and syslogd is not running. Syslog-ng doesn't choke on the config, so I'm assuming its not a syntax problem, but I just can't seem to get it to use 514 as its source port?! Does anyone have any ideas what I may be doing wrong here? Or is it just silently ignoring my request to use port 514? Thanks.

4 3