- syslog-ng - lists.balabit.hu

1.9.9 segmentation fault
by Martin, David M 22 Mar '06

22 Mar '06

testing with syslog-ng 1.9.9 recently configured a pipe destination (config below) log processing to the pipe occurs as expected - then get a segmentation fault have seen it run perfectly for days, other times hours/minutes Linux distribution is FC4 (also RHEL 3 and 4) server has handled 500k-2M transactions daily over the past week (works fine when logging to file). below if the info I've collected so far. Let me know if you need additional. any ideas? strace -p {pid} output: time([1142745045]) = 1142745045 time([1142745045]) = 1142745045 gettimeofday({1142745045, 307328}, NULL) = 0 poll([{fd=6, events=POLLOUT, revents=POLLOUT}, {fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 3, 550317) = 1 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 585) = 585 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 746) = 746 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 644) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1142745045, 310203}, NULL) = 0 time(NULL) = 1142745045 time(NULL) = 1142745045 gettimeofday({1142745045, 310286}, NULL) = 0 time(NULL) = 1142745045 time(NULL) = 1142745045 --- SIGSEGV (Segmentation fault) @ 0 (0) --- kill(12488, SIGSEGV) = 0 sigreturn() = ? (mask now []) --- SIGSEGV (Segmentation fault) @ 0 (0) --- internal shows nothing out of the ordinary. daemon in foreground w/debug: Reducing stack by rule 2 (line 188), stmt ';' stmts -> stmts Stack now 0 9 25 Entering state 74 Reducing stack by rule 2 (line 188), stmt ';' stmts -> stmts Stack now 0 Entering state 8 Reducing stack by rule 1 (line 184), stmts -> start Stack now 0 Entering state 7 Now at end of input. Segmentation fault Here is the config: #------------------------------------------------------------------------- options { chain_hostnames(no); #default yes keep_hostname(yes); #default no create_dirs(no); #default use_dns(no); #default yes use_fqdn(no); #default owner(root); group(root); perm(0600); }; ############################################################################### # sources source abs_udp514 { udp(ip(0.0.0.0) port(514)); }; source abs_localhost { internal(); }; ############################################################################### # destinations destination abd_localhost { file( "/var/log/localhost.log" ); }; destination abd_mysql { pipe( "/tmp/mysql.pipe" template("INSERT INTO logs ( host, facility, priority, datetime, date, time, year, month, day, hour, min, sec, week, unixtime, tzoffset, tz, program, msg ) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$YEAR', '$MONTH', '$DAY', '$HOUR', '$MIN', '$SEC', '$WEEK', '$UNIXTIME', '$TZOFFSET', '$TZ', '$PROGRAM', '$MSG');\n" ) template_escape(yes) ); }; ############################################################################### # logs log { source(abs_udp514); destination(abd_mysql); }; log { source(abs_localhost); destination(abd_localhost); }; # #------------------------------------------------------------------------- Thanks, The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 1

netmask
by Olleg Samoylov 18 Mar '06

18 Mar '06

After upgrade from 1.6.x to 1.9.9 netmask filter rule don't work. Write "syntax error at 188" on "netmask(149.49.32.121/255.255.255.255);". -- Olleg Samoylov

2 2

Multiple IP filters
by R King 18 Mar '06

18 Mar '06

I am trying to setup filters for multiple network devices. Below are excerts to my syslog-ng.conf file. filter f_netdev { host("192.168.23.1") or host("192.168.23.2") or host("192.168.23.4") or host("192.168.23.8") or host("192.168.23.40") or host("192.168.23.75"); }; destination d_host-date { file("/var/log/network/$HOST/$MONTH$DAY" group(watch) perm(0640)); }; log { source(s_udp); filter(f_netdev); destination(d_host-date); }; The problem is when I log host IPs, the filter for "192.168.23.1" includes "192.168.23.11", "192.168.23.12", "192.168.23.135", etc... How can I specify 192.168.23.1 only? --------------------------------- Yahoo! Mail Use Photomail to share photos without annoying attachments.

2 1

Running ON INSERT UPDATE (MySQL)
by Dukes Clayton 17 Mar '06

17 Mar '06

Hii, I'm trying to update a table when I do an insert, do I have the syntax correct here? destination db_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, datetime, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$MSG' ) ON INSERT UPDATE NumRows SET NumRowsLogs=NumRowsLogs+1;\n") template-escape(yes)); }; TIA! -Clayton

2 1

Solaris syslog-ng tuning
by Cary, Kim 17 Mar '06

17 Mar '06

In summary, tuning the solaris udp hiwat and max_buf had no effect on the continuous udpInOverflows loss. At present, changing some parameters in syslog-ng.conf has helped, it seems. We have now gone from log_fifo_size(100000); gc_busy_threshold(3000000); gc_idle_threshold(300); and having A1) udpInOverFlows increment by 3000 every 4 sec A2) Garbage collecting while busy for 3 sec every half hour at peak to log_fifo_size(1000000); # up an order of magnitude gc_busy_threshold(300000); # down an order of magnitude gc_idle_threshold(500); # log was reporting objects 310-322 range, so some headroom and having B1) udpInOverflows increment by 50 every half hour B2) Garbage collecting while busy for a fraction of a second every 4-8 minutes This seems better. Hope that helps someone out there. My first question - we are losing messages during GC while busy - do we only lose the messages that arrive between these two log entries: Mar 16 20:00:30 syslog-ng[9688]: Garbage collecting while busy... Mar 16 20:00:31 syslog-ng[9688]: Objects alive: 310, garbage collected: 299695 My second question is, what can we do better to tune this further? Thanks to all (& Balasz for the wonderful software).

2 1

Re: syslog-ng Digest, Vol 11, Issue 23
by dwarnock＠harbrook.net 16 Mar '06

16 Mar '06

I am on holiday on Wednesday 15th March returning on Thursday 16th. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734. Regards David Warnock

1 0

sylog-ng hangs after upgrade to 1.9.9
by Cristian Livadaru 16 Mar '06

16 Mar '06

Hi, just did an apt-get upgrade today and installed syslog-ng 1.9.9 after this I was fighting some hours with my system. syslog-ng logged a few minutes then it hanged, all logfiles showing the same timestamp. This also caused my mail system ( postfix ) to hang and also caused other problems. After a restart of syslog-ng the problem was gone, also 2 times sylog-ng started logging again after about 15-20 minutes for a short while. I found something on a old mailinglist about 2 sources from the same unix-stream which was the case in my logfile. I removed the second one but the problem still occured. I have also read something about klog runing, which isn't the case on my machine. I have now downgraded syslog-ng to 1.6.9 and it's working fine again. My system is a Debian linux Linux 2.4.31-grsec #21 SMP Fri Dec 2 08:49:48 CET 2005 i686 athlon-4 i386 GNU/Linux Best regards, Cristian Livadaru

3 6

snmptrapd to syslog-ng
by Mike Tremaine 15 Mar '06

15 Mar '06

Does anyone have a simple snmptrapd.conf that can push the traps to syslog-ng (via shell script, logger, pipe or whatever?) Sorry if this has been asked before but I didn't see in the FAQ -Mike

1 1

Regex subpattern macros [1.9.9]
by Mel Collins 15 Mar '06

15 Mar '06

Hi, I'm using syslog-ng's match filter to extract parts of messages for use in parts of the destination; specifically in the log file name and template sections. The messages sent to syslog-ng for this purpose are all of the format: "<filename> <message>" (plus whatever sl-ng prepends). .conf extract: filter f_logsplitter { match ("^.*?: ([^ ]*) (.*)$"); }; destination d_logfiles { file ("/our/logs/$1" template ("$2\n") ... etc. Eg. For the message "root: foo/bar.log some text", the log file /our/logs/foo/bar.log gets "some text" appended to it. Now this seems to work fine for low traffic, but if I send 10k messages in a tight loop - either locally or remotely via TCP - $2 contains the wrong string for many/most of them, eg: 07/03/2006 16:56:34 SEEK #3 07/03/2006 16:56:35 SEEK #3 07/03/2006 16:56:35 SEEK #3 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #8 07/03/2006 16:56:35 SEEK #11 07/03/2006 16:56:35 SEEK #11 07/03/2006 16:56:35 SEEK #11 ...where the #number should be consecutive. If I use $MSGONLY, or such, the correct message line is used. It seems to me like the subpattern macros - $1, $2, etc - aren't maintained on a per-message basis, but are global..? Given that they aren't documented (that I can find), are the presence of $1, etc. an unintended side-effect of using a regex? Am I doing something I shouldn't by using them? Or did I find me a bug? ;o) Takk, - Mel C

3 6

Re: syslog-ng Digest, Vol 11, Issue 22
by dwarnock＠harbrook.net 15 Mar '06

15 Mar '06

I am on holiday on Wednesday 15th March returning on Thursday 16th. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734. Regards David Warnock

1 0