- syslog-ng - lists.balabit.hu

Syslog-ng email message setup
by R King 23 Mar '06

23 Mar '06

I am trying to set up syslog-ng to send messages based on specific events. The only time it generates a messages and sends it is when I reload the service. For testing, I have set up filters based on cron jobs (run every ten minutes), ssh access, and even allowed any event to send a message. I have included the actual command in the script as well as used the "program" destination to a script file. The only message generated is when the syslog-ng service is restarted. Everything has been updates: Gentoo (latest kernel), syslog-ng, postfix (sendmail). Here are the configurations for my different attempts: Command located in syslog-ng.conf file: source s_udp { udp(); }; destination d_email { program("echo -e \"subject: Log Alert\nA log message has been generated\n.\" | sendmail -f syslog(a)domain.com user(a)domain.com"); }; filter f_ssh { facility(auth,authpriv); }; log { source(src); destination(d_email); }; #log { source(src); filter(f_ssh); destination(d_email); }; Here is my config when attempting to redirect to a script: source s_udp { udp(); }; destination d_email { program("/usr/local/sbin/syslog-mail"); }; filter f_ssh { facility(auth,authpriv); }; log { source(src); destination(d_email); }; #log { source(src); filter(f_ssh); destination(d_email); }; Script file: #!/bin/bash # This script is designed to sedn messages based on syslog errors echo -e "subject: Log Alert\nA log message has been generated by the script\n." | sendmail -f syslog(a)domain.com users(a)domain.com Any help will be greatly appreciated.... Ron King --------------------------------- Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo! Messenger with Voice.

1 0

Syslog-ng email message setup
by R King 23 Mar '06

23 Mar '06

I am trying to set up syslog-ng to send messages based on specific events. The only time it generates a messages and sends it is when I reload the service. For testing, I have set up filters based on cron jobs (run every ten minutes), ssh access, and even allowed any event to send a message. I have included the actual command in the script as well as used the "program" destination to a script file. The only message generated is when the syslog-ng service is restarted. Everything has been updates: Gentoo (latest kernel), syslog-ng, postfix (sendmail). Here are the configurations for my different attempts: Command located in syslog-ng.conf file: source s_udp { udp(); }; destination d_email { program("echo -e \"subject: Log Alert\nA log message has been generated\n.\" | sendmail -f syslog(a)domain.com user(a)domain.com"); }; filter f_ssh { facility(auth,authpriv); }; log { source(src); destination(d_email); }; #log { source(src); filter(f_ssh); destination(d_email); }; Here is my config when attempting to redirect to a script: source s_udp { udp(); }; destination d_email { program("/usr/local/sbin/syslog-mail"); }; filter f_ssh { facility(auth,authpriv); }; log { source(src); destination(d_email); }; #log { source(src); filter(f_ssh); destination(d_email); }; Script file: #!/bin/bash # This script is designed to sedn messages based on syslog errors echo -e "subject: Log Alert\nA log message has been generated by the script\n." | sendmail -f syslog(a)domain.com users(a)domain.com Any help will be greatly appreciated.... Ron King --------------------------------- Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo! Messenger with Voice.

1 0

starting syslog with unreachable loghost
by luksedj luksedj 23 Mar '06

23 Mar '06

Hello I'm trying some stuff with syslog-ng 1.9.9. Is it possible to start the syslog-ng daemon when one of the remote hosts to which messages have to be send is not reachable? it seems not to be the case, but my config might be a little bit too thin... I have two destinations for my logs, one is on the local disk, the other is a host in the network. It seems that when the host is not available syslog-ng won't start, although it should better start logging to the local disk. What happens when the remote host connection gets broken while syslog-ng is running? thx for any pointers ... pieter

3 4

Creating a named pipe (FIFO)
by Cary, Kim 22 Mar '06

22 Mar '06

We create our pipes by placing the appropriate commands in the syslog-ng startup file (/etc/init.d/syslog-ng). [Actually, we test to see if the pipe already exists; if not, we create it. We do this creation in /var/run and here the pipe needs to be recreated after every boot. We don't want to recreate it when we restart the process (syslog-ng) that is feeding it, since recreating it requires us to bounce the process that is reading from it (fisq.pl)]. Kim Cary Infrastructure Security Administrator M-F 7-4 ~ 310 506 6655

2 1

Solaris syslog-ng tuning
by Cary, Kim 22 Mar '06

22 Mar '06

Vincent! Thanks much for confirming the issue and repeating the link to me. Well, the only _intended_ udp traffic to the system is syslog. Currently, the system is logging from a PIX on one GigE interface, and from a few servers plus a less active PIX on another GigE. We send the PIX logs, separately, to pipes. And log everything to file. # vmstat 5 4 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr s0 s1 s3 -- in sy cs us sy id 0 0 4 2793888 737144 34 5 232 2 1 0 0 0 3 0 0 124 18 107 11 9 80 0 0 36 2680456 673440 5 6 0 0 0 0 0 3 5 0 0 1897 15728 2516 9 15 76 0 0 36 2680456 673440 5 5 0 0 0 0 0 0 2 0 0 1612 13349 2268 10 10 80 0 0 36 2680456 673440 5 5 0 0 0 0 0 0 3 0 0 1854 15740 2520 13 15 73 # iostat 5 4 tty sd0 sd1 sd30 nfs1 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 37 5 0 11 421 3 22 0 0 0 0 0 0 11 9 0 80 0 47 0 0 0 259 2 29 0 0 0 0 0 0 11 14 0 75 0 16 28 4 17 334 5 25 0 0 0 0 0 0 15 16 1 69 0 16 2 0 10 293 2 40 0 0 0 0 0 0 11 11 0 78 At: ndd /dev/udp udp_max_buf 33554432 (32Mb!) We have these time/counter readings for udpInOverflows: 00 - 645628929 33 - 645630391 96 - 645632008 Or, about 1924 packets/minute lost. At udp_max_buf 64Mb (!!!), 2713 packets/minute lost. I am FAR from out of memory 700Mb free. 1) Am I reading that loss right?? 2) Any tips from Solaris/syslog-ng tuners would be appreciated! Kim On Mar 6, 2006, at 8:49 AM, syslog-ng-request(a)lists.balabit.hu wrote: > Le Mon Mar 6 07:45:39 2006, Cary, Kim a ecrit: > | Syslog-ng 1.6.4 on Solaris 9: > | > | IPv4 > | udpInOverflows =640473547 > | > | UDP > | udpInDatagrams =409687632 udpInErrors = 0 > | udpOutDatagrams =466811 udpOutErrors = 0 > | > | Does the udpInOverflows indicate I'm losing packets? > > Yes, as mentioned in this link > http://www.29west.com/docs/THPM/udp-buffer-sizing.html given today by > Mike, it means that some udp packets could not be inserted in the > sockets buffers. > > Be careful, it means you are losing udp packets, not only syslog > packets... > > Vincent.

4 5

"MSG_DONTWAIT undeclared"-error during compiling libol-0.3.17 on HP-UX 11.00/11.11
by Tob_Sch＠gmx.de 22 Mar '06

22 Mar '06

Hi, compiling libol-0.3.16 (and syslog-ng-1.6.8) on HP-UX 11.00/11.11 worked without any problems. Today, I tried to compile libol-0.3.17 (and syslog-ng-1.6.9). I got following errors during 'make': Making all in src make all-am source='io.c' object='io.lo' libtool=yes \ depfile='.deps/io.Plo' tmpdepfile='.deps/io.TPlo' \ depmode=hp /bin/sh /tmp/libol-0.3.17/depcomp \ /bin/sh ../libtool --mode=compile cc -DHAVE_CONFIG_H -I. -I/tmp/libol-0.3.17/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c -o io.lo `test -f 'io.c' || echo '/tmp/libol-0.3.17/src/'`io.c cc -DHAVE_CONFIG_H -I. -I/tmp/libol-0.3.17/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c io.c -Wp,-M.deps/io.TPlo +Z -DPIC -o .libs/io.o cc: "io.c", line 493: error 1588: "MSG_DONTWAIT" undefined. cc: "io.c", line 493: warning 563: Argument #4 is not the correct type. *** Error exit code 1 Stop. *** Error exit code 1 Stop. *** Error exit code 1 Stop. It seems, that there is no MSG_DONTWAIT variable defined in the socket.h file on HP-UX like for example on SUSE 10.0. It seems, that the reference to MSG_DONTWAIT came new with libol-0.3.17. -- Lust, ein paar Euro nebenbei zu verdienen? Ohne Kosten, ohne Risiko! Satte Provisionen f�r GMX Partner: http://www.gmx.net/de/go/partner

4 8

RE: [syslog-ng] Creating a named pipe (FIFO)
by Timothy A. Holmes 22 Mar '06

22 Mar '06

Michael - -I created it and pointed syslog-ng at it and its working great -- thank you very much TIM Timothy A. Holmes IT Manager / Network Admin / Web Master / Computer Teacher Medina Christian Academy A Higher Standard... Jeremiah 33:3 Jeremiah 29:11 Esther 4:14 > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng- > bounces(a)lists.balabit.hu] On Behalf Of Michael Clark > Sent: Wednesday, March 22, 2006 11:36 AM > To: 'Syslog-ng users' and developers' mailing list' > Subject: RE: [syslog-ng] Creating a named pipe (FIFO) > > Timothy, > > Technically, it's not a "file" but a named pipe. That means it doesn't > take > up space on the disk. It's a stream that allows you to connect to and > read > information that is being presented on it. Once you have the > /var/syslog-ng/syslog_fifo named pipe created, you can refer to it by name > in the configuration files. I've never used a named pipe in syslog-ng (I > have in other programs), so I don't know if you use the same command that > you would to reference a file name. In the Micromuse Syslog Probe, you > have > to use a different parameter name to specify a named pipe than you would > for > a file. > > Check the syslog-ng docs to see what the command would be to specify a > named > pipe. It's possible it's the same command. > > -----Original Message----- > From: Timothy A. Holmes [mailto:tholmes@mcaschool.net] > Sent: Wednesday, March 22, 2006 9:15 AM > To: holtzl.peter(a)balabit.hu; Syslog-ng users' and developers' mailing list > Subject: RE: [syslog-ng] Creating a named pipe (FIFO) > > Does that just create the file, which is then used as the destination? > > Is that how it works? > > > Timothy A. Holmes > IT Manager / Network Admin / Web Master / Computer Teacher > > Medina Christian Academy > A Higher Standard... > > Jeremiah 33:3 > Jeremiah 29:11 > Esther 4:14 > > > -----Original Message----- > > From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng- > > bounces(a)lists.balabit.hu] On Behalf Of HÖLTZL Péter > > Sent: Wednesday, March 22, 2006 9:35 AM > > To: Syslog-ng users' and developers' mailing list > > Subject: RE: [syslog-ng] Creating a named pipe (FIFO) > > > > Hi Timothy, > > > > Insted of putting mkfifo into the syslog-ng.conf, add the command > > 'mkfifo /var/syslog-ng/syslog_fifo' from a command prompt when you are > > root. > > > > regards, > > > > Péter > > > > > > > Does the name of the named pipe need to be the same as the destination > > line > > > mkfifo splunk; > > > > > > -- > > BalaBit IT Bizt. Kft | Tel: +36 1 371-0540 | GnuPG Fingerprint: > > holtzl.peter(a)balabit.hu | Mobil: +36 20 366-9667 | 796B C9D3 E492 B006 > > C8B2 > > http://www.balabit.hu/ | Fax: +36 1 208-0875 | 4D1F 5320 28E3 9A1B > > 3FC6 > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] Creating a named pipe (FIFO)
by Timothy A. Holmes 22 Mar '06

22 Mar '06

Does that just create the file, which is then used as the destination? Is that how it works? Timothy A. Holmes IT Manager / Network Admin / Web Master / Computer Teacher Medina Christian Academy A Higher Standard... Jeremiah 33:3 Jeremiah 29:11 Esther 4:14 > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng- > bounces(a)lists.balabit.hu] On Behalf Of HÖLTZL Péter > Sent: Wednesday, March 22, 2006 9:35 AM > To: Syslog-ng users' and developers' mailing list > Subject: RE: [syslog-ng] Creating a named pipe (FIFO) > > Hi Timothy, > > Insted of putting mkfifo into the syslog-ng.conf, add the command > 'mkfifo /var/syslog-ng/syslog_fifo' from a command prompt when you are > root. > > regards, > > Péter > > > > Does the name of the named pipe need to be the same as the destination > line > > mkfifo splunk; > > > -- > BalaBit IT Bizt. Kft | Tel: +36 1 371-0540 | GnuPG Fingerprint: > holtzl.peter(a)balabit.hu | Mobil: +36 20 366-9667 | 796B C9D3 E492 B006 > C8B2 > http://www.balabit.hu/ | Fax: +36 1 208-0875 | 4D1F 5320 28E3 9A1B > 3FC6

2 1

RE: 1.9.9 segmentation fault
by Balazs Scheidler 22 Mar '06

22 Mar '06

On Wed, 2006-03-22 at 07:43 -0600, Martin, David M wrote: > here you go. let me know if you need additional info. And here's the patch which should fix it: --- orig/src/logwriter.c +++ mod/src/logwriter.c @@ -327,6 +327,7 @@ log_writer_flush_log(LogWriter *self, FD line = NULL; if (errno != EAGAIN) goto write_error; + return TRUE; } else { -- Bazsi

1 0

Creating a named pipe (FIFO)
by Timothy A. Holmes 22 Mar '06

22 Mar '06

Good Morning All: I have hit a bit of a problem with the set up of my Splunk system. I am trying to set it up for using a named pipe to get my data into splunk. I have the logging server receiving data and logging it to files with no problem, but, someplace, I have gotten the named pipe configuration messed up. Here is my syslog-ng.conf file # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo, v 1.5 2005/05/12 05:46:10 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux # contributed by Michael Sterrett options { chain_hostnames(off); sync(0); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; source remote { udp(); }; destination splunk { pipe("/var/syslog-ng/syslog_fifo"); }; log{ source(remote); destination(splunk); }; destination hosts { file("/var/log/hosts/$HOST/messages" owner(root) group(logs) perm(0640) dir_perm(0750) create_dirs(yes) ); }; log { source(remote); destination(hosts); }; # By default messages are logged to tty12... destination console_all { file("/dev/tty12"); }; # ...if you intend to use /dev/console for programs like xconsole # you can comment out the destination line above that references /dev/tty12 # and uncomment the line below. #destination console_all { file("/dev/console"); }; log { source(src); destination(messages); }; log { source(src); destination(console_all); }; srvnms-01 ~ # As the file is currently configured, syslog-ng starts with no problems and runs logging to the files as specified in the destination hosts directive. The data never appears in the named pipe. I did some investigation and asked a few questions on IRC, and someone told me that I needed to use the mkpipe command, however I have been unable to locate (using google) the proper syntax for the mkpipe command. Another person told me that once I got it set up, I would need to configure logrotate as well. Any and all pointers that you can provide would be most welcome. I have gone over the FAQ and also the users manual. As well as googling about for the proper answers. If you prefer to just point me to resources and let me figure it out myself, That is also quite cool, I like to learn. Thank you Timothy A. Holmes IT Manager / Network Admin / Web Master / Computer Teacher Medina Christian Academy A Higher Standard... Jeremiah 33:3 Jeremiah 29:11 Esther 4:14

2 1

1.9.9 segmentation fault
by Martin, David M 22 Mar '06

22 Mar '06

testing with syslog-ng 1.9.9 recently configured a pipe destination (config below) log processing to the pipe occurs as expected - then get a segmentation fault have seen it run perfectly for days, other times hours/minutes Linux distribution is FC4 (also RHEL 3 and 4) server has handled 500k-2M transactions daily over the past week (works fine when logging to file). below if the info I've collected so far. Let me know if you need additional. any ideas? strace -p {pid} output: time([1142745045]) = 1142745045 time([1142745045]) = 1142745045 gettimeofday({1142745045, 307328}, NULL) = 0 poll([{fd=6, events=POLLOUT, revents=POLLOUT}, {fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 3, 550317) = 1 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 625) = 625 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 585) = 585 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 746) = 746 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1279, ...}) =0 write(6, "INSERT INTO logs (\n host,\n "..., 644) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1142745045, 310203}, NULL) = 0 time(NULL) = 1142745045 time(NULL) = 1142745045 gettimeofday({1142745045, 310286}, NULL) = 0 time(NULL) = 1142745045 time(NULL) = 1142745045 --- SIGSEGV (Segmentation fault) @ 0 (0) --- kill(12488, SIGSEGV) = 0 sigreturn() = ? (mask now []) --- SIGSEGV (Segmentation fault) @ 0 (0) --- internal shows nothing out of the ordinary. daemon in foreground w/debug: Reducing stack by rule 2 (line 188), stmt ';' stmts -> stmts Stack now 0 9 25 Entering state 74 Reducing stack by rule 2 (line 188), stmt ';' stmts -> stmts Stack now 0 Entering state 8 Reducing stack by rule 1 (line 184), stmts -> start Stack now 0 Entering state 7 Now at end of input. Segmentation fault Here is the config: #------------------------------------------------------------------------- options { chain_hostnames(no); #default yes keep_hostname(yes); #default no create_dirs(no); #default use_dns(no); #default yes use_fqdn(no); #default owner(root); group(root); perm(0600); }; ############################################################################### # sources source abs_udp514 { udp(ip(0.0.0.0) port(514)); }; source abs_localhost { internal(); }; ############################################################################### # destinations destination abd_localhost { file( "/var/log/localhost.log" ); }; destination abd_mysql { pipe( "/tmp/mysql.pipe" template("INSERT INTO logs ( host, facility, priority, datetime, date, time, year, month, day, hour, min, sec, week, unixtime, tzoffset, tz, program, msg ) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$YEAR', '$MONTH', '$DAY', '$HOUR', '$MIN', '$SEC', '$WEEK', '$UNIXTIME', '$TZOFFSET', '$TZ', '$PROGRAM', '$MSG');\n" ) template_escape(yes) ); }; ############################################################################### # logs log { source(abs_udp514); destination(abd_mysql); }; log { source(abs_localhost); destination(abd_localhost); }; # #------------------------------------------------------------------------- Thanks, The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 1

netmask
by Olleg Samoylov 18 Mar '06

18 Mar '06

After upgrade from 1.6.x to 1.9.9 netmask filter rule don't work. Write "syntax error at 188" on "netmask(149.49.32.121/255.255.255.255);". -- Olleg Samoylov

2 2

Multiple IP filters
by R King 18 Mar '06

18 Mar '06

I am trying to setup filters for multiple network devices. Below are excerts to my syslog-ng.conf file. filter f_netdev { host("192.168.23.1") or host("192.168.23.2") or host("192.168.23.4") or host("192.168.23.8") or host("192.168.23.40") or host("192.168.23.75"); }; destination d_host-date { file("/var/log/network/$HOST/$MONTH$DAY" group(watch) perm(0640)); }; log { source(s_udp); filter(f_netdev); destination(d_host-date); }; The problem is when I log host IPs, the filter for "192.168.23.1" includes "192.168.23.11", "192.168.23.12", "192.168.23.135", etc... How can I specify 192.168.23.1 only? --------------------------------- Yahoo! Mail Use Photomail to share photos without annoying attachments.

2 1

Running ON INSERT UPDATE (MySQL)
by Dukes Clayton 17 Mar '06

17 Mar '06

Hii, I'm trying to update a table when I do an insert, do I have the syntax correct here? destination db_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, datetime, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$MSG' ) ON INSERT UPDATE NumRows SET NumRowsLogs=NumRowsLogs+1;\n") template-escape(yes)); }; TIA! -Clayton

2 1

Solaris syslog-ng tuning
by Cary, Kim 17 Mar '06

17 Mar '06

In summary, tuning the solaris udp hiwat and max_buf had no effect on the continuous udpInOverflows loss. At present, changing some parameters in syslog-ng.conf has helped, it seems. We have now gone from log_fifo_size(100000); gc_busy_threshold(3000000); gc_idle_threshold(300); and having A1) udpInOverFlows increment by 3000 every 4 sec A2) Garbage collecting while busy for 3 sec every half hour at peak to log_fifo_size(1000000); # up an order of magnitude gc_busy_threshold(300000); # down an order of magnitude gc_idle_threshold(500); # log was reporting objects 310-322 range, so some headroom and having B1) udpInOverflows increment by 50 every half hour B2) Garbage collecting while busy for a fraction of a second every 4-8 minutes This seems better. Hope that helps someone out there. My first question - we are losing messages during GC while busy - do we only lose the messages that arrive between these two log entries: Mar 16 20:00:30 syslog-ng[9688]: Garbage collecting while busy... Mar 16 20:00:31 syslog-ng[9688]: Objects alive: 310, garbage collected: 299695 My second question is, what can we do better to tune this further? Thanks to all (& Balasz for the wonderful software).

2 1

Re: syslog-ng Digest, Vol 11, Issue 23
by dwarnock＠harbrook.net 16 Mar '06

16 Mar '06

I am on holiday on Wednesday 15th March returning on Thursday 16th. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734. Regards David Warnock

1 0

sylog-ng hangs after upgrade to 1.9.9
by Cristian Livadaru 16 Mar '06

16 Mar '06

Hi, just did an apt-get upgrade today and installed syslog-ng 1.9.9 after this I was fighting some hours with my system. syslog-ng logged a few minutes then it hanged, all logfiles showing the same timestamp. This also caused my mail system ( postfix ) to hang and also caused other problems. After a restart of syslog-ng the problem was gone, also 2 times sylog-ng started logging again after about 15-20 minutes for a short while. I found something on a old mailinglist about 2 sources from the same unix-stream which was the case in my logfile. I removed the second one but the problem still occured. I have also read something about klog runing, which isn't the case on my machine. I have now downgraded syslog-ng to 1.6.9 and it's working fine again. My system is a Debian linux Linux 2.4.31-grsec #21 SMP Fri Dec 2 08:49:48 CET 2005 i686 athlon-4 i386 GNU/Linux Best regards, Cristian Livadaru

3 6

snmptrapd to syslog-ng
by Mike Tremaine 15 Mar '06

15 Mar '06

Does anyone have a simple snmptrapd.conf that can push the traps to syslog-ng (via shell script, logger, pipe or whatever?) Sorry if this has been asked before but I didn't see in the FAQ -Mike

1 1

Regex subpattern macros [1.9.9]
by Mel Collins 15 Mar '06

15 Mar '06

Hi, I'm using syslog-ng's match filter to extract parts of messages for use in parts of the destination; specifically in the log file name and template sections. The messages sent to syslog-ng for this purpose are all of the format: "<filename> <message>" (plus whatever sl-ng prepends). .conf extract: filter f_logsplitter { match ("^.*?: ([^ ]*) (.*)$"); }; destination d_logfiles { file ("/our/logs/$1" template ("$2\n") ... etc. Eg. For the message "root: foo/bar.log some text", the log file /our/logs/foo/bar.log gets "some text" appended to it. Now this seems to work fine for low traffic, but if I send 10k messages in a tight loop - either locally or remotely via TCP - $2 contains the wrong string for many/most of them, eg: 07/03/2006 16:56:34 SEEK #3 07/03/2006 16:56:35 SEEK #3 07/03/2006 16:56:35 SEEK #3 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #7 07/03/2006 16:56:35 SEEK #8 07/03/2006 16:56:35 SEEK #11 07/03/2006 16:56:35 SEEK #11 07/03/2006 16:56:35 SEEK #11 ...where the #number should be consecutive. If I use $MSGONLY, or such, the correct message line is used. It seems to me like the subpattern macros - $1, $2, etc - aren't maintained on a per-message basis, but are global..? Given that they aren't documented (that I can find), are the presence of $1, etc. an unintended side-effect of using a regex? Am I doing something I shouldn't by using them? Or did I find me a bug? ;o) Takk, - Mel C

3 6

Re: syslog-ng Digest, Vol 11, Issue 22
by dwarnock＠harbrook.net 15 Mar '06

15 Mar '06

I am on holiday on Wednesday 15th March returning on Thursday 16th. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734. Regards David Warnock

1 0

DOMAIN Name Variable
by Breitman, Jason 15 Mar '06

15 Mar '06

I am using the below line in my syslog-ng configuration file. destination maillog { file("/var/log/clients/$YEAR/$MONTH/$HOST/mail.log"); }; I am looking to add a variable for the domain name and have been unable to find one. Can someone help me out? Thank you. Jason Breitman jason.breitman(a)blackrock.com THE INFORMATION CONTAINED IN THIS MESSAGE AND ANY ATTACHMENT MAY BE PRIVILEGED, CONFIDENTIAL, PROPRIETARY OR OTHERWISE PROTECTED FROM DISCLOSURE. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, copying or use of this message and any attachment is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and permanently delete it from your computer and destroy any printout thereof.

6 8

RE: [syslog-ng] solaris UDP loss
by SOLIS, ALEX 14 Mar '06

14 Mar '06

What exactly is Garbage collecting? I don't get those in my logs. Alex ________________________________ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Cary, Kim Sent: Tuesday, March 14, 2006 12:30 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] solaris UDP loss Finally got around to reading the docs and applying the recommendations. FIRST - thanks much! NOW - my results: # ndd /dev/udp udp_recv_hiwat 65536 # ndd /dev/udp udp_max_buf 1073741824 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:16 udpInCksumErrs = 0 udpInOverflows =677996405 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:19 udpInCksumErrs = 0 udpInOverflows =677996571 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:22 udpInCksumErrs = 0 udpInOverflows =677996726 I'm pretty sure this is just the Firewall sending more than 55M lines of connection/deny logs over UDP. IF I was to switch the FW over to TCP logging from UDP, would this be a better solution? BTW, here are a few words from syslog-ng: Mar 14 09:32:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 09:42:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 09:52:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:02:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:04:53 syslog-ng[20685]: Garbage collecting while busy... Mar 14 10:04:58 syslog-ng[20685]: Objects alive: 310, garbage collected: 2999695 Mar 14 10:12:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:16:22 syslog-ng[20685]: Garbage collecting while idle... Mar 14 10:16:24 syslog-ng[20685]: Objects alive: 318, garbage collected: 1129970 Mar 14 10:22:32 syslog-ng[20685]: STATS: dropped 0 and its config: source s_udp_inetpix { udp(ip(192.168.0.1)); }; On Mar 7, 2006, at 3:01 PM, syslog-ng-request(a)lists.balabit.hu wrote: | 1) Am I reading that loss right?? Probably, you might however want to snoop on the interface to see what kind of udp packets come on your interface. | 2) Any tips from Solaris/syslog-ng tuners would be appreciated! udp_max_buf does not set the queue length of the udp socket, which by the way can have a different value for each socket... You could have a look at: http://sunsolve.sun.com/search/document.do?assetkey=1-30-3218-1 basically: increasing udp_max_buf without increasing udp_recv_hiwat has no meaning. Furthermore, you can increase you socket buffer that way up to 64k (Solaris 8 & 9), if you want to increase it further up you must use the setsockopt call (up to udp_max_buf which has a maximum value of 1GB). Here is the official SUN documentation regarding this: http://docs.sun.com/app/docs/doc/817-0404/6mg74vsb5?a=view#gbtag Now regarding your packet loss issue. I would increase udp_recv_hiwat -> 65536 udp_max_buf -> 1073741824 (you will never get here anyway) Then I would try to play with syslog-ng config: log_fifo_size, log_iw_size and log_fetch_limit. But here I'd appreciate a syslog-ng expert to step in and tell us what to do more preceisely. This e-mail contains Omaha Public Power District's confidential and proprietary information and is for use only by the intended recipient. Unless explicitly stated otherwise, this e-mail is not a contract offer, amendment, nor acceptance. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

2 1

RE: [syslog-ng] DOMAIN Name Variable
by Breitman, Jason 14 Mar '06

14 Mar '06

To clarify what I want servername.domainname.com or servername.city.domainame.com I believe the first name is the server name and everything else is the domain name. ================================================ #!/bin/sh if [ $# != 1 ] ; then echo "USAGE: $0 fqdn" exit 1 fi FQDN=$1 DOMAINNAME=`echo ${FQDN} | cut -d. -f2,3,4,5` echo ${DOMAINNAME} ================================================ I would like to put all servers in the same domain under one directory. This will allow me to allow access to different user groups based on domain name. Also, If I wanted, I can leave out the host name and have one big file for each domain name. -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Rob Munsch Sent: Tuesday, March 14, 2006 4:03 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] DOMAIN Name Variable oh yes, of course i forgot. we certainly wouldn't want the US to stick .us on the end of their domains and create the impression of a global standard or anything, sigh... David Anderson wrote: >Rob, > >Monday, March 13, 2006, 11:47:52 PM, you wrote: > > > >>that would be the TLD, yes: but i got the impression jason means >>'domain' more like xyz.com foo.edu how.de >> >> > > > >>so therefore 'starting with the string before the last dot.' >> >> > >That doesn't work, as many domains outside of the US have a 3-part >domain, as in xyz.co.uk. > > > -- Rob Munsch Solutions For Progress IT _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html THE INFORMATION CONTAINED IN THIS MESSAGE AND ANY ATTACHMENT MAY BE PRIVILEGED, CONFIDENTIAL, PROPRIETARY OR OTHERWISE PROTECTED FROM DISCLOSURE. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, copying or use of this message and any attachment is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and permanently delete it from your computer and destroy any printout thereof.

2 1

RE: [syslog-ng] solaris UDP loss
by SOLIS, ALEX 14 Mar '06

14 Mar '06

Just in case: If the firewall is a Cisco PIX you might want to reconsider. If the PIX looses its ability to send TCP messages to its loghost then it denies all traffic until loghost connectivity is restored. ________________________________ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Cary, Kim Sent: Tuesday, March 14, 2006 12:30 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] solaris UDP loss Finally got around to reading the docs and applying the recommendations. FIRST - thanks much! NOW - my results: # ndd /dev/udp udp_recv_hiwat 65536 # ndd /dev/udp udp_max_buf 1073741824 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:16 udpInCksumErrs = 0 udpInOverflows =677996405 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:19 udpInCksumErrs = 0 udpInOverflows =677996571 # date ; netstat -s | grep udpInOverflows Tue Mar 14 10:15:22 udpInCksumErrs = 0 udpInOverflows =677996726 I'm pretty sure this is just the Firewall sending more than 55M lines of connection/deny logs over UDP. IF I was to switch the FW over to TCP logging from UDP, would this be a better solution? BTW, here are a few words from syslog-ng: Mar 14 09:32:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 09:42:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 09:52:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:02:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:04:53 syslog-ng[20685]: Garbage collecting while busy... Mar 14 10:04:58 syslog-ng[20685]: Objects alive: 310, garbage collected: 2999695 Mar 14 10:12:32 syslog-ng[20685]: STATS: dropped 0 Mar 14 10:16:22 syslog-ng[20685]: Garbage collecting while idle... Mar 14 10:16:24 syslog-ng[20685]: Objects alive: 318, garbage collected: 1129970 Mar 14 10:22:32 syslog-ng[20685]: STATS: dropped 0 and its config: source s_udp_inetpix { udp(ip(192.168.0.1)); }; On Mar 7, 2006, at 3:01 PM, syslog-ng-request(a)lists.balabit.hu wrote: | 1) Am I reading that loss right?? Probably, you might however want to snoop on the interface to see what kind of udp packets come on your interface. | 2) Any tips from Solaris/syslog-ng tuners would be appreciated! udp_max_buf does not set the queue length of the udp socket, which by the way can have a different value for each socket... You could have a look at: http://sunsolve.sun.com/search/document.do?assetkey=1-30-3218-1 basically: increasing udp_max_buf without increasing udp_recv_hiwat has no meaning. Furthermore, you can increase you socket buffer that way up to 64k (Solaris 8 & 9), if you want to increase it further up you must use the setsockopt call (up to udp_max_buf which has a maximum value of 1GB). Here is the official SUN documentation regarding this: http://docs.sun.com/app/docs/doc/817-0404/6mg74vsb5?a=view#gbtag Now regarding your packet loss issue. I would increase udp_recv_hiwat -> 65536 udp_max_buf -> 1073741824 (you will never get here anyway) Then I would try to play with syslog-ng config: log_fifo_size, log_iw_size and log_fetch_limit. But here I'd appreciate a syslog-ng expert to step in and tell us what to do more preceisely. This e-mail contains Omaha Public Power District's confidential and proprietary information and is for use only by the intended recipient. Unless explicitly stated otherwise, this e-mail is not a contract offer, amendment, nor acceptance. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

1 0

Re: syslog-ng Digest, Vol 11, Issue 20
by dwarnock＠harbrook.net 14 Mar '06

14 Mar '06

I am on holiday on Wednesday 15th March returning on Thursday 16th. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734. Regards David Warnock

2 1