August 2002 - syslog-ng - lists.balabit.hu

RE: [syslog-ng] logging pauses and log entry truncation
by Dustin Trammell 14 Aug '02

14 Aug '02

One difference I noticed between your log entries and mine is, yours appear to either get mangled and/or truncated, and then that's it, it continues on the next line with a new log entry. I've seen a bit different behavior in mine, where it will mangle and/or truncate a line, but then on the next entry, it continues with the rest of the previous entry. It's like when reading from /proc/kmsg it's erroneously detecting an end of line, sending that bit of data to the destinations, then continues reading from /proc/kmsg, starting where it cut off. For example: Aug 14 10:17:46 loghost01 fwlog input drop cleanup IN=eth0 OUT= MAC=00:06: Aug 14 10:17:46 loghost01 5b:84:60:7e:00:50:04:a5:11:1f:08:00 SRC=172.30.10.33 DST=172.30.8.40 LEN=60 TOS=0x00 PREC=0x00 TTL=219 ID=61008 DF PROTO=TCP SPT=54189 DPT=32601 WINDOW=5840 RES=0x00 SYN URGP=0 Aug 14 10:17:52 loghost01 <4>fwlog input drop cleanup IN=eth0 OUT= MAC=00: Aug 14 10:17:52 loghost01 06:5b:84:60:7e:00:50:04:a5:11:1f:08:00 SRC=172.30.10.33 DST=172.30.8.40 LEN=60 TOS=0x00 PREC=0x00 TTL=219 ID=36311 DF PROTO=TCP SPT=54189 DPT=32601 WINDOW=5840 RES=0x00 SYN URGP=0 As you can see, the first line gets truncated, then on the next line, it continues with the rest of the log entry starting where the previous one got cut off. While writing this, I just saw the most mangled entry I've seen yet: Aug 14 10:16:15 loghost01 fwlog input drop clenup Ib:84:602. PREC=0x00 TTL=79 ID=T=TCP S SYN TCP i OUT MAC=00:06:5b:84172.3072.=5840 RES=0x00 N=0 As opposed to what it /should/ look like: Aug 14 10:17:46 loghost01 fwlog input drop cleanup IN=eth0 OUT= MAC=00:06:5b:84:60:7e:00:50:04:a5:11:1f:08:00 SRC=172.30.10.33 DST=172.30.8.40 LEN=60 TOS=0x00 PREC=0x00 TTL=219 ID=45827 DF PROTO=TCP SPT=38821 DPT=32606 WINDOW=5840 RES=0x00 SYN URGP=0 Weird weird stuff. Hopefully some of this discussion and related log snippets will help Balazs pinpoint the problem. --- Dustin D. Trammell Information Security Specialist Penson Financial Services, Inc. -----Original Message----- From: Caylan Van Larson [mailto:caylan@cs.und.edu] Sent: Wednesday, August 14, 2002 10:08 To: 'syslog-ng(a)lists.balabit.hu' Subject: RE: [syslog-ng] logging pauses and log entry truncation Yes, similar, but also erratic. I have it happening in the MAC addy and toward the end. Sometimes it seems as they are folded in on itself. I have placed a sample on www.cs.und.edu/~caylan/kern for your viewing pleasure. It is not my sytle to make things like this publicly accessible but I want to get this issue resolved. In case you did not see the above URL or are too lazy :P Aug 14 10:02:30 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08: Aug 14 10:02:31 smack IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=63.151.197.164 DST=134.129.212.30 LEN=48 TOS=0x00 PREC=0x00TT14 2103DF PRTCP SW=163 Aug 14 10:02:32 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08: Aug 14 10:02:32 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc: Aug 14 10:02:32 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.215.35 DST=1PSPT7 DPT=137 L Aug 14 10:02:32 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08: Aug 14 10:02:33 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.215.35 DST=134.129.212.30 LEN=9 138 N=20 Aug 14 10:02:33 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08: Thats a pretty damn good sample! Good luck, Caylan Van Larson Unix Administrator - Systems Team Member University of North Dakota (Aerospace College) caylan(a)cs.und.edu 701-777-6151 (work)

2 1

RE: [syslog-ng] logging pauses and log entry truncation
by Dustin Trammell 14 Aug '02

14 Aug '02

I found out why syslog-ng was doing the pausing... I had accidentally left two source entries for /proc/kmsg in my syslog-ng.conf file (I was originally trying to see if using file() vs. pipe() made any difference in the log entry truncation), so I assume it was a locking issue with both of them trying to read from the same place. After removing one of them, I haven't had syslog-ng pause on me once, but I am still getting the log entry truncation on entries from iptables. It usually truncates on iptables log entries that have an empty OUT= tag (no out interface), and truncates just after the OUT= tag and before the MAC= tag, or somewhere halfway through the MAC address in the MAC= tag. Are you getting the truncation at the same place on the same types of log entries? --- Dustin D. Trammell Information Security Specialist Penson Financial Services, Inc. -----Original Message----- From: Caylan Van Larson [mailto:caylan@cs.und.edu] Sent: Monday, August 12, 2002 15:54 To: Dustin Trammell Cc: 'syslog-ng(a)lists.balabit.hu' Subject: Re: [syslog-ng]logging pauses and log entry truncation I am having very similar truncating going on. Bazsi is working on a fix. However, my logs never paused, maybe for a little bit (5-10seconds) but that is prolly just net traffic jumps. Good luck to Bazsi!!!

2 1

Re: [syslog-ng]1.5.19 and tcp
by Abraham, Ajamu 14 Aug '02

14 Aug '02

-------------------------- Sent from my BlackBerry Wireless Handheld

1 0

klogd and sysctl
by Frank Crawford 10 Aug '02

10 Aug '02

Folks, One thing I noticed on eliminating klogd from my Red Hat 7.3 system was that kernel messages were then being printed on the console. Looking further into it, it seems that klogd invokes a sysctl option to suppress certain messages (in particular packet filtering messages). The same effect can be managed with the command: /sbin/sysctl -n -w kernel.printk=6 on starting syslog-ng, and: /sbin/sysctl -n -w kernel.printk=7 on shutdown. One other item I did to make it a total drop in was write the pid file to /var/run/syslogd.pid, by invoking syslog-ng with: /usr/local/sbin/syslog-ng -p /var/run/syslogd.pid (These are all items that could be in the FAQ.) Frank Crawford -- ac3 Suite G16, Bay 7, Locomotive Workshop Phone: 02 9209 4600 Australian Technology Park Fax: 02 9209 4611 Eveleigh NSW 1430

1 0

Penny L Creech/Nationwide/NWIE is out of the office.
by CREECHP＠Nationwide.com 10 Aug '02

10 Aug '02

I will be out of the office starting 08/09/2002 and will not return until 08/19/2002. I will respond to your message when I return.

1 0

RE: [syslog-ng]Installing on a non-compiler system
by Clausing, James A (Jim), SOLCM 09 Aug '02

09 Aug '02

I build packages to do this all the time (the only way I'll do it). I would be happy to share my Makefile (which gunzips, untars, configures, makes, installs, and pkgmks). I haven't yet added the logic to it to also build & install libol, but I only have that installed on my development/build systems, not where syslog-ng is actually deployed. ---Jim -----Original Message----- From: Nicholas Berry [mailto:nberry@ragingwire.com] Sent: Tuesday, July 30, 2002 11:26 AM To: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]Installing on a non-compiler system You could build syslog-ng in a Solaris pkg format, but I'm not sure how much of a problem you would have installing the package on your Solaris 2.6 system (might not have a problem if you're up-to-date with the recommended patch cluster). Nicholas Berry Systems Engineer RagingWire Telecommunications, Inc. nberry(a)ragingwire.com -----Original Message----- From: Lightfoot.Michael [mailto:Lightfoot.Michael@comcare.gov.au] Sent: Monday, July 29, 2002 8:02 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Installing on a non-compiler system I have built syslog-ng on a development server (Solaris 9) and now wish to install it on another system which doesn't have development tools (it is a fairly standard Solaris 9 installation.) I will also be installing a Solaris 2.6 build on systems in our DMZ which have been hardened and have very little other than the OS. The problem I have struck is that after copying the built source to the target server, "make install" fails with the following error: # make install Making install in src Making install in . make: Fatal error: Don't know how to make target `/usr/local/src/syslog-ng-1.4.15/configure.in' Current working directory /export/home/syslog-ng-1.4.15/src *** Error code 1 make: Fatal error: Command failed for target `install-recursive' Current working directory /export/home/syslog-ng-1.4.15/src *** Error code 1 make: Fatal error: Command failed for target `install-recursive' Any ideas on how I get around this easily? NFS is one option, but will not be for the DMZ systems (and is not used in production on any systems here.) I suppose I could install it on the development server using a filesystem diff before and after trick I have used in the past, but again there are problems doing this with the Solaris 2.6 build (which has to be done on a production server as I don't have a Solaris 2.6 dev machine any more.) Michael Lightfoot Unix Consultant ISG Host Systems Comcare +61 2 62750680 NOTICE: This e-mail message and attachments may contain confidential information. If you are not the intended recipient you should not use or disclose any information in the message or attachments. If received in error, please notify the sender by return email immediately. Comcare does not waive any confidentiality or privilege. Comcare's Canberra and Melbourne offices have moved to new premises. We hope this notice makes it easier for you to update your records. New street addresses: Canberra office Level 1 14 Moore street CANBERRA CITY ACT 2601 Melbourne Office Level 2 121 William Street MELBOURNE VIC 3001 Claims Fax Number 03 9620 4721 Please call Comcare on 1300 366 979 if you have any queries. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

CRITICAL: disk flooding bug - disabling of STATS seems to be broken in 1.5.19
by Dominique Kaiser 09 Aug '02

09 Aug '02

Disabling STATS reporting with options "{ stats(0); }" doesn't work anymore! Instead syslog-ng gets busy reporting STATS all the time without pause - and succeeds in eating up all free disk space in a matter of minutes. If you don't have an extra mount for /var/log your whole system is rendered inoperatible... the upstream Changelog shows an entry from 2002-06-17 concerning STATS which most likely describes the change causing the bug. I didn't fully study the sources but on first sight it seems to me that the error is in main_loop() in src/main.c in the lines 188 and 204. I think it should be if (s.timeout > 0) instead of if (s.timeout <= 0) especially as the callback timers seem to get deleted in line 201. The way it is now STATS might be disabled by providing any positive config value??!! The problem might also be solved if one could use "stats(-1);" to deactivate STATS reporting but syslog-ng doesn't seem to accept negative values (which might be another bug on itself?). In the default installation the default value of 600 seconds is used, so this bug is not a problem if syslog-ng is used with unchanged config options. But whoever does want his/her disks tpo spin down at all will have tried to disable STATS... Cheers, Dominique. P.S.: also reported as Bug #156045 to the Debian BTS... P.P.S.: I'm not a member of the syslog-ng mainling list...

3 2

Re: [syslog-ng][PATCH] please test: syslog-ng message mangling fix
by Balazs Scheidler 09 Aug '02

09 Aug '02

On Thu, Aug 08, 2002 at 08:47:01AM -0500, Caylan Van Larson wrote: > Mr. Scheidler, > > May I ask you the correct way to apply this patch. I have applied kernel > patches before but I am not aware of this syntax (not that it is different > I just am curious on your "Right Way"). I thought it would be best to ask > you instead of trying it 10 different ways hoping it worked. unpack a vanilla 1.5.19 syslog-ng cd syslog-ng-1.5.19 patch -p1 <path to file> touch src/sources.c.x (the last one is needed if you don't have scsh, to satisfy make dependencies) rebuild. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

2 4

"message repeated N times"
by Andan Lauber 09 Aug '02

09 Aug '02

heya, i've not seen it do it, but i've only the one host running syslog-ng as a central loghost, with generic syslogs reporting to it, but does syslog-ng deal with repeat messages like generic syslog does, in this fashion? host.domain.tld message repeated 4 times or does it just spew out the message again? thanks. (and as a side, if any know of a way to STOP generic syslog from doing this, i'd love to hear it. i'm hoping the answer is "install syslog-ng") thanks. -- Andan Lauber Redback Networks, Inc. Daemonologist http://www.redback.com

2 1

Re: [syslog-ng]1.5.19 and tcp
by CHENNES＠Nationwide.com 09 Aug '02

09 Aug '02

The truss output also shows that the process is ultimately killed. Are you saying you suspect an "external force" is killing the process. Balazs Scheidler <bazsi(a)balabit.hu> T To: syslog-ng(a)lists.balabit.hu Sent by: cc: syslog-ng-admin(a)lists.b alabit.hu bcc: Subject: Re: [syslog-ng]1.5.19 and tcp 08/08/02 03:28 AM Please respond to syslog-ng On Wed, Aug 07, 2002 at 02:09:06PM -0400, CHENNES(a)Nationwide.com wrote: > > Here's the gdb backtrace: > > gdb syslog-ng -c core > GNU gdb 4.18 > Copyright 1998 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "sparc-sun-solaris2.8"... > Core was generated by `/test/syslog/bin/syslog-ng -f /etc/syslog-ng.conf -p > /etc/s'. > Program terminated with signal 9, Killed. > Reading symbols from /usr/lib/libpthread.so.1...done. > Reading symbols from /usr/lib/libdoor.so.1...done. > Reading symbols from /usr/lib/libl.so.1...done. > Reading symbols from /usr/lib/libsocket.so.1...done. > Reading symbols from /usr/lib/libnsl.so.1...done. > Reading symbols from /usr/lib/libxnet.so.1...done. > Reading symbols from /usr/lib/libc.so.1...done. > Reading symbols from /usr/lib/libdl.so.1...done. > Reading symbols from /usr/lib/libmp.so.2...done. > Reading symbols from > /usr/platform/SUNW,Sun-Fire-880/lib/libc_psr.so.1...done. > Reading symbols from /usr/lib/libthread.so.1...done. > #0 0xff141d00 in _malloc_unlocked () from /usr/lib/libc.so.1 > (gdb) bt > #0 0xff141d00 in _malloc_unlocked () from /usr/lib/libc.so.1 > #1 0xff141ad0 in malloc () from /usr/lib/libc.so.1 > #2 0x2219c in xalloc () > #3 0x17978 in make_log_info (length=0, > msg=0x3d8d8 "<13>Aug 7 13:59:21 test/test user: [ID 702911 > user.notice] test message number 3\n<13>Aug 7 13:59:21 test/ > test user: [ID 702911 user.notice] test message number 4\n<13>Aug 7 > 13:59:21 "..., prefix=0x0) at log.c:263 > #4 0x15be8 in do_handle_line (self=0xffbefa90, length=89, > data=0x3d8d8 "<13>Aug 7 13:59:21 test/test user: [ID 702911 > user.notice] test message number 3\n<13>Aug 7 13:59:21 test > /test user: [ID 702911 user.notice] test message number 4\n<13>Aug 7 > 13:59:21 "..., addr=0x3d8d8, addrlen=0) > at sources.c:67 > #5 0x15de0 in do_read_line (h=0x3d8a0, read=0xffbefa90) at sources.c:134 > #6 0x202f4 in read_callback () > #7 0x1fed4 in io_iter () > #8 0x152dc in main_loop (backend=0x395d8) at main.c:191 > #9 0x159ec in main (argc=6, argv=0xffbefd54) at main.c:513 > (gdb) hmm.. this is not a segfault. the process is killed with signal SIGKILL. the original truss output indicated a segfault. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1