Re: [syslog-ng][PATCH] please test: syslog-ng message mangling fix
On Thu, Aug 08, 2002 at 08:47:01AM -0500, Caylan Van Larson wrote:
Mr. Scheidler,
May I ask you the correct way to apply this patch. I have applied kernel patches before but I am not aware of this syntax (not that it is different I just am curious on your "Right Way"). I thought it would be best to ask you instead of trying it 10 different ways hoping it worked.
unpack a vanilla 1.5.19 syslog-ng cd syslog-ng-1.5.19 patch -p1 <path to file> touch src/sources.c.x (the last one is needed if you don't have scsh, to satisfy make dependencies) rebuild. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
Hello, I tried to apply the patch and got this: [root@smack src]# patch -p0 < patchto.bazsi patching file sources.c Hunk #1 FAILED at 80. Hunk #2 FAILED at 114. 2 out of 2 hunks FAILED -- saving rejects to file sources.c.rej [root@smack src]# patch -p1 < patchto.bazsi missing header for unified diff at line 8 of patch can't find file to patch at input line 8 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- |Index: sources.c |=================================================================== |RCS file: /var/cvs/syslog-ng/syslog-ng/src/sources.c,v |retrieving revision 1.34 |diff -u -r1.34 sources.c |--- sources.c 18 Jul 2002 13:18:02 -0000 1.34 |+++ sources.c 8 Aug 2002 10:01:18 -0000 -------------------------- File to patch: sources.c patching file sources.c Hunk #1 FAILED at 80. Hunk #2 FAILED at 114. 2 out of 2 hunks FAILED -- saving rejects to file sources.c.rej I know what was going on here so I just edited sources.c by hand. Unfortunately after a complete deletion of the src directory, unpacking, copied over sources.c, ./configure, touch src/sources.c.x, make, make install. I restarted syslog-ng had a look at /var/log/kern and these entries were still happening: Aug 8 12:29:23 smack IPTABLES UDP-IN:etOUT= MAC44:00:05:01:fb:e3:fc:08:00 SRC=6EN=141 TOS=0x00 PREC=0x00 UDP SP3014 LEN=121 Aug 8 12:29:28 smack IPTABLES UDP-INOUT= MAC=00:03:47:4e:3:05:01:fb::00 SRC=64.12.51.129 DST=134.129 LEN=141 TOSPROTO=UDP SPT=53 DPTIN: IN=eth1 OUT= MAC=00:03:47:4eRC=64.12.51.129 D0x00 PREC=0x00 L=45 ID=47526 PROTO=UDP SPT=53 D<6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:005:fbSR SP<6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3fc:08 SRC=1329.9DS89 <6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.217.172 DST=134.129.212.30 LEN=244 TOS=0x00 PREC=0x00 TTL=127 ID=60809 PROTO=UDP SPT=138 DPT=138 LEN=224 Aug 8 12:29:29 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.212.30 LEN=78 TOS=0x00 PREC=0x00 TTL=2 I404LEN= Aug 8 12:29:29 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.12.C=0x0TL=12ID6 PRO Aug 8 12:29:30 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.212.30 LEN=78 TOS=0x00 PREC=0x00 TT=127 44047 PROTO=UDP SP37 DPT=137 LEN=58 Aug 8 12:29:33 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08: I have no clue :( Any ideas? Caylan Van Larson Unix Administrator - Systems Team Member University of North Dakota (Aerospace College) caylan@cs.und.edu 701-777-6151 (work) On Thu, 8 Aug 2002, Balazs Scheidler wrote:
On Thu, Aug 08, 2002 at 08:47:01AM -0500, Caylan Van Larson wrote:
Mr. Scheidler,
May I ask you the correct way to apply this patch. I have applied kernel patches before but I am not aware of this syntax (not that it is different I just am curious on your "Right Way"). I thought it would be best to ask you instead of trying it 10 different ways hoping it worked.
unpack a vanilla 1.5.19 syslog-ng cd syslog-ng-1.5.19 patch -p1 <path to file> touch src/sources.c.x
(the last one is needed if you don't have scsh, to satisfy make dependencies)
rebuild.
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Thu, Aug 08, 2002 at 12:31:20PM -0500, Caylan Van Larson wrote:
Aug 8 12:29:23 smack IPTABLES UDP-IN:etOUT= MAC44:00:05:01:fb:e3:fc:08:00 SRC=6EN=141 TOS=0x00 PREC=0x00 UDP SP3014 LEN=121 Aug 8 12:29:28 smack IPTABLES UDP-INOUT= MAC=00:03:47:4e:3:05:01:fb::00 SRC=64.12.51.129 DST=134.129 LEN=141 TOSPROTO=UDP SPT=53 DPTIN: IN=eth1 OUT= MAC=00:03:47:4eRC=64.12.51.129 D0x00 PREC=0x00 L=45 ID=47526 PROTO=UDP SPT=53 D<6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:005:fbSR SP<6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3fc:08 SRC=1329.9DS89 <6> IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.217.172 DST=134.129.212.30 LEN=244 TOS=0x00 PREC=0x00 TTL=127 ID=60809 PROTO=UDP SPT=138 DPT=138 LEN=224 Aug 8 12:29:29 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.212.30 LEN=78 TOS=0x00 PREC=0x00 TTL=2 I404LEN= Aug 8 12:29:29 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.12.C=0x0TL=12ID6 PRO Aug 8 12:29:30 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.120 DST=134.129.212.30 LEN=78 TOS=0x00 PREC=0x00 TT=127 44047 PROTO=UDP SP37 DPT=137 LEN=58 Aug 8 12:29:33 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:
I have no clue :(
ok, let me try to narrow the problem. Are these messages coming from /proc/kmsg directly, or they are forwarded over an UDP channel from another box? how long are these lines when they are not mangled? I had reports that there are some kind of line mangling, but all of them indicated 'rare' mangling. The log line you quoted above indicate that all of your log lines are mangled. Are these mangled also if you are using klogd and not using syslog-ng to fetch kernel messages? -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
Mr. Bazsi, I will try to clarify this for you. If I miss something just shoot me an email and I will answer it immeidately. When I say "mangled" the logs are cut off prematurely or squished in some spots, almost as if they are being folded. This "mangled" effect has happened when I use pipe or file to gather the kernel logs. Using the command "demsg" I am confronted with a console full of nice looking iptable logs. When these are compared to the logged versions you can see the apparent squishing. When syslog-ng is stopped and syslogd is started /var/log/messages is populated with perfect iptable logs. The syslog.conf file is at default configuration, while syslog-ng.conf is customized and attached below. The logs that I have been presenting to this list are from the local filesystem, although this machine is configured to send these out to a log host. I read that syslog-ng could replace klogd and I did this. However, the effects still showed that w/ only syslog-ng running the logs are still bad. Here are some examples of good iptable logs (from dmesg, line wrapped of course): IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=138.12.5.2 DST=134.129.212.30 LEN=127 TOS=0x00 PREC=0x00 TTL=243 ID=6173 DF PROTO=UDP SPT=53 DPT=33014 LEN=107 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.194 DST=134.129.212.30 LEN=252 TOS=0x00 PREC=0x00 TTL=127 ID=56906 PROTO=UDP SPT=138 DPT=138 LEN=232 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.223.141 DST=134.129.212.30 LEN=232 TOS=0x00 PREC=0x00 TTL=127 ID=16276 PROTO=UDP SPT=138 DPT=138 LEN=212 IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.201.29 DST=134.129.212.30 LEN=60 TOS=0x00 PREC=0x00 TTL=61 ID=10830 DF PROTO=TCP SPT=2715 DPT=53 WINDOW=32120 RES=0x00 SYN URGP=0 IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.111.111 DST=134.129.212.30 LEN=60 TOS=0x00 PREC=0x00 TTL=57 ID=49241 DF PROTO=TCP SPT=3475 DPT=53 WINDOW=32120 RES=0x00 SYN URGP=0 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.223.237 DST=134.129.212.30 LEN=229 TOS=0x00 PREC=0x00 TTL=127 ID=14673 PROTO=UDP SPT=138 DPT=138 LEN=209 Here is an example of what would show up in the logs (/var/log/kern): Aug 8 12:34:04 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 S=0PREC=0x00 TL=127 ID=5 0426 PROTO=UDSPT=137T=137 LEN=58 Aug 8 12:34:05 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 TOS=0x00 PRE Aug 8 12:34:08 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.215.54 DS 0x00 TTL=127 ID=36747 OTDP SPT=138PT= LEN=222 Aug 8 12:34:08 smack IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.139 DST=134.129.212.30 LEN=60 TO=0 PREC=0x00 TTL=63 ID=29205 DF PROTO=TCP 347 DPT=53 WINDOW=5840 RES=0x00 URGP=0 Aug 8 12:34:08 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 T0x00 PREC=0x00 TTL=127 ID=50430 PROTO=UDP SPT=137 DPT=137 LEN=58
Are these mangled also if you are using klogd and not using syslog-ng to fetch kernel messages?
About that... Hmm, what is the best way to just have klogd log kernel messages? Maybe they are being "interpreted" twice and thus mucking something up. I am willing to try anything that you suggest :) If there is anything else I can do for you let me know. I am very eager to put this to rest. Also, did you see any errors with the way I patched the source.c file? I might have possible made a mistake and the patched version was never built properly thus making the patch a viable option to continue persuing? Thanks a million! (syslog-ng.conf attached below) Caylan --SNIP # global options # options { use_dns(yes); use_fqdn(no); use_time_recvd(no); chain_hostnames(no); mark(0); sync(0); }; # sources # source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg"); }; # facility filters # filter f_authpriv { facility(authpriv); }; filter f_auth { facility(auth); }; filter f_boot { facility(local7); }; filter f_2511 { facility(local5); }; filter f_6509-1-log { facility(local4); }; filter f_6509-2-log { facility(local3); }; filter f_cron { facility(cron); }; filter f_kern { facility(kern); }; filter f_user { facility(user); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_daemon { facility(daemon); }; filter f_messages { priority(info..emerg) and not facility(mail, news, authpriv, cron, local1, local2, local3, local4, local5, local6); }; filter f_news { facility(news); }; # priority filters # filter f_emerg { priority(emerg); }; filter f_crit { priority(crit..emerg); }; filter f_crit_only { priority(crit); }; filter f_err { priority(err..emerg); }; filter f_err_only { priority(err); }; filter f_warn { priority(warning..emerg); }; filter f_notice { priority(notice..emerg); }; filter f_info { priority(info..emerg); }; filter f_debug { priority(debug..emerg); }; # host filters # filter f_smack { host(smack); }; #destination filters # # *network* destination d_tcp { tcp("134.129.212.33"); }; destination d_udp { udp("134.129.212.33"); }; # *everyone* destination d_all { usertty("*"); }; # *console* destination d_console { file("/dev/console"); }; # *boot* destination d_smacboot { file("/var/log/bootlog"); }; # *cron* destination d_smaccron { file("/var/log/cron"); }; # *mail* destination d_smacmail { file("/var/log/maillog"); }; # *messages* destination d_smacmsg { file("/var/log/messages"); }; # *secure (auth & authpriv)* destination d_smacsec { file("/var/log/secure"); }; # *user* destination d_smacuser { file("/var/log/user"); }; # *kern* destination d_smackern { file("/var/log/kern"); }; # *daemon* destination d_smacdaemon { file("/var/log/daemon"); }; # *spool (lpr)* destination d_smacspool { file("/var/log/spooler"); }; #Everyone gets emergency messages log { source(s_all); filter(f_emerg); destination(d_all); }; #Log messages from Smack log { source(s_all); filter(f_cron); filter(f_debug); filter(f_smack); destination(d_smaccron); destination(d_tcp); }; log { source(s_all); filter(f_authpriv); filter(f_debug); filter(f_smack); destination(d_smacsec); destination(d_tcp); }; log { source(s_all); filter(f_mail); filter(f_warn); filter(f_smack); destination(d_smacmail); destination(d_tcp); }; log { source(s_all); filter(f_boot); filter(f_debug); filter(f_smack);destination(d_smacboot); destination(d_tcp); }; # fw-iptables logs at NOTICE <5> (fragments/unknown protocols) and INFO <6> (known udp/tcp/icmp) # This line will log ALL of kern locally log { source(s_all); filter(f_kern); filter(f_messages); filter(f_debug); filter(f_smack); destination(d_smackern); }; # This line will only remotely log NOTICE <5> and above (5,4,3,2,1,0) log { source(s_all); filter(f_kern); filter(f_messages); filter(f_notice); filter(f_smack); destination(d_tcp); }; log { source(s_all); filter(f_user); filter(f_debug); filter(f_smack); destination(d_smacuser); destination(d_tcp); }; log { source(s_all); filter(f_lpr); filter(f_debug); filter(f_smack); destination(d_smacspool); destination(d_tcp); }; log { source(s_all); filter(f_daemon); filter(f_notice); filter(f_smack); destination(d_smacdaemon); destination(d_tcp); }; --SNAP Some line wrapping occured in PINE. Thanks ~ Caylan
On Thu, Aug 08, 2002 at 01:53:29PM -0500, Caylan Van Larson wrote:
Mr. Bazsi,
I will try to clarify this for you. If I miss something just shoot me an email and I will answer it immeidately.
When I say "mangled" the logs are cut off prematurely or squished in some spots, almost as if they are being folded. This "mangled" effect has happened when I use pipe or file to gather the kernel logs. Using the command "demsg" I am confronted with a console full of nice looking iptable logs. When these are compared to the logged versions you can see the apparent squishing. When syslog-ng is stopped and syslogd is started /var/log/messages is populated with perfect iptable logs. The syslog.conf file is at default configuration, while syslog-ng.conf is customized and attached below. The logs that I have been presenting to this list are from the local filesystem, although this machine is configured to send these out to a log host.
I read that syslog-ng could replace klogd and I did this. However, the effects still showed that w/ only syslog-ng running the logs are still bad.
Here are some examples of good iptable logs (from dmesg, line wrapped of course): IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=138.12.5.2 DST=134.129.212.30 LEN=127 TOS=0x00 PREC=0x00 TTL=243 ID=6173 DF PROTO=UDP SPT=53 DPT=33014 LEN=107 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.194 DST=134.129.212.30 LEN=252 TOS=0x00 PREC=0x00 TTL=127 ID=56906 PROTO=UDP SPT=138 DPT=138 LEN=232 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.223.141 DST=134.129.212.30 LEN=232 TOS=0x00 PREC=0x00 TTL=127 ID=16276 PROTO=UDP SPT=138 DPT=138 LEN=212 IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.201.29 DST=134.129.212.30 LEN=60 TOS=0x00 PREC=0x00 TTL=61 ID=10830 DF PROTO=TCP SPT=2715 DPT=53 WINDOW=32120 RES=0x00 SYN URGP=0 IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.111.111 DST=134.129.212.30 LEN=60 TOS=0x00 PREC=0x00 TTL=57 ID=49241 DF PROTO=TCP SPT=3475 DPT=53 WINDOW=32120 RES=0x00 SYN URGP=0 IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.223.237 DST=134.129.212.30 LEN=229 TOS=0x00 PREC=0x00 TTL=127 ID=14673 PROTO=UDP SPT=138 DPT=138 LEN=209
Here is an example of what would show up in the logs (/var/log/kern): Aug 8 12:34:04 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 S=0PREC=0x00 TL=127 ID=5 0426 PROTO=UDSPT=137T=137 LEN=58 Aug 8 12:34:05 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 TOS=0x00 PRE Aug 8 12:34:08 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.215.54 DS 0x00 TTL=127 ID=36747 OTDP SPT=138PT= LEN=222 Aug 8 12:34:08 smack IPTABLES TCP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.139 DST=134.129.212.30 LEN=60 TO=0 PREC=0x00 TTL=63 ID=29205 DF PROTO=TCP 347 DPT=53 WINDOW=5840 RES=0x00 URGP=0 Aug 8 12:34:08 smack IPTABLES UDP-IN: IN=eth1 OUT= MAC=00:03:47:4e:32:44:00:05:01:fb:e3:fc:08:00 SRC=134.129.214.231 DST=134.129.212.30 LEN=78 T0x00 PREC=0x00 TTL=127 ID=50430 PROTO=UDP SPT=137 DPT=137 LEN=58
I've installed syslog-ng on a host with kernel 2.4 and iptables installed. I added a rule iptables -A INPUT -j LOG to log all traffic. I generated network traffic using ping, and ssh, all messages were logged fine to local logs. can you strace syslog-ng while running? for example try a simple configuration: source src { file("/proc/kmsg"); internal(); }; destination dst { file("/var/log/kern.log"); }; log { source(src); destination(dst); }; this syslog-ng doesn't handle local messages only reads kernel messages. stop any daemons that use /proc/kmsg (klogd, the other syslog-ng) run this syslog-ng (syslog-ng -f <pathtonewsyslog-ngconf>) and start strace: strace -o syslog-ng.strace -s 256 -p <syslogngpid> generate some iptables lines that are mangled. please send me the logfile, the strace output, and a dmesg output in private. Today is the only chance to find the problem, as I'm leaving for holidays tomorrow. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
participants (2)
-
Balazs Scheidler
-
Caylan Van Larson