cisco tcp syslog weirdness/merkwuerdigkeit
When a certain cisco router is set to UPD syslog delivery it creates a unique message for each unique message, duh ! But when this same router is set to TCP syslog, it takes all messages and tacks them back to back until syslog-ng runs out of buffer space in one line. I kept increasing the message log size, but the real problem is that the messages would have to be parsed out of this massive long line. My router guy says he can't make the cisco router behave any differently. How do I handle this problem ? Ideas ? Thanks
On Mon, 2007-08-20 at 17:12 -0400, Blurry wrote:
When a certain cisco router is set to UPD syslog delivery it creates a unique message for each unique message, duh ! But when this same router is set to TCP syslog, it takes all messages and tacks them back to back until syslog-ng runs out of buffer space in one line. I kept increasing the message log size, but the real problem is that the messages would have to be parsed out of this massive long line. My router guy says he can't make the cisco router behave any differently. How do I handle this problem ? Ideas ?
Is there any kind of line separator? Can you post a tcpdump or something similar that shows what is sent by the router? Thanks. -- Bazsi
Here is a sample, first some nice ones Jul 25 13:43:04 144.49.126.22/144.49.126.22 GET Jul 25 13:43:07 144.49.126.22/144.49.126.22 HELLO Jul 25 13:43:13 144.49.126.22/144.49.126.22 quit then Aug 20 09:59:13 tcpgateway@thishost syslog-ng[12107]: Message length overflow, line is split, log_msg_size=8192 Aug 20 10:27:53 router01/router01 ernet1/0<191>11463: Aug 20 10:25:52.617 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11464: aid:144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64274 from FastEthernet1/0<191>11465: Aug 20 10:26:02.617 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11466: aid: 144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64276 from FastEthernet1/0<191>11467: Aug 20 10:26:12.625 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11468: aid:144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64278 from FastEthernet1/0<191>11469: Aug 20 10:26:22.625 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11470: aid:14.1.0.0 chk:0 aut:2 keyid:1 seq:0xC6427A from and continues on for a very long time on one line and then cuts off. There doesn't seem to be a field sep that I can tell in the file. I will try a tcpdump also. Thanks On 8/22/07, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Mon, 2007-08-20 at 17:12 -0400, Blurry wrote:
When a certain cisco router is set to UPD syslog delivery it creates a unique message for each unique message, duh ! But when this same router is set to TCP syslog, it takes all messages and tacks them back to back until syslog-ng runs out of buffer space in one line. I kept increasing the message log size, but the real problem is that the messages would have to be parsed out of this massive long line. My router guy says he can't make the cisco router behave any differently. How do I handle this problem ? Ideas ?
Is there any kind of line separator? Can you post a tcpdump or something similar that shows what is sent by the router?
Thanks.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Wed, 2007-08-22 at 08:21 -0400, Blurry wrote:
Here is a sample, first some nice ones
Jul 25 13:43:04 144.49.126.22/144.49.126.22 GET Jul 25 13:43:07 144.49.126.22/144.49.126.22 HELLO Jul 25 13:43:13 144.49.126.22/144.49.126.22 quit
then
Aug 20 09:59:13 tcpgateway@thishost syslog-ng[12107]: Message length overflow, line is split, log_msg_size=8192 Aug 20 10:27:53 router01/router01 ernet1/0<191>11463: Aug 20 10:25:52.617 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11464: aid:144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64274 from FastEthernet1/0<191>11465: Aug 20 10:26:02.617 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11466: aid: 144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64276 from FastEthernet1/0<191>11467: Aug 20 10:26:12.625 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11468: aid:144.1.0.0 chk:0 aut:2 keyid:1 seq:0xC64278 from FastEthernet1/0<191>11469: Aug 20 10:26:22.625 EDT: OSPF: rcv. v:2 t:1 l:48 rid:144.63.255.232<191>11470: aid:14.1.0.0 chk:0 aut:2 keyid:1 seq:0xC6427A from
and continues on for a very long time on one line and then cuts off. There doesn't seem to be a field sep that I can tell in the file. I will try a tcpdump also.
A tcpdump would be helpful, as syslog-ng might filter out some characters as it writes to the output. If there's no linetermination, then I'm afraid I cannot help here. The message itself can contain <NNN> sequences, so I can't split lines there. -- Bazsi
I am not sure what to expect from tcp dump, but I don't see much that matches between the log file and the tcp dump file expect hostnames and timestamps. 15:00:14.401603 IP (tos 0x0, ttl 250, id 24720, offset 0, flags [none], proto TC P (6), length 576) router.57230 > loghost.1514: . 40597:41133(536) ack 1 win 4128 15:00:14.415798 IP (tos 0x0, ttl 64, id 48307, offset 0, flags [DF], proto TCP ( 6), length 40) loghost.1514 > router.57230: ., cksum 0x61bb (incorrect (-> 0x0b66), 1:1(0) ack 41133 win 48776 15:00:14.416512 IP (tos 0x0, ttl 250, id 24721, offset 0, flags [none], proto TCP (6), length 571) router.57230 > loghost.1514: P 41133:41664(531) ack 1 win 4128 15:00:14.465815 IP (tos 0x0, ttl 64, id 48308, offset 0, flags [DF], proto TCP ( 6), length 40) loghost.1514 > router.57230: ., cksum 0x61bb (incorrect (-> 0x073b), 1:1(0) ack 41664 win 49312 I still get one very long line in the log file. The router guy says that he just turns on 'TCP; syslog and it all comes in one line. Very frustrating. Thanks
A tcpdump would be helpful, as syslog-ng might filter out some characters as it writes to the output.
If there's no linetermination, then I'm afraid I cannot help here. The message itself can contain <NNN> sequences, so I can't split lines there.
On Wed, 2007-08-22 at 15:27 -0400, Blurry wrote:
I am not sure what to expect from tcp dump, but I don't see much that matches between the log file and the tcp dump file expect hostnames and timestamps.
try this on the syslog-ng host: # tcpdump -s0 -w /tmp/syslog-ng.dump dst port 514 then attach the dump file in an email. -- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot
Ok that output was quite different, with some non-printable chars. I didn't want to meial it to everyone, but it is quite small. it is here http://20v.org/tmp/cap.gz looks a bit like Aug 22 16:47:56.298 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47303: Aug 22 16:47:56.298 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47304: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47305: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47306: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47307: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0F .202, dst 155.2.254.250<47>47308: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47309: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250 Thanks On 8/22/07, Matt Zagrabelny <mzagrabe@d.umn.edu> wrote:
On Wed, 2007-08-22 at 15:27 -0400, Blurry wrote:
I am not sure what to expect from tcp dump, but I don't see much that matches between the log file and the tcp dump file expect hostnames and timestamps.
try this on the syslog-ng host:
# tcpdump -s0 -w /tmp/syslog-ng.dump dst port 514
then attach the dump file in an email.
-- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2
On Wed, 2007-08-22 at 17:03 -0400, Blurry wrote:
Ok that output was quite different, with some non-printable chars. I didn't want to meial it to everyone, but it is quite small. it is here http://20v.org/tmp/cap.gz
Something went awry. I get the following error from Wireshark: The file "/tmp/cap" is a capture for a network type that Wireshark doesn't support. (pcap: network type 4095878165 unknown or unsupported) Try again. (hit ^C when done capturing) # tcpdump -s0 -w /tmp/syslog-ng.dump dst port 514
looks a bit like Aug 22 16:47:56.298 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47303: Aug 22 16:47:56.298 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47304: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47305: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47306: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47307: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0F .202, dst 155.2.254.250<47>47308: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250<47>47309: Aug 22 16:47:56.302 EDT: ICMP: echo reply rcvd, src 77.22.0.202, dst 155.2.254.250
Thanks
On 8/22/07, Matt Zagrabelny <mzagrabe@d.umn.edu> wrote:
On Wed, 2007-08-22 at 15:27 -0400, Blurry wrote:
I am not sure what to expect from tcp dump, but I don't see much that matches between the log file and the tcp dump file expect hostnames and timestamps.
try this on the syslog-ng host:
# tcpdump -s0 -w /tmp/syslog-ng.dump dst port 514
then attach the dump file in an email.
-- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html -- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2
He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot
participants (3)
-
Balazs Scheidler
-
Blurry
-
Matt Zagrabelny