Some device doesn't write to file
Hi, I just upgrade the hardware and software of the syslog-ng server to 1.60rc4 from 1.5 to log about 30 firewalls syslog. After upgarde, I did find that nearly half of the firewalls log doesn't write to the file. I did check with tcpdump and it did receive the tons of logs but did't log into the file. The iptables/ipchains has all been disabled. Is there any way to identify the source of problem. Thanks for your help. B. Regards, Santa
On Mon, Oct 27, 2003 at 04:36:21PM +0800, Santa Lau wrote:
Hi,
I just upgrade the hardware and software of the syslog-ng server to 1.60rc4 from 1.5 to log about 30 firewalls syslog. After upgarde, I did find that nearly half of the firewalls log doesn't write to the file. I did check with tcpdump and it did receive the tons of logs but did't log into the file. The iptables/ipchains has all been disabled. Is there any way to identify the source of problem. Thanks for your help.
I think you should attach strace to the syslog-ng process and check whether it really receives log messages (you should see recvfrom() lines for each message received), it might also be possible that syslog-ng blocks on DNS for example. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@lists.balabit.hu> Sent: Monday, October 27, 2003 4:41 PM Subject: Re: [syslog-ng]Some device doesn't write to file
On Mon, Oct 27, 2003 at 04:36:21PM +0800, Santa Lau wrote:
Hi,
I just upgrade the hardware and software of the syslog-ng server to 1.60rc4 from 1.5 to log about 30 firewalls syslog. After upgarde, I did find that nearly half of the firewalls log doesn't write to the file. I did check with tcpdump and it did receive the tons of logs but did't log into the file. The iptables/ipchains has all been disabled. Is there any way to identify the source of problem. Thanks for your help.
I think you should attach strace to the syslog-ng process and check whether it really receives log messages (you should see recvfrom() lines for each message received), it might also be possible that syslog-ng blocks on DNS for example.
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Thanks for your tips. I use strace to trace the network activity(strace -e network syslog-ng -F). I only found the IP which has logs. It is different from the result of tcpdump. B. Regards, Santa Lau Result from strace: .85.129.136")}}, [16]) = 237 recvfrom(3, "<144>HK1CUSTFW01: NetScreen devi"..., 2048, 0, {sin_family=AF_INET, sin_port=htons(2053), sin_addr=inet_addr("202.85.129.136")}}, [16]) = 238 recvfrom(3, "<180>Ocean_shores: NetScreen dev"..., 2048, 0, {sin_family=AF_INET, sin_port=htons(2053), sin_addr=inet_addr("202.82.19.35")}}, [16]) = 232 recvfrom(3, "<180>Ocean_shores: NetScreen dev"..., 2048, 0, {sin_family=AF_INET, sin_port=htons(2053), sin_addr=inet_addr("202.82.19.35")}}, [16]) = 232 recvfrom(3, "<180>Ocean_shores: NetScreen dev"..., 2048, 0, {sin_family=AF_INET, sin_port=htons(2053), sin_addr=inet_addr("202.82.19.35")}}, [16]) = 232 recvfrom(3, "<180>Ocean_shores: NetScreen dev"..., 2048, 0, {sin_family=AF_INET, sin_port=htons(2053), sin_addr=inet_addr("202.82.19.35")}}, [16]) = 232 Result from tcpdump: 16:54:09.842696 202.85.129.145.syslog > 202.85.170.92.syslog: udp 158 (ttl 250, id 45138, len 186) 16:54:09.843394 202.85.171.101.syslog > 202.85.170.92.syslog: udp 136 (ttl 253, id 28061, len 164) 16:54:09.850701 202.85.129.145.syslog > 202.85.170.92.syslog: udp 158 (ttl 250, id 45141, len 186) 16:54:09.862894 202.85.129.145.syslog > 202.85.170.92.syslog: udp 255 (ttl 250, id 45144, len 283) 16:54:09.864625 202.85.129.145.syslog > 202.85.170.92.syslog: udp 189 (ttl 250, id 45147, len 217) 16:54:09.869982 202.85.129.145.syslog > 202.85.170.92.syslog: udp 255 (ttl 250, id 45150, len 283) 16:54:09.878462 203.194.198.221.2053 > 202.85.170.92.syslog: udp 300 (ttl 59, id 40259, len 328) 16:54:09.880661 203.194.198.221.2053 > 202.85.170.92.syslog: udp 300 (ttl 59, id 40260, len 328) 16:54:09.889413 202.85.129.145.syslog > 202.85.170.92.syslog: udp 255 (ttl 250, id 45153, len 283) 16:54:09.895356 202.85.129.143.syslog > 202.85.170.92.syslog: udp 155 (ttl 250, id 13539, len 183) 16:54:09.908718 202.85.129.145.syslog > 202.85.170.92.syslog: udp 255 (ttl 250, id 45156, len 283) 16:54:09.920173 202.85.129.145.syslog > 202.85.170.92.syslog: udp 187 (ttl 250, id 45159, len 215) 16:54:09.925052 202.85.129.143.syslog > 202.85.170.92.syslog: udp 155 (ttl 250, id 13542, len 183) 16:54:09.926965 202.85.129.145.syslog > 202.85.170.92.syslog: udp 158 (ttl 250, id 45162, len 186) 16:54:09.928272 202.85.129.143.syslog > 202.85.170.92.syslog: udp 155 (ttl 250, id 13545, len 183)
On Mon, Oct 27, 2003 at 04:54:51PM +0800, Santa Lau wrote:
I think you should attach strace to the syslog-ng process and check whether it really receives log messages (you should see recvfrom() lines for each message received), it might also be possible that syslog-ng blocks on DNS for example.
Thanks for your tips. I use strace to trace the network activity(strace -e network syslog-ng -F). I only found the IP which has logs. It is different from the result of tcpdump.
Maybe your packet filter drops those messages? -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@lists.balabit.hu> Sent: Monday, October 27, 2003 5:12 PM Subject: Re: [syslog-ng]Some device doesn't write to file
On Mon, Oct 27, 2003 at 04:54:51PM +0800, Santa Lau wrote:
I think you should attach strace to the syslog-ng process and check whether it really receives log messages (you should see recvfrom() lines for each message received), it might also be possible that syslog-ng blocks on DNS for example.
Thanks for your tips. I use strace to trace the network activity(strace -e network syslog-ng -F). I only found the IP which has logs. It is different from the result of tcpdump.
Maybe your packet filter drops those messages?
--
Well. The ipchains/iptables has all been disabled. Is there any other locations which I should pay attention? B. Regards, Santa
On Mon, Oct 27, 2003 at 05:42:21PM +0800, Santa Lau wrote:
Well. The ipchains/iptables has all been disabled. Is there any other locations which I should pay attention?
If syslog-ng does not receive messages via recvfrom, but the box receives it, it can mean many things: 1) the packet filter drops packets 2) rp_filter drops packets 3) the destination IP is not local 4) the IP is local but syslog-ng listens on a different IP 5) the port is not correct 6) the UDP receive buffer overflows The first four cases are easy to confirm, please check that the packet headers as seen in tcpdump are destined to the box, syslog-ng listens on the correct interface/port (check via netstat -an). Can you see ICMP port unreachables as you receive messages? The last case is also possible, though I'm a bit skeptic as you told me that only specific hosts are missing from the log files. Check the recvq column in the netstat -an output. If this recvq value is never 0 you should increase the receive buffer size by increasing the values in /proc/sys/net/core/rmem_default and /proc/sys/net/core/rmem_max -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
participants (2)
-
Balazs Scheidler
-
Santa Lau