RE: [syslog-ng] Syslog-ng 1.6.9 just stops...
I had klogd off already, but this got me thinking about something else. My config file's sources are: source s_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); udp(); internal(); }; source t_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); tcp(); internal(); }; Could it be that both sources, attempting to read /proc/kmsg and /dev/log (and internal()?) are causing this? Thanks a lot! Tony -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Wednesday, March 01, 2006 15:25 To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng 1.6.9 just stops... On Wed, 2006-03-01 at 10:01 -0500, Andreoli, Tony A. USNUNK NAVAIR B1490, R215 wrote:
I'm using 1.6.9 (upgraded from 1.6.6 because I was seeing the same problem). I have it running on 8 different servers at different locations, some are SMP, some aren't. On these hosts, we have anywhere from 2 to 14 devices logging to the servers, some via 514/tcp, others via 514/udp. All of the loggers typically sit with a load average < 1 (usually not even registering), and a cpu idle of 99%. 7 of these remote loggers also log to our local machine, but only 5 lines every 2 minutes (for stats).
What I've noticed (and I've seen this on all of them at one time or another), is that syslog-ng just stops. ps shows it running, but the log file (/logs/messages) never changes. If I tcpdump on the interface that it's listening on, I see traffic, and it seems that the
act of tcpdumping causes the log file to start to grow again, then a little while later, it may stop again. It's sporadic though, on one of my systems, it hasn't done it in over 2 months, on another, it's done it 3 times today.
I've pulled out my last hair and still haven't come any closer to a solution. I've recompiled the source, loaded 3 different versions, etc. The only thing common is that all of these systems are running RHEL3.
Don't you happen to read /proc/kmsg by both syslog-ng and klogd ? That is a known bad situation and the symptoms are exactly what you describe. (poll indicates readability but by the time syslog-ng gets to read the file the data has already been read) This is documented in the FAQ as well. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Andreoli, Tony A. USNUNK NAVAIR B1490, R215 wrote:
I had klogd off already, but this got me thinking about something else. My config file's sources are:
source s_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); udp(); internal(); }; source t_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); tcp(); internal(); };
Could it be that both sources, attempting to read /proc/kmsg and /dev/log (and internal()?) are causing this?
Never use the same log source (pipe, stream, diagram) two or more times. You should reorganize your source sections: source s_common { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream("/dev/log"); internal(); }; source s_udp { udp(); }; source s_tcp { tcp(); }; And modify your log sections accordingly. -- Sandor Geller wildy@balabit.hu
Thanks for the info. Could this be the cause of my hangs? -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu on behalf of Sandor Geller Sent: Thu 3/2/2006 3:20 AM To: Syslog-ng users' and developers' mailing list Cc: Subject: Re: [syslog-ng] Syslog-ng 1.6.9 just stops... Andreoli, Tony A. USNUNK NAVAIR B1490, R215 wrote: > I had klogd off already, but this got me thinking about something else. > My config file's sources are: > > source s_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream > ("/dev/log"); udp(); internal(); }; > source t_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream > ("/dev/log"); tcp(); internal(); }; > > Could it be that both sources, attempting to read /proc/kmsg and > /dev/log (and internal()?) are causing this? Never use the same log source (pipe, stream, diagram) two or more times. You should reorganize your source sections: source s_common { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream("/dev/log"); internal(); }; source s_udp { udp(); }; source s_tcp { tcp(); }; And modify your log sections accordingly. -- Sandor Geller wildy@balabit.hu _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Thu, 2006-03-02 at 07:47 -0500, Andreoli, Tony A. USNUNK NAVAIR B1490, R215 wrote:
Thanks for the info. Could this be the cause of my hangs?
yes. -- Bazsi
On Thu, Mar 02, 2006 at 07:47:02AM -0500, Andreoli, Tony A. USNUNK NAVAIR B1490, R215 wrote:
Thanks for the info. Could this be the cause of my hangs?
Yes, it's in the FAQ as well: http://www.campin.net/syslog-ng/faq.html#dupe_sources -- Nate "If you torture the data enough, it will confess." - Ronald Coase.
I've recently used this mailing list to get help setting up logging for my cisco pix's, now I want to be able to log my Dell and HP Procurve Switches, is there anything special i need to do for those? Is there something predefined, is there a website where I can find all the info for these devices?? Thank you all for your help. Andrew I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
You didn't get any documentation with your switches? Seriously, configuring syslog isn't rocket science, you just need the most basic understanding of your network's layout. The Dell web interfaces's have a prominent section called 'Global Log Parameters'. The command line interface has command line options under 'logging'. I found configuration examples for both with a mere Google of: "<platform name here> configuration guide". You can too. - billn On Thu, 2 Mar 2006, Andrew Meyer wrote:
I've recently used this mailing list to get help setting up logging for my cisco pix's, now I want to be able to log my Dell and HP Procurve Switches, is there anything special i need to do for those? Is there something predefined, is there a website where I can find all the info for these devices??
Thank you all for your help. Andrew
I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan.
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Well then I guess I'm not understanding how to use this. I am looking up the info on what to put in the config file and I find nothing. Do I need to add a line to source? Should I tell it to capture from a certain IP addy? Is there a avlue in there like there is for a Cisco PIX? I do not see any of this documented specifically. Also, I have no value in the system name of the Dell Switch, do I need to have something in there? Could that be the reason? Andrew --- Bill Nash <billn@odyssey.billn.net> wrote:
You didn't get any documentation with your switches? Seriously, configuring syslog isn't rocket science, you just need the most basic understanding of your network's layout.
The Dell web interfaces's have a prominent section called 'Global Log Parameters'. The command line interface has command line options under 'logging'.
I found configuration examples for both with a mere Google of: "<platform name here> configuration guide". You can too.
- billn
On Thu, 2 Mar 2006, Andrew Meyer wrote:
I've recently used this mailing list to get help setting up logging for my cisco pix's, now I want to be able to log my Dell and HP Procurve Switches, is there anything special i need to do for those? Is there something predefined, is there a website where I can find all the info for these devices??
Thank you all for your help. Andrew
I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan.
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Is there any reason you would have all your devices coming into local0?? Or does it matter, when I switch the Dell switch to local1 I no longer get packets coming into the syslog server. Why is that? --- Bill Nash <billn@odyssey.billn.net> wrote:
You didn't get any documentation with your switches? Seriously, configuring syslog isn't rocket science, you just need the most basic understanding of your network's layout.
The Dell web interfaces's have a prominent section called 'Global Log Parameters'. The command line interface has command line options under 'logging'.
I found configuration examples for both with a mere Google of: "<platform name here> configuration guide". You can too.
- billn
On Thu, 2 Mar 2006, Andrew Meyer wrote:
I've recently used this mailing list to get help setting up logging for my cisco pix's, now I want to be able to log my Dell and HP Procurve Switches, is there anything special i need to do for those? Is there something predefined, is there a website where I can find all the info for these devices??
Thank you all for your help. Andrew
I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan.
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
I'm worth a million in prizes With my torture film Drive a GTO Wear a uniform On a government loan. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
participants (6)
-
Andreoli, Tony A. USNUNK NAVAIR B1490, R215
-
Andrew Meyer
-
Balazs Scheidler
-
Bill Nash
-
Nate Campi
-
Sandor Geller