- syslog-ng - lists.balabit.hu

Re: [syslog-ng] runaway programs started by syslog-ng
by Dev Man 27 Oct '06

27 Oct '06

No, the program does not fork or execute any other programs. It just processes input from syslog and does some inserts into a database. Due to the out of control restarts, they end up hogging up database connections, too. :( --===============0443184984== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C6FA0C.AE82AFEE" This is a multi-part message in MIME format. Simple question which may help: does your program fork() or daemon() into a "background" process? This would explain the behavior you're seeing, and should be relatively easy to work around. Hope that helps. Good luck, Peter Nahas MRV Communications - Boston Division ________________________________ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Dev Man Sent: Friday, October 27, 2006 3:38 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] runaway programs started by syslog-ng Hi Bazsi, How would having that switch help? The problem is we do want it to automatically restart. It just appears that there are some scenarios which we haven't figure out.. in which the program gets restarted repeatedly leaving apparently multiple instances running. How do you determine when to restart? Thanks! On Fri, 2006-10-27 at 10:59 -0700, Dev Man wrote: > > Hi, > > We have experienced cases of syslog-ng apparently launching multiple > instances of the same program unexpectedly, causing the machine to > become unusable. Some of the launched processes can end up defunct, > but not all appeared to be when we checked the running processes. > > Isn't syslog-ng only supposed to start a program if it was terminated? > > In one case, we were seeing a constant stream of "duplicate stats > counter" messages like: > syslog.1:Oct 26 06:27:10 in-mta syslog-ng[29833]: Duplicate stats > counter; counter='program(/home/readlog/readlog.php)' > > One machine was using syslog-ng version 1.9.11 and another version > 2.0rc1. 2.0.x (and 1.9.x which was development versions for 2.0) restart program() destinations by default. I'll add a switch to disable that behaviour as it is hardwired at the moment. -- Bazsi [Attachment #3 (text/html)] <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META http-equiv=Content-Type content="text/html; charset=us-ascii"> <META content="MSHTML 6.00.2900.2963" name=GENERATOR></HEAD> <BODY> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2>Simple question which may help: does your program fork() or daemon() into a "background" process?  This would explain the behavior you're seeing, and should be relatively easy to work around.</FONT></SPAN></DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2>Hope that helps.</FONT></SPAN></DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2>Good luck,</FONT></SPAN></DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2>Peter Nahas</FONT></SPAN></DIV> <DIV dir=ltr align=left><SPAN class=510391021-27102006><FONT face=Arial color=#0000ff size=2>MRV Communications - Boston Division</FONT></SPAN><BR></DIV> <DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left> <HR tabIndex=-1> <FONT face=Tahoma size=2><B>From:</B> syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] <B>On Behalf Of </B>Dev Man<BR><B>Sent:</B> Friday, October 27, 2006 3:38 PM<BR><B>To:</B> syslog-ng(a)lists.balabit.hu<BR><B>Subject:</B> Re: [syslog-ng] runaway programs started by syslog-ng<BR></FONT><BR></DIV> <DIV></DIV> <DIV> </DIV> <DIV>Hi Bazsi,</DIV> <DIV> </DIV> <DIV>How would having that switch help?</DIV> <DIV>The problem is we do want it to automatically restart.  It just appears that there are some scenarios which we haven't figure out.. in which the program gets restarted repeatedly leaving apparently multiple instances running. </DIV> <DIV> </DIV> <DIV>How do you determine when to restart?</DIV> <DIV> </DIV> <DIV>Thanks!</DIV> <DIV><PRE>On Fri, 2006-10-27 at 10:59 -0700, Dev Man wrote: > > Hi, > > We have experienced cases of syslog-ng apparently launching multiple > instances of the same program unexpectedly, causing the machine to > become unusable. Some of the launched processes can end up defunct, > but not all appeared to be when we checked the running processes. > > Isn't syslog-ng only supposed to start a program if it was terminated? > > In one case, we were seeing a constant stream of "duplicate stats > counter" messages like: > syslog.1:Oct 26 06:27:10 in-mta syslog-ng[29833]: Duplicate stats > counter; counter='program(/home/readlog/readlog.php)' > > One machine was using syslog-ng version 1.9.11 and another version > 2.0rc1. 2.0.x (and 1.9.x which was development versions for 2.0) restart program() destinations by default. I'll add a switch to disable that behaviour as it is hardwired at the moment. -- Bazsi</PRE></DIV></BODY></HTML> _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] runaway programs started by syslog-ng
by Peter Nahas 27 Oct '06

27 Oct '06

Simple question which may help: does your program fork() or daemon() into a "background" process? This would explain the behavior you're seeing, and should be relatively easy to work around. Hope that helps. Good luck, Peter Nahas MRV Communications - Boston Division ________________________________ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Dev Man Sent: Friday, October 27, 2006 3:38 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] runaway programs started by syslog-ng Hi Bazsi, How would having that switch help? The problem is we do want it to automatically restart. It just appears that there are some scenarios which we haven't figure out.. in which the program gets restarted repeatedly leaving apparently multiple instances running. How do you determine when to restart? Thanks! On Fri, 2006-10-27 at 10:59 -0700, Dev Man wrote: > > Hi, > > We have experienced cases of syslog-ng apparently launching multiple > instances of the same program unexpectedly, causing the machine to > become unusable. Some of the launched processes can end up defunct, > but not all appeared to be when we checked the running processes. > > Isn't syslog-ng only supposed to start a program if it was terminated? > > In one case, we were seeing a constant stream of "duplicate stats > counter" messages like: > syslog.1:Oct 26 06:27:10 in-mta syslog-ng[29833]: Duplicate stats > counter; counter='program(/home/readlog/readlog.php)' > > One machine was using syslog-ng version 1.9.11 and another version > 2.0rc1. 2.0.x (and 1.9.x which was development versions for 2.0) restart program() destinations by default. I'll add a switch to disable that behaviour as it is hardwired at the moment. -- Bazsi

1 0

Re: [syslog-ng] runaway programs started by syslog-ng
by Dev Man 27 Oct '06

27 Oct '06

Hi Bazsi, How would having that switch help? The problem is we do want it to automatically restart. It just appears that there are some scenarios which we haven't figure out.. in which the program gets restarted repeatedly leaving apparently multiple instances running. How do you determine when to restart? Thanks! On Fri, 2006-10-27 at 10:59 -0700, Dev Man wrote: > > Hi, > > We have experienced cases of syslog-ng apparently launching multiple > instances of the same program unexpectedly, causing the machine to > become unusable. Some of the launched processes can end up defunct, > but not all appeared to be when we checked the running processes. > > Isn't syslog-ng only supposed to start a program if it was terminated? > > In one case, we were seeing a constant stream of "duplicate stats > counter" messages like: > syslog.1:Oct 26 06:27:10 in-mta syslog-ng[29833]: Duplicate stats > counter; counter='program(/home/readlog/readlog.php)' > > One machine was using syslog-ng version 1.9.11 and another version > 2.0rc1. 2.0.x (and 1.9.x which was development versions for 2.0) restart program() destinations by default. I'll add a switch to disable that behaviour as it is hardwired at the moment. -- Bazsi

1 0

runaway programs started by syslog-ng
by Dev Man 27 Oct '06

27 Oct '06

Hi, We have experienced cases of syslog-ng apparently launching multiple instances of the same program unexpectedly, causing the machine to become unusable. Some of the launched processes can end up defunct, but not all appeared to be when we checked the running processes. Isn't syslog-ng only supposed to start a program if it was terminated? In one case, we were seeing a constant stream of "duplicate stats counter" messages like: syslog.1:Oct 26 06:27:10 in-mta syslog-ng[29833]: Duplicate stats counter; counter='program(/home/readlog/readlog.php)' One machine was using syslog-ng version 1.9.11 and another version 2.0rc1. Any suggestions? Thanks. D

2 1

Re: [syslog-ng] use syslog-ng to monitor just couple of processes
by Hari Sekhon 27 Oct '06

27 Oct '06

I've not used rsyslog myself, but it seems a lot closer to the old syslog than syslog-ng which is quite different and very flexible. Rsyslog seems to do most of the things that syslog-ng does so I guess it might be down to which you personally prefer. Syslog-ng seems more prevalent, and although rsyslog seems to have built in mysql support, whereas syslog-ng uses a tiny glue shell script, the syslog-ng way works well enough too. I run my central logserver this way and it is reliable. Neither of these things really support web interfaces to my knowledge. My web interface is an interface to mysql, nothing to do with syslog-ng.... -h Hari Sekhon Tiger Peng wrote: > Hi, Hari: > Thanks a lot, since I also consider to dump the > syslog-ng log to database (MySQL or Oracle), do you > think it is better to choose the syslog-ng, or > rsyslog, it looks like the syslog-ng has better web > interface supporting, but rsyslog has the better built > in MySQL database support. > Definitely, I will take your advice to read through > all the syslog-ng documents and FAQ. Thanks again. > > > > David > > = = = Original message = = = > > ? > > You don't use syslog-ng in this way. For what you want > to do you should make your application output the > information you want to the standard syslog and use > syslog-ng as your logger. Then use filters to extract > the messages you want the launch an external program > to email them off to you or something. That is what I > did. A few lines of code, some regex filtering with > syslog-ng definitions is all it takes. > > Now you have all the pointers, you can read the docs > on filtering, program(), etc. > Although it sounds like you don't have a good grasp of > the whole logging thing yet so my advice is to read > all the syslog-ng docs and faq. They are worth the > read. > > -h > > > > Hari Sekhon > > Tiger Peng wrote: > > I have two questions about how to use the > syslog-ng to monitor just couple of processes. > In my case, I try to use some kind of utility to > just monitor 4 to 8 different processes, some of them > are system resource, but some of them are new > developed processes. Here are my questions. > > 1. can I start a seperate syslog-ng, I mean if I can > keep syslogd running, but start another syslog-ng > process to dedicate the service for my special > purpose. > > 2. I guess syslog-ng has some kind of filter which can > filter the information, but how can I expand it > ability to receive some specific information or more > field, such as message ID. Is this configurable, or I > must modify the source to support it. > > Thanks. > > > > David > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at > http://www.campin.net/syslog-ng/faq.html > > ___________________________________________________________ > Sent by ePrompter, the premier email notification > software. > Free download at http://www.ePrompter.com. > >

1 0

use syslog-ng to monitor just couple of processes
by Tiger Peng 27 Oct '06

27 Oct '06

I have two questions about how to use the syslog-ng to monitor just couple of processes. In my case, I try to use some kind of utility to just monitor 4 to 8 different processes, some of them are system resource, but some of them are new developed processes. Here are my questions. 1. can I start a seperate syslog-ng, I mean if I can keep syslogd running, but start another syslog-ng process to dedicate the service for my special purpose. 2. I guess syslog-ng has some kind of filter which can filter the information, but how can I expand it ability to receive some specific information or more field, such as message ID. Is this configurable, or I must modify the source to support it. Thanks. David

2 1

beginner's question about how to use Java to write log to syslog-ng
by Tiger Peng 26 Oct '06

26 Oct '06

I am the beginner of the syslog-ng, and have a very basic question. How to use Java to write the system log to syslog-ng? Is the same syslogger wrapper can be used here? Another suggestion is that the list didn't have the search ability, it should be very helpful to have it, I believe a lot of people do like to see it. Thanks. David

2 1

building 2.0 on freebsd
by Vivek Khera 25 Oct '06

25 Oct '06

I'm attempting to install 2.0 daily snapshot from 12 october onto a freebsd 6.1 system. From the FAQ I see that it needs to have glib installed, which was easy. However, the configure is not able to find the eventlib library. I cannot find what provides this library. My searches all keep pointing to something which is part of bind, which seems not right to me. What do I need to install to get eventlib on this system?

2 4

A database connection problem was encountered.
by tagarsi, arun 25 Oct '06

25 Oct '06

Hi, I have installed syslog-ng on a server running RHEL4 without much problems but so far I have not been able to on a RHEL3.8 system. from what I can tell the mysql server is running ok , I can do all the command line queries to the databases but for some reason I cannot get the web access up and running . On this server I have loaded phpMyAdmin-2.8.2 and this is ok and I am able to use this to look and manage the databases . Can someone point out why or where I can look for any log information regarding the error reported. I know the config file is ok as it is . I tried to load up the latest version 2.9.2 using the installer, and this does complain there is something not quite right as on the pre check it says - Pre-installation check for: Php-Syslog-NG 2.9.2a (r13) Stable [ cdukes ] 23-Sep-2006 15:10 EST If any of these items are highlighted in red then please take actions to correct them. Failure to do so could lead to your PHP-Syslog-NG installation not functioning correctly. PHP version >= 4.1.0 Yes - zlib compression support Available - XML support Available - GD support Available - MySQL support Unavailable ************************************** config.php Writeable Session save path /tmp, Writeable So it is complaining about not having MySQl support . Unfortunately I cannot see what it is that it needs and what is missing from my mysql install. The weird thing is that mysql seems to be ok on this server as I can access all my databases using other means. Does anyone know what could be missing. Here is a list of the RPMS loaded for mysql and php. mysql-bench-3.23.58-16.RHEL3.1 php-mysql-4.3.2-36.ent libdbi-dbd-mysql-0.6.5-5 mysql-server-3.23.58-16.RHEL3.1 mysql-3.23.58-16.RHEL3.1 mysql-devel-3.23.58-16.RHEL3.1 mod_auth_mysql-20030510-2.ent php-imap-4.3.2-36.ent php-mysql-4.3.2-36.ent php-ldap-4.3.2-36.ent php-odbc-4.3.2-36.ent php-4.3.2-36.ent php-devel-4.3.2-36.ent Thanks Arun Tagarsi

1 0

Re: [syslog-ng] AIX startup
by Nathan Campi 25 Oct '06

25 Oct '06

Is that a question for me? Easy answer, I don't run AIX. :) I probably should have mentioned that. Back a couple years ago when installing openssh on AIX I found it easy to set it up under the SRC, syslog-ng should be easy. On Wed, Oct 25, 2006 at 12:09:14PM -0400, Schernau, Ed wrote: > On AIX, why not just use the system resource controller (SRC) ? > > Ed > > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu > [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Nathan Campi > Sent: Wednesday, October 25, 2006 11:50 AM > To: Syslog-ng users' and developers' mailing list > Subject: Re: [syslog-ng] AIX startup > > I run syslog-ng under daemontools on a couple loghosts where I don't > want it down for more than a second. > > I already have cfengine install daemontools on all my systems anyways, > so it is trivial for me to automate that part as well. The only problem > is that daemontools comes up after all the rc scripts, which leaves you > without logging during all of startup. > > Workable solution is to still have an rc script that starts syslog and > have the daemontools run script kill off any already running syslog-ng > daemons when it comes up later. Might be overkill to supervise it in the > > first place - judgement call I suppose. > > -- > Nate Campi > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > ----------------------------------------- > Use of email is inherently insecure. Confidential information, > including account information, and personally identifiable > information, should not be transmitted via email, or email > attachment. In no event shall Citizens or any of its affiliates > accept any responsibility for the loss, use or misuse of any > information including confidential information, which is sent to > Citizens or its affiliates via email, or email attachment. Citizens > does not guarantee the accuracy of any email or email attachment, > that an email will be received by Citizens or that Citizens will > respond to any email. > > This email message is confidential and/or privileged. It is to be > used by the intended recipient only. Use of the information > contained in this email by anyone other than the intended recipient > is strictly prohibited. If you have received this message in error, > please notify the sender immediately and promptly destroy any > record of this email. > > -- Nate "If a trainstation is where the train stops, what's a workstation...?" -Anon.

1 0