April 2004 - syslog-ng - lists.balabit.hu

cannot get sec.pl to exit after syslog-ng does
by atonns＠mail.ivillage.com 28 Apr '04

28 Apr '04

I've tried the various configurations in the FAQ and I still get sec.pl spawned by a shell. Then after killing syslog-ng, sec.pl doesn't exit. Below are the configs and the output from ptree. I'm using Solaris 7, syslog-ng 1.6.2, sec 2.2.2 and perl 5.00503. I'm thinking of adding stats(60) to the syslog-ng config and then using the -input_timeout and -timeout_script options for sec.pl, but I have figured out how the sub-script would tell the difference between the abandoned sec.pl and the new one that's a child of the current syslog-ng. as per FAQ: ----------- destination d_sec { program("/usr/local/sbin/sec.pl -intevents -input=\"-\" -conf=/usr/local/etc/sec.conf"); }; 25879 /usr/local/sbin/syslog-ng 25880 /bin/sh -c /usr/local/sbin/sec.pl -intevents -input="-" -conf=/usr/local/etc/se 25881 /usr/local/bin/perl -w /usr/local/sbin/sec.pl -intevents -input=- -conf=/usr/lo another config: --------------- destination d_sec { program("/usr/local/sbin/sec.pl -intevents -input=\"-\" -conf=/usr/local/etc/sec.conf > /var/log/sec.log 2>&1"); }; 26428 /usr/local/sbin/syslog-ng 26429 /bin/sh -c /usr/local/sbin/sec.pl -intevents -input="-" -conf=/usr/local/etc/se 26430 /usr/local/bin/perl -w /usr/local/sbin/sec.pl -intevents -input=- -conf=/usr/lo pure desperation: ----------------- destination d_sec { program("/usr/local/bin/perl -w /usr/local/sbin/sec.pl -intevents -input=\"-\" -conf=/usr/local/etc/sec.conf"); }; 27478 /usr/local/sbin/syslog-ng 27479 /bin/sh -c /usr/local/bin/perl -w /usr/local/sbin/sec.pl -intevents -input="-" 27480 /usr/local/bin/perl -w /usr/local/sbin/sec.pl -intevents -input=- -conf=/usr/lo -- "Computer science is as much about computers as astronomy is about telescopes" -- Edsger Dijkstra --------------------------------------------------------- Anthony Tonns, UNIX Administrator - atonns(a)mail.ivillage.com iVillage Inc., 500 Seventh Avenue, New York, NY 10018 - iVillage Inc. is a leading women's media company that includes iVillage.com, Women.com, gURL.com, Astrology.com, Promotions.com, iVillage Parenting Network, The Newborn Channel, Lamaze Publishing, Business Women's Network, Diversity Best Practices, Best Practices in Corporate Communications, and iVillage Consulting. The information contained in this communication may be confidential, is intended only for the use of the recipient named above, and may be construed under applicable law to be a commercial email. If you have received this communication in error, please delete this message from your computer system. If you are the recipient named above and do not wish to receive any future commercial emails, please reply to the sender with a message stating such preference.

2 2

No facility/level logged
by me 28 Apr '04

28 Apr '04

Hello, I've compiled syslog-ng for AIX 5L and want the facility/level information be logged to the log file(s) (I assume this shuold be included in the message part as described in the syslog protocol rfc). Well, my investigations lend me to the "syslog" subroutine, which is called during the log of an event. This concerns in fact the os and not really syslog-ng. But maybe someone had the same requirement... Thanks for any feedback! Btw: Solaris syslogd logs this way, including the facility/level information. -- "Sie haben neue Mails!" - Die GMX Toolbar informiert Sie beim Surfen! Jetzt aktivieren unter http://www.gmx.net/info

3 3

program destination driver and template option
by Jim Mozley 28 Apr '04

28 Apr '04

I'm using syslog-ng 1.6.2 which I believe from the changelog supports the template option for the program driver. When I comment/uncomment the line from the config file below the destination will work with the template options commented out, but not with them uncommented. destination d_add_message { program("/path/to/my/test.pl" #template("$HOST $MESSAGE") template_escape(no) ); }; No parse errors are reported when syslog-ng starts. I have googled for examples of correct usage (although I cannot get through to the faq site at present) and cannot see anything wrong with the above, although I assume the problem is with the conf file definition. Jim Mozley

1 0

unsubscribe
by Minson, John Spawar 28 Apr '04

28 Apr '04

unsubscribe -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu]On Behalf Of me Sent: Wednesday, April 28, 2004 8:29 AM To: syslog-ng(a)lists.balabit.hu Cc: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]No facility/level logged Hi, that will work for sure, as soon as I have compiled syslog-ng-1.9.1... Thank you very much! /me > 2004-04-27, k keltezéssel 09:56-kor me ezt írta: > > Hello, > > I've compiled syslog-ng for AIX 5L and want the facility/level > information > > be logged to the log file(s) (I assume this shuold be included in the > > message part as described in the syslog protocol rfc). Well, my > > investigations lend me to the "syslog" subroutine, which is called > during > > the log of an event. This concerns in fact the os and not really > syslog-ng. > > But maybe someone had the same requirement... > > > > Thanks for any feedback! > > probably you want to define a file template > > destination d_messages { file("/var/log/messages" template("<$PRI> $DATE > $HOST $MSG\n"); }; > > -- > Bazsi > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C > 8EB1 > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > -- "Sie haben neue Mails!" - Die GMX Toolbar informiert Sie beim Surfen! Jetzt aktivieren unter http://www.gmx.net/info _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Using a perl script with the pipe destination driver
by Jim Mozley 28 Apr '04

28 Apr '04

I have a perl script that reads from a pipe using the following construct: while ( 1 ) { die "$pipe is not a pipe!" unless ( -p $pipe ); open PIPE, $pipe or die "Cannot open $pipe: $!"; my @lines = <PIPE>; close PIPE; for ( @lines ) { # do some stuff } } The "do some stuff part" loads info into a database (the messages are processed before being put into the database and two hashes keep cached information relevant to this in order to speed up the process). I can test using: cat syslog > pipe and the script does its stuff, loading single syslog messages or a days worth (around 3000 messages in 8 seconds). I then try to add a destination within syslog-ng.conf: destination d_pipe { pipe("/path/to/pipe"); }; and log { source(all); destination(d_pipe); }; This doesn't work as I expected. It seems as if I have some kind of locking problem; as I send test syslog messages nothing happens, then if I stop syslog-ng all the test messages I have added (using logger) are then processed by the script. I can then restart syslog-ng, send messages and again they are not read by the script until syslog-ng is stopped. I was able to use the construct: open PIPE, $pipe or die "Cannot open $pipe: $!"; while ( <LOGPIPE> ) { # do something e.g. test write to file }; as a test and this worked OK, seeming to process each message as I entered it (although I didn't try a volume test). Based on what I'd found so far I had a few questions: - can anyone explain what I think is the locking problem with the first approach (apologies if I'm asking a general unix or perl question, but testing the pipe outside of syslog-ng seemed to work ok)? - If I use the second approach are log messages written atomically such that I will get them one at a time? - One of the reasons I adopted the first approach is that I can throw a lot of messages at the pipe and still have them processed and put in the DB in a short space of time. I was concerned that if I used the second approach and syslog-ng was passing messages into the pipe faster than they could be processed, I would end up loosing messages. I'm sure that there is either an OS or syslog-ng buffer that is configurable but I assume its finite. We are logging a lot of events from network equipment and in the event of a "message storm" when network problems occur I wanted to ensure all of the messages are captured. I'm using perl 5.8 and syslog-ng 1.6.2 on Solaris 8. Jim Mozley

2 2

killed by HUP signal on Mac OS X 10.3
by Jeremy Mates 28 Apr '04

28 Apr '04

The syslog-ng daemon dies on Mac OS X when a HUP signal is sent: $ sudo /usr/local/sbin/syslog-ng -d io.c: Preparing fd 4 for reading io.c: Preparing fd 5 for reading syslog-ng version 1.6.2 starting io.c: Preparing fd 7 for writing io.c: Preparing fd 8 for writing SIGHUP received, restarting syslog-ng poll.c:282: failed assertion `pArray != (struct pollfd *) NULL' zsh: abort sudo /usr/local/sbin/syslog-ng -d Software versions involved: syslog-ng 1.6.2 libol 0.3.13 gcc 3.3 20030304 (Apple Computer, Inc. build 1495) Mac OS X 10.3.3 (build 7F44) Running ktrace shows the following: 12661 syslog-ng RET write 149/0x95 12661 syslog-ng CALL select(0x9,0xbffff3a0,0xbffff420,0xbffff4a0,0xbffff520) 12661 syslog-ng RET select -1 errno 4 Interrupted system call 12661 syslog-ng PSIG SIGHUP caught handler=0x2a94 mask=0x0 code=0x0 12661 syslog-ng CALL sigaction(0x1,0xbfffecb0,0xbfffed20) 12661 syslog-ng RET sigaction 0 12661 syslog-ng CALL #184(0xbfffee58,0x1) 12661 syslog-ng RET #184 JUSTRETURN 12661 syslog-ng CALL write(0x2,0xbffff540,0x26) 12661 syslog-ng GIO fd 2 wrote 38 bytes "SIGHUP received, restarting syslog-ng " 12661 syslog-ng RET write 38/0x26 12661 syslog-ng CALL write(0x2,0xbfffeb50,0x40) 12661 syslog-ng GIO fd 2 wrote 64 bytes "poll.c:282: failed assertion `pArray != (struct pollfd *) NULL' " 12661 syslog-ng RET write 64/0x40 12661 syslog-ng CALL sigprocmask(0x3,0xbffff324,0) 12661 syslog-ng RET sigprocmask 0 12661 syslog-ng CALL getpid 12661 syslog-ng RET getpid 12661/0x3175 12661 syslog-ng CALL kill(0x3175,0x6) 12661 syslog-ng RET kill 0 12661 syslog-ng PSIG SIGIOT SIG_DFL

3 3

help me for the drop messages
by Luigi Augello 27 Apr '04

27 Apr '04

Hello at all sorry Bazsi I had copied teh wrong address. rewrite the messages Hello at all I have a problem with a message STATS dropped probably because the database is busy, syslog-ng can not write fast enough and drop messages. Anyone have the solution. I'm desperado!!! Is possible to configure syslo-ng opportunely? Tanks a lot

1 0

Corruption in program destination
by Andy Kirkpatrick 27 Apr '04

27 Apr '04

Hi everyone, I have syslog-ng set up to accept syslog entries over UDP from remote hosts: source src { unix-dgram("/dev/log"); internal(); udp(); }; The log entries in question come from various machines, all arrive with facility local6 and get written to /var/log/local6 nicely: destination local6 { file("/var/log/local6" owner("root") group("adm") perm("640")); }; filter local6 { facility(local6); }; log { source(src); filter(local6); destination(local6); }; However the same data when written to a program destination gets corrupted: destination logprocess { program("/usr/local/bin/logprocess.pl"); }; log { source(src); filter(local6); destination(logprocess); }; Whereas in /var/log/local6 I get each line whole and pristine, my logprocess script gets lines that are occasionally overwritten, with one log entry truncated and another starting halfway through it. The lines are all under 120 bytes in length so its not a log_msg_size issue. This only seems to occur when more than one remote host is sending to syslog-ng at once. Thinking it might be a buffer overrun when the server is busy I increased log_fifo_size to 8000 but that made no difference. Any ideas? Thanks, Andy Kirkpatrick

1 0

RE: [syslog-ng]Program destination ?
by Andy Kirkpatrick 27 Apr '04

27 Apr '04

Hi David, This is just a guess, but the output buffer of syslog-ng would be holding more than one line, which is set with option log_fifo_size. Consoles tend to be line buffered while pipes are block/chunk/arbitrarily buffered. Try sending a few hundred messages and see if they come through all at once. Or it could be more related to the sync option, the number of lines buffered before being written to file (not sure if that applies to program targets). I can't see anything wrong with your config, we have a similar setup where I work and I haven't seen the problem you mention. HTH, Andy Kirkpatrick -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of David Vaartjes Sent: Friday, 23 April 2004 8:16 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Program destination ? Hello list, I'm testing syslog-ng to send certain messages to the stdin of a program. I've set the following lines in "syslog-ng.conf" to accomplish this: <CONFIG-LINES> destination alerter { program("/home/dv/download/perl-test.pl"); }; filter f_special { match("special"); }; log { source(src); filter(f_special); destination(alerter); }; </CONFIG-LINES> The perl-test.pl looks like: <PERL-TEST.PL> #!/usr/bin/perl while (<>) { print $_ . "\n"; } </PERL-TEST.PL> If I do a 'logger -p daemon.notice "special"' the message is not send to the stdin of perl-test.pl. However, if i do a 'echo "test" > /proc/3240/fd/0" the data is send to the stdin of perl-test.pl and shown on the console by perl-test.pl as expected. I can't figure out why syslog isn't sending messages to perl-test.pl. If i change the "destination" in syslog-ng.conf to write to a file, everything works fine. Hope someone can shed some light on this one. Thanks in advance, David _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Program destination ?
by David Vaartjes 25 Apr '04

25 Apr '04

Hello list, I'm testing syslog-ng to send certain messages to the stdin of a program. I've set the following lines in "syslog-ng.conf" to accomplish this: <CONFIG-LINES> destination alerter { program("/home/dv/download/perl-test.pl"); }; filter f_special { match("special"); }; log { source(src); filter(f_special); destination(alerter); }; </CONFIG-LINES> The perl-test.pl looks like: <PERL-TEST.PL> #!/usr/bin/perl while (<>) { print $_ . "\n"; } </PERL-TEST.PL> If I do a 'logger -p daemon.notice "special"' the message is not send to the stdin of perl-test.pl. However, if i do a 'echo "test" > /proc/3240/fd/0" the data is send to the stdin of perl-test.pl and shown on the console by perl-test.pl as expected. I can't figure out why syslog isn't sending messages to perl-test.pl. If i change the "destination" in syslog-ng.conf to write to a file, everything works fine. Hope someone can shed some light on this one. Thanks in advance, David

2 1