[syslog-ng] destination program is killed due to broken pipe

[prmarino1 at gmail.com]

well the script on the surface sounds like a good simple idea, the simple way is not always the best or most relyable way. There are several better options open to you that should work better and shouldn't be too hard to impliment. The first is to write to a log file then use something like a "tail -f" in your script. Another option open to you is if you know how to write unix network program's in any languages  (I use perl for this kind of thing my self) it wouldn't be difficult to write a simple application read from a socket, or even udp syslog on an alternitive port on the loopback address. Syslog is a relitivly simple protocal to write a listener for and there are several perl modules that can do most of the work for you.
-----------------------
Sent from my Treo(r) smartphone

-----Original Message-----
From: "midnight codder" <midnight.codder at gmail.com>
Date: Tuesday, Aug 19, 2008 5:08 pm
Subject: [syslog-ng] destination program is killed due to broken pipe
To: syslog-ng at lists.balabit.huReply-To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>

I am trying to redirect all log messages to a destination program (written in java).
This did not do much so I redirected all messages to a single catch-all log file AND also to a bash shell script that runs an endless loop, reads a line and spit it out to a text file.
 The shell script is killed by syslog-ng after the 5th message. The catch-all file, however, keeps getting all of the messages (that's how I know who is killing who).

Here is the config file for syslog-ng:
 
options {
        ts_format(iso);
        frac_digits(3);
};

template bds_template{
        template('<pri=$PRI> <date=$ISODATE> <host=$HOST> <msg=$MSG>\n');
template_escape(no);
};

source src{
        internal();
        unix_stream('/dev/log');
        unix-dgram('/dev/log');
        file('/proc/kmsg' log_prefix('kernel:'));
        udp();
        tcp();
};


destination test_adapter{
program('bash /usr/local/bds/logger/syslogAdapterTest.sh' template(bds_template));
};

log { source(src); destination(test_adapter); };

destination catchAll{
file('/var/log/syslog-ng-catch-all.log', template(bds_template));
};

log { source(src); destination(catchAll); };

Here is the bash script:

 #!/bin/bash

file='/root/syslogAdapterTest'
counter=1
limit=3


while true
do
 read line
echo '$line' > $file.$counter
((counter++))
done

and finally the content of the catch-all file, in blue are the log messages that the shell script still captures and spits to text files, in red are the messages are the messages the shell does not get:

 <priE> <date 08-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Server license found, starting in server mode; customer='University of Illinois', serial='49544f30-573e-11dd-9892-0007e9195011', limit='25'>
<priF> <date 08-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Starting destination program; cmdline='bash /usr/local/bds/logger/syslogAdapterTest.sh'>
<priE> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: syslog-ng starting up; version='2.1.11a'>
<priF> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Initializing destination file writer; template='/var/log/syslog-ng-catch-all.log', filename='/var/log/syslog-ng-catch-all.log'>
<priE> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: License validity expires in one week, logging service will terminate at that point; error='License outside the validity period (option=syslog-ng, not-before 08/07/21, not-after 08/08/21)'>
<priC> <date 08-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: I/O error occurred while writing; fd='11', error='Broken pipe (32)'>
<priF> <date 08-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Sending child a TERM signal; child_pid='27972'>

Any idea?