[syslog-ng] Example 3.4. Script to feed a growing logfileinto syslog-ng

[Alan Edmonds]

This is how I capture garbage collections with tomcat.

On the tomcat host in the startup.sh script.
Create a named pipe, catalinia.gc
Run logger to read named pipe and forward to syslog-ng host.
I use a "tag" (or program) to identify it as a gc.out.
I created an $INSTANCE for the different tomcats running on a single
host.

..
rm -f "$CATALINA_BASE"/logs/catalina.gc
mkfifo "$CATALINA_BASE"/logs/catalina.gc

...
  "$_RUNJAVA" $JAVA_OPTS $CATALINA_OPTS \
      -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH"
\
      -Dcatalina.base="$CATALINA_BASE" \
      -Dcatalina.home="$CATALINA_HOME" \
      -Djava.io.tmpdir="$CATALINA_TMPDIR" \
      -Xloggc:"$CATALINA_BASE"/logs/catalina.gc -XX:-DisableExplicitGC \
      org.apache.catalina.startup.Bootstrap "$@" start 2>&1 | logger -p
local2.info -t tomcat$INSTANCE &

      if [ ! -z "$CATALINA_PID" ]; then
        echo $! > $CATALINA_PID
      fi
  fi
  logger -p local3.info -t GC$INSTANCE <
"$CATALINA_BASE"/logs/catalina.gc &
..

On the syslog-ng server, it sorts it out like this:

filter f_gc { facility(local3); };

destination host-program-out {
        file("/logs/portal/$HOST/$PROGRAM.out"
        template ("$MSG\n")
        template_escape(no)
        );
};

log {
        source(local);
        filter(f_gc);
        destination(host-program-out);
};

you get a GC$INSTANCE.out file in $HOST/ on the syslog-ng server. But,
you have to strip the GC$INSTANCE: from each line of the .out file
before running it through HPJtune, etc.  you get the idea.

Alan Edmonds





-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu
[mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Balazs
Scheidler
Sent: 25 July 2008 08:35
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Example 3.4. Script to feed a growing
logfileinto syslog-ng


On Thu, 2008-07-24 at 16:10 -0400, Chris Pratt wrote:
> Thanks for your suggestion.  This is for Tomcat.  Unfortunately I am 
> stuck reading from the log because the syslog appender in the log4j 
> version on our app does not capture stack traces while the file 
> appenders do.  In other cases we want to capture garbage collections 
> which are only written to stdout.  I tried sending my file appends to 
> /dev/log but that didn't work so I am stuck trying to read from files.

> Only my central log server runs syslog-ng.  All of my Tomcat hosts 
> just use syslogd and send *.* @mylogserver.  Otherwise I'd probably 
> try # source s_tail { file("log.log" follow_freq(1) flags(no-parse)); 
> }; to see if it kept it in order any better.
> 
> What I could really use is a logger that has the ability to read and 
> forward to syslog the existing timestamp...

syslog-ng could that if it was running on the client hosts too.

but otherwise what about this dirty trick:

tail -f <logfile> | sed -e 's/^/<133>/' | nc syslog-server 514

If the timestamps are otherwise in syslog-ng processable format, the
above scriptlet would work.

The sed sets local6.notice as a priority.

However the difficult part is in doing the "tail -f" part robust enough,
but I guess you achieved that already.

-- 
Bazsi

________________________________________________________________________
______
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html

T-Mobile International UK Limited
Company Registered Number: 3951860
Registered Office Address: Hatfield Business Park, Hatfield, Hertfordshire, AL10 9BW
Registered in England and Wales
 
NOTICE AND DISCLAIMER
 
This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose