[syslog-ng] Growing log files (i.e. access.log) and regex.

[sawall]

Why not use SEC (http://www.estpak.ee/~risto/sec/) to monitor the log
file?  I use SEC with my syslog-ng files as well as a few other key
log files.

You could probably then use SEC to look for specific data then call an
external script to put it into your DB.

Chris


On 10/11/05, Peter Nahas <pnahas at mrv.com> wrote:
> Perhaps I am missing something here, but I believe that the "logger:"
> can be easily replaced with "jboss:" using the -t flag for logger.  So
> modify your script file as follows:
>
> #!/bin/sh
> tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info
> -t jboss
>
> Another way to tackle this problem is to avoid logger completely by
> tailing the log into a pipe (mkfifo(1)) and use the pipe() source for
> syslog-ng.  Thus, you could modify your script to be:
>
> #!/bin/sh
> tail -f /opt/jboss/server/default/log/boot.log > /dev/pipe_to_syslog
>
>
>   Peter Nahas
>   Software Engineer
>   MRV Communications, InReach Division
>
> Ken Garland wrote:
>
> > sed -n 's/\(.*\)logger:.*$/\1jboss: DEBUG/p'
> >
> > I use syslog2mysql.sh which came with the version of syslog-ng that i
> > installed from http://www.phpwizardry.com
> >
> > - Ken
> >
> > Robert.Becker at motoristsgroup.com wrote:
> >
> >>
> >>
> >> Here's the scenario.  We have a JBoss server that outputs to *.log
> >> files.
> >> After doing all the research I could, I found out the only way to get
> >> these
> >> logs sent to syslog-ng was through a tail & logger command. The problem
> >> with that is logger appends the date and some other information to that
> >> logged message.  When attempting to put that information in a mysql
> >> database I'm getting errors.  So, what I would like to do is use some
> >> form
> >> of regex to alter the statement so it can be inserted into a database.
> >> There are basically two things that need to be changed.  First, logger
> >> appends the date/time it receives the message and it also says it is
> >> coming
> >> from logger.  I would like to remove that date and switch logger to
> >> jboss
> >> or something similar.
> >>
> >> Here is an example of what I am doing.
> >>
> >> This is a line from the jboss log file:
> >> 2005-10-11 13:14:15,848 DEBUG
> >> [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover
> >> notifying pools, interval: 450000
> >>
> >> Then I send the log through logger to syslog-ng:
> >>
> >> #!/bin/sh
> >> tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info
> >>
> >> The log gets turned into this:
> >> Oct 11 13:14:15 src at linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG
> >> [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover
> >> notifying pools, interval: 450000
> >>
> >> What I need is:
> >>
> >> Oct 11 13:14:15 src at linuxp17 jboss: DEBUG
> >> [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover
> >> notifying pools, interval: 450000
> >>
> >>
> >> Now, I've thought of using awk in the tail/logger script to remove the
> >> date, but that does not take care of the logger/jboss substitution.
> >> Anyone
> >> have any ideas?  Can syslog-ng do this with the match() function?
> >>
> >> -Rob Becker
> >> Systems Engineer
> >> Motorists Insurance
> >>
> >>
> >> **********************************************************************
> >> The information contained in this message is confidential and is
> >> intended for the addressee(s) only. If you have received this message
> >> in error or there are any problems please notify the originator
> >> immediately. The unauthorized use, disclosure, copying or alteration
> >> of this message is strictly forbidden. Motorists Insurance Group will
> >> not be liable for direct, special, indirect or consequential damages
> >> arising from the alteration of the contents of this message by a
> >> third party or as a result of any virus being passed on.
> >>
> >> **********************************************************************
> >> _______________________________________________
> >> syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> >> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >>
> >>
> >>
> >
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >
>
> _______________________________________________
> syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
>
>