[syslog-ng] Parsing message in unstructured rows

[Scheidler, Balázs]

filter extract_date { message("Date: ([0-9]+)"  flags(store-matches));

The important part is store-matches because by default syslog-ng doesn't
store the values of the groups.

By default syslog-ng stores the value as $1, but that can be changed using
the pcre syntax.

(?P<name>group)

http://regular-expressions.mobi/named.html
On Sep 7, 2015 9:25 PM, "Giovanni Mancuso" <giovanni.mancuso at par-tec.it>
wrote:

> Il 07 settembre 2015 21:10:59 "Scheidler, Balázs" <
> balazs.scheidler at balabit.com> ha scritto:
>
> > On Sep 7, 2015 6:11 PM, "Giovanni Mancuso" <giovanni.mancuso at par-tec.it>
> > wrote:
> > >
> > > Hi,
> > >
> > > I have an application that log date in every rows. The problem is that
> > the string isn't in specific part of MESSAGE, but it could be the first
> > element or the last element, or in the middle :-) :-)
> > >
> > > For example (only MESSAGE):
> > > User: user1 at example.com Date: 12/12/2014 Status: OK
> > > User: user2 at example.com ID: 1234 Status: DEL .......  Date: 03/05/2014
> > > Date: 05/08/2015 User: user3 at example.com ....... Stauts: OK
> > > .........
> > >
> > > I want split the log in more files arranged in different path builded
> by
> > the date information, for example /LOGS/YYYY/MM/DD/mylog.log.
> > >
> > > I do:
> > > rewrite r_rewrite_set {
> > >    set("$(python get_data)", value("APP.DATE"));
> > > };
> > >
> > > python{
> > > import re
> > > def get_data(logmsg):
> > >   out=None
> > >   vars(logmsg)
> > >   out = re.findall(" Date: (\d\d/\d\d/\d\d\d\d) ", logmsg.MESSAGE)
> > >   if len(out) == 1:
> > >       return out[0]
> > >   else:
> > >       raise Exception("Invalid match")
> > > };
> > >
> > > In this way i have in APP.DATE the date.
> > > Now i have a some questions:
> > > 1) Is there another way to do this without python?
> >
> > Well, this should be possible with a simple regexp filter.
>
> I try to use a filter regexp, but i don't find the solution. Have you an
> example?
>
> >
> > > 2) In this way for every message, syslog-ng forks and exec a python
> > interpreter?
> >
> > No, it embeds a Python interpreter.
> >
> > > 3) Is there a way to add custom SDATA field from python? Or is there a
> > way to create APP.DATE from python without rewrite rule?
> >
> > Not right now.
> >
> > > 4) Is there a documentation about python{}? I only found a post in a
> blog.
> >
> > It's being prepared by the tech writer team in BalaBit
> >
> > >
> > > Thanks
> > >
> > >
> > >
> >
> ______________________________________________________________________________
> > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > > Documentation:
> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> > >
> > >
>
> Inviato con AquaMail per Android
> http://www.aqua-mail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20150908/37871c2c/attachment.htm