[syslog-ng] Using source tags in rewrites?

[Steve Barnes]

Patrick

I did consider the program-override together with the $PROGRAM macro, but
I was concerned that ~70 different $PROGRAM types would come back to haunt
me some where down the line. Having a single $PROGRAM to summarise the log
file related traffic felt like a less "risky" / more manageable approach.
Saying that, it looks like I don't have much choice given that I'm
presently using the OSE version.

Thanks for the suggestion, I'll give it a whirl and see what the end
result looks like :-)

Cheers

Steve

> Ok, I see what youre doing now :-)
> There's another simple solution that pops into mind; the file() source
> driver supports the 'program-override()' option. Basically any logs read
> from that file would have whatever $PROGRAM you specified. And that
> $PROGRAM macro will be propagated from the client syslog-ng to your
> server.
> -Patrick
> Sent: Tuesday, August 10, 2010 5:12:10 PM
> From: Steve Barnes mailto:steve at echo.id.au <steve at echo.id.au>
> To: Syslog-ng users' and developers' mailing list
> mailto:syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
> Subject: Re: [syslog-ng] Using source tags in rewrites?
> To what end? Rewrites are used to change or set key/value pairs. Flags
> arent key/value pairs, theyre flags, meaning you either have the flag, or
> you dont. Think of it this way; each incoming log statment has a list of
> flag names associated with it, you can see if a certain flag name is in
> that list, but thats it. It would help to know what youre trying to
> accomplish though. -Patrick
> Patrick I have ~70 different log files containing the results of a daily
> processing cycle. None of these log files are Syslog formatted. Unlike an
> Apache log file entry, each line of these log files is ambiguous - unless
> you referred to a portion of the file name, you'd have no way to
> differentiate a line from one file with another line from another file.
> The plan is to have SyslogNG set up as a client on a machine, tracking the
> contents of these files as they are updated. Interesting lines are
> filtered on the client with matching lines pushed across the network to a
> SyslogNG server. The server is responsible for processing lines against a
> pattern DB and then stuffing the useful information into a PostgreSQL
> backend. What I was hoping to do on the client side is tag each source in
> syslog-ng.conf with a meaningful name and then include that tag in either
> the IETF structured data or, by using a rewrite, prefix each log line
> before transmission with the tag value. This would give the SyslogNG
> server at the other end someway to differentiate messages as they arrived.
> Further reading yesterday uncovered the following in the 3.2 PE
> documentation:
> http://www.balabit.com/dl/html/syslog-ng-pe-v3.2-guide-admin-en.html/tagging_messages.html
> http://www.balabit.com/dl/html/syslog-ng-pe-v3.2-guide-admin-en.html/tagging_messages.html
> "To include the tags in the message, use the $TAGS macro in a template.
> Alternatively, if you are using the IETF-syslog message format, you can
> include the $TAGS macro in the .SDATA.meta part of the message. Note that
> the $TAGS macro is available only in syslog-ng PE 3.1.1 and later." So it
> looks like I've found the answer to my question? Before I start along the
> path seeking purchase approval for SyslogNG PE, I wanted to make sure
> there was no other way to do what I hand in mind? Cheers Steve
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
> http://www.campin.net/syslog-ng/faq.html
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>