[syslog-ng] Using source tags in rewrites?

[syslogng at feystorm.net]

You can do something like "program-override('logfile/category/subcat')". 
that way later down the line, if you need to test for entries coming 
from these sources, you can do "program('logfile')" inside a filter, or 
"program('logfile/category')" to match a certain category of the log 
files or whatnot.

-Patrick

Sent: Wednesday, August 11, 2010 7:34:16 AM
From: Steve Barnes <steve at echo.id.au>
To: Syslog-ng users' and developers' mailing list 
<syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Using source tags in rewrites?
> Patrick
>
> I did consider the program-override together with the $PROGRAM macro, but
> I was concerned that ~70 different $PROGRAM types would come back to haunt
> me some where down the line. Having a single $PROGRAM to summarise the log
> file related traffic felt like a less "risky" / more manageable approach.
> Saying that, it looks like I don't have much choice given that I'm
> presently using the OSE version.
>
> Thanks for the suggestion, I'll give it a whirl and see what the end
> result looks like :-)
>
> Cheers
>
> Steve
>
>   
>> Ok, I see what youre doing now :-)
>> There's another simple solution that pops into mind; the file() source
>> driver supports the 'program-override()' option. Basically any logs read
>> from that file would have whatever $PROGRAM you specified. And that
>> $PROGRAM macro will be propagated from the client syslog-ng to your
>> server.
>> -Patrick
>> Sent: Tuesday, August 10, 2010 5:12:10 PM
>> From: Steve Barnes mailto:steve at echo.id.au <steve at echo.id.au>
>> To: Syslog-ng users' and developers' mailing list
>> mailto:syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
>> Subject: Re: [syslog-ng] Using source tags in rewrites?
>> To what end? Rewrites are used to change or set key/value pairs. Flags
>> arent key/value pairs, theyre flags, meaning you either have the flag, or
>> you dont. Think of it this way; each incoming log statment has a list of
>> flag names associated with it, you can see if a certain flag name is in
>> that list, but thats it. It would help to know what youre trying to
>> accomplish though. -Patrick
>> Patrick I have ~70 different log files containing the results of a daily
>> processing cycle. None of these log files are Syslog formatted. Unlike an
>> Apache log file entry, each line of these log files is ambiguous - unless
>> you referred to a portion of the file name, you'd have no way to
>> differentiate a line from one file with another line from another file.
>> The plan is to have SyslogNG set up as a client on a machine, tracking the
>> contents of these files as they are updated. Interesting lines are
>> filtered on the client with matching lines pushed across the network to a
>> SyslogNG server. The server is responsible for processing lines against a
>> pattern DB and then stuffing the useful information into a PostgreSQL
>> backend. What I was hoping to do on the client side is tag each source in
>> syslog-ng.conf with a meaningful name and then include that tag in either
>> the IETF structured data or, by using a rewrite, prefix each log line
>> before transmission with the tag value. This would give the SyslogNG
>> server at the other end someway to differentiate messages as they arrived.
>> Further reading yesterday uncovered the following in the 3.2 PE
>> documentation:
>> http://www.balabit.com/dl/html/syslog-ng-pe-v3.2-guide-admin-en.html/tagging_messages.html
>> http://www.balabit.com/dl/html/syslog-ng-pe-v3.2-guide-admin-en.html/tagging_messages.html
>> "To include the tags in the message, use the $TAGS macro in a template.
>> Alternatively, if you are using the IETF-syslog message format, you can
>> include the $TAGS macro in the .SDATA.meta part of the message. Note that
>> the $TAGS macro is available only in syslog-ng PE 3.1.1 and later." So it
>> looks like I've found the answer to my question? Before I start along the
>> path seeking purchase approval for SyslogNG PE, I wanted to make sure
>> there was no other way to do what I hand in mind? Cheers Steve
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>> http://www.campin.net/syslog-ng/faq.html
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>     
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100811/397bacc9/attachment.htm