[syslog-ng] is it exist some way to extract the filename listened by file() ?

[Scheidler, Balázs]

Hi,

It seems indeed ugly. We do have a FILE_NAME macro that gets set to the
name of the file the message was read from.

With a quick search I didn't find it documented.

On Apr 8, 2017 07:27, "Jorge Pereira" <jpereiran at gmail.com> wrote:

> Hi Team,
>
> Well, I am working on a POC using the syslog-ng 3.7.1, basically, I have
> many of log files that the filename is /path/<file> and I need to append
> the file name into the syslog payload.
>
> My current approach is.
>
> 1. I have the below destination() receiving the file name as a parameter.
>
> <snip>
> block destination d_collector_with_fn(__filename("")) {
>     tcp("192.168.2.44"
>         port(514)
>         keep-alive(on)
>         template("$DATE $HOST $MSGHDR $(format-json --scope
> selected_macros             \
>                                                     --exclude TAGS
>              \
>                                                     --exclude DATE
>              \
>                                                     --exclude PRIORITY
>              \
>                                                     --exclude FACILITY
>              \
>                                                     --exclude SOURCEIP
>              \
>                                                     --exclude PROGRAM
>               \
>                                                     --pair
> SYSLOG_WEBAPP_DOMAIN='`__filename`'  \
>                                                     --pair SOURCE=${SOURCE}
>         )\n")
>         template-escape(no)
>     );
> };
> </snip>
>
>
> 2. My simple script called by confgen create some dynamic "log {}"
> statements listening to the files and appending the filename as a parameter
> to the d_collector_with_fn()
>
> <snip>
> log {
>         source {
>                 file("/path/thisisafile001.net"
>                         program_override("mytag")
>                         follow_freq(1)
>                         flags(no-parse)
>                 );
>         };
>         destination {
>                 d_collector_with_fn(__filename("thisisafile001.net"));
>         };
> };
>
> log {
>         source {
>                 file("caipirinha4ever.net"
>                         program_override("mytag")
>                         follow_freq(1)
>                         flags(no-parse)
>                 );
>         };
>         destination {
>                 d_collector_with_fn(__filename("caipirinha4ever.net"));
>         };
> };
>
> .........................
> </snip>
>
> But, I have more than 5k files and my current approach creating multiples
> log { } statement resulting in one connection to the collector by each
> file!!! in this case, I have 5k connections... this is terrible, someone
> has some other suggestion? exist some way to catch the filename by some
> internal ${variable} and pass for a single destination()?
>
> --
> Jorge Pereira
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170408/5196baa5/attachment.html>