[syslog-ng] is it exist some way to extract the filename listened by file() ?

[Scheidler, Balázs]

Hi,

I've just sent a PR to implement basename and dirname for your use-case:

https://github.com/balabit/syslog-ng/pull/1420

You can probably apply the same to 3.7, but still you'd need to recompile
from source.


-- 
Bazsi

On Sat, Apr 8, 2017 at 9:09 AM, Jorge Pereira <jpereiran at gmail.com> wrote:

> Btw, somebody knows which the best way to extract only the file name? I
> mean something like the function basename() ?
>
> --
> Jorge Pereira
>
> On Sat, Apr 8, 2017 at 3:50 AM, Jorge Pereira <jpereiran at gmail.com> wrote:
>
>> Hi,
>>
>> Thanks so much!! exactly, I didn find it.
>>
>> --
>> Jorge Pereira
>>
>> On Sat, Apr 8, 2017 at 3:10 AM, Scheidler, Balázs <
>> balazs.scheidler at balabit.com> wrote:
>>
>>> Hi,
>>>
>>> It seems indeed ugly. We do have a FILE_NAME macro that gets set to the
>>> name of the file the message was read from.
>>>
>>> With a quick search I didn't find it documented.
>>>
>>> On Apr 8, 2017 07:27, "Jorge Pereira" <jpereiran at gmail.com> wrote:
>>>
>>>> Hi Team,
>>>>
>>>> Well, I am working on a POC using the syslog-ng 3.7.1, basically, I
>>>> have many of log files that the filename is /path/<file> and I need to
>>>> append the file name into the syslog payload.
>>>>
>>>> My current approach is.
>>>>
>>>> 1. I have the below destination() receiving the file name as a
>>>> parameter.
>>>>
>>>> <snip>
>>>> block destination d_collector_with_fn(__filename("")) {
>>>>     tcp("192.168.2.44"
>>>>         port(514)
>>>>         keep-alive(on)
>>>>         template("$DATE $HOST $MSGHDR $(format-json --scope
>>>> selected_macros             \
>>>>                                                     --exclude TAGS
>>>>                  \
>>>>                                                     --exclude DATE
>>>>                  \
>>>>                                                     --exclude PRIORITY
>>>>                  \
>>>>                                                     --exclude FACILITY
>>>>                  \
>>>>                                                     --exclude SOURCEIP
>>>>                  \
>>>>                                                     --exclude PROGRAM
>>>>                 \
>>>>                                                     --pair
>>>> SYSLOG_WEBAPP_DOMAIN='`__filename`'  \
>>>>                                                     --pair
>>>> SOURCE=${SOURCE}
>>>>         )\n")
>>>>         template-escape(no)
>>>>     );
>>>> };
>>>> </snip>
>>>>
>>>>
>>>> 2. My simple script called by confgen create some dynamic "log {}"
>>>> statements listening to the files and appending the filename as a parameter
>>>> to the d_collector_with_fn()
>>>>
>>>> <snip>
>>>> log {
>>>>         source {
>>>>                 file("/path/thisisafile001.net"
>>>>                         program_override("mytag")
>>>>                         follow_freq(1)
>>>>                         flags(no-parse)
>>>>                 );
>>>>         };
>>>>         destination {
>>>>                 d_collector_with_fn(__filename("thisisafile001.net"));
>>>>         };
>>>> };
>>>>
>>>> log {
>>>>         source {
>>>>                 file("caipirinha4ever.net"
>>>>                         program_override("mytag")
>>>>                         follow_freq(1)
>>>>                         flags(no-parse)
>>>>                 );
>>>>         };
>>>>         destination {
>>>>                 d_collector_with_fn(__filename("caipirinha4ever.net"));
>>>>         };
>>>> };
>>>>
>>>> .........................
>>>> </snip>
>>>>
>>>> But, I have more than 5k files and my current approach creating
>>>> multiples log { } statement resulting in one connection to the collector by
>>>> each file!!! in this case, I have 5k connections... this is terrible,
>>>> someone has some other suggestion? exist some way to catch the filename by
>>>> some internal ${variable} and pass for a single destination()?
>>>>
>>>> --
>>>> Jorge Pereira
>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170408/d8b462a8/attachment.html>