On Apr 8, 2017 07:27, "Jorge Pereira" <jpereiran@gmail.com> wrote:

Hi Team,

Well, I am working on a POC using the syslog-ng 3.7.1, basically, I have many of log files that the filename is /path/<file> and I need to append the file name into the syslog payload.

My current approach is.

1. I have the below destination() receiving the file name as a parameter.

<snip>
block destination d_collector_with_fn(__filename("")) {
tcp("192.168.2.44"
port(514)
keep-alive(on)
template("$DATE $HOST $MSGHDR $(format-json --scope selected_macros \
--exclude TAGS \
--exclude DATE \
--exclude PRIORITY \
--exclude FACILITY \
--exclude SOURCEIP \
--exclude PROGRAM \
--pair SYSLOG_WEBAPP_DOMAIN='`__filename`' \
--pair SOURCE=${SOURCE}
)\n")
template-escape(no)
);
};
</snip>

2. My simple script called by confgen create some dynamic "log {}" statements listening to the files and appending the filename as a parameter to the d_collector_with_fn()

<snip>
log {
source {
file("/path/thisisafile001.net"
program_override("mytag")
follow_freq(1)
flags(no-parse)
);
};
destination {
d_collector_with_fn(__filename("thisisafile001.net"));
};
};

log {
source {
file("caipirinha4ever.net"
program_override("mytag")
follow_freq(1)
flags(no-parse)
);
};
destination {
d_collector_with_fn(__filename("caipirinha4ever.net"));
};
};

.........................
</snip>

But, I have more than 5k files and my current approach creating multiples log { } statement resulting in one connection to the collector by each file!!! in this case, I have 5k connections... this is terrible, someone has some other suggestion? exist some way to catch the filename by some internal ${variable} and pass for a single destination()?

--
Jorge Pereira

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq