Testing filter rules, How can I log structure and data to a fifo ?
Searched for 2 hours this morning and could not find the answer. Testing message filters and I'd like to be able to log all the meta data value pairs to a fifo to make troubleshooting easier. Like RSYSLOG_DebugFormat
Hi, "Scot" <scotrn@gmail.com> írta 2016-02-15 09:58-kor:
Searched for 2 hours this morning and could not find the answer.
Testing message filters and I'd like to be able to log all the meta data value pairs to a fifo to make troubleshooting easier.
Like RSYSLOG_DebugFormat
I typically use this: destination d_net_flat { file("/var/log/net/$HOST" template("$ISODATE $(format_json --scope nv_pairs,all_macros,core,selected_macros,everything,core --exclude R_* --exclude S_* )\n")); }; Later in /var/log/net I watch the interesting file with this: tail -f 10.21.255.141 | perl -MJSON -MData::Dumper -ne 's/.*?\{/{/ ; my $j=JSON::decode_json $_; print Data::Dumper::Dumper($j) if 1 ; print "$j->{FULLDATE};$j->{HOST};$j->{MSGHDR}$j->{MSG}\n"; ' The filename in the tail -f can be any hosts's file. The other part: If I do not use the "$ISODATE " prefix in the template, then even this perl one-liner is not neccessary. You can use json_pp as a filter after tail too. The advantage of this perl "oneliner" is, that I can modify the always true 1 expression as the print's conditional, so I have a better chance to filter out lines, which interests me. I hope, this is kind of a solution, which may help you. Cheers, Gyu
Wow... I figured there was a simple log headers option or something I wasn't finding. I have about 3000 unique hosts so that solution may be a bit more involved for simple filter troubleshooting. Thanks for the feedback. I'll give it a try On Mon, Feb 15, 2016 at 10:06 AM, PÁSZTOR György < pasztor@linux.gyakg.u-szeged.hu> wrote:
Hi,
"Scot" <scotrn@gmail.com> írta 2016-02-15 09:58-kor:
Searched for 2 hours this morning and could not find the answer.
Testing message filters and I'd like to be able to log all the meta data value pairs to a fifo to make troubleshooting easier.
Like RSYSLOG_DebugFormat
I typically use this: destination d_net_flat { file("/var/log/net/$HOST" template("$ISODATE $(format_json --scope nv_pairs,all_macros,core,selected_macros,everything,core --exclude R_* --exclude S_* )\n")); };
Later in /var/log/net I watch the interesting file with this:
tail -f 10.21.255.141 | perl -MJSON -MData::Dumper -ne 's/.*?\{/{/ ; my $j=JSON::decode_json $_; print Data::Dumper::Dumper($j) if 1 ; print "$j->{FULLDATE};$j->{HOST};$j->{MSGHDR}$j->{MSG}\n"; '
The filename in the tail -f can be any hosts's file. The other part: If I do not use the "$ISODATE " prefix in the template, then even this perl one-liner is not neccessary. You can use json_pp as a filter after tail too. The advantage of this perl "oneliner" is, that I can modify the always true 1 expression as the print's conditional, so I have a better chance to filter out lines, which interests me.
I hope, this is kind of a solution, which may help you.
Cheers, Gyu
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Starting with syslog-ng 3.7 there's an interactive debugger console as well. It's only the beginning but still pretty useful for config problems. Just launch syslog-ng with -I and if you have Python enabled you'll also have command completion.
participants (3)
-
PÁSZTOR György
-
Scheidler, Balázs
-
Scot