Filtering binary data fields and catch all
Hi, My log source sends data over syslog in CEF (Arcsight format). I have a simple parser written in Syslog-NG that takes advantage of the CEF format (that is all messages are in eight fields separated by a pipe char). -----------xxxxxxxxxxxxxxxxx------------------- parser p_cef { csv-parser(columns("cef.ff","cef.vendor","cef.product","cef.c4","cef.c5","cef.c6","cef.c7","cef.c8") delimiters("|") flags(drop-invalid) ); }; destination d_file { file("/var/log/net/${cef.vendor}/${cef.product}/logfile"); }; log { source(s_tcp); parser(p_cef); destination(d_file); }; -----------xxxxxxxxxxxxxxxxx------------------- Some events coming in contain binary data that get translated into control characters and create thousands of directories with garbage in the names. How do I handle binary data in the message? Should I use the sanitize function in the destination/file definition or is there a better way to do it? Also, how do I create a catch-all destination for all the messages that do not match my parser? Thanks!
On Saturday, May 18, 2013 02:35 CEST, Xuri Nagarin <secsubs@gmail.com> wrote:
Hi,
My log source sends data over syslog in CEF (Arcsight format).
I have a simple parser written in Syslog-NG that takes advantage of the CEF format (that is all messages are in eight fields separated by a pipe char). -----------xxxxxxxxxxxxxxxxx-------------------
parser p_cef {
csv-parser(columns("cef.ff","cef.vendor","cef.product","cef.c4","cef.c5","cef.c6","cef.c7","cef.c8") delimiters("|") flags(drop-invalid) ); };
destination d_file { file("/var/log/net/${cef.vendor}/${cef.product}/logfile"); };
log { source(s_tcp); parser(p_cef); destination(d_file); }; -----------xxxxxxxxxxxxxxxxx-------------------
Some events coming in contain binary data that get translated into control characters and create thousands of directories with garbage in the names.
How do I handle binary data in the message? Should I use the sanitize function in the destination/file definition or is there a better way to do it?
Hi, I currently do not know about any way to handle binary data within a messages.
Also, how do I create a catch-all destination for all the messages that do not match my parser?
Create a log statement that uses the flags(catchall) option. Regards, Robert
Thanks!
Thanks Robert. I figured as much. I am going to look at some intermediary that can transform binary to base64 and then stream to syslog. On Tue, May 21, 2013 at 11:48 AM, Fekete Róbert <frobert@balabit.hu> wrote:
On Saturday, May 18, 2013 02:35 CEST, Xuri Nagarin <secsubs@gmail.com> wrote:
Hi,
My log source sends data over syslog in CEF (Arcsight format).
I have a simple parser written in Syslog-NG that takes advantage of the CEF format (that is all messages are in eight fields separated by a pipe char). -----------xxxxxxxxxxxxxxxxx-------------------
parser p_cef {
csv-parser(columns("cef.ff","cef.vendor","cef.product","cef.c4","cef.c5","cef.c6","cef.c7","cef.c8")
delimiters("|") flags(drop-invalid) ); };
destination d_file { file("/var/log/net/${cef.vendor}/${cef.product}/logfile"); };
log { source(s_tcp); parser(p_cef); destination(d_file); }; -----------xxxxxxxxxxxxxxxxx-------------------
Some events coming in contain binary data that get translated into
control
characters and create thousands of directories with garbage in the names.
How do I handle binary data in the message? Should I use the sanitize function in the destination/file definition or is there a better way to do it?
Hi, I currently do not know about any way to handle binary data within a messages.
Also, how do I create a catch-all destination for all the messages that do not match my parser?
Create a log statement that uses the flags(catchall) option.
Regards,
Robert
Thanks!
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
Fekete Róbert
-
Xuri Nagarin