[syslog-ng] Windows Events multi-line

Thu Jul 13 14:23:16 UTC 2017

Hi Nagy,

Thanks for checking that. That was probably one of the reasons it did not work. That is too bad since Windows servers using the syslog-ng agent are the source. I would have thought that would be working since bot products are from Balabit. Windows logs also send a lot of garbage with Information lines that it would also be good to remove since that is just going to fill up the log files with junk. I am getting 8-10 gigs per day from a single Windows Domain Controller and I need to add in about 25 more. The log file would be immense.

I tested out using the syslog driver and was poking around with the multi-line-prefix and multi-line-garbage and they don't give any syntax errors, as long as I don't use the multi-line-mode option. Does syslog-ng ignore these options if the multi-line-mode option is not specified? Basically, do all the multi-line-xxx options not work like this?

It seems like the only option that works for syslog or network is the flags(no-multi-line) flag.

Is that the case?

Are there any other alternatives? I would hate to write the syslogs to a file and then parse that file again to get the logs I want. That is double work for the server. But that seems like that might be the only option I have. 

Hmmm...maybe I can log to a VM syslog-ng and have that one parse the data first using the file driver and then I can relay to my main systems.

Paul C Smith

-----Original Message-----
From: syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of syslog-ng-request at lists.balabit.hu
Sent: Thursday, July 13, 2017 7:00 AM
To: syslog-ng at lists.balabit.hu
Subject: syslog-ng Digest, Vol 147, Issue 14

Send syslog-ng mailing list submissions to
	syslog-ng at lists.balabit.hu

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.balabit.hu/mailman/listinfo/syslog-ng
or, via email, send a message with subject or body 'help' to
	syslog-ng-request at lists.balabit.hu

You can reach the person managing the list at
	syslog-ng-owner at lists.balabit.hu

When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..."

Today's Topics:

   1. Re:  Windows Events multi-line (Nagy)

----------------------------------------------------------------------

Message: 1
Date: Thu, 13 Jul 2017 10:50:03 +0200
From: Nagy, Gábor <gabor.nagy at balabit.com>
To: "Syslog-ng users' and developers' mailing list"
	<syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Windows Events multi-line
Message-ID:
	<CAETAYnCft3KAyWF_5pB3g4+jodrn6+SpXTYJqqyfnFoUFzzaEw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,

Checking your configuration I see that you are using multi-line mode for a network source driver.
Unfortunately, currently this is not supported, only the file() and pipe() source drivers support multi-line messages.

I have checked the documentation and found that although it is mentioned which source drivers support multi-line mode (under flags option "no-multi-line"), the multi-line-* options are included in the network source driver options page.
This is a documentation bug, which will be fixed soon. Sorry for the inconveniences.

Best Regards,
Gabor

On Mon, Jul 10, 2017 at 8:15 PM, Smith, Paul (Sr. Admin-InfoSec) < Paul.C.Smith at snapon.com> wrote:

> Hello,
>
>
>
> I’m having problems getting Windows events on a single line on 
> syslog-ng OSE. I’ve scoured the interwebs and not found what I need to 
> get this exact. I am guessing this is not an uncommon problem but I 
> can’t seem to find quite what I need. I am guessing I am just missing 
> some simple thing here.
>
>
>
> Here are my details.
>
>
>
> Using syslog-ng OSE 3.9.1
>
>
>
> Have syslog-ng Windows Agent 6.0.6 running on Windows 2012 server
>
>
>
> Have a tcp source and that writes direct to the log file. Works fine 
> with no options set.
>
>
>
> Getting multiple lines per event.
>
>
>
> I’ve added what I think are the correct settings for multi-line, it 
> does not work. I don’t think it is the regex syntax, but something else?
>
>
>
> ___________
>
> Syslog.ng.conf
>
>
>
> source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp")
>
>                               multi-line-mode(regexp)
>
>                               multi-line-prefix("^[0-9]{3,5} 
> \s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][
> 0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})")
>
>                               flags(no-parse)); };
>
>
>
> ___________
>
> Here is the error when trying to start syslog-ng or run syslog-ng –s:
>
>
>
> Error parsing afsocket, syntax error, unexpected LL_IDENTIFIER, 
> expecting KW_NORMALIZE_HOSTNAMES or KW_USE_DNS or KW_USE_FQDN or 
> KW_DNS_CACHE in /etc/syslog-ng/syslog-ng.conf at line 38, column 76:
>
>
>
> source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp")
> multi-line-mode(regexp) multi-line-prefix("^[0-9]{3,5} 
> \s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][
> 0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})")
> flags(no-parse)); };
>
>
> ^^^^^^^^^^^^^^^
>
> ___________
>
> Sample of the event log part I am matching regex on:
>
>
>
> Jul 10 12:11:19 x.x.x.x 912 <133>1 2017-07-10T12:11:18-05:00 
> computername Microsoft_Windows_security_auditing. 6260 - [win at 18372.4 
> EVENT_CATEGORY="Logoff" EVENT_FACILITY="16" EVENT_ID="4634" EVENT_LEVEL="0"
> EVENT_NAME="Security" EVENT_REC_NUM="573516592" EVENT_SID="N/A"
> EVENT_SOURCE="Microsoft Windows security auditing." EVENT_TASK="Logoff"
> EVENT_TYPE="Success Audit" EVENT_USERNAME="domain\\userid"][meta
> sequenceId="10817278" sysUpTime="-198876"] domain\userid: Security 
> Microsoft Windows security auditing.: [Success Audit] An account was 
> logged off.
>
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170713/826dd963/attachment-0001.html>

------------------------------

Subject: Digest Footer

_______________________________________________
syslog-ng maillist  -  syslog-ng at lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

------------------------------

End of syslog-ng Digest, Vol 147, Issue 14
******************************************