[syslog-ng] parsing what gets into syslog from windows machines.

[Martin Holste]

Patterndb is probably your best bet, you can use pdbtool patternize to
create patterns.  The best way to do this is to dump many of these
logs into a file and then run patternize like:
pdbtool patternize <file>
Which should spit out a properly formatted patterndb XML file with a
pattern like this:
<patterns>
          <pattern>account was successfully logged on. Subject:
Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0
Logon Type: 3 New Logon: Security ID: @ESTRING:: @Account Name:
@ESTRING:: @Account Domain: ACCOUNTS Logon ID: @ESTRING:: @Logon GUID:
@ESTRING:: @Process Information: Process ID: 0x0 Process Name: -
Network Information: Workstation Name: Source Network Address:
@ESTRING:: @Source Port: @ESTRING:: @Detailed Authentication
Information: Logon Process: Kerberos Authentication Package: Kerberos
Transited Services: - Package Name (NTLM only): - Key Length: 0 This
event is generated when a logon session is created. It is generated on
the computer that was accessed. The subject fields indicate the
account on the local system which requested the logon. This is most
commonly a service such as the Server service</pattern>
</patterns>

Then you can insert your macro names into the ESTRING's, like this:
<patterns>
          <pattern>account was successfully logged on. Subject:
Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0
Logon Type: 3 New Logon: Security ID: @ESTRING:id: @Account Name:
@ESTRING:account_name: @Account Domain: ACCOUNTS Logon ID:
@ESTRING:logon_id: @Logon GUID: @ESTRING:guid: @Process Information:
Process ID: 0x0 Process Name: - Network Information: Workstation Name:
Source Network Address: @ESTRING:srcip: @Source Port:
@ESTRING:$srcport: @Detailed Authentication Information: Logon
Process: Kerberos Authentication Package: Kerberos Transited Services:
- Package Name (NTLM only): - Key Length: 0 This event is generated
when a logon session is created. It is generated on the computer that
was accessed. The subject fields indicate the account on the local
system which requested the logon. This is most commonly a service such
as the Server service</pattern>
</patterns>

Which will set the macros $id, $account_name, $logon_id, $guid,
$srcip, $srcport to be used in rewriting $MESSAGE to what you actually
want.

On Wed, Feb 1, 2012 at 3:24 PM, greg whynott <greg.whynott at gmail.com> wrote:
> Hello,
>
>  We have all our windows servers/controllers send system messages (using
> snare)  to our central log server which is a linux machine running
> syslog-ng.  They are very verbose, multi line messages.   Is there a way to
> have syslog-ng trim these down before the are wrote out to the log file?  Or
> perhaps you know of another method I should be looking at?  Thanks very
> much.
>
> here is a typical windows server message:
>
> Feb  1 16:08:31 10.111.1.11 MSWinEventLog       1       Security
> 5043398 Wed Feb 01 16:08:29 2012        4624
> Microsoft-Windows-Security-Auditing     NT AUTHORITY\ANONYMOUS LOGON
> N/A     Success Audit   AD02.domain.com Logon            An account was
> successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:
> -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    New Logon:
> Security ID:  S-1-5-7   Account Name:  ANONYMOUS LOGON   Account Domain:  NT
> AUTHORITY   Logon ID:  0x45ab55d   Logon GUID:
> {00000000-0000-0000-0000-000000000000}    Process Information:   Process
> ID:  0x0   Process Name:  -    Network Information:   Workstation Name:
> AC010   Source Network Address: 10.101.2.44   Source Port:  60711
> Detailed Authentication Information:   Logon Process:  NtLmSsp
> Authentication Package: NTLM   Transited Services: -   Package Name (NTLM
> only): NTLM V1   Key Length:  128    This event is generated when a logon
> session is created. It is generated on the computer that was accessed.
> The subject fields indicate the account on the local system which requested
> the logon. This is most commonly a service such as the Server service, or a
> local process such as Winlogon.exe or Services.exe.    The logon type field
> indicates the kind of logon that occurred. The most common types are 2
> (interactive) and 3 (network).    The New Logon fields indicate the account
> for whom the new logon was created, i.e. the account that was logged on.
> The network fields indicate where a remote logon request originated.
> Workstation name is not always available and may be left blank in some
> cases.    The authentication information fields provide detailed information
> about this specific logon request.   - Logon GUID is a unique identifier
> that can be used to correlate this event with a KDC event.   - Transited
> services indicate which intermediate services have participated in this
> logon request.   - Package name indicates which sub-protocol was used among
> the NTLM protocols.   - Key length indicates the length of the generated
> session key. This will be 0 if no session key was requested.       5040030
>
>
>
> I'd like to parse out the redundant data which is in every similar
> message,   several times a second...  the ultimate goal would be for that
> message to be trimmed down to:
>
>
>
> Feb  1 16:08:31 10.111.1.11        5043398  4624
> Microsoft-Windows-Security-Auditing     NT AUTHORITY\ANONYMOUS LOGON
> N/A     Success Audit   AD02.domain.com Logon    Workstation Name: AC110
> Source Network Address: 10.101.2.110 Source Port:  60711
>
> or similar...
>
>
> I can define aregx to do this,  but I am not sure where the best place would
> be to intercept and parse this before it is wrote out,  or if syslog-ng has
> the parts to allow this to happen.
>
> thanks for your time,
>
> greg
>
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>