[syslog-ng] RE syslog-ng OSE parsing

[Nagy, Gábor]

Hi Daniel!

Your log does not conform to either RFC-3164 or RFC-5424, it looks like
kind-of cisco format to me.
I've tried one or two ideas, using our `default-network-drivers()` which
includes the cisco-parser() but it is not dealing with it properly.

On second try, I've experimented with `kv-parser()` so the key-value parts
in the message will be parsed in a structured format and after that you can
use a rewrite rule to modify HOST field of the message.
I had to adjust the pair-separator option to get an optimal result.
`parser { kv-parser( prefix(".kv.") pair-separator("]")); };`

After a successful parsing, you can use a rewrite rule to set the HOSTNAME.
rewrite { set("${.kv.ID}" value("HOST")); };
There is one catch: you need to know the name of the key: "Node ID" or
"Hostname", which comes from the log message.
So, if this expression is not fix, then your config will not work.

You can also see that I've added the key "ID" rather than "Node ID", since
your log message does not quote the keys in it and
in this case kv-parser will only use the "ID" from "Node ID" part which
leads to that similar key-values are overwritten: "App ID", "Cluster ID"

I will think about it, but hopefully others will come up with a better
idea. :)

Regards,
Gabor

On Fri, Jun 15, 2018 at 6:37 AM, Daniel Ehrlich <Daniel.Ehrlich at usq.edu.au>
wrote:

> Hi,
>
>
>
> Hoping you can assist me, I haven’t really come across anything that makes
> full sense to me in my searching of various sites/forums.
>
>
>
> This is an example log.
>
> Jun 14 11:57:27 PM.685 UTC :  %UC_LOGIN-4-AuthenticationFailed: %[Login
> Date/Time=06/15/2018 at 09:57:27][Login IP Address/Hostname=10.25.1.16][Login
> Interface=cucm-uds][Login UserID=POBAR][App ID=Cisco Tomcat][Cluster
> ID=][Node ID=cucmsub-prd-t2]: Login Authentication failed.
>
>
>
> Syslog-ng reads the $HOST as PM.685 ; can I get it to rewrite host as
> cucmsub-prd-t2 ? i.e. Node ID=
>
>
>
> Thanks you
>
>
>
> Kind Regards,
>
>
>
> *Daniel Ehrlich*
>
> *MastInfoSysSec, DipBA, SSCP, F5-CA, Splunk CA*
>
> ICT Security Officer
> ICT Client Services|Infrastructure Services
> Phone: +61 7 4687 5600 Email: Daniel.Ehrlich at usq.edu.au
>
> Toowoomba | Queensland | 4350 | Australia
>
>
>
> _____________________________________________________________
> This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email.
>
> The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt.
>
> The University of Southern Queensland is a registered provider of education with the Australian Government.
> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=
> syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>

On Fri, Jun 15, 2018 at 6:37 AM, Daniel Ehrlich <Daniel.Ehrlich at usq.edu.au>
wrote:

> Hi,
>
>
>
> Hoping you can assist me, I haven’t really come across anything that makes
> full sense to me in my searching of various sites/forums.
>
>
>
> This is an example log.
>
> Jun 14 11:57:27 PM.685 UTC :  %UC_LOGIN-4-AuthenticationFailed: %[Login
> Date/Time=06/15/2018 at 09:57:27][Login IP Address/Hostname=10.25.1.16][Login
> Interface=cucm-uds][Login UserID=POBAR][App ID=Cisco Tomcat][Cluster
> ID=][Node ID=cucmsub-prd-t2]: Login Authentication failed.
>
>
>
> Syslog-ng reads the $HOST as PM.685 ; can I get it to rewrite host as
> cucmsub-prd-t2 ? i.e. Node ID=
>
>
>
> Thanks you
>
>
>
> Kind Regards,
>
>
>
> *Daniel Ehrlich*
>
> *MastInfoSysSec, DipBA, SSCP, F5-CA, Splunk CA*
>
> ICT Security Officer
> ICT Client Services|Infrastructure Services
> Phone: +61 7 4687 5600 Email: Daniel.Ehrlich at usq.edu.au
>
> Toowoomba | Queensland | 4350 | Australia
>
>
>
> _____________________________________________________________
> This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email.
>
> The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt.
>
> The University of Southern Queensland is a registered provider of education with the Australian Government.
> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180615/6837b51b/attachment-0001.html>