<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Thanks Kokan for the options. I’ll start with the add-contextual-data option first and go from there.<div class=""><br class=""></div><div class="">Regards,</div><div class="">Max<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar 8, 2019, at 1:49 AM, Péter, Kókai <<a href="mailto:peter.kokai@oneidentity.com" class="">peter.kokai@oneidentity.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">Hello,<div class=""><br class=""></div><div class="">There are a few ways you could achieve that, but neither of them is built directly to handle *severity*; in a sense that they are general text handling options.</div><div class=""><br class=""></div><div class="">Option1: (the slowest one) use *if* - *elif* - *else* chain match/set (You can see a few example in scl)</div><div class="">Option2: (I would recommend this option) add-contextual-data , this provides a way to enrich the logs based on a csv</div><div class="">Option3: (This is really risky) I've created a PR (<a href="https://github.com/balabit/syslog-ng/pull/2602" class="">https://github.com/balabit/syslog-ng/pull/2602</a>) "fixing" the level value of level macro with *cisco-parser*, with this patch it should set the *.cisco.severity* so you could use the old macros</div><div class=""><br class=""></div><div class="">Please not that for the last option you have to compile syslog-ng yourself, and there is no guarantee that it is going to be upstreamed at all.</div><div class=""><br class=""></div><div class="">Note: there are possible other ways with template as well.</div><div class=""><br class=""></div><div class="">--</div><div class="">Kokan</div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 7, 2019 at 8:22 PM N. Max Pierson <<a href="mailto:nmaxpierson@gmail.com" class="">nmaxpierson@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class="">Hi Kokan,<div class=""><br class=""></div><div class="">I didn't catch that previously, so thanks for pointing that out for me. I made the changes to reflect the ${.cisco.severity} macro but now I am not getting anything in my database table. It seems to work if I log it to a file, but for some reason when I log to mysql, it doesn't put anything in the column I had for level. My config is below. Also, is there any way to show the text version of the severity instead of the number?</div><div class=""><br class=""></div><div class="">source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); };<br class=""></div><div class=""><br class=""></div><div class="">template t_cisco { template("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC} ${HOST} ${.cisco.severity} ${MESSAGE}\n"); };<br class=""></div><div class=""><br class=""></div><div class="">destination d_test { file("/var/log/network-test" template(t_cisco)); };<br class=""></div><div class=""><br class=""></div><div class=""><div class="">destination d_mysql {</div><div class=""> sql(type(mysql)</div><div class=""> host("127.0.0.1")</div><div class=""> username("syslog-ng")</div><div class=""> password("password")</div><div class=""> database("syslog")</div><div class=""> table("messages_${HOST}")</div><div class=""> columns("datetime datetime", "host varchar(50)", "level varchar(10)", "message text")</div><div class=""> values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "${HOST}", "${.cisco.severity}", "${MESSAGE}")</div><div class=""> indexes("datetime", "level")</div><div class=""> );</div><div class="">};</div></div><div class=""><br class=""></div><div class="">log { source(s_network); parser(p_cisco); destination(d_test); };<br class=""></div><div class=""><br class=""></div><div class="">log { source(s_network); parser(p_cisco); destination(d_mysql); };<br class=""></div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Max</div><div class=""><br class=""></div></div></div></div></div></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 7, 2019 at 1:50 AM Péter, Kókai <<a href="mailto:peter.kokai@oneidentity.com" target="_blank" class="">peter.kokai@oneidentity.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">Hello,<div class=""><br class=""></div><div class="">The *cisco-parser* as of now cannot set the LEVEL macro, as that is a *hard* macro, those cannot be set directly from configuration setters (for example: rewrite { set("lvl" value("LEVEL"); };).</div><div class="">The header is parsed into the ".prefix.*" macros, the default *prefix* is ".cisco.".</div><div class=""><br class=""></div><div class="">I think with shifting *parser*s implemented as syslog-ng configuration rather then C, it might be a good idea to lift the soft vs hard macro via a proper setters.</div><div class=""><br class=""></div><div class="">Note1: if you are using *flags(no-parse)* it is going to set a default level, changing at least that default level is possible per source "network( default_level(...) flags(no-parse) ...);"</div><div class=""><br class=""></div><div class="">Note2: if you really want to set LEVEL macro, well you can do it but not by design, and it might break at any point. It is possible to shadow hard macros setting patterns, instead of LEVEL you could set LEVLE[L] pattern, well which is kinda only matches LEVEL anyway. The way it works it creates a LEVEL soft macro. IMPORTANT: this leaves the message in a semi-state, as some code is going to use ${LEVEL} and some code is going to use the internal level representation which still stays the same old level.</div><div class=""><br class=""></div><div class="">--</div><div class="">Kokan</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 6, 2019 at 9:45 PM N. Max Pierson <<a href="mailto:nmaxpierson@gmail.com" target="_blank" class="">nmaxpierson@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><h3 class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-iw" style="overflow:hidden;white-space:nowrap;font-size:0.75rem;margin:inherit;text-overflow:ellipsis;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Balázs,,</span></span></h3><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal"><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">I have decided to not worry about anything without the cisco mnemonic and use the cisco parser until I can come up with a better solution. I made that modification to the plugin.conf file and now I am able to receive NX-OS messages but I have discovered an issue that seems to be with the cisco parser itself. When I enable it, all of my messages are coming in with a "notice" level so it seems as though the cisco parser isn't properly parsing the header correctly. I even moved back to the default plugin.conf file to make sure my modifications didn't break it in the process. Here is my config if that helps.....</span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal"><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class="">source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); };</span></font><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class=""><br class=""></span></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class="">template t_cisco { template("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC} ${HOST} ${LEVEL} ${MESSAGE}\n"); };</span><br class=""></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class=""><br class=""></span></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class="">destination d_test { file("/var/log/network-test" template(t_cisco)); };</span><br class=""></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class=""><br class=""></span></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class="">parser p_cisco { cisco-parser(); };</span><br class=""></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class=""><br class=""></span></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="display:inline;vertical-align:top;line-height:20px"><font color="#202124" class=""><span style="font-size:14px;letter-spacing:0.2px" class="">log { source(s_network); parser(p_cisco); destination(d_test); };</span><br class=""></font></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal"><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Regards,</span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Max</span></span></div></div></div></div></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 4, 2019 at 2:00 PM Scheidler, Balázs <<a href="mailto:balazs.scheidler@oneidentity.com" target="_blank" class="">balazs.scheidler@oneidentity.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class="">It was a bit strange that nxos starts with a colon and that a cisco message didn't have a mnemonic. Cisco messages by my experience always contain a triplet and this one isn't, and doesn't seem to have a timestamp either. If you say that this was recorded the way you mentioned then i believe you :), they just looked a bit odd.<div dir="auto" class=""><br class=""></div><div dir="auto" class="">Do all messages from the same device look the same? Or these are just the exceptions?</div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Also what is nxos?</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Sat, Mar 2, 2019, 21:37 Scheidler, Balázs <<a href="mailto:balazs.scheidler@oneidentity.com" target="_blank" class="">balazs.scheidler@oneidentity.com</a> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class="">Can you please include the log sample as it was received by syslog-ng? This seems to have been mangled already.</div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, Mar 1, 2019, 23:16 N. Max Pierson <<a href="mailto:nmaxpierson@gmail.com" rel="noreferrer" target="_blank" class="">nmaxpierson@gmail.com</a> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class="">Here are 2 different samples of logs without the mnemonic, one is IOS and the other is NX-OS<div class=""><br class=""></div><div class=""><div class="">IOS, no-parse log to file</div><div class="">--</div><div class="">Mar 1 16:06:07 hostname1 <185>78919861: -Traceback= 151644C 151763C 17F0094 1B8BDE8 1B82858</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">NX-OS, no-parse log to file</div><div class="">--</div><div class="">Mar 1 16:10:41 hostname1 <189>: 2019 Mar 1 16:10:41 CST: last message repeated 3 times</div></div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Max</div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 28, 2019 at 5:45 PM Scheidler, Balázs <<a href="mailto:balazs.scheidler@oneidentity.com" rel="noreferrer noreferrer" target="_blank" class="">balazs.scheidler@oneidentity.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class="">Log samples would be appreciated.</div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Thu, Feb 28, 2019, 18:11 N. Max Pierson <<a href="mailto:nmaxpierson@gmail.com" rel="noreferrer noreferrer" target="_blank" class="">nmaxpierson@gmail.com</a> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class=""><h3 class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-iw" style="overflow:hidden;white-space:nowrap;font-size:0.75rem;margin:inherit;text-overflow:ellipsis;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Balázs,</span></span></h3><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal"><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">I modified the plugin.conf XML file (<a href="https://pastebin.com/9BruYy0S" rel="noreferrer noreferrer noreferrer" target="_blank" class="">https://pastebin.com/9BruYy0S</a> - added lines 62-65) for the cisco-parser and I was able to capture NX-OS syslogs. I also noticed that the default parser is not matching syslogs that do not include the Cisco mnemonic (both for IOS and NX-OS). While I was able to fix the NX-OS issues (which was a date parsing problem), i'm not sure what needs to be changed/added in this file so that even logs that do not contain the mnemonic are matched. Can anyone shed some light on what part of this parser needs to be changed to resolve the issue? I can post some examples of a log that doesn't contain the mnemonic should you need them.</span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal"><br class=""></span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Regards,</span></span></div><div class=""><span class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-qu" style="outline:none"><span name="Scheidler, Balázs" class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-gD" style="color:rgb(32,33,36);font-size:0.875rem;display:inline;vertical-align:top;letter-spacing:0.2px;line-height:20px;font-weight:normal">Max</span></span></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 26, 2019 at 11:14 AM Scheidler, Balázs <<a href="mailto:balazs.scheidler@oneidentity.com" rel="noreferrer noreferrer noreferrer" target="_blank" class="">balazs.scheidler@oneidentity.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class="">The cisco-parser() should take care about messages from cisco routers assuming they were received using flags(no-parse). It will automatically detect timestamp formats, but its not perfect, so if you encounter something that it doesn't properly parse, do let us know, so we can add it.<div dir="auto" class=""><br class=""></div><div dir="auto" class="">Also, default-network-drivers() makes it possible to receive both cisco and non-cisco logs on the same port, automatically recognizing the appropriate format. This driver relies on the app-parser() framework, which can be extended by application specific parsers. With that you can construct your specific source driver configuration if you find default-network-drivers () too complicated.</div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">You can find all of these in the documentation, and their source in the syslog-ng configuration library (scl for short, usually in /usr/share/syslog-ng/include/scl).</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Tue, Feb 26, 2019, 18:08 Sandor Geller <<a href="mailto:sandor.geller@ericsson.com" rel="noreferrer noreferrer noreferrer" target="_blank" class="">sandor.geller@ericsson.com</a> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" class="">
<div class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-m_5001367715343377916m_2558471824089517427moz-cite-prefix">Hello,<br class="">
<br class="">
When the no-parse flag is used then the macros referencing various
parts of the message aren't filled in. HOST could get looked up
using a reverse DNS lookup unless the keep_hostname option is set.
The syslog priority is set to user.notice when parsing is
disabled.<br class="">
<br class="">
Did you take a look at the existing cisco parser? Using it or
adopting it should ease your job. If you could configure the
Ciscos to use other port than anything else (which speaks syslog,
Cisco devices usually aren't such...) would be even better.<br class="">
<br class="">
Regards,<br class="">
Sandor<br class="">
<br class="">
On 02/26/2019 05:36 PM, N. Max Pierson wrote:<br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">Hi List,
<div class=""><br class="">
</div>
<div class="">I have been trying to get something in place that
can parse syslogs from various Cisco devices. The
message format is almost the same with a few
exceptions. Here is what I have tried and it works but
now it has created another problem I do not know how
to troubleshoot.</div>
<div class=""><br class="">
</div>
<div class="">So that I could see exactly what was being parsed,
I disabled the default parsing using the below.</div>
<div class=""><br class="">
</div>
<div class="">source s_network { udp(ip(0.0.0.0) port(514)
flags(no-parse)); };<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">rewrite r_cisco{
subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s',
"", value("MESSAGE"), type("pcre"),
flags("ignore-case")); };<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">destination d_mysql {</div>
<div class=""> sql(type(mysql)</div>
<div class=""> host("127.0.0.1")</div>
<div class=""> username("syslog-ng")</div>
<div class=""> password("password")</div>
<div class=""> database("syslog")</div>
<div class=""> table("messages_${HOST}")</div>
<div class=""> columns("datetime datetime", "host
varchar(50)", "level varchar(10)", "message text")</div>
<div class=""> values("${R_YEAR}-${R_MONTH}-${R_DAY}
${R_HOUR}:${R_MIN}:${R_SEC}", "${HOST}", "${LEVEL}",
"${MESSAGE}")</div>
<div class=""> indexes("datetime", "level")</div>
<div class=""> );</div>
<div class="">};</div>
</div>
<div class=""><br class="">
</div>
<div class="">log { source(s_network); rewrite(r_cisco);
destination(d_mysql); };<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">This works perfectly as it formats the message as I
want and covers IOS and NX-OS devices. The problem is
when I turned off the default parser, now all of my
logs show "notice" in the $LEVEL macro and doesn't
reflect the real message header level. The $HOST macro
still works fine however.</div>
<div class=""><br class="">
</div>
<div class="">Is this the expected behavior that the message
header fields are not parsed as well as the $MESSAGE
itself not being parsed? How can map the header level
field properly to the $LEVEL marco if I disable the
default parser?</div>
<div class=""><br class="">
</div>
<div class="">Regards,</div>
<div class="">Max</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="">
<fieldset class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-m_5001367715343377916m_2558471824089517427mimeAttachmentHeader"></fieldset>
<br class="">
<pre class="">______________________________________________________________________________
Member info: <a class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-m_5001367715343377916m_2558471824089517427moz-txt-link-freetext" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation: <a class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-m_5001367715343377916m_2558471824089517427moz-txt-link-freetext" href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a class="gmail-m_-4455728725286707607gmail-m_-6605974198270267173gmail-m_-2303564284846740693gmail-m_-2060649226236921190m_9050550322518824090m_-5038579878582832643gmail-m_-1307092612606575568m_-7753228101678025534gmail-m_5001367715343377916m_2558471824089517427moz-txt-link-freetext" href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a>
</pre>
</blockquote><p class=""><br class="">
</p>
</div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class="">
<br class="">
</blockquote></div>
______________________________________________________________________________<br class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class=""><br class=""></div></blockquote></div><br class=""></div></body></html>