[syslog-ng] syslog-ng is skipping syslog events with no PRI

[Asif Iqbal]

On Tue, Mar 20, 2018 at 7:21 AM, Nagy, Gábor <gabor.nagy at balabit.com> wrote:

> Hi Asif!
>
> I think the problem with the first message comes from the message
> structure and the filter statement.
> Its structure does not conform to either syslog RFC standards (RFC3164 or
> RFC5424).
> Syslog-ng still tries to parse the log message by its internal heuristics
> and the "alarmLog" text is parsed as the 'program' field.
> You can debug syslog-ng parsing with the format-json template in the
> destination:
>         `template("$(format-json -s syslog-proto)\n")`
>
> The output for this was:
> {"PROGRAM":"alarmLog,","PRIORITY":"notice","MESSAGE":"applianceName=Branch-UC1,
> tenantName=DEMO-CORP, alarmType=sla-not-met, alarmKey=INTERNET,
> generateTime=1521503387, applianceId=1, vsnId=0, tenantId=2,
> alarmCause=datapathState, alarmClearable=yes, alarmClass=cleared,
> alarmKind=symptom, alarmEventType=equipmentAlarm, alarmSeverity=cleared,
> alarmOwner=tenant, alarmSeqNo=1568, alarmText=delay:9 msec,
> siteName=Branch-UC1","HOST":"+0000","FACILITY":"user","DATE":"Mar 13
> 23:49:48"}
>
> The filter statement uses the `match()` filter which works on both the
> header and message part of the log message and thus would match for the
> first log message if the `value("MESSAGE")` part would not be there.
> With that you restricted the filter to match only on the message part.
> If you remove the value("MESSAGE") from the filter statement it will work.
>

That was it. It is working now!!

BTW, is there a way to generate a feed a pcap to some program on the
terminal to generate a json formatted output like this short from modifying
the syslog-ng config
for destination to a template like you are showing?

Thanks for your help and I see the logs being written to the file now!


>
> Regards,
> Gabor
>
>
> On Tue, Mar 20, 2018 at 1:52 AM, Asif Iqbal <vadud3 at gmail.com> wrote:
>
>> syslog-ng is *NOT* writing syslog like this to a file which has no <*PRI*
>> >
>>
>> 23:49:48.306587 IP 192.168.1.100.39567 > 192.168.100.100.514: [|syslog]
>> E..... at .>..U...g..........T.2018-03-19T23:49:48+0000 alarmLog,
>> applianceName=Branch-UC1, tenantName=DEMO-CORP, alarmType=sla-not-met,
>> alarmKey=INTERNET, generateTime=1521503387, applianceId=1, vsnId=0,
>> tenantId=2, alarmCause=datapathState, alarmClearable=yes,
>> alarmClass=cleared, alarmKind=symptom, alarmEventType=equipmentAlarm,
>> alarmSeverity=cleared, alarmOwner=tenant, alarmSeqNo=1568,
>> alarmText="delay:9 msec", siteName=Branch-UC1
>> ................
>>
>>
>> syslog-ng is writing syslog like this to a file *OK *
>>
>> 23:50:26.930023 IP 192.168.1.100.55078 > 192.168.100.100.514: SYSLOG
>> mail.info, length: 76
>> E..h.B at .>......g.....&...Tt.<22>Mar 19 23:50:26 SVL-remotehost-02 root:
>> this is third test alarmLog................
>>
>>
>> Here is my syslog-ng config
>>     source s_udp { udp(ip(0.0.0.0) port(514)); };
>>     destination d_alarm { file("/var/log/alarms.log"); };
>>     filter f_alarm { match("alarmLog" value("MESSAGE")); };
>>     log { source(s_udp); filter(f_alarm); destination(d_alarm); };
>>
>> I am using syslog-ng version 3.5.6 on centos 7
>>
>> Any idea why syslog-ng is writing the first log event into a file?
>>
>> Appreciate any help!
>>
>>
>> --
>> Asif Iqbal
>> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
>> A: Because it messes up the order in which people normally read text.
>> Q: Why is top-posting such a bad thing?
>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>


-- 
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180320/e7e987ce/attachment-0001.html>