[syslog-ng] syslog-ng has add extra field
Balazs Scheidler
bazsi77 at gmail.com
Fri Mar 26 09:11:03 UTC 2021
Hi,
This is a yet another date format, that our rfc3164 parser doesn't
recognize.
You will need to receive this via flags(no-parse), which is done by
default-network-drivers (), but there's no application adapter for this
kind of log to automatically recognize this yet.
An adapter something along the like of:
application huawei[syslog-raw] {
filter { message("%%"); };
parser { ... };
};
The filter part should contain a filter that identifies huawei candidate
messages, it doesn't need to be perfect (can even be absent), it just
excludes messages that would definitely not be huawei messages, and is
basically a performance optimization.
The parser piece (which is dotted out for now), needs both identify the
message and to extract fields that you want to make available within
syslog-ng itself.
If you look at the cisco-parser() source, it similarly recognizes a set of
timestamps (cisco isn't very standards friendly either), and then extracts
the cisco log triplet (severity, subsystem and mnemonic).
If you come up with the parser expression, this would be a worthwhile
addition to syslog-ng, so please submit it as a pull request to our scl
directory.
Thanks
Bazsi
On Fri, Mar 26, 2021, 09:42 Ivan Nepryahin - Bercut <
Ivan.Nepryahin at bercut.com> wrote:
> I've made pcap file and did send to you.
>
>
> Piece of output (for history)
>
>
> ####### with +03:00
> 11:11:45.057726 IP (tos 0x0, ttl 239, id 17, offset 0, flags [none], proto
> UDP (17), length 297)
> 192.168.102.1.38514 > 172.18.0.2.syslog: SYSLOG, length: 269
> Facility local7 (23), Severity notice (5)
> Msg: Mar 26 2021 11:11:44+03:00 HUAWEI-CORE-IMAQLIQ-1
> %%01CLI/5/CMDRECORD(s):CID=0x80ca2713;Recorded command information.
> (Task=VTY0, RemoteIp=192.168.55.41, VpnName=_public_, User=nepryahin,
> AuthenticationMethod="Tacacs", Command="system-view",
> LocalIp=192.168.102.1.)
>
>
> ####### without +03:00
> 11:23:37.882898 IP (tos 0x0, ttl 239, id 50, offset 0, flags [none], proto
> UDP (17), length 284)
> 192.168.102.1.38514 > 172.18.0.2.syslog: SYSLOG, length: 256
> Facility local7 (23), Severity notice (5)
> Msg: Mar 26 2021 08:23:37 HUAWEI-CORE-IMAQLIQ-1
> %%01CLI/5/CMDRECORD(s):CID=0x80ca2713;Recorded command information.
> (Task=VTY0, RemoteIp=192.168.55.41, VpnName=_public_, User=nepryahin,
> AuthenticationMethod="Tacacs", Command="quit", LocalIp=192.168.102.1.)
>
>
>
> Thank you for you kindness!
>
>
> best regards,
> *Nepryahin Ivan*
> IT Department
> *Phone**: *+7 812 327 32 33
> *Mobile: *+7 911 291 81 68
>
> ------------------------------
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
> SZIGETVÁRI János <jszigetvari at gmail.com>
> *Sent:* Thursday, March 25, 2021 8:55:36 PM
> *To:* Syslog-ng users' and developers' mailing list
> *Subject:* Re: [syslog-ng] syslog-ng has add extra field
>
> Hi Ivan,
>
> Okay, I see what you mean.
> The thing is that the default-network-drivers() source in your config is
> automatically and seamlessly expanded to a series of syslog() and network()
> sources definitions.
> The syslog(...) and network(...) statements I mentioned eariler were
> source definitions as well.
>
> In order to get a more thorough look at your use-case, I would kindly
> request you to send (either to me privately, or to this mailing list) a
> packet dump of the Huawei device's logs so that I can identify which port
> the traffic is destined to (and through which protocol), and what the
> actual format of the logs is.
>
> You can either do this by
> * running tcpdump ( tcpdump -peni any -v -w /tmp/syslog.pcap "port 514 or
> port 601" ), or
> * by downloading (if the script wasn't already deployed along your
> syslog-ng installation) and running syslog-ng-debun
> <https://raw.githubusercontent.com/syslog-ng/syslog-ng/master/contrib/syslog-ng-debun>
> ( syslog-ng-debun -r -p -t 300 ) (this will run the information gathering
> script for 5 minutes, and it will try to do the packet capture for you)
>
> Thanks!
>
> Best Regards,
> János Szigetvári
>
> Ivan Nepryahin - Bercut <Ivan.Nepryahin at bercut.com> ezt írta (időpont:
> 2021. márc. 25., Cs, 16:38):
>
>> Thanks for your reply, János !
>>
>>
>> If you can explain me please what does that mean?
>>
>>
>> >network(transport(tcp|udp))
>> >or
>> >syslog() or network(transport(tcp|udp) flags(syslog-protocol))
>>
>>
>>
>>
>> ======================================================================
>>
>>
>> my config:
>>
>>
>>
>> @version: 3.18
>> @include "scl.conf"
>>
>> source s_local {
>> internal();
>> };
>>
>> source s_network {
>> default-network-drivers(
>> # NOTE: TLS support
>> #
>> # the default-network-drivers() source driver opens the
>> TLS
>> # enabled ports as well, however without an actual
>> key/cert
>> # pair they will not operate and syslog-ng would display a
>> # warning at startup.
>> #
>> #tls(key-file("/path/to/ssl-private-key")
>> cert-file("/path/to/ssl-cert"))
>> );
>> };
>>
>> destination d_local {
>> # file("/var/log/messages");
>> file("/var/log/messages-kv_${YEAR}-${MONTH}-${DAY}.log"
>> template("$ISODATE $HOST $(format-welf --scope all-nv-pairs)\n")
>> frac-digits(3));
>> file("/var/log/messages_${HOST}.log"
>> perm(0644)
>> );
>> };
>>
>> destination d_logstore {
>> file(
>> "/var/log/remote/${HOST}/${HOST}_${YEAR}-${MONTH}-${DAY}.log"
>> create-dirs(yes)
>> );
>> };
>>
>> log {
>> source(s_local);
>> source(s_network);
>> destination(d_local);
>> destination(d_logstore);
>> # destination(d_sorted);
>> };
>>
>>
>>
>> best regards,
>> *Nepryahin Ivan*
>> IT Department
>> *Phone**: *+7 812 327 32 33
>> *Mobile: *+7 911 291 81 68
>>
>> ------------------------------
>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>> SZIGETVÁRI János <jszigetvari at gmail.com>
>> *Sent:* Thursday, March 25, 2021 6:24:09 PM
>> *To:* Syslog-ng users' and developers' mailing list
>> *Subject:* Re: [syslog-ng] syslog-ng has add extra field
>>
>> Hello Ivan,
>>
>> Most commonly there may be two main formats of logs that you may
>> encounter.
>> One is the traditional BSD-style syslog, described in RFC 3164:
>> https://tools.ietf.org/html/rfc3164
>> The other is the IETF-style log format, described in RFC 5424:
>> https://tools.ietf.org/html/rfc5424
>>
>> In case of syslog-ng you would have to either use
>> network(transport(tcp|udp))
>> or
>> syslog() or network(transport(tcp|udp) flags(syslog-protocol))
>> respectively.
>>
>> The sample logs you included seem to resemble the IETF-style.
>> What type of source do you have configured in your syslog-ng setup?
>> (Could you please share your config file?)
>>
>> Best Regards,
>> János
>> --
>> Janos SZIGETVARI
>> RHCE, License no. 150-053-692
>> <https://www.redhat.com/rhtapps/verify/?certId=150-053-692>
>>
>> LinkedIn: linkedin.com/in/janosszigetvari
>> E-mail: janos at szigetvari.com, jszigetvari at gmail.com
>> Web: janos.szigetvari.com
>>
>> __ at __˚V˚
>> Make the switch to open (source) applications, protocols, formats now:
>> - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice
>> - msn -> jabber protocol (Pidgin, Google Talk)
>> - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp
>>
>>
>> Ivan Nepryahin - Bercut <Ivan.Nepryahin at bercut.com> ezt írta (időpont:
>> 2021. márc. 25., Cs, 14:56):
>>
>>> Hi all!
>>>
>>>
>>>
>>> I think I have a stupid question, but I really dont know how this make.
>>>
>>> Situation:
>>> When I send syslog message with timestamp in format "1Mar 25 2021
>>> 16:35:49" everything works great, but when I send message with timestamp
>>> in format "1Mar 25 2021 16:35:49*+03:00*", syslog-ng adding two extra
>>> fields with timestamp and IP address and due that break down file naming.
>>>
>>> Question:
>>> How can I say to syslog-ng server do not add extra fields when he get
>>> message with +03:00 in timestamp?
>>>
>>> message without +03:00
>>> Mar 25 13:11:57 HUAWEI-CORE-OFFICE-1 <bla bla bal>
>>>
>>> mesage with +03:00
>>> Mar 25 13:46:45 192.168.100.34 Mar 25 2021 16:46:45*+03:00*
>>> HUAWEI-CORE-OFFICE-1 <bla bla bla>
>>>
>>>
>>>
>>> I will be appreciate for any advice!
>>>
>>>
>>>
>>> P.s sorry for bad english it is not my native language
>>>
>>>
>>>
>>> best regards,
>>> *Nepryahin Ivan*
>>> IT Department
>>> *Phone**: *+7 812 327 32 33
>>> *Mobile: *+7 911 291 81 68
>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20210326/db1171ae/attachment-0001.html>
More information about the syslog-ng
mailing list