[syslog-ng] Cisco ASA parsing with patterndb/elasticsearch
Evan Rempel
erempel at uvic.ca
Wed Feb 28 01:26:07 UTC 2018
Welcome to the horrible world of Cisco logging :-(
The issue you are bumping into is that when these log lines are parsed
by syslog-ng (or any
syslog daemon that does not specifically understand Cisco logs) there is
no program name, or the
program name is the full %ASA-4-106023 part of the log line. There are
so many ways that Cisco
can log incorrectly it is not possible to say without seeing your exact
log lines.
Cisco can add a * to the date/time stamp to indicate that there is no
time server configured on the device.
That makes the date/time invalid as far as parsing is concerned.
Cisco can add a period (.) to the date/time stamp to indicate that there
is a time server configured on
the device, but the time server can not be reached. Again, this makes
the date/time invalid as far as parsing
is concerned.
Cisco can add a sequence number at the start of the log line rather than
starting the line with a date/6time stamp.
Invalid parsing again.
Cisco can leave out the sequence number but still include the trailing
colon from the sequence number.
You get the idea. So many ways to get it wrong, and they never get it
right :-(
Evan.
On 02/27/2018 04:46 PM, Tim Ghetti wrote:
>
> Hi – having some trouble getting paterndb functional and looking for
> some help. I would like to use patterndb to parse my cisco ass
> firewall logs before sending it to elasticsearch. However when the
> messages get to elasticsearch, I don’t see the messages being parsed.
> Running pdbtool against the logs seems to work.
>
> # pdbtool match -p /etc/syslog-ng/patterndb.d/ciscoasa.pdb -P %ASA -f
> /var/log/asatest.log |more
>
> HOST=X.X.X.X
>
> MESSAGE=Built dynamic TCP translation from INSIDE:X.X.X.X/X to
> OUTSIDE:X.X.X.X/X
>
> PROGRAM=%ASA-6-305011
>
> LEGACY_MSGHDR=%ASA-6-305011:
>
> .classifier.class=system
>
> .classifier.rule_id=e075efdc-c25f-5e49-a208-7661e3b5a29b
>
> Protocol=TCP
>
> GlobalIP=X.X.X.X
>
> GlobalPort=X
>
> LocalIP=X.X.X.X
>
> LocalPort=X
>
> TAGS=.classifier.system
>
> **********************
>
> SYSLOG-NG CONF FILE
>
> @version: 3.11
>
> source s_network { tcp(); udp(); };
>
> destination d_elastic {
>
> elasticsearch2(
>
> client-mode("http")
>
> cluster("ITESCL001")
>
> index("logstash-syslogng_${YEAR}.${MONTH}.${DAY}")
>
> cluster-url("http://X.X.X.X:9200")
>
> type("syslog")
>
> flush-limit("1")
>
> );
>
> };
>
> destination d_catchall { file("/var/log/catchall.log"); };
>
> filter f_ciscoasa { host("X.X.X.X"); };
>
> parser p_ciscoasa
> {db-parser(file("/etc/syslog-ng/patterndb.d/ciscoasa.pdb"));};
>
> log { source(s_network); filter(f_ciscoasa); parser(p_ciscoasa);
> destination(d_elastic); flags(final, flow-control); };
>
> log { source(s_network); destination(d_catchall); };
>
> **********************
>
> PATTERNDB FILE
>
> <?xml version='1.0' encoding='UTF-8'?>
>
> <patterndb version='4' pub_date='2018-02-19'>
>
> <ruleset name='%ASA' id='a300d776-8bd7-834d-a4a9-23eb81a4b3ba'>
>
> <pattern>%ASA</pattern>
>
> <description>
>
> This ruleset covers the Cisco ASA firewalls
>
> </description>
>
> <rules>
>
> <rule provider="%ASA" id="b3de7699-8213-c744-944e-9413298afe86"
> class="system">
>
> <!-- support: 1594 -->
>
> <patterns>
>
> <pattern>Teardown @ESTRING:Protocol: @connection for faddr
> @IPv4:SrcIP:/@@ESTRING:SrcPort: @gaddr
> @IPv4:GlobalIP:/@@ESTRING:GlobalPort: @laddr
> @IPv4:LocalIP:/@@ESTRING:LocalPort:@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Teardown ICMP connection
> for faddr X.X.X.X/X gaddr X.X.X.X/X laddr X.X.X.X/X</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='90d0f8c9-7591-d44e-b886-2f7e5cb17ce6' class='system'
> provider='%ASA'>
>
> <!-- support: 1369 -->
>
> <patterns>
>
> <pattern>Teardown dynamic @ESTRING:Protocol: @translation
> from @ESTRING:::@@IPv4:LocalIP:/@@ESTRING:LocalPort: @to
> @ESTRING:::@@IPv4:GlobalIP:/@@ESTRING:GlobalPort:
> @duration at ANYSTRING::@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Teardown dynamic UDP
> translation from any:X.X.X.X/X to outside:X.X.X.X/X duration
> 0:00:00</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='8f0a8d57-80c6-4745-8a8a-5ce018bb0d87' class='system'
> provider='%ASA'>
>
> <!-- support: 1254 -->
>
> <patterns>
>
> <pattern>Teardown @ESTRING:Protocol: @connection @ESTRING::
> @for @ESTRING:::@@IPv4:DstIP:/@@ESTRING:DstPort: @to
> @ESTRING:::@@IPv4:SrcIP:/@@ESTRING:SrcPort: @@ESTRING::@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Teardown UDP connection
> 55101037 for outside:X.X.X.X/X to inside:X.X.X.X/X duration 0:00:00
> bytes 132</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='00c0732d-1e34-7340-a75f-21198bf71137' class='system'
> provider='%ASA'>
>
> <!-- support: 1256 -->
>
> <patterns>
>
> <pattern>Built outbound @ESTRING:Protocol: @connection
> @ESTRING:: @for @ESTRING:::@@IPv4:DstIP:/@@ESTRING:DstPort:
> @(@ESTRING::)@ to @ESTRING:::@@IPv4:SrcIP:/@@ESTRING:SrcPort:
> @(@ESTRING::)@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Built outbound UDP
> connection 55101037 for outside:X.X.X.X/X (X.X.X.X/X) to
> inside:X.X.X.X/X (X.X.X.X/X)</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='4a586711-ebe2-dc4d-bf6e-e512666d8c5d' class='system'
> provider='%ASA'>
>
> <!-- support: 1594 -->
>
> <patterns>
>
> <pattern>Built inbound @ESTRING:Protocol: @connection for
> faddr @IPv4:SrcIP:/@@ESTRING:SrcPort: @gaddr
> @IPv4:GlobalIP:/@@ESTRING:GlobalPort: @laddr
> @IPv4:LocalIP:/@@ESTRING:LocalPort:@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Built inbound ICMP
> connection for faddr X.X.X.X/X gaddr X.X.X.X/X laddr
> X.X.X.X/X</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='8be7928d-66e7-7042-abd5-869d6b49c56e' class='system'
> provider='%ASA'>
>
> <!-- support: 1763 -->
>
> <patterns>
>
> <pattern>Built inbound @ESTRING:Protocol: @connection
> @ESTRING:: @for @ESTRING::@@IPv4:SrcIP:/@@ESTRING:SrcPort:
> @(@ESTRING::)@ to identity:@IPv4:DstIP:/@@ESTRING:DstPort:
> @(@ESTRING::)@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Built inbound UDP
> connection 55101078 for inside:X.X.X.X/X (X.X.X.X/X) to
> identity:X.X.X.X/X (X.X.X.X/X)</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='20aee256-b4f0-8b4d-93cb-263d5338fd21' class='system'
> provider='%ASA'>
>
> <!-- support: 1539 -->
>
> <patterns>
>
> <pattern>Teardown @ESTRING:Protocol: @connection @ESTRING::
> @for @ESTRING:::@@IPv4:SrcIP:/@@ESTRING:SrcPort: @to
> identity:@IPv4:DstIP:/@@ESTRING:DstPort: @duration at ANYSTRING::@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Teardown UDP connection
> 55101084 for inside:X.X.X.X/X to identity:X.X.X.X/X duration 0:02:01
> bytes 88</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule id='e075efdc-c25f-5e49-a208-7661e3b5a29b' class='system'
> provider='%ASA'>
>
> <!-- support: 3648 -->
>
> <patterns>
>
> <pattern>Built dynamic @ESTRING:Protocol: @translation from
> @ESTRING:::@@IPv4:LocalIP:/@@ESTRING:LocalPort: @to
> @ESTRING:::@@IPv4:GlobalIP:/@@ESTRING:GlobalPort:@</pattern>
>
> </patterns>
>
> <examples>
>
> <example>
>
> <test_message program='%ASA'>Built dynamic TCP
> translation from any:X.X.X.X/X to outside:X.X.X.X/X</test_message>
>
> </example>
>
> </examples>
>
> </rule>
>
> <rule provider='%ASA' class='system' id='39'>
>
> <patterns>
>
> <pattern>Cleared @ESTRING:: @urgent flag from
> @ESTRING:::@@ESTRING::/@@NUMBER::@ to @ESTRING:
> ::@@ESTRING::/@@NUMBER::@</pattern>
>
> <pattern>regular translation creation failed for @ESTRING::
> @src @ESTRING:::@@ESTRING:: @dst @ESTRING: ::@@ESTRING:: @(type
> @NUMBER::@, code @NUMBER::@</pattern>
>
> </patterns>
>
> </rule>
>
> </rules>
>
> </ruleset>
>
> </patterndb>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180227/15ccaa22/attachment-0001.html>
More information about the syslog-ng
mailing list