[syslog-ng] Cisco ACS logs truncated

[Evan Rempel]

Before I put this out into the public I thought I would familiarize 
myself with it again, and I think there is a mistake that never seems to 
get invoked anyways. Perhaps Balázs could review.

Attached is the correlate.xml patterndb for correlating the messages.

I think the mistake is the the two "action condition" clauses where the 
MESSAGE
is set to "${csologline}@1". I think this should be "${csologline}". It 
is working in my environment,
so perhaps the clause "$(context-length)" == "$max" is never true and it 
is the timeout that always gets triggered.

The other consideration is that if these unwrapped log messages are sent 
on to an analysis framework that understands the native format of these 
messages, the patterndb should maintain the message number, max and 
counters. To do that the rule IDs *_0 should set csologline with

<value name="csologline">CSCOacs_Failed_Attempts $msgnum 1 0 $line</value>
<value name="csologline">CSCOacs_Passed_Authentications $msgnum 1 0 
$line</value>

I can not do that in my environment because I have already rewritten 
these log lines to give them a program name.

Here are the details.

1. I use the rewrite.xml pattern database to detect the acs log lines 
and give them a program name.

template t_rewrite { template("$MSGHDR$MESSAGE"); template_escape(no); };
parser p_rewrite {
         db_parser(
                 file("rewrite.xml")
                 inject_mode(internal)
                 template(t_rewrite)
         );
};

2. Then I use the correlate.xml pattern database to unwrap the log 
messages. This only works because I have already added the program name.

parser p_correlate {
         db_parser(
                 file("correlate.xml")
                 inject_mode(pass-through)
         );
};


The final bit of configuration to get this all working.

filter f_correlate_drop {
         tags("CORRELATE_DROP") and not tags("CORRELATE_CONTINUE");
};


log {
         source(s_network);
         parser(p_rewrite);
         parser(p_correlate);
         # if the p_correlate parser is unwrapping lines, then the line 
snippits need to be dropped
         log {
                filter(f_correlate_drop);
                flags(final);
         };
... whatever else you need for logging
};


I hope this makes it into some official docs, blog or repository for 
everyone to use.

Evan


On 11/15/2017 09:01 AM, Scheidler, Balázs wrote:
> Yup, I might even add this use case to my latedt application parsers 
> framewrok.
>
> On Nov 15, 2017 17:57, "Kókai Péter" <peter.kokai at balabit.com 
> <mailto:peter.kokai at balabit.com>> wrote:
>
>     Hello,
>
>     It would be really useful if you could share it (Y).
>
>     Kokan
>
>     On Wed, Nov 15, 2017 at 5:18 PM Evan Rempel <erempel at uvic.ca
>     <mailto:erempel at uvic.ca>> wrote:
>
>         Answered out of band because the details are messy.
>         If there is sufficient interest I can clean it up and post it
>         to the list.
>
>
>         Evan.
>
>
>         On 11/15/2017 04:26 AM, Scot wrote:
>>         Thanks Evan,
>>         Didn't see much in term of cisco documentation of the
>>         format.  Is that 1st number in the message header unique to
>>         each message and do you share patterns ?
>>
>>         Scot
>>
>>         On Tue, Nov 14, 2017 at 8:36 PM, Evan Rempel <erempel at uvic.ca
>>         <mailto:erempel at uvic.ca>> wrote:
>>
>>             At our side we used a patterndb to unwrap the ACS logs
>>             into single long line messages. These long lines seem to
>>             be wrapped at the source (Cisco device) before sending to
>>             the syslog server.
>>
>>             Evan.
>>
>>
>>             On 11/14/2017 02:03 PM, Scot wrote:
>>
>>                 Hi,
>>
>>                  Has anyone worked with ACS logs and solved the
>>                 message header limit ?
>>                 We can get syslog working but as expected the message
>>                 gets truncated.
>>
>>                 Local logs on the ACS have the entire payload.
>>
>>                 Thinking there may be a way to script a log fetch or
>>                 something.
>>
>>                 Thanks
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171115/33d88917/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: correlate.xml
Type: text/xml
Size: 3609 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171115/33d88917/attachment.xml>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rewrite.xml
Type: text/xml
Size: 1572 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171115/33d88917/attachment-0001.xml>