<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style>.EmailQuote {
BORDER-LEFT: #800000 2px solid; PADDING-LEFT: 4pt; MARGIN-LEFT: 1pt
}
</style><style title="owaParaStyle"><!--P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
--></style>
</head>
<body ocsi="x">
<div dir="ltr"><font color="#000000" size="2" face="Tahoma">Since we are part of a govt project, we aren't allowed to compile code on the RHEL platforms as this will break our STIG/DISA requirements so we can only use RPM builds. (Yea, I know it sucks.).
</font></div>
<div dir="ltr"><font color="#000000" size="2" face="Tahoma"></font> </div>
<div dir="ltr"><font color="#000000" size="2" face="Tahoma">As per the syslog-ng -V, it seems that PCRE is disabled in the RPM build so that option won't be working.</font></div>
<div dir="ltr"><font size="2" face="tahoma"></font> </div>
<div dir="ltr">[root@syslog-svr sbin]# ./syslog-ng -V<br>
syslog-ng 3.0.8<br>
Installer-Version: 3.0.8<br>
Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#c88009fc97b5014e330ecc8ead747691a05b7e97<br>
Compile-Date: Jul 9 2010 08:32:54<br>
Enable-Threads: on<br>
Enable-Debug: off<br>
Enable-GProf: off<br>
Enable-Memtrace: off<br>
Enable-Sun-STREAMS: off<br>
Enable-Sun-Door: off<br>
Enable-IPv6: on<br>
Enable-Spoof-Source: on<br>
Enable-TCP-Wrapper: off<br>
Enable-SSL: on<br>
Enable-SQL: on<br>
Enable-Linux-Caps: on<br>
Enable-Pcre: off<br>
</div>
<div dir="ltr"> </div>
<div dir="ltr"><font size="2" face="tahoma"></font> </div>
<div style="DIRECTION: ltr" id="divRpF83230">
<hr tabindex="-1">
<font color="#000000" size="2" face="Tahoma"><b>From:</b> syslog-ng-bounces@lists.balabit.hu [syslog-ng-bounces@lists.balabit.hu] On Behalf Of Alan McKinnon [Alan.McKinnon@is.co.za]<br>
<b>Sent:</b> Tuesday, September 21, 2010 5:22 PM<br>
<b>To:</b> syslog-ng@lists.balabit.hu<br>
<b>Subject:</b> Re: [syslog-ng] [Bug 93] New: filter() functionality between 2.1 to 3.0 not consistent<br>
</font><br>
</div>
<div></div>
<font size="2">
<div class="PlainText">You seem to have simple logic errors, I do not think regexes are your problem.
<br>
However it will be good to make sure that you configure pcre support when you <br>
built syslog-ng<br>
<br>
Here's the relevant parts of your config.<br>
<br>
filter M_audit { not message("Audit daemon rotating log files" flags(ignore-<br>
case)); };<br>
filter M_repeat { not message("last message repeated" flags(ignore-case)); };<br>
filter M_stats { not message("Log statistics" flags(ignore-case)); };<br>
<br>
log { source(s_general);<br>
filter(M_audit);<br>
filter(M_repeat);<br>
filter(M_stats);<br>
destination(d_general);<br>
};<br>
<br>
<br>
<br>
First, filters are ANDed. So it's a case of a log is sent to d_general only if <br>
M_audit is true AND M_repeat is true AND M_stats is true. It is not OR!<br>
<br>
Secondly, M_repeat is false if the string "last message repeated" IS in the <br>
log. It does not say that M_repeat is true if the string is not in the log. <br>
Those things are very different - some simply boolean algebra will show it.<br>
<br>
Moving onto your log statement, all 3 filters must be true for the message to <br>
be logged to the destination. This evaluates to none of the three strings must <br>
be present. When you run "logger last message repeated", the message() <br>
evaluates to true, the not makes it false and the log does not match.<br>
<br>
These booleans get tricky as you do not have the entire spectrum of boolean <br>
operations available in the syslog-ng syntax. In combinations, it gets even <br>
worse and quickly becomes unmaintainable. And negative matches are even worse <br>
than that. I find a much better approach is to make individual tests on <br>
messages, discard the ones you are not interested in and then write positive <br>
match rules to log what is left (all of those are then by definition logs you <br>
want to keep). Try this approach instead:<br>
<br>
<br>
filter M_audit { message("Audit daemon rotating log files" flags(ignore-<br>
case)); };<br>
filter M_repeat { message("last message repeated" flags(ignore-case)); };<br>
filter M_stats { message("Log statistics" flags(ignore-case)); };<br>
<br>
log { source(s_general); filter(M_audit); flags(final); };<br>
log { source(s_general); filter(M_repeat); flags(final); };<br>
log { source(s_general); filter(M_stats); flags(final); };<br>
<br>
Add here filter and log statement for messages you wish to keep.<br>
<br>
<br>
On Tuesday 21 September 2010 22:38:59 Worsham, Michael wrote:<br>
> One of my co-workers lent a pair of eyes found something rather unique<br>
> between the two versions I hadn't looked at. Right now, the v3.0.8 build,<br>
> we are using has the following filter configuration modified from the v2.1<br>
> build:<br>
> <br>
> filter M_audit { not message("Audit daemon rotating log files"); };<br>
> filter M_repeat { not message("last message repeated"); };<br>
> filter M_stats { not message("Log statistics"); };<br>
> Now, if I were to login to the syslog-ng client server and do the following<br>
> 'logger last message repeated', the entire message is dropped from<br>
> actually showing up on the remote syslog-ng server (which it should).<br>
> However, if I do 'logger repeated' (a portion cut from the actual full<br>
> message that is to be filtered), then the remote syslog-ng server will<br>
> record it.<br>
> <br>
> The filters, in the way they are configured, are NOT allowing for portions<br>
> of the messages to be detected -- it's either the entire message is<br>
> matched or none of it, not pieces. What do I need to do to allow the<br>
> pieces to be detected and filtered out just like the full message?<br>
> <br>
> -- M<br>
> <br>
> ________________________________<br>
> From: syslog-ng-bounces@lists.balabit.hu<br>
> [syslog-ng-bounces@lists.balabit.hu] On Behalf Of Worsham, Michael Sent:<br>
> Tuesday, September 21, 2010 10:48 AM<br>
> To: Syslog-ng users' and developers' mailing list<br>
> Subject: Re: [syslog-ng] [Bug 93] New: filter() functionality between 2.1<br>
> to 3.0 not consistent<br>
> <br>
> This is what my current configuration looks like now (disabled TLS to allow<br>
> for debugging):<br>
> <br>
> <a href="http://www.murpe.com/syslog-ng-v3.conf2.txt" target="_blank">http://www.murpe.com/syslog-ng-v3.conf2.txt</a><br>
> <br>
> -- M<br>
> <br>
> ________________________________<br>
> From: syslog-ng-bounces@lists.balabit.hu<br>
> [syslog-ng-bounces@lists.balabit.hu] On Behalf Of Alan McKinnon<br>
> [Alan.McKinnon@is.co.za] Sent: Tuesday, September 21, 2010 10:35 AM<br>
> To: syslog-ng@lists.balabit.hu<br>
> Subject: Re: [syslog-ng] [Bug 93] New: filter() functionality between 2.1<br>
> to 3.0 not consistent<br>
> <br>
> I think you are running into the changed definitions of match() between 2.0<br>
> and 3.0<br>
> <br>
> <a href="http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-gui" target="_blank">
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-gui</a><br>
> de- admin-en.html/reference_filters.html<br>
> <br>
> especially match() and message()<br>
> <br>
> Of your samples, the 3rd and 4th should work. What is the full config that<br>
> applies to how you use this filter? Are you perhaps applying two filters?<br>
> Be careful with that - filters are always ANDed in a log statement, not<br>
> ORed.<br>
> <br>
> On Tuesday 21 September 2010 16:16:57 bugzilla@bugzilla.balabit.com wrote:<br>
> > <a href="https://bugzilla.balabit.com/show_bug.cgi?id=93" target="_blank">https://bugzilla.balabit.com/show_bug.cgi?id=93</a><br>
> > <br>
> > Summary: filter() functionality between 2.1 to 3.0 not<br>
> > <br>
> > consistent Product: syslog-ng<br>
> > <br>
> > Version: 3.0.x<br>
> > <br>
> > Platform: PC<br>
> > <br>
> > OS/Version: Linux<br>
> > <br>
> > Status: NEW<br>
> > <br>
> > Severity: major<br>
> > Priority: unspecified<br>
> > <br>
> > Component: syslog-ng<br>
> > <br>
> > AssignedTo: bazsi@balabit.hu<br>
> > ReportedBy: mworsham@scires.com<br>
> > <br>
> > Type of the Report: bug<br>
> > <br>
> > Estimated Hours: 0.0<br>
> > <br>
> > In the older syslong-ng v2.1, this line works perfectly:<br>
> > <br>
> > filter M_audit { not match("Audit daemon rotating log files"); };<br>
> > <br>
> > Under 3.0.8, none of the following are working (if added one line at a<br>
> > time) and the daemon restarted:<br>
> > <br>
> > filter M_audit { not match("Audit daemon rotating log files"<br>
> > value("MSGONLY") flags(ignore-case)); }; filter M_audit { not<br>
> > match("MSGONLY" value("Audit daemon rotating log files")<br>
> > flags(ignore-case)); }; filter M_audit { not match("Audit daemon<br>
> > rotating log files" value(MSGONLY) flags(ignore-case)); }; filter<br>
> > M_audit { not match("Audit daemon rotating log files" value(MSGONLY));<br>
> > }; filter M_audit<br>
> > <br>
> > { not match("MSGONLY" value("Audit daemon rotating log files")); };<br>
> > <br>
> > What I am looking to do is if any incoming data has the following<br>
> > keywords (i.e. "Audit daemon rotating log files") being detected, it<br>
> > should be filtered (i.e. dropped), and then not show up in the actual<br>
> > message log file.<br>
> > <br>
> > For example, if I go over on the syslog-ng client and do 'logger daemon':<br>
> > - The older v2.1 syslog-ng server detects the embedded keyword and<br>
> > <br>
> > drops the message. <-- This is correct - On the v3.0.8, the message<br>
> > passes through and found in the data file. <-- This is incorrect<br>
> > <br>
> > Attempted to use the following URLs to correct the formatting, but it<br>
> > still not working: -<br>
> > <a href="http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-g" target="_blank">
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-g</a><br>
> > u ide-admin-en.html/configuring_filters.html -<br>
> > <a href="http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-g" target="_blank">
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-g</a><br>
> > u ide-admin-en.html/reference_macros.html<br>
> <br>
> --<br>
> Alan McKinnon<br>
> Systems Engineer^W Technician<br>
> Infrastructure Services<br>
> Internet Solutions<br>
> <br>
> +27 11 575 7585<br>
> <br>
> Please note: This email and its content are subject to the disclaimer as<br>
> displayed at the following link<br>
> <a href="http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm" target="_blank">
http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm</a><br>
> . Should you not have Web access, send a mail to disclaimers@is.co.za and a<br>
> copy will be emailed to you.<br>
> __________________________________________________________________________<br>
> ____ Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
> Documentation:<br>
> <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">
http://www.balabit.com/support/documentation/?product=syslog-ng</a> FAQ:<br>
> <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
> <br>
> <br>
> ________________________________<br>
> CONFIDENTIALITY NOTICE: This email and any attachments are intended solely<br>
> for the use of the named recipient(s). This email may contain confidential<br>
> and/or proprietary information of Scientific Research Corporation. If you<br>
> are not a named recipient, you are prohibited from reviewing, copying,<br>
> using, disclosing or distributing to others the information in this email<br>
> and attachments. If you believe you have received this email in error,<br>
> please notify the sender immediately and permanently delete the email, any<br>
> attachments, and all copies thereof from any drives or storage media and<br>
> destroy any printouts of the email or attachments.<br>
> <br>
> EXPORT COMPLIANCE NOTICE: This email and any attachments may contain<br>
> technical data subject to U.S export restrictions under the International<br>
> Traffic in Arms Regulations (ITAR) or the Export Administration<br>
> Regulations (EAR). Export or transfer of this technical data and/or<br>
> related information to any foreign person(s) or entity(ies), either within<br>
> the U.S. or outside of the U.S., may require advance export authorization<br>
> by the appropriate U.S. Government agency prior to export or transfer. In<br>
> addition, technical data may not be exported or transferred to certain<br>
> countries or specified designated nationals identified by U.S. embargo<br>
> controls without prior export authorization. By accepting this email and<br>
> any attachments, all recipients confirm that they understand and will<br>
> comply with all applicable ITAR, EAR and embargo compliance requirements.<br>
> <br>
> ________________________________<br>
> CONFIDENTIALITY NOTICE: This email and any attachments are intended solely<br>
> for the use of the named recipient(s). This email may contain confidential<br>
> and/or proprietary information of Scientific Research Corporation. If you<br>
> are not a named recipient, you are prohibited from reviewing, copying,<br>
> using, disclosing or distributing to others the information in this email<br>
> and attachments. If you believe you have received this email in error,<br>
> please notify the sender immediately and permanently delete the email, any<br>
> attachments, and all copies thereof from any drives or storage media and<br>
> destroy any printouts of the email or attachments.<br>
> <br>
> EXPORT COMPLIANCE NOTICE: This email and any attachments may contain<br>
> technical data subject to U.S export restrictions under the International<br>
> Traffic in Arms Regulations (ITAR) or the Export Administration<br>
> Regulations (EAR). Export or transfer of this technical data and/or<br>
> related information to any foreign person(s) or entity(ies), either within<br>
> the U.S. or outside of the U.S., may require advance export authorization<br>
> by the appropriate U.S. Government agency prior to export or transfer. In<br>
> addition, technical data may not be exported or transferred to certain<br>
> countries or specified designated nationals identified by U.S. embargo<br>
> controls without prior export authorization. By accepting this email and<br>
> any attachments, all recipients confirm that they understand and will<br>
> comply with all applicable ITAR, EAR and embargo compliance requirements.<br>
<br>
-- <br>
Alan McKinnon<br>
Systems Engineer^W Technician<br>
Infrastructure Services<br>
Internet Solutions<br>
<br>
+27 11 575 7585<br>
<br>
Please note: This email and its content are subject to the disclaimer as displayed at the following link
<a href="http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm" target="_blank">
http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm</a>. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you.<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">
http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
</div>
</font><br>
<hr>
<font face="Arial" color="Gray" size="1">CONFIDENTIALITY NOTICE: This email and any attachments are intended solely for the use of the named recipient(s). This email may contain confidential and/or proprietary information of Scientific Research Corporation.
If you are not a named recipient, you are prohibited from reviewing, copying, using, disclosing or distributing to others the information in this email and attachments. If you believe you have received this email in error, please notify the sender immediately
and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts of the email or attachments.<br>
<br>
EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical
data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data
may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will
comply with all applicable ITAR, EAR and embargo compliance requirements.<br>
</font>
</body>
</html>