<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body>
<div> I think it would depend on the filter. If it uses something that is readily available then maybe filter first. If is a match within the message, then possibly not. </div><div><br></div><div>The patterndb parser is very fast. I have a single system pulling 24 fields out of a 7000 event per second load and it shows no sign of stress. </div><div><br></div><div>Jim</div><div><br></div><div><br></div><div><br></div><div id="composer_signature"><div style="font-size:85%;color:#575757">Sent from my Verizon Wireless 4G LTE smartphone</div></div><br><br>-------- Original message --------<br>From: Mikkel Leth Carlsen <mlca@tdc.dk> <br>Date: 04/10/2015 2:11 AM (GMT-05:00) <br>To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> <br>Subject: Re: [syslog-ng] db-parser reuse for multiple logs? <br><br>Hi Evan<br><br>Thanks! I'm not too familiar with how syslog-ng actually implements the configuration and processes syslog messages, but I would think that parsing before filtering could have an impact on performance? I.e. I would want to exclude unwanted data before applying the parser - which I assume is a more costly operation even if it does not match? Perhaps by inserting:<br><br>filter(filter_host1_or_host2);<br><br>before the parser below? <br><br>/Mikkel<br><br>> -----Oprindelig meddelelse-----<br>> Fra: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-<br>> bounces@lists.balabit.hu] På vegne af Evan Rempel<br>> Sendt: 8. april 2015 17:38<br>> Til: syslog-ng@lists.balabit.hu<br>> Emne: Re: [syslog-ng] db-parser reuse for multiple logs?<br>> <br>> I don't have an answer to your actual question, however, you can work<br>> around it by<br>> <br>> log {<br>> source(src_udp);<br>> parser(myparser);<br>> log {<br>> filter(filter_host1);<br>> destination(dst_host1);<br>> flags(final);<br>> };<br>> log {<br>> filter(filter_host2);<br>> destination(dst_host2);<br>> flags(final);<br>> };<br>> };<br>> <br>> Hope that helps.<br>> <br>> Evan.<br>> <br>> On 04/08/2015 04:31 AM, Mikkel Leth Carlsen wrote:<br>> > Hi<br>> ><br>> > Are db-parsers defined in syslog-ng configurations not reusable for<br>> multiple logs? A simplified example (syslog 3.6.2):<br>> ><br>> > parser myparser {<br>> > db_parser(<br>> > file("/usr/local/etc/patterndb.d/myparser.xml")<br>> > );<br>> > };<br>> ><br>> > template mytemplate {<br>> > template("${A};${B};${C}\n");<br>> > }<br>> ><br>> > filter filter_host1 {<br>> > netmask(10.0.0.1/255.255.255.255);<br>> > };<br>> ><br>> > filter filter_host2 {<br>> > netmask(10.0.0.2/255.255.255.255);<br>> > };<br>> ><br>> > destination dst_host1 {<br>> > file("host1.log" perm(0644) template(mytemplate));<br>> > };<br>> ><br>> ><br>> > destination dst_host2 {<br>> > file("host2.log" perm(0644) template(mytemplate));<br>> > };<br>> ><br>> > log {<br>> > source(src_udp);<br>> > filter(filter_host1);<br>> > parser(myparser);<br>> > destination(dst_host1);<br>> > flags(final);<br>> > };<br>> ><br>> > log {<br>> > source(src_udp);<br>> > filter(filter_host2);<br>> > parser(myparser);<br>> > destination(dst_host2);<br>> > flags(final);<br>> > };<br>> ><br>> > This seems to work as expected and 'syslog-ng -s' does not report any<br>> problems, but I see the following in the syslog-ng internal log:<br>> ><br>> > Internal error, duplicate configuration elements refer to the same<br>> persistent config; name='db-<br>> parser(/usr/local/etc/patterndb.d/myparser.xml)'<br>> > Internal error, duplicate configuration elements refer to the same<br>> persistent config; name='db-<br>> parser(/usr/local/etc/patterndb.d/myparser.xml)'<br>> ><br>> > /Mikkel<br>> ><br>> _______________________________________________________________________<br>> _______<br>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<br>> > Documentation:<br>> http://www.balabit.com/support/documentation/?product=syslog-ng<br>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq<br>> ><br>> _______________________________________________________________________<br>> _______<br>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<br>> Documentation:<br>> http://www.balabit.com/support/documentation/?product=syslog-ng<br>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq<br><br>______________________________________________________________________________<br>Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<br>Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<br>FAQ: http://www.balabit.com/wiki/syslog-ng-faq<br><br></body></html>