<div dir="ltr">i am not sure if you mean about transaction with batch messages but i saw that the older syslog-ng doest support it and that there was a fix so it will support. I added the fix to the libdbi_drivers SRPM and created a new RPM. the transaction work now. i can see the change. before my fix it couldnt handle more them 100 messages per second. now it can handle more then 700 per second.<div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 17, 2017 at 3:59 AM, Balazs Scheidler <span dir="ltr"><<a href="mailto:bazsi77@gmail.com" target="_blank">bazsi77@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The problem is that older syslog-ng versions didn't have transaction support, so even if you set flush-lines() higher, it may not be starting transactions.<br></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Tue, Feb 14, 2017 at 11:57 PM, work ran <span dir="ltr"><<a href="mailto:work.greeran@gmail.com" target="_blank">work.greeran@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">thanks for the response<div>For now our architecture is limiting us to an old syslog-ng version so we cannot upgrade. </div><div>as for the flush we are running it with 200 and with timeout 200 also. we are trying to increase the flush-lines to 1000 and see if it helps.</div><div>i also googled the problem and saw a couple of mails that said that there was a bug with syslog-ng <span style="font-size:12.8px">3.2.5/4 </span>and pcre that created a bug with the cpu usage. does someone if it might be the problem and how to fix it.</div><div><br></div><div>thanks</div><span class="m_4694173726941204226HOEnZb"><font color="#888888"><div>Ran</div><div> </div></font></span></div><div class="m_4694173726941204226HOEnZb"><div class="m_4694173726941204226h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 14, 2017 at 5:53 PM, Scheidler, Balázs <span dir="ltr"><<a href="mailto:balazs.scheidler@balabit.com" target="_blank">balazs.scheidler@balabit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">It might be because of transaction size. Sqlite is very inefficient if the transaction size is small.<div dir="auto"><br></div><div dir="auto">Newer syslog-ng versions support batching, set flush-lines to something like 100 and it would improve a lot.</div></div><div class="m_4694173726941204226m_-4073757383783350458HOEnZb"><div class="m_4694173726941204226m_-4073757383783350458h5"><div class="gmail_extra"><br><div class="gmail_quote">On Feb 14, 2017 06:31, "Fekete, Róbert" <<a href="mailto:robert.fekete@balabit.com" target="_blank">robert.fekete@balabit.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi, <div><br></div><div>It is not normal, it might be a bug. However, version 3.2 is ancient, I'd recommend upgrading to a newer version. You can find release tarballs on github, or links to binaries here: <a href="https://syslog-ng.org/3rd-party-binaries/" target="_blank">https://syslog-ng.org/3r<wbr>d-party-binaries/</a></div><div><br></div><div>Regards, </div><div><br></div><div>Robert</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 14, 2017 at 3:19 PM, work ran <span dir="ltr"><<a href="mailto:work.greeran@gmail.com" target="_blank">work.greeran@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">hi<div>i am running syslog-ng with sqlite</div><div>i am sending 50 logs per second and i am getting 80% cpu usage.</div><div>could someone tell me how i can fix this or is it normal</div><div>thanks</div><div>Ran</div><div><br></div><div><br></div><div><br></div><div><div>syslog-ng 3.2.5</div><div>Installer-Version: 3.2.5</div><div>Revision: ssh+git://bazsi@git.balabit//v<wbr>ar/scm/git/syslog-ng/syslog-ng<wbr>-ose--mainline--3.2#master#9d4<wbr>bea28198bd731df1a61e980a2af5b8<wbr>8d81116</div><div>Compile-Date: Jul 25 2014 15:20:50</div><div>Enable-Threads: on</div><div>Enable-Debug: off</div><div>Enable-GProf: off</div><div>Enable-Memtrace: off</div><div>Enable-Sun-STREAMS: off</div><div>Enable-IPv6: on</div><div>Enable-Spoof-Source: on</div><div>Enable-TCP-Wrapper: on</div><div>Enable-SSL: off</div><div>Enable-SQL: on</div><div>Enable-Linux-Caps: off</div><div>Enable-Pcre: on</div><div>Enable-Pacct: off</div></div><div><br></div><div><br></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br>
<br>
<br></blockquote></div></div>
</div></div><br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>
</div></div><br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=<wbr>syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br>
<br>
<br></blockquote></div><br><br clear="all"><br></div></div><span class="HOEnZb"><font color="#888888">-- <br><div class="m_4694173726941204226gmail_signature" data-smartmail="gmail_signature">Bazsi</div>
</font></span></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>