On Wed, 2011-03-09 at 16:21 +0100, Zoltán Pallagi wrote:
Hi,
If the problem is permanent after using Sandor solution as well, maybe you should use flow-control in server, because it can happen that the client syslog-ng will forward the logs as fast as it can, but the server will drop the incoming lines if it cannot process them in time and all buffers are full.
Use this config:
log { source(s_mysrc); filter(f_filter); flags(flow-control); destination(d_mydest); };
I guess this is important on the client, not on the server, right? ;) Or preferably both.
On 2011-03-09 15:49, Sandor Geller wrote:
Hi,
On Wed, Mar 9, 2011 at 2:50 PM, Hidayath Basha <hidayath.basha@saventech.com> wrote:
Hi all,
I'm trying to transmit a huge log file (of about 80k lines) to a centralized syslog server over TCP 80k lines is piece of cake unless the average line length is quite big :)
yeah, I could see syslog-ng chewing 800k messages/second on my test environment. I only need to get some time to push that out. :( But I guess 3.3alpha2 will come first.
But, on the syslog server, I'm receiver only the last part of the log file (of about 7000 lines)
How can I transmit the whole log file syslog-ng keeps track where it left off reading a file to avoid sending the whole file again when it gets restarted so my guess is that you fired up syslog-ng a few times.
To confirm this could you stop syslog-ng, delete /var/lib/syslog-ng.persist (or where your persist file lives) and start syslog-ng?
If the problem persists then run syslog-ng under strace and show the relevant parts (file opens, seeks, reads).
-- Bazsi