Balazs Scheidler wrote:
On Wed, 2008-03-26 at 09:46 -0700, Evan Rempel wrote:
Another instance of dropped messages, but all statics records from syslog-ng indicate that no messages were dropped.
Here is the logging path
Server C - syslog-ng 2.0.8 - file destination - tcp destination A - tcp destination B
Server A - syslog-ng 2.0.8 Linux 2.6.x - tcp source - pipe destination Server A - syslog-ng 2.0.8 linux 2.6.x - pipe source with flow-control - file destination
Server B - syslog-ng 2.0.8 Linux 2.6.x - tcp source - pipe destination Server B - syslog-ng 2.0.8 linux 2.6.x - pipe source with flow-control - file destination
Server C logged 608955 messages Server A logged 351502 messages Server B logged 179874 messages
None of the servers record any dropped messages. syslog-ng does not report dropped messages on file destinations, but with the flow-control any drops should occur at the server that writes to the pipe.
Anyone want to propose an explanation?
Isn't this the same as the previous one? E.g. the tcp connections could not be established at startup from Server C -> Server A and Server C -> Server B
No, not in this case. All servers were up and connected, and logging some information on the channel, however, many messages never got do disk on the receiving servers.
Otherwise I can't see where it could lose messages, it should at least report it.
I'm glad that you think it should. This was my understanding to. What about dropping at the incoming buffer? Is there something related to log_iw_size that would result in messages begin discarded, but since it is not related to a destination, no drop counter is increased? Are the upper limits to the buffer size numbers such as 64K, 2G, 4G etc. We use very large buffers (20 million) and log_is_size(10 million)
I'm going to add a "global" drop counter, just as in 1.6.x; it might give a clue in these situations.
A long time ago you mentioned that you wanted to implement a different statistics reporting mechanism. Have you made any progress in that area? I would like to see drop counters for file destinations. If a filesystem becomes full, I want to know how many messages were dropped, or none if it gets fixed prior to buffers being exhausted. I need to confirm that no messages were dropped. Evan. -- Evan Rempel erempel@uvic.ca Senior Programmer Analyst 250.721.7691 Computing Services University of Victoria