[syslog-ng]buffer limitations and TCP compression

[Andreas Schulze]

Jason Haar wrote:
> 
> Can someone explain to me how the tcp/udp destination options handle 
> network outages/congestion? If I set my fifo to 10000, does that mean if 
> the remote server is down (I assume this only affects TCP - not UDP 
> BTW), the "client" will hold up to 10000 records before dropping... the 
> oldest? Just what does happen then?

As fas as I know, the messages are dropped silently.

> Also, does anyone think adding a compress option to the tcp destination 
> option is a good idea? It could really cut down on the traffic (esp if 
> you could force a *minimum* fifo queue size - to increase your chance of 
> repeated data). It could make centralized logging over WANs much more 
> attractive.

I think compression isn't the factor for centralized logging
in WAN environments. But maybe its nice to have.
Imagine your normal messages size is approx. <512Bytes.
How many messages you must create/send to flood a 64KB/128KB
leased line?

The important factor in centralized logging is in most cases
the perfomance of the center (the log server).
Compression will decrease log servers performance.

We are logging >5000 devices with >15.000.000 messages per day
to a centralized syslog-ng server over WAN.
Problems we observed are mostly on the central size. The WAN
isn't really the bottleneck in most scenarios.

-- 
Best regards --Andreas Schulze
                [phone: +49.5246.80.1275, fax: +49.5246.80.2275]

| I believe, it was Dennis Ritchie who said something like:
|   "C is rarely the best language for a given task,
|    but it's often the second-best".
| The implication being that: "[...]"
|     http://www.ioccc.org/1990/dds.c