[syslog-ng] Message loss (probably) within syslog-ng

[Vincent Haverlant]

Le Mon Mar  6 08:07:51 2006, Mike a écrit:
| 
| >The default receive buffer size is set using the parameter
| >udp_recv_hiwat which is currently set to its maximum value of 65536. To
| >increase it the only way is the setsockopt syscall.
| >
| >Anyway I don't get that many udpInOverflows (a few everyday) but it does
| >not explain the few thousand messages I miss in my logs files.
| >
| >Regards,
| >Vincent.
| 
| 
| hmmm. strange..on Linux it seems to use that max value without modifying applications. I could be wrong tho.
| 

I think it is the same issue on linux: (extract from man 7 socket)
   SO_RCVBUF
       Sets  or gets the maximum socket receive buffer in bytes.
       The default value is set by the rmem_default sysctl
       and the maximum allowed value is set by the rmem_max
       sysctl.

The buffer size is controled by two parameters: rmem_default and
rmem_max. The difference with Solaris is that these two parameters are
global to all ip protocols except set otherwise for tcp with tcp_rmem.

| options {
| .
| .
| .
|  use_dns (yes);
|  dns_cache (yes);
|  dns_cache_size(3000);
|  use_fqdn (no); # utilisation du nom court de la machine
| .
| .
| .
| .
| };
| 
| hmm..I am kinda wondering about the DNS usage now tho. I have never used this feature before, but from what the docs say, syslog-ng will block on DNS 
| queries...can you tell if any of DNS queries are failing?

I saw that too, unfortunately even after turning it off, I still miss
between 5 to 15% messages in my test, which I will admit is a burst test
but bursts can happen when you have 2500 hosts.

| maybe you could add in some options here:
|  dns_cache_expire(n)
|            Number of seconds while a successful lookup is cached.
| 
| 
|       dns_cache_expire_failed(n)
|            Number of seconds while a failed lookup is cached.
| 
| but if syslog-ng blocks on DNS queries, I would imagine that you would see your udpInOverflows value increase....
| 
| anyone know the default value of dns_cache_expire() off the top of your heads?

dns_cache_expire -> 3600
dns_cache_expire_failed -> 60

I'm quite at loss as to what to do now...
Vincent.
-- 
   .~.     Vincent Haverlant  -- Galadril -- #ICQ: 35695155   
   /V\      MSN: vincent_msn at haverlant.org  -- http://www.haverlant.org/
  /( )\      Parinux member: http://www.parinux.org/
  ^^-^^       GPG: 8FEA 52C2 5C54 A201 2375  0FA5 AF2E 1881 92D0 EE84