Why does syslog-ng do a PTR lookup for EVERY message? It uses them for $HOST stuff? But why not cache these and avoid the lookup? At first I thought, the caching belongs in the libc, possibly in nscd. But despite nscd running (this is Linux with glibc-2.1), gethostbyaddr() still does a full name server PTR lookup for every single message. I started running a slave name server on the localhost just to avoid the bandwidth hit doing all these queries :) But now there is a bunch of extra CPU usage (syslog-ng has accumulated 600 minutes of CPU time after running for just one day; this is a 333Mhz machine and there are only a couple dozen logging hosts, most of which comes relayed through other syslog-ngs anyways). If I were to set use_dns to `no' this would probably make the problem go away. Is there any reason not to do this? Would I then get IPs for $HOST instead of names? Incidentally, the assert fix definitely got rid of the crashing, and the string free definitely fixed some memory leakage. But I have only been running it for one day and it's up to 6 meg VSZ so far (would have been 30 by now before the fix) ...hopefully this is just some data structures which will plateau out once it's been running for some time (xinetd does this in a big way ;)
Hi,
Why does syslog-ng do a PTR lookup for EVERY message?
Just as you describe below, I think caching DNS results belongs to libc. syslog-ng rewrites hostnames by default, and for this it needs a DNS lookup. You have several options: 1) disable hostname rewriting by using the keep_hostname(yes) option 2) disable dns usage by use_dns(no) 3) do not accept remote messages :) Installing nscd and a caching bind seems to be a good solution.
If I were to set use_dns to `no' this would probably make the problem go away. Is there any reason not to do this? Would I then get IPs for $HOST instead of names?
yes
Incidentally, the assert fix definitely got rid of the crashing, and the string free definitely fixed some memory leakage. But I have only been running it for one day and it's up to 6 meg VSZ so far (would have been 30 by now before the fix) ...hopefully this is just some data structures which will plateau out once it's been running for some time (xinetd does this in a big way ;)
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
"Balazs" == Balazs Scheidler <bazsi@balabit.hu> writes: Balazs> Installing nscd and a caching bind seems to be a good Balazs> solution. I do not recommend nscd. A caching bind is one possibilty. Another solution I'd like to recommend is djbdns. See http://cr.yp.to/djbdns.html In particular, I'd recommend installing dnscache (from that package) on the same machine as syslog-ng. I think you'll find this to work better than using bind. j.
participants (3)
-
Balazs Scheidler
-
Jay Soffian
-
Scott McDermott