- syslog-ng - lists.balabit.hu

Feature request: Default template
by Gert Menke 28 Mar '02

28 Mar '02

Hello, Balazs! I need a configuration option for a default template which is used whenever I don't specify one explicitly. If you don't have the time for implementing it in the near future I would very much appreciate any hints on how to do it myself. Thank you! Greetings, Gert

1 0

Re: [syslog-ng]DNS caching
by Ted_Rule＠flextech.co.uk 21 Mar '02

21 Mar '02

A couple of things... > You also mention "syslog-ng blocks on DNS queries, so enabling DNS may lead > to a Denial of Service attack." in your documentation. Does this mean that > syslog messages which are received by the NIC, while syslog-ng performs a > synchronous DNS lookup, are stored in the kernels receive buffer or are > dropped? My belief is that your presumption is indeed correct, and that the kernel will store such inbound UDP messages in the local socket buffer ( usually of the order of 10's of kbytes big ) whilst performing gethostbyaddr() or equivalent. Reworking any such daemon to be multi-threaded enough to use the re-entrant version gethostbyaddr_r() instead, and thereby allow only the given source socket whence the original syslog message arrived to be blocked pending the DNS response, is something I would imagine syslog-ng has not yet been encoded to perform. However, I have come across a even worse problem with syslog ( non-NG syslogd that is ) and DNS, namely a terrifying full blown hard deadlock. This for Linux kernel 2.2.19 + latest RedHat syslogd ( with Unix DGRAM sockets preferred ) + local bind 8.2.3 server, whilst acting as remote syslog receiver. The problem only appeared to arise under extreme conditions, once in a blue moon. After months of tortuous attempts at debugging the problem, the issue appears to be that syslogd ( as I said this is not syslog-ng but it might yet apply which is why I raise the issue here ) deadlocks when it attempts to perform gethostbyaddr() ( and hence res_query() ) to localhost:53, whilst at exactly the same instant, the DNS server sitting on localhost:53 perform a syslog operation in respect of something/anything/perhaps completely unrelated, ( such as logging a Zone Transfer complete ). At this point syslogd/gethostbyddr() are blocked waiting on the DNS server, and the DNS server is blocked waiting on syslogd(). Note that this flies in the face of what one might deduce might happen, namely that gethostbyaddr() times out waiting for localhost:53 to reply ( because the DNS server is temporarily deadlocked waiting for syslogd itself ), and works round the deadlock by performing the DNS query against the secondary DNS server in /etc/resolv.conf. Once this deadlock has occurred, any given process which potentially uses syslog deadlocks on the next occasion it attempts to use syslog(). Over a matter of minutes or less, the whole system deadlocks on syslogd, BUT snmpd ( which very rarely syslogs ) and ping ( which is unlikely to syslog unless something in ipchains or similar performs a printk() ) continue to function Ok. Thus simplistic monitoring of the host indicates nothing untoward via SNMP / ping but end-user experience is deadly. The best solution I have so far is to ensure syslogd never uses a local DNS server, but instead always uses a remote DNS server to perform the lookups. Under these conditions, any such deadlock appears to be recoverable, as gethostbyaddr() never blocks waiting on the local DNS server. Whilst using nscd improves the situation - make it less likely to occur - and including a small DNS cache within the syslogd process itself, make it less likely still, neither workround completely fixes the potential deadlock. And of course even trying to debug for such a deadlock is fraught with difficulties. I feel the really proper solution where one strongly desires to run a local DNS server, as intimated above, is to multi-thread the syslog daemon, use a fully threaded resolver library, and only block the socket the message arrived on whilst performing the gethostbyaddr_() - and I haven't had enough strong coffee all year to face attempting that nightmare rewrite! As to why the deadlock really occurs - i.e. why gethostbyaddr() doesn't simply timeout, fall through to backup DNS server, and thereby release the deadlock - I have no idea. Maybe it somehow got fixed in Kernel 2.4? Again, testing the theory is tricky. If anyone has further ideas on releasing the deadlock, I'd be only too happy to hear from you. By the by, I've been looking into migrating to syslog-ng for other reasons anyway, mainly due to its more fine-grained filtering structure, and despite some minor reservations I'm happily impressed with what I've seen. Many thanks to all involved in its development so far. Ted Rule, Flextech Television. *************************************************************************************************** This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. If you are not the intended recipient, any review, retransmission, disclosure, copying, modification or other use of this E-mail message or attachments is strictly forbidden. If you have received this E-mail message in error, please contact the author and delete the message and any attachments from your computer. You are also advised that the views and opinions expressed in this E-mail message and any attachments are the author's own, and may not reflect the views and opinions of FLEXTECH Television Limited. ***************************************************************************************************

1 0

Re: [syslog-ng]DNS caching
by Michael Renner 21 Mar '02

21 Mar '02

At 09:01 20.03.2002 +1000, you wrote: >On Tue, 19 Mar 2002 at 10:24am (+0100), Balazs Scheidler wrote: >[...] > > > > > can you check this patch, whether it fixes your problem ? > > > > diff -u -r1.37 cfgfile.c > > --- cfgfile.c 2001/09/03 16:42:23 1.37 > > +++ cfgfile.c 2002/03/19 09:23:55 > >[...] > >Thank you... it appears to be working as advertised now. When running with >NSCACHE_DEBUG enabled we get lots of messages like... > >.... which looks good. CPU usage for the syslog process has dropped from >~80% to ~40% (hazzah!) and the named process that was doing local caching >has dropped from ~10% to almost nill. Dear Bazsi, Thanks, also works flawless here, I get hardly any hits on my dnscache. Btw. the default option of dns_cache is "on", maybe you should document this or change it to "no", otherwise people who upgrade to newer versions (with the fixed dns_cache) may be confused by the "new" behaviour of syslog-ng. You also mention "syslog-ng blocks on DNS queries, so enabling DNS may lead to a Denial of Service attack." in your documentation. Does this mean that syslog messages which are received by the NIC, while syslog-ng performs a synchronous DNS lookup, are stored in the kernels receive buffer or are dropped? best regards, -- Michael Rennner Junior System Engineer Inode Telekommunikationsdienstleistungs GmbH - http://www.inode.at support(a)inode.at, Tel.: 05 9999-0, Fax.: 05 9999-2699

2 1

position dependencie of rules and "flags(final);"
by Volker Apelt 20 Mar '02

20 Mar '02

Hello, Are syslog rules position dependend? Is "flags(final);" position dependend? Is example #1 different from #2 in what it logs? Theory: # example 1 log { source; filter1; destination1; } log { source; filter1; filter2; destination2; flags(final); } # example 2 log { source; filter1; filter2; destination2; flags(final); } log { source; filter1; destination1; } Let's say, that filter2 matches only a sub set of the messages caught by filter1 and define: set1 := messages that matched by filter1 set2 := messages that matched by filter2 setLeftover := set1 without messages of set2 which means set1 == setLeftover + set2 In example #2 destination1 receives the messages of setLeftover and destination2 receives set2. But if rules+final are position dependend, then destination1 in example #1 will log _all_ messages of source1, _including_ those of filter2. So, destination1 receives the messages of set1 If they are not position dependend both destinations will receive the same message sets as in example #2. Is syslog-ng implemented in a way to Or as a practical example from my configuration: Will this configuration do what it is intended to? ## in pseudo code of what I am trying to do. if ( is it from "ssh" ) { if ( is loglevel equal or worse than "error" ){ send it to D_udp_network and D_tty_all } send it to D_ssh final ; # ie: dont send it any place else } ## I'll try to solve that with two different rules, one with a flag(final) and one without. I try to log all messages of ssh into one file and all messages of ssh above a certain log level (error or worse) shall be send to another destination (network + alert tty), too. All messages caught by the second rule must be logged to the first destination and No message from ssh shall be logged any place else, and I don't want to write a filter(f_no_ssh); into each and every log{} rule. # A message from ssh with loglevel emergency If syslog rules are postion dependend #### options { long_hostnames(off); sync(0); }; source S_local { unix-stream("/dev/log"); file("/proc/kmsg"); internal();}; ## destination D_udp_network{ udp( 192.168.51.2 port(514)); }; destination D_ssh { file("/var/log/ssh" perm(0640) owner(root) group(log)); }; ## filter l_normal_or_worse { level(normal..emerg); }; filter l_err_or_worse { level(err..emerg); }; ## filter f_ssh { program("ssh.*") or program("su"); }; ##### TWO RULES accepting messages from ssh ##### first without "final", second with "final" ## ## selected ssh messages of level error or worse ## send critical ssh messages to all tty's log { source(S_local); filter(f_ssh); filter(l_err_or_worse); destination( D_udp_network ); destination(D_tty_net); }; ## secure login / ssh / su ... ## shall store all messages (normal ... emerg) of ssh log { source(S_local); filter(f_ssh); filter(l_normal_or_worse); destination(D_ssh); flags(final); }; What kind of output on D_ssh and D_udp_network should I expect if there are these two messages? ## example messages #loglevel message [normal] datum host sshd[550]: some normal message of sshd [error] datum host sshd[550]: some error output of sshd Thank you, Volker Apelt -- Volker Apelt volker_apelt .@. yahoo.de (remove the dots, please) Dipl. Chem. +49 6172 31126

2 3

template option for tcp destination
by Nam 20 Mar '02

20 Mar '02

Does the template() option work for tcp destination? Help docs only list the option for file destination, but it also works with pipe. I would like to be able to apply formatting on the output for data inputing into a vendor product without having to use netcat (feeding from a piped destination). Nam __________________________________________________________ Get your FREE personalized e-mail at http://www.canada.com

2 1

DNS caching
by Matthew Melvin 20 Mar '02

20 Mar '02

Hello, I'm loath to ask a question right off the blocks but I wasn't able to find too much in the archives about this... I'm using syslog-ng 1.5.14 and am trying to get the 'dns_cache(yes);' option to work for me. When compiled with -DNSCACHE_DEBUG I get this message to start with... initialising nscache: 1007 entries, 3600/60 expiration. ... but no messages to suggest nscache_lookup() or any of the other cache functions are ever being called. Is there more to this option than meets the eye? Other recent discussions have talked about using djb's dnscache or nscd to do this instead and questioned the wisdom of using caching internal to syslog-ng. Should I not be using this feature? Does it, in fact, work at all? M. -- WebCentral Pty Ltd Australia's #1 Internet Web Hosting Company Level 1, 96 Lytton Road. Network Operations - Systems Engineer PO Box 4169, East Brisbane. phone: +61 7 3249 2500 Queensland, Australia. pgp key id: 0x900E515F

2 2

Re: big packets
by Balazs Scheidler 19 Mar '02

19 Mar '02

On Tue, Mar 05, 2002 at 05:15:11PM +0100, p(a)netoides.com wrote: > Hello syslog-ng developers. > > We are using your wonderful syslog-ng to receive > some big message chunks. We noticed that syslog-ng drops > bytes above 1Kb when using UDP. So, we changed to > TCP and, to our surprise, we found that syslog-ng > splits messages in chunks of 1Kb (approx). > > Is it possible to configure syslog-ng so that > each tcp message goes to only one line? > If this is not possible, what about modifing > syslog-ng to add this capability? > We can contribute development effort. after applying the following patch, you can change the maximum line length of syslog-ng by changing the MAX_LOG_LINE constant in syslog-ng.h (currently 2048 bytes) Index: sources.c =================================================================== RCS file: /var/cvs/syslog-ng/src/sources.c,v retrieving revision 1.30 diff -u -r1.30 sources.c --- sources.c 2002/02/28 13:05:37 1.30 +++ sources.c 2002/03/05 18:38:35 @@ -49,7 +49,7 @@ (vars (dgram simple UINT32) (pos simple UINT32) - (buffer array UINT8 MAX_LINE) + (buffer array UINT8 MAX_LOG_LINE) (next object log_handler))) */ @@ -81,11 +81,11 @@ size_t salen = sizeof(sabuf); if (!closure->dgram) { - n = A_READ(read, MAX_LINE - closure->pos, closure->buffer + closure->pos); + n = A_READ(read, MAX_LOG_LINE - closure->pos, closure->buffer + closure->pos); salen = 0; } else - n = A_RECV(read, MAX_LINE - closure->pos, closure->buffer + closure->pos, (abstract_addr *) &sabuf, &salen); + n = A_RECV(read, MAX_LOG_LINE - closure->pos, closure->buffer + closure->pos, (abstract_addr *) &sabuf, &salen); switch(n) { case 0: -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

6 10

Re: [syslog-ng]DNS caching
by Michael Renner 19 Mar '02

19 Mar '02

At 12:04 19.03.2002 +1000, you wrote: >Other recent discussions have talked about using djb's dnscache or nscd to >do this instead and questioned the wisdom of using caching internal to >syslog-ng. Should I not be using this feature? Does it, in fact, work at >all? It seems that this function is currently in development (or even deprecated, no changes were made from 1.5.14 to 1.5.15, it isn't documented at all, nobody even bothered to mention it when I asked a similiar question) and not working at the moment (hits to dnscache haven't decreased after activating it, i'm currently using 1.5.14 too). I for my part prefer dnscache over nscd because dnscache is much more transparent in its workings (no "weird" or "_very_ old cached" responses, very detailed logs). As for the internal dns cache, i don't think that it's that complicated at all, because the main part of dns caching, the resolver, is already implemented and working fine; adding a cache with fixed expiry times isn't that much additional work (at least it doesn't seem so to me, correct me if i'm wrong). The only problem I can think of are DNS records which have a TTL lower than the cache expiry time; the dns cache should ignore the ttl fields because here it gets way too complicated for a syslogger :). best regards, -- Renner Michael Junior System Engineer Inode Telekommunikationsdienstleistungs GmbH - http://www.inode.at support(a)inode.at, Tel.: 05 9999-0, Fax.: 05 9999-2699

1 0

Default template?
by Gert Menke 15 Mar '02

15 Mar '02

Hi! Is there a possibility to define a default template, so that I don't have to set my favorite template for every destination explicitly? Thanks in advance! Greetings Gert

1 0

load balanced syslog-ng
by Nicholas Berry 14 Mar '02

14 Mar '02

I'm running syslog-ng 1.5.13 (w/libol-0.3.1) under Solaris 8 and have two servers load balanced by two Foundry ServerIrons. This has been a rock solid solution for load balancing syslog reporting, but about once every few days both load balancers report a loss of port 514/udp. After checking syslog-ng for each syslog server, the daemon is still running and nothing was reported in syslog messages. Any ideas? Nicholas Berry Systems Engineer RagingWire Telecommunications, Inc. 2710 Gateway Oaks Dr., Suite 300 South Sacramento, CA 95833 Phone: 916.286.4048 Fax: 916.921.4148 E-mail: nberry(a)ragingwire.com www.ragingwire.com

1 0