August 2002 - syslog-ng - lists.balabit.hu

Urgent: Mangled messages in pipe() destinations in version 1.5.19
by Andreas Schulze 20 Mar '03

20 Mar '03

Hello Balazs, in version 1.5.19 (and in other versions between >1.5.7 and <=1.5.19, too) we observing obscure message mangling in pipe() destinations. Imagine the config below and the scenario: syslog-ng --write(2)--> pipe() --read(2)--> application All four log paths below, writes lines to the pipe '/tmp/pipe'. With 1.5.7 we read from the pipe exactly that, what syslog-ng (using the config below) writes to it. But with all newer versions (espec. 1.5.19) we read (with the same config): message_1 message_2 messagmessage_4 !!! ... message_n if we try to let syslog-ng write: message_1 message_2 message_3 message_4 ... message_n (The numbers and terms are only examples, because of the different sources and high message load.) Seems, that write(2) to fifo skips some message ends now. (espec. the terminating newline) > # example config: > # > source sys { sun-streams ("/dev/log" door("/var/run/syslog_door")); }; > source net { udp (ip(w.x.y.z) port(n)); }; > destination d_cisco { pipe("/tmp/pipe" template("@cisco: | $MSG\n")); }; > destination d_icmp { pipe("/tmp/pipe" template("@icmpd: | $MSG\n")); }; > log { source(net); filter(fa); destination(da); destination(d_cisco); }; > log { source(net); filter(fb); destination(db); destination(d_cisco); }; > log { source(sys); filter(f1); destination(d1); destination(d_icmp); }; > log { source(sys); filter(f2); destination(d2); destination(d_icmp); }; We are running a config similar to this above, with 1.5.7 since a lot of months very successfully in production ennvironment with very high message load. Since 1.5.1? the message mangling occurs. But now, some of your newer features becomes mission critical for us. Have you any idea, what happens? Versions <= 1.5.7 works well. Thanks a lot. -- Best regards --Andreas Schulze [phone: +49.5246.80.1275, fax: +49.5246.80.2275] | I believe, it was Dennis Ritchie who said something like: | "C is rarely the best language for a given task, | but it's often the second-best". | The implication being that: "[...]" | | sh# cat>$$.c<<EOT | main(l,a,n,d)char**a;{for(d=atoi(a[1])/10*80-atoi(a[2])/5-596;n="@NK\ | ACLCCGZAAQBEAADAFaISADJABBA^SNLGAQABDAXIMBAACTBATAHDBANZcEMMCCCCAAhE\ | IJFAEAAABAfHJETBdFLDAANEfDNBPHdBcBBBEA_AL H E L L O, W O R L D! " | [l++-3];)for(;n-->64;)putchar(!d+++33^l&1);} | EOT | gcc -o$$ $$.c;clear;./$$ 52 8;rm -f $$*

6 19

RE: [syslog-ng]Errors Compiling 1.5.19 with TCP_WRAPPERS
by Sangbutsarakum, Patai 16 Sep '02

16 Sep '02

I can't success also. I got suggestion to use other feature of syslog-ng that can do access-list like as TCP_WRAPPER. Hope this help Pat. -----Original Message----- From: Victor [mailto:victord@paid.com] Sent: Thursday, August 22, 2002 10:15 To: Syslog-ng Subject: [syslog-ng]Errors Compiling 1.5.19 with TCP_WRAPPERS I am trying to compile with tcp_wrappers, but I get this error: gcc -O2 -march=i686 -pipe -Wall -I/root/syslog-ng/work/src/libol-0.3.3/src -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o -lfl /root/syslog-ng/work/src/libol-0.3.3/src/.libs/libol.a -lnsl afinet.o: In function `do_open_afinet_connection': afinet.o(.text+0x3c3): undefined reference to `request_init' afinet.o(.text+0x3cf): undefined reference to `sock_host' afinet.o(.text+0x3d8): undefined reference to `hosts_access' collect2: ld returned 1 exit status make[3]: *** [syslog-ng] Error 1 Shouldn't it have something like -lwrap on the line above? Also, I heard on the mailing list that there is a patch for 1.5.19? Is there a link to it? What's the proper patch command to apply it? Thanks _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

4 3

file buffer flushing
by Joerg Sommer 09 Sep '02

09 Sep '02

Hi, can syslog-ng delay the flush of a file buffer? I have noflushd running and if syslog-ng flushes his buffers, my disc spins up. But I will see new messages in the log files. Is it possible, that syslog writes the messages in the file, but don't call flush() on this file. Joerg.

3 3

Incomplete library checking when building 1.4.15 on Sol7/SPARC
by Adam Steffes 30 Aug '02

30 Aug '02

While trying to build syslog-ng v1.4.15 on a rather plain Solaris 7/SPARC machine, I found that the configure script doesn't look everywhere it should to find yywrap. My system had it in /usr/lib/libl.a while the configure script seemed to look only in /usr/lib/libfl.a. I basically hardcoded the configure script to look at libl.a in a hurried attempt to get it to compile. Is this something anyone has looked at before? I admit I am not a developer by any means, and certainly do not want to insult anyone. It seems a small bit of code could be added to the 'case' statement in the configure script to perform a more extensive search for yywrap. Here is how I found the location of yywrap on my machine: $ for lib in /usr/lib/*.a ; do echo $lib ; nm $lib | grep yywrap ; done <other libs snipped> /usr/lib/libintl.a /usr/lib/libkrb.a /usr/lib/libl.a /usr/lib/libl.a[yywrap.o]: [5] | 0| 8|FUNC |GLOB |0 |2 |yywrap [1] | 0| 0|FILE |LOCL |0 |ABS |yywrap.c /usr/lib/libldfeature.a /usr/lib/libm.a /usr/lib/libmail.a /usr/lib/libmalloc.a <the rest snipped> I hope this helps. -Adam

1 0

Priming the database.
by Scott A. McIntyre 30 Aug '02

30 Aug '02

Hi, I'm pondering moving towards mysql logging for syslog-ng, but was wondering if anyone has already got a tool/script laying around to take current logs and import into the database accurately and correctly. Thanks for any pointers, Scott

1 0

syslog over TCP
by BONIKOWSKE, SCOTT 28 Aug '02

28 Aug '02

--- Received from WPS.SBONIKOW (608) 226-2630 08-28-02 1021a Hello, I am new to syslog-ng, and I had a couple questions that I have not seen answered before in the archives (from what I saw), and I was wondering if anyone out there could help. I hope this is the right area for these questions. First, Does TCP increase the load on the CPU alot over UDP? the situation is, we are looking at using syslog-ng on a midsized company (overall about 6,000+ people). we would like to monitor all syslog activity from NT,2000, XP, HPUX, AIX, Cisco routers/switches/PIXs, linux. The windows machiens are mostly file/application servers, the linux boxes are file/web/application servers, the HPUX and AIX boxes are application servers also. There is a lot of logging that will need to be watched because of security reasons (including file access, logins, etc.), so we are expecting a lot of traffic. The end goal will be to have all of the logs filtered by machine (in some cases even more granular than that like cisco errors going into the main log, and a seperate error log for that machine, and an error log for all networking devices), and maybe even searchable if I have enough time. I am only going into high detail above becase I would like to know opinions about using TCP vs UDP for load on the CPU, and to see if anyone else has maybe done something like this themselves for a company close to the same size. Any help and comments would be welcome. Thanks Scott Bonikowske Open Systems Analyst WPS ---- 08-28-02 1021a ---- Sent to ------------------------------------ -> syslog-ng(a)lists.balabit.hu

1 0

syslog-ng documentation and some minor niggles
by Ted_Rule＠flextech.co.uk 28 Aug '02

28 Aug '02

Many thanks to Stephen Frost and Balazs for the useful information and suggestions regarding PIX syslogging. Having had a slightly deeper read of the latest src tree, I'd like to just make a few suggestions and comments. Several of the options available have Default Settings in their relevant subsection of the HTML tree within: /usr/src/syslog-ng/syslog-ng-1.5.19/doc/sgml/syslog-ng.html However, several options don't have defaults listed therein. Perhaps the next round of documentation updates could double check that all defaults are fully listed in the html pages. The Main HTML Doc Pages at http://www.balabit.hu/static/syslog-ng/reference/book1.html contain no reference to recent dns_cache options - even though this option appears to have been available for about 18 months. As I think several others have mentioned on the list, it would seem reasonable for a more up to date copy of the html pages to be glued up onto the website. The syslog-ng man pages explicitly state that destinations are closed and re-opened on SIGHUP, as per dear-old syslogd's behaviour. There have been references on the mailing list and the FAQs to the corresponding close / re-open of sources, but no explicit statement in the docs. Could I request that an explicit statement of source file descriptor open/close behaviour on SIGHUP be added to the docs/FAQ, please. Others on the list have noted that closing /dev/log on HUP causes certain programs grief - notably sendmail which requires a further HUP to itself to properly heal the broken syslog file descriptor. If syslog-ng does indeed close descriptors on HUP, can we please have an option to explicitly keep certain descriptors open on HUP. ( The docs suggest this may already be possible for TCP sockets, and Unix SOCK_STREAM sockets, but not Unix SOCK_DGRAM sockets ). See ensuing paragraphs for why I'd like the option to be available on Unix SOCK_DGRAM. In all the syslog-ng docs, Linux platforms are documented as having a default log listener socket of /dev/log SOCK_STREAM. However, the most recent RedHat sysklogd RPMs open /dev/log in SOCK_DGRAM mode. I believe this was done as part of a security patch to avoid a potential DoS attack against the stream socket. To the best of my recollection, this behaviour changed about a year ago. This seems to bring RedHat - at least - in line with other BSD variants. glibc openlog() call on RedHat opens SOCK_DGRAM first, falling back to SOCK_STREAM if that fails. Hence RedHat's glibc still supports either socket type, and hence syslog-ng's documented Linux defaults still work. Likewise, Perl's Syslog.pm module calls SOCK_DGRAM first falling back to SOCK_STREAM. Sometime during the change from kernel 2.0 to 2.2 I believe, the Unix DGRAM socket type behaviour was adjusted to blocking ( this assumption purely based on the perceived behaviour of sysklogd ). Thus it should be impossible to drop a message across a Unix DGRAM socket , as compared with the ease of such a message being dropped across an UDP DGRAM socket when the system is really busy. Hence the change from SOCK_STREAM to SOCK_DGRAM within sysklogd should not make the system more prone to dropping locally logged messages, as might have been the case with earlier kernels? Given all of the above, is it reasonable to change the preferred documented socket type from STREAM to DGRAM on /dev/log for syslog-ng on Linux? "logger-ng" - I believe others may also have asked for this - does such a thing exist? to allow one to create test messages and the like over TCP etc etc etc. Sys::Syslog-NG.pm ?? does such a thing exist to support Perl Syslogging over TCP and suchlike? Ted Rule, Flextech Television *************************************************************************************************** This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. If you are not the intended recipient, any review, retransmission, disclosure, copying, modification or other use of this E-mail message or attachments is strictly forbidden. If you have received this E-mail message in error, please contact the author and delete the message and any attachments from your computer. You are also advised that the views and opinions expressed in this E-mail message and any attachments are the author's own, and may not reflect the views and opinions of FLEXTECH Television Limited. ***************************************************************************************************

1 0

Please help fast...
by Keven Belanger 28 Aug '02

28 Aug '02

What is incorrect in this line ??? log { source(s_udp); destination(d_std); flags(fallback); }; Thanx... Kéven Belanger Analyste en solutions de sécurité Logicon Inc. - Division Sécurité 819.825.8049 x7717 800.567.6399 x7717

2 1

2 more question...please help
by Keven Belanger 27 Aug '02

27 Aug '02

Okay! We have 2 problems; 1- The hostname of the syslog receiver is SECU-Rcv1 and SECU-Rcv2 So with a destinations like destination: d_std { file("/export/disk1/log/$HOST/$YEAR/$MONTH/messages.log"); }; We receive the log in /export/disk1/log/SECU-Rcv2 on the local host why not the ip like other host ?? 2- We have 2 syslog collector and a big config file with each host (customer) defined line by line and a rule with unknown, the problem is that each host log correctly to the respective directory but they also log to my unknown directory, I want to be able to say, if no filter match, log to unknown... Thanx a lot! Kéven Belanger Analyste en solutions de sécurité Logicon Inc. - Division Sécurité 819.825.8049 x7717 800.567.6399 x7717

1 0

some questions
by Joerg Sommer 27 Aug '02

27 Aug '02

Hi, * What's the difference between long_hostnames() and use_fqdn()? * How get I names instead of ips in the logs? With use_dns()? How dangerouse is use of use_dns()? * Can I write a hostname instead of a ip in tcp()? * I miss the default values of the options in the docu. What's default for use_dns(), stats(),...? * When will encryption and authentication come? Joerg.

3 4