October 2004 - syslog-ng - lists.balabit.hu

Re: [syslog-ng]syslog-ng loses messages
by Aaron watkins 20 Oct '04

20 Oct '04

On Wed, 20 Oct 2004 14:45:50 +0100, Watkins, Aaron <aaron.watkins(a)newsint.co.uk> wrote: > > Dave Johnson syslog-ng(a)lists.balabit.hu > Mon, 11 Oct 2004 14:43:24 -0500 > > ------------------------------------------------------------------------ > -------- > > The two questions are: > 1) where in the process do messages 0,1 get lost > 2) where in the process do messages x get lost when high load > This is exactly the question. Did you manage to set up the test case on your system. If so, what sort of results did you get? > In theory, since using TCP this shouldn't be the problem... ? > I am assured that we have a highly reliable network, and as you mentioned, with TCP it shouldn't be an issue. The fact that it is consistently two messages that get dropped implies that it is unlikely to be a network problem. However, maybe it is dropped at the kernel level? > *--------------------------* > * Can we try using "logger" as opposed to "event_pipe" in the while > loop? > * Can we `snoop -p -o /tmp/outout port 1515` on server2 and see if the > packets get sent? > * If that doesn't work, can we try without the templates? > * Where are the messages lost? system1 local files? system2 local > files? system2 pipe output? > * Solarisms? nscd running? kernel-revs? > > ---- It is worth noting that the system1 local copy of the events includes these lost events; it is somewhere in between sys1 and sys2 that it goes. Unfortunately, we have one of those corporate setups where I don't have the privileges to be able to run most of the other options, maybe someone else will have some ideas? Thanks, Aaron > > Anybody else have any ideas? > > On Fri, 8 Oct 2004 12:22:45 +0100, Aaron watkins > <aaron.j.watkins(a)gmail.com> wrote: > > Dave, > > > > > Dave Johnson syslog-ng(a)lists.balabit.hu > > > Thu, 7 Oct 2004 10:47:14 -0500 > > > > > > We have a couple solaris boxes that do logging with pipes and we log > a > > > large amount of data, thats why I'm interested in these tests. I > can > > > see if I can setup a quick testbed next week. > > > > > > ********** > > > > > > Otherwise, I'm wondering if the packets are throttled, what > happens.. IE: > > > > > > --------------------------- > > > cat > /tmp/smallsleep.c <<ENDHERE > > > #include <unistd.h> > > > > > > main () { > > > usleep(50000); /* 20 hz */ > > > } > > > ENDHERE > > > gcc -o /tmp/smallsleep /tmp/smallsleep.c > > > ----------------------------- > > > in the do -> done segment of your for loop add `/tmp/smallsleep` > > > > > > How does this impact lost messages? > > > > > > > In the situation where the receiving server is killed while the > > sending server has stopped sending messages (ie. during the sleep 1 > > command), there is no affect (ie. Message 0 and 1 are still lost). > > > > In the situation where the receiving server is killed while the > > sending server is still sending messages, the throttling you suggested > > reduces the number of messages that are lost to 2 (similar to the > > first test). > > > > Test 1: Message 12 and 13 lost > > Test 2: Message 25 and 26 lost > > > > It's an improvement, but unfortunately, its still not quite reliable > > enough and when used in practice, I'm unsure if our events (which will > > be asynchronous by their very nature) could possibly be throttled... > > > > Do you have any more ideas??? > > > > Thanks, > > > > > > Aaron Watkins > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > > > > The Newspaper Marketing Agency: Opening Up Newspapers: > > www.nmauk.co.uk > > This e-mail and all attachments are confidential and may be privileged. If you have received this e-mail in error, notify the sender immediately. Do not use, disseminate, store or copy it in any way. Statements or opinions in this e-mail or any attachment are those of the author and are not necessarily agreed or authorised by News International (NI). NI Group may monitor emails sent or received for operational or business reasons as permitted by law. NI Group accepts no liability for viruses introduced by this e-mail or attachments. You should employ virus checking software. News International Limited, 1 Virginia St, London E98 1XY, is the holding company for the News International group and is registered in England No 81701 > >

1 0

Problem compiling syslog-ng 1.5.26
by Chris Vaughan 20 Oct '04

20 Oct '04

Hello, I am trying to compile syslog-ng 1.5.26 on a Red Hat Linux 7.3 system, in order to use template() destinations so that I can log to a mysql database. While running the make command, it fails as follows: syslog-ng-1.5.26]# make Making all in src make[1]: Entering directory `/usr/local/install/syslog-ng-1.5.26/src' make all-recursive make[2]: Entering directory `/usr/local/install/syslog-ng-1.5.26/src' Making all in . make[3]: Entering directory `/usr/local/install/syslog-ng-1.5.26/src' gcc -g -O2 -Wall -I/usr/local/install/libol-0.3.14/src -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o -lresolv -lfl /usr/local/install/libol-0.3.14/src/.libs/libol.a -lnsl sources.o: In function `do_read_line': /usr/local/install/syslog-ng-1.5.26/src/sources.c:91: undefined reference to `MIN' sources.o: In function `make_log_reader': /usr/local/install/syslog-ng-1.5.26/src/sources.c:169: undefined reference to `MAX' cfg-lex.o: In function `append_string': /usr/local/install/syslog-ng-1.5.26/src/cfg-lex.l:239: undefined reference to `MIN' affile.o: In function `do_init_dest_writer': /usr/local/install/syslog-ng-1.5.26/src/affile.c:287: undefined reference to `MAX' affile.o: In function `append_string': /usr/local/install/syslog-ng-1.5.26/src/affile.c:558: undefined reference to `MIN' affile.o: In function `expand_macro': /usr/local/install/syslog-ng-1.5.26/src/affile.c:654: undefined reference to `MIN' afinet.o: In function `do_init_afinet_dest': /usr/local/install/syslog-ng-1.5.26/src/afinet.c:461: undefined reference to `MAX' collect2: ld returned 1 exit status make[3]: *** [syslog-ng] Error 1 make[3]: Leaving directory `/usr/local/install/syslog-ng-1.5.26/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/install/syslog-ng-1.5.26/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/local/install/syslog-ng-1.5.26/src' make: *** [all-recursive] Error 1 What is going wrong here? What should I do to correct this? *************************************************************** This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of the Department of Lands. This email message has been swept by MIMEsweeper for the presence of computer viruses. ***************************************************************

2 1

Syslog forwarding with TCP.
by Davis, Jay 20 Oct '04

20 Oct '04

We are using syslog-ng in a test lab environment to get syslog messages out of the testlab into another environment. So several systems are forwarding messages to a bastion host of sorts, which then forwards by traversing a firewall to another host. The messages have to traverse a firewall (NAT). We decided to use TCP because it is a bit easier to get past the security guys. We have noticed behavior which is the nature of TCP. Of several (lines) of a syslog message ending up in the same packet (using a sniffer). Although interesting, this is not the problem. We have noticed that some messages are getting lost. So the questions. a) Can we change something to have one message per packet with syslog-ng settings. b) Has anyone else seen this behavior (or is anyone even using TCP)? c) Any other clues as to why a TCP session that is stable would drop packets?

1 0

Stats dropped ??
by Jörgen Ekstedt 19 Oct '04

19 Oct '04

Hi everyone! I have a problem with syslog-ng reporting stats-dropped? What does this mean ?? Anyone got any clues??

2 1

syslog-ng configure and tcp_wrappers
by José Pedro Oliveira 19 Oct '04

19 Oct '04

Hi, Right now syslog-ng is built using the dynamic tcp_wrappers library. $ ldd syslog-ng libnsl.so.1 => /lib/libnsl.so.1 (0x0583f000) libresolv.so.2 => /lib/libresolv.so.2 (0x0029d000) libwrap.so.0 => /usr/lib/libwrap.so.0 (0x00d87000) ^^^^^^^^ libc.so.6 => /lib/tls/libc.so.6 (0x00bcb000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00bb3000) In some configurations the /usr/lib directory may not be available (eg: mount by nfs) when the daemon is started. Would it be possible to modify the syslog-ng configure file to use the static tcp_wrappers library instead? A possible patch is attached. Note that it may not be the best solution as I am not a autotools expert (it does work in Red Hat/Fedora distros though). Thanks in advance, jpo -- José Pedro Oliveira * mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo * * gpg fingerprint = F9B6 8D87 859D 1C94 48F0 84C0 9749 9EB5 91BD 851B *

1 0

syslog-ng snapshots broken
by José Pedro Oliveira 19 Oct '04

19 Oct '04

Hi, The last syslog-ng snapshot is from 2004-10-06. Could this be fixed? And also (maybe) delete libol-0.3.13 and syslog-ng-1.6.4 snapshots? Thanks in advance, jpo Snapshots directory: http://www.balabit.com/downloads/syslog-ng/1.6/src-snapshot/ -- José Pedro Oliveira * mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo * * gpg fingerprint = F9B6 8D87 859D 1C94 48F0 84C0 9749 9EB5 91BD 851B *

1 0

Re: problem with syslog-ng and door
by jesse.keefe＠convergys.com 18 Oct '04

18 Oct '04

Balazs, An Etherlite is a console concentrator. Basically, it's a serial connection from the server back to the etherlite. We use a piece of software called conserver to access our system consoles over this etherlite connection. When one of these ports hang, syslog-ng hangs (I'm assuming because it's trying to write out to the console) and this causes su commands and others to hang. If you'd like, send me an email offline and I can give you some more background. Thanks for responding to this. Jesse Keefe Sr Analyst, Open Systems Convergys Corp 513-723-3689

1 0

syntax error line 1
by Chet Harvey 16 Oct '04

16 Oct '04

Hmmm...getting an error: $ /usr/local/sbin/syslog-ng syntax error at 1 Parse error reading configuration file, exiting. (line 1) here is what the conf looks like: options { use_fqdn(yes); keep_hostname(yes); use_dns(yes); long_hostnames(off); use_time_recvd(no); chain_hostnames(no); log_fifo_size(300); mark(0); sync(0); }; source src { unix-dgram("/var/run/log"); udp(); internal(); file ("/dev/klog"); }; # local0.* %/var/log/filter.log filter f_1 { facility(local0) and level(debug...emerg); }; destination d_1 { file("/var/log/filter.log"); }; log { source(local); filter(f_1); destination(d_1); }; # local7.* %/var/log/dhcpd.log filter f_2 { facility(local7) and level(debug...emerg); }; destination d_2 { file("/var/log/dhcpd.log"); }; log { source(local); filter(f_2); destination(d_2); }; # *.notice;kern.debug;lpr.info;mail.crit;news.err;local0.none;local7.none %/var/log/system.log filter f_3 { facility(mail) and level(crit...emerg); }; filter f_4 { facility(news) and level(err...emerg); }; filter f_5 { not facility(local0); }; filter f_6 { not facility(local7); }; filter f_7 { level(notice...emerg); }; filter f_8 { facility(kern) and level(debug...emerg); }; filter f_9 { facility(lpr) and level(info...emerg); }; destination d_3 { file("/var/log/system.log"); }; log { source(local); filter(f_3); filter(f_4); filter(f_5); filter(f_6); filter (f_7); filter(f_8); filter(f_9); destination(d_3); }; # security.* %/var/log/system.log filter f_10 { facility(security) and level(debug...emerg); }; log { source(local); filter(f_10); destination(d_3); }; # auth.info;authpriv.info;daemon.info %/var/log/system.log filter f_11 { facility(auth) and level(info...emerg); }; filter f_12 { facility(authpriv) and level(info...emerg); }; filter f_13 { facility(daemon) and level(info...emerg); }; log { source(local); filter(f_11); filter(f_12); filter(f_13); destination (d_3); }; # *.emerg * filter f_14 { level(emerg); }; destination d_4 { file("*"); }; log { source(local); filter(f_14); destination(d_4); }; TIA, Chet

1 0

RE: [syslog-ng]syslog2ng util?
by Walt Rich 15 Oct '04

15 Oct '04

I've installed syslog-ng both Solaris and Red Hat Linux. That utility should be located in the 'contrib' subdirectory, where you untarred syslog-ng. For example: Syslog-ng-1.6.5\contrib -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Chet Harvey Sent: Friday, October 15, 2004 4:41 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]syslog2ng util? Hey all, I am haveing a heck of a time converting our syslog.confs to syslog-ng. I keep reading about a util that seemed to have been available in 1.5.x called syslog2ng. I am running freebsd syslog-ng version 1.6.5 but cant seem to find it.... any suggestions?? Chet _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

3 3

syslog2ng util?
by Chet Harvey 15 Oct '04

15 Oct '04

Hey all, I am haveing a heck of a time converting our syslog.confs to syslog-ng. I keep reading about a util that seemed to have been available in 1.5.x called syslog2ng. I am running freebsd syslog-ng version 1.6.5 but cant seem to find it.... any suggestions?? Chet

2 2