- syslog-ng - lists.balabit.hu

Weird issue
by Didier Conchaudron 03 May '06

03 May '06

Hi folks, I've a loghost receiving tons of logs from many servers and I'm experiencing problems regarding 2 destinations: a log forwarding to another server, and written logs for each servers. The logs forwarding work for most of the servers described into the filter written bellow, but some logs from some servers are not forwarded. First I think about a mistake into my regexp. For example, for the server ptcassidc01.ad5.ad.foo.com , I tried host("ptcassidc01.ad5.ad.foo.com"), host("ptcassidc01*"), and host("ptcassidc01") but none works. For the written log files I've the same problem, the logs which are not forwarded are not written too. So I'm thinking of a filter issue. Maybe it's too big and it can't be parsed rightly. It's not a network issue, the network load is about 250 Kb/sec. Regards, Didier Conchaudron Here're the destinations: destination dest_soc_pilot_project { udp("an ip address" port(514) ); }; destination dest_soc_exaprotect_agent { file("/data/logs/exaprotect/$HOST" owner(syslog) group(syslog) perm(0644)); }; Here's the filter I use: filter f_soc_pilot_project { level(info...emerg) and not facility(mail) and ( host("bcv51se9") or host("bcv50s56") or host("bcv50s9b") [ many many others, maybe 100 ] or host("plwarsidc01") or host("ptcassidc01") ); }; And finaly the log statements: log { source(net); filter(f_soc_pilot_project); destination(dest_soc_pilot_project); }; log { source(local); filter(f_soc_pilot_project); destination(dest_soc_pilot_project); }; log { source(net); filter(f_soc_pilot_project); destination(dest_soc_exaprotect_agent); }; log { source(local); filter(f_soc_pilot_project); destination(dest_soc_exaprotect_agent); }; Here're the main options: options { owner("root"); group("root"); dir_perm(0750); perm(0640); log_fifo_size(10000); chain_hostnames(no); keep_hostname(yes); check_hostname(yes); use_dns(yes); dns_cache(yes); dns_cache_size(1000); dns_cache_expire(36000); dns_cache_expire_failed(900); use_fqdn(yes); create_dirs(yes); use_time_recvd(no); stats(6000); sync(50); }; ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

1 0

Remote SYSLOG-NG logging - can't log from remote
by rlubbers＠sysctl.net 03 May '06

03 May '06

I am at my wits end with this. Some devices are logging remotely, but others are not. I can see the packets arrive on UDP port 514 using tcpdump, but the packets don't get recorded in the proper file. I have tried nearly everything, and I am certain I am doing something silly, but maybe you can help out. Here is my syslog-ng.conf.file: ************************************************************* source local { unix-dgram("/var/run/log"); udp(ip(0.0.0.0) port(514)); internal(); }; ### SECURITY LOG - This logs filter f_9 { facility(security) and level(debug..emerg); }; destination d_3 { file("/var/log/security" create_dirs(yes)); }; log { source(local); filter(f_9); destination(d_3); }; ### MAIL LOG - This logs filter f_12 { facility(mail) and level(info..emerg); }; destination d_5 { file("/var/log/maillog" create_dirs(yes)); }; log { source(local); filter(f_12); destination(d_5); }; ### PHONE DACS LOGS - This also logs filter f_40 { level(debug..emerg) and host("172.12.67.28"); }; destination d_60 { file("/var/log/dacs/ftldgaaw_dac_1.log" create_dirs(yes)); }; log{ source(local); filter(f_40); destination(d_60); }; ### ROUTER LOG - This does NOT log filter f_19 { host("192.168.1.128"); }; destination d_19 { file("/var/log/netrouter_pisst.log" create_dirs(yes)); }; log{ source(local); filter(f_19); destination(d_19); };

2 1

how to start the syslog-ng demeaon
by wayne yu 03 May '06

03 May '06

Hi: I configured my syslog-ng on the solaris 9 computer.' when I did /etc/init.d/syslog start, and then ps -ef |grep syslog-ng, nothing is list. How can i find out how the syslog-ng works. Thanks Wayne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

2 1

syslog-ng hanging?
by Tony Davis 03 May '06

03 May '06

Hi, I recently converted our RedHat EL servers (Release 3 Update 6) form the sysklog package to syslog-ng. On some of the servers syslog-ng appears to be occasionally hanging, i.e. messages stop getting written to the various log files. This also has a knock on effect of stopping logins, cron jobs hanging, etc, presumably because they are waiting to write to a log file. I read in the archives that this can be caused by something else using /dev/log but there's no other syslogd or minilogd processes running and lsof gives the following: COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME syslog-ng 1384 root 5u unix 0xf778c880 1703 /dev/log syslog-ng 1384 root 6u unix 0xf777e8c0 1715 /dev/log I'm using the RedHat conf file bundled in the contribs directory that comes with the source code. Any ideas would be welcome. Thanks Tony ***************************************************** You can find us at www.voca.com ***************************************************** This communication is confidential and intended for the exclusive use of the addressee only. You should not disclose its contents to any other person. If you are not the intended recipient please notify the sender named above immediately. Registered in England, No 1023742, Registered Office: Voca Limited Drake House, Three Rivers Court, Homestead Road, Rickmansworth, Hertfordshire, WD3 1FX This message has been checked for all email viruses by MessageLabs.

3 2

syslog-ng 1.6.10 release broken
by Balazs Scheidler 01 May '06

01 May '06

Hi, It came to my attention that syslog-ng 1.6.10 broke file("/proc/kmsg") support with the recent performance improvement patches as /proc/kmsg does not support nonblocking mode. The issue might cause the complete system to deadlock. Non-Linux platforms, or installation where /proc/kmsg is not directly processed by syslog-ng is not affected. I'm going to release 1.6.11 as soon as possible to fix this issue. -- Bazsi

1 0

Re: HP-UX pipe read errors
by Andy G. 01 May '06

01 May '06

On 04/25/06 09:08, Andrew Gill wrote: > Hello, > > I'm new to syslog-ng and am trying to configure it to run on HP-UX 10.20 > and 11.11. It seems to have problems reading from the /dev/log pipe. > Each time a message is received on the pipe, syslog-ng hits an I/O error > and closes the filehandle, and no more messages are received. However, > the first message is always processed correctly. > > Here is my syslog-ng.conf: > source src { > pipe("/dev/log" pad_size(2048)); > internal(); > }; > > destination d_file { > file("/var/adm/syslog-ng/messages" > owner(root) > group(systaff) > perm(0640) > ); > }; > > destination d_net { > tcp(testy port(1514)); > }; > > log { > source(src); > destination(d_file); > destination(d_net); > }; > > When I run syslog-ng in the foreground with debug and verbose options, > this is what I see: > > io.c: Preparing fd 3 for reading > io.c: connecting using fd 6 > connecting fd 6 to inetaddr testy, port 1514 > syslog-ng version 1.6.10 starting > io.c: Preparing fd 6 for writing > io.c: Preparing fd 7 for writing > io.c: do_read: read() failed (errno 11), Resource temporarily > unavailable > fd = 3, buffer = 00098b0, length = 2048 > Marking fd 3 for closing. > Closing fd 3. > > The do_read line is when the message is received. syslog-ng continues > to run, and internal messages (e.g. STATS) continue to show up on the > local host and the remote loghost. > > Any thoughts would be greatly appreciated. > > -Andy > > I think I figured this out...in libol's src/io.c an assumption is made that EAGAIN == EWOULDBLOCK, but this is not the case for HP-UX. > egrep "EAGAIN|EWOULD" /usr/include/sys/errno.h #define EAGAIN 11 /* No more processes */ #define EWOULDBLOCK 246 /* Operation would block */ I applied the following change to src/io.c and recompiled libol, and it works correctly now. -Andy -- io.c.orig Mon Mar 13 10:01:31 2006 +++ io.c Mon Mar 13 10:01:31 2006 @@ -361,6 +361,7 @@ case EINTR: continue; /* FIXME: Is it really worth looping here, * instead of in the select loop? */ + case EAGAIN: case EWOULDBLOCK: /* aka EAGAIN */ return 0; case EPIPE: @@ -415,6 +416,7 @@ continue; /* FIXME: Is it really worth looping here, * instead of in the select loop? */ case EWOULDBLOCK: /* aka EAGAIN */ + case EAGAIN: return 0; case EPIPE: werror("io.c: recvfrom() returned EPIPE! Treating it as EOF.\n");

2 1

Compiling & Using syslog-ng 1.9.10 on Solaris 10
by Philippe JOYEZ 01 May '06

01 May '06

Hello all, I've finally succedeed in compiling syslog-ng on my new Solaris 10 box using the following tasks: Modifications src/Makefile (Add -lrt) 75c75 < LIBS = -lpthread -lnsl -lsocket -ldoor -ll -L/usr/local/lib -lglib-2.0 -lintl -L/usr/local/lib -levtlog --- > LIBS = -lpthread -lrt -lnsl -lsocket -ldoor -ll -L/usr/local/lib -lglib-2.0 -lintl -L/usr/local/lib -levtlog Modification src/filter.c (Adding macros): /* Define G_LIKELY() and G_UNLIKELY() so that they are available when * using GLib 1.2 as well. These allow optimization by static branch * prediction with GCC. */ #ifndef G_LIKELY #define G_LIKELY(x) (__builtin_expect(x, 1)) #define G_UNLIKELY(x) (__builtin_expect(x, 0)) #endif /* !G_LIKELY */ But when I start syslog-ng, I have the following error message: Apr 26 09:50:55 sewerrat syslog-ng[752]: syslog-ng starting up; version='1.9.10' Apr 26 09:50:55 sewerrat syslog-ng[752]: Insufficient buffer space for retrieving STREAMS log message; res='ffffffff' Any idea? Best regards, Philippe JOYEZ -- Disclaimer ------------------------------------ Ce message ainsi que les eventuelles pieces jointes constituent une correspondance privee et confidentielle a l'attention exclusive du destinataire designe ci-dessus. Si vous n'etes pas le destinataire du present message ou une personne susceptible de pouvoir le lui delivrer, il vous est signifie que toute divulgation, distribution ou copie de cette transmission est strictement interdite. Si vous avez recu ce message par erreur, nous vous remercions d'en informer l'expediteur par telephone ou de lui retourner le present message, puis d'effacer immediatement ce message de votre systeme. *** This e-mail and any attachments is a confidential correspondence intended only for use of the individual or entity named above. If you are not the intended recipient or the agent responsible for delivering the message to the intended recipient, you are hereby notified that any disclosure, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender by phone or by replying this message, and then delete this message from your system.

2 1

RE: [PATCH] Re: [syslog-ng] time_sleep()
by Vaibhav Goel 30 Apr '06

30 Apr '06

That did the trick. I was going through the code and I suspected that was the problem in cfg-grammar.y but I am no code monkey so I let it be ;-) Thanks again for your help. Now I hope that this parameter will help bring down the CPU load on one of my loghosts (it has a whole bunch of parallel connections). Thanks, -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Roberto Nibali Sent: April 26, 2006 2:19 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [PATCH] Re: [syslog-ng] time_sleep() ADDENDUM: > The grammar was missing an entry, the attached patch should fix it. > Please try and report back if this solves the problem for you. And my patch was missing the documentation update, which I've obtained by fraud from the marvellous syslog-ng-2 tree. > ------------------------------------------------------------------------ > > --- syslog-ng-1.6.10/src/cfg-grammar.y 2006-03-14 00:34:13.000000000 +0100 > +++ syslog-ng-1.6.10-ratz/src/cfg-grammar.y 2006-04-26 22:58:20.000000000 +0200 > @@ -631,6 +631,7 @@ > | KW_USE_DNS '(' yesno ')' { configuration->use_dns = $3; } > | KW_TIME_REOPEN '(' NUMBER ')' { configuration->time_reopen = $3; } > | KW_TIME_REAP '(' NUMBER ')' { configuration->time_reap = $3; } > + | KW_TIME_SLEEP '(' NUMBER ')' { configuration->time_sleep = $3; } > | KW_LOG_FIFO_SIZE '(' NUMBER ')' { configuration->log_fifo_size = $3; } > | KW_GC_BUSY_THRESHOLD '(' NUMBER ')' { configuration->gc_busy_threshold = $3; } > | KW_GC_IDLE_THRESHOLD '(' NUMBER ')' { configuration->gc_idle_threshold = $3; } And attached is the better version for Bazsi to apply if he's inclined to do so. This is about the third time I've chased down a parser-based bug, so I wonder if using (f)lex/yacc for this really pays off ;)? Considering my state of mind, I reckon I could be really off the track with this patch as well ... Cheers, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc

2 1

RE: [syslog-ng] time_sleep()
by Vaibhav Goel 30 Apr '06

30 Apr '06

I can't post the whole thing since its too big but I am posting the relevant portion of it: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (on); use_dns (yes); use_fqdn (no); create_dirs (no); owner(mdirect); group(mdirect); keep_hostname (yes); log_msg_size(20480); }; The rest of the file is just a bunch of source, destination, filter and log entries for various stuff Also: [root@clxd01md syslog-ng]# strings /sbin/syslog-ng | grep time_sleep time_sleep So looks like there could be some time_sleep code in the binary at least... -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of John Morrissey Sent: April 25, 2006 1:48 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] time_sleep() On Tue, Apr 25, 2006 at 01:00:17PM -0700, Vaibhav Goel wrote: > I did exactly that... > > [root@clxd01md syslog-ng]# /etc/init.d/syslog-ng restart > Shutting down system logger: [ OK ] > Starting system logger: syntax error at 20 > Parse error reading configuration file, exiting. (line 20) > [FAILED] > [root@clxd01md syslog-ng]# cat /etc/syslog-ng/syslog-ng.conf | grep time > time_reopen (10); > time_sleep(30); > > [root@clxd01md syslog-ng]# /sbin/syslog-ng -V > syslog-ng 1.6.10 > > > Am I doing something wrong? Strange. I backported the time_sleep change to the Debian package I'm running (1.6.9) so I can't speak for the pristine 1.6.10 source, but it's been working fine for us. I glanced at the 1.6.10 tarball and see some time_sleep code there. Could you post your entire syslog-ng.conf? john -- John Morrissey _o /\ ---- __o jwm(a)horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

3 2

RE: [syslog-ng] HUP does not work
by Lumir Unzeitig (DHL CZ) 30 Apr '06

30 Apr '06

Piece of command output strace -s 256 -o /tmp/output-file -p 4893 --------------------------------------------- poll([{fd=29, events=0}, {fd=17, events=0}, {fd=16, events=0}, {fd=18, events=0}, {fd=15, events=POLLOUT, revents=POLLOUT}, {fd=14, events=0}, {fd=13, events=0}, {fd=4, events=POLLOUT, revents=POLLOUT}, {fd=12, events=POLLIN}, {fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 17, 100) = 2 recv(15, 0xbfffaf7b, 1, MSG_PEEK|MSG_DONTWAIT) = -1 ENOTSOCK (Socket operation on non-socket) write(15, "Apr 20 16:56:38 czchols020 sendmail[13901]: k3KEuWpW013859: to=<mrk(a)erwerwerwwer.com>, delay=00:00:04, xdelay=00:00:03, mailer=esmtp, pri=88225, relay=mrucontsv00001.mru-co.mu.asdasda.cz. [152.141.145.16], dsn=2.0.0, stat=Sent (mrucontsv00001.mru-co.mu.uuuu"..., 293) = 293 recv(4, 0xbfffaf7b, 1, MSG_PEEK|MSG_DONTWAIT) = -1 ENOTSOCK (Socket operation on non-socket) write(4, "Apr 20 16:56:38 czchols020 sendmail[13901]: k3KEuWpW013859: to=< mrk(a)erwerwerwwer.com >, delay=00:00:04, xdelay=00:00:03, mailer=esmtp, pri=88225, relay=mrucontsv00001.mru-co.mu.asdasda.cz. [152.141.145.16], dsn=2.0.0, stat=Sent (mrucontsv00001.mru-co.mu.uuuu"..., 293) = 293 time(NULL) = 1145544998 poll([{fd=29, events=0}, {fd=17, events=0}, {fd=16, events=0}, {fd=18, events=0}, {fd=15, events=0}, {fd=14, events=0}, {fd=13, events=0}, {fd=4, events=0}, {fd=12, events=POLLIN, revents=POLLIN}, {fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 17, 100) = 1 recvfrom(12, "<22>sendmail[13954]: k3KEub8A013945: to=<werwerw(a)asdasdasd.cz>, delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=43032, relay=smtp-out-vip.asdasda.cz [85.72.196.13], dsn=2.0.0, stat=Sent (OK - Data received)\n", 2048, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("2.252.103.89")}, [16]) = 215 time(NULL) = 1145544998 time(NULL) = 1145544998 time(NULL) = 1145544998 time(NULL) = 1145544998 time(NULL) = 1145544998 poll([{fd=29, events=0}, {fd=17, events=0}, {fd=16, events=0}, {fd=18, events=0}, {fd=15, events=POLLOUT, revents=POLLOUT}, {fd=14, events=0}, {fd=13, events=0}, {fd=4, events=POLLOUT, revents=POLLOUT}, {fd=12, events=POLLIN}, {fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 17, 100) = 2 recv(15, 0xbfffaf7b, 1, MSG_PEEK|MSG_DONTWAIT) = -1 ENOTSOCK (Socket operation on non-socket) write(15, "Apr 20 16:56:38 czchols020 sendmail[13954]: k3KEub8A013945: to=<rrr(a)rrrr.co.uk>, delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=43032, relay=smtp-out-vip.asdasda.cz [85.72.196.13], dsn=2.0.0, stat=Sent (OK - Data received)\n", 238) = 238 recv(4, 0xbfffaf7b, 1, MSG_PEEK|MSG_DONTWAIT) = -1 ENOTSOCK (Socket operation on non-socket) write(4, "Apr 20 16:56:38 czchols020 sendmail[13954]: k3KEub8A013945: to=<sarah.queen(a)city-link.co.uk>, delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=43032, relay=smtp-out-vip.asdasda.cz [85.72.196.13], dsn=2.0.0, stat=Sent (OK - Data received)\n", 238) = 238 time(NULL) = 1145544998 poll([{fd=29, events=0}, {fd=17, events=0}, {fd=16, events=0}, {fd=18, events=0}, {fd=15, events=0}, {fd=14, events=0}, {fd=13, events=0}, {fd=4, events=0}, {fd=12, events=POLLIN}, {fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 17, 100) = -1 EINTR (Interrupted system call) --- SIGHUP (Hangup) @ 0 (0) --- rt_sigaction(SIGHUP, {0x804a7b0, [HUP], SA_RESTORER|SA_RESTART, 0xaa20d8}, {0x804a7b0, [HUP], SA_RESTORER|SA_RESTART, 0xaa20d8}, 8) = 0 sigreturn() = ? (mask now []) open("/etc/resolv.conf", O_RDONLY) = 19 fstat64(19, {st_mode=S_IFREG|0644, st_size=52, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb75e3000 read(19, "search prg-dc.asdasda.cz asdasda.cz\n\nnameserver 127.0.0.1\n", 1024) = 52 read(19, "", 1024) = 0 close(19) = 0 munmap(0xb75e3000, 4096) = 0 write(2, "SIGHUP received, restarting syslog-ng\n", 38) = 38 time(NULL) = 1145544998 time(NULL) = 1145544998 poll([{fd=29, events=0}, {fd=17, events=0}, {fd=16, events=0}, {fd=18, events=0}, {fd=15, events=0}, {fd=14, events=0}, {fd=13, events=0}, {fd=4, events=0}, {fd=12, events=POLLIN, revents=POLLIN}, {fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 17, 100) = 1 recvfrom(12, "<22>sendmail[13950]: k3KEucD1013950: from=<gematracing(a)asdasda.cz>, size=747, class=0, nrcpts=1, msgid=<5228949.1145544998753.JavaMail.weblogic(a)czhs0119.prg-dc.asdasda.c z>, proto=ESMTP, daemon=MTA, relay=localhost.localdomain [127.0.0.1]\n", 2048, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("2.252.103.89")}, [16]) = 232 time(NULL) = 1145544998 time(NULL) = 1145544998 time(NULL) = 1145544998 time(NULL) = 1145544998 --------------------------------- >And an idea, isn't it postfix that stops logging? No, it is remotely logged sendmail 8.12.11. -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Thursday, April 20, 2006 3:55 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] HUP does not work On Thu, 2006-04-20 at 13:47 +0200, Lumir Unzeitig (UUUU CZ) wrote: > The problem is: A central syslog server logs huge number of maillog > messages from a server. HUP signal is sent to the syslog-ng in > postrotate instructions when logrotate runs. All log files are rotated > properly except maillog. The old maillog messages are in the proper > file, but no new file is created and thus nothing is logged for the > facility . When I stop/start syslog-ng daemon it's working good. > We would need some more information about syslog-ng when in this bad state. Please attach to the syslog-ng process with strace right before sending a HUP signal and wait till some mail log is generated. Use the following strace command: strace -s 256 -o /tmp/output-file -p <syslog-ng pid> And an idea, isn't it postfix that stops logging? -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1