- syslog-ng - lists.balabit.hu

preparing for 2.0.8
by Balazs Scheidler 28 Jan '08

28 Jan '08

Hi, I'm preparing a 2.0.8 release, but I'd like to avoid past mistakes with recent releases, thus I'd be very grateful if some of you could test the current snapshot. There are two patches that I haven't received real feedback yet, the others are so trivial that I doubt they'd cause problems. The candidates that should be tested are below. I've also included a description how I think the patches should be tested, to make it easier for you to perform testing: SIGHUP did not reopen files as needed for logrotation, fixed that (a8b9878ab38b10d24df7b773c8c580d341b22383) * the problem this patch fixes is that syslog-ng did not properly reopen destination files, it kept them opened and wrote into them even if a SIGHUP was sent to it * this was a regression in 2.0.7 and didn't affect earlier versions * to test whether the fix was ok, you should start the new version of syslog-ng, generate a continous stream of messages (for instance with loggen), and send rotate the output files using logrotate, or by hand * if syslog-ng starts a new file after rotation, then the fix was ok * if syslog-ng writes to the new file, or crashes or behaves oddly in any other was, then the fix was bad. and: fixed checking the DNS cache size limit 40f999eaff5fdb217656ab6af03e336e8e703cbf * the problem was related to the support for a persistent set of DNS cache entries (dns-cache-hosts() option) * if the hosts file specified for dns-cache-hosts() contained over dns-cache-size() entries, syslog-ng quickly dumped core right after the reception of the first message * to test this, please enable dns-cache-hosts(), populate the file with over 1007 ip->name mappings * start syslog-ng, generate some messages and check whether: * syslog-ng behaves correctly (no crash, no other odd behaviour) * syslog-ng does not try to resolve the names that are already in the dns-cache, you can check this by adding an entry that has no DNS record and check whether syslog-ng successfully resolves the name (keep_hostname(no), use_dns(yes), dns_cache(persist-only)) I've now checked that the latest snapshot (syslog-ng-2.0.7+20080128.tar.gz) contains both of these patches. Both patches run the regression test suite well, but as you know real world testing is the only thing that matters at the end. I'm going to release 2.0.8 this week, any kind of help on the patches above would help a lot. Thanks in advance. -- Bazsi

1 0

Win syslog-agent PRI
by Tiago Gomes da Silva Mendo 28 Jan '08

28 Jan '08

Hi, I have syslog-ng-premium-edition (2.1.8) on an Debian etch and multiple linuxs sending syslog messages to this server, using diferents PRIs. The problem is with the windows agent (2.1.4). In the windows agent I have an message format like this: "<182>$DATE $HOST $EVENT_SOURCE: $MSG", but at the server the received PRI is not 182. At the server I get messages with the correct PRI when the syslog-ng agent is restarted: "Jan 28 10:57:41 10.176.25.108 LogRelay: Application started", with local6 and info, but every message I send through syslog-ng agent arrives at the server with user/notice (PRI 13). Any ideas?

2 1

[patch] Fixes for 2.0.7 on not ipv6 aware systems
by Corinna Vinschen 28 Jan '08

28 Jan '08

Hi, I'm maintaining syslog-ng as part of the Cygwin net distro. Today I tried to build 2.0.7 and came across a couple of problems, which were mostly due to the fact that the current Cygwin release 1.5.25 is not ipv6 aware. While this is rectified in Cygwin's CVS, the current release is still based on older code. Below you find a patch which solves the following problems: - Two functions in src/afsocket.c use AF_INET6 even if ENABLED_IPV6 is not defined. - The new loggen tool has a typo in usage(), printing -i instead of -I for the interval arg. - loggen has a bug in the option handling. Due to a missing colon in the option string, it crashes when using the -s option. - loggen uses getaddrinfo/freeaddrinfo even on systems on which HAVE_GETADDRINFO is undefined. I created replacement code using gethostbyname/gertservbyname on systems lacking getaddrinfo. ChangeLog and diff below. Another problem I have is this: Given that loggen is in the tests subdir, I'm wondering if it's correct that loggen is installed to ${bindir} when running `make install'. Is that really supposed to happen or should `make install' skip loggen? Thanks, Corinna * src/afsocket.c (afsocket_sd_process_connection): Allow building with tcp_wrappers if AF_INET6 is unknown. (afsocket_dd_format_stats_name): Guard AF_INET6 case with ENABLE_IPV6. * tests/loggen/loggen.c diff -rup syslog-ng-2.0.7/src/afsocket.c syslog-ng-2.0.7-1/src/afsocket.c --- syslog-ng-2.0.7/src/afsocket.c 2007-11-26 09:26:38.000000000 +0100 +++ syslog-ng-2.0.7-1/src/afsocket.c 2008-01-07 18:38:31.088449800 +0100 @@ -359,7 +359,11 @@ gboolean afsocket_sd_process_connection(AFSocketSourceDriver *self, GSockAddr *peer_addr, gint fd) { #if ENABLE_TCP_WRAPPER - if (peer_addr && (peer_addr->sa.sa_family == AF_INET || peer_addr->sa.sa_family == AF_INET6)) + if (peer_addr && (peer_addr->sa.sa_family == AF_INET +#if ENABLE_IPV6 + || peer_addr->sa.sa_family == AF_INET6 +#endif + )) { struct request_info req; @@ -693,9 +697,11 @@ afsocket_dd_format_stats_name(AFSocketDe case AF_INET: driver_name = !!(self->flags & AFSOCKET_STREAM) ? "tcp" : "udp"; break; +#if ENABLE_IPV6 case AF_INET6: driver_name = !!(self->flags & AFSOCKET_STREAM) ? "tcp6" : "udp6"; break; +#endif } g_snprintf(stats_name, sizeof(stats_name), "%s(%s)", diff -rup syslog-ng-2.0.7/tests/loggen/loggen.c syslog-ng-2.0.7-1/tests/loggen/loggen.c --- syslog-ng-2.0.7/tests/loggen/loggen.c 2008-01-03 07:29:56.000000000 +0100 +++ syslog-ng-2.0.7-1/tests/loggen/loggen.c 2008-01-07 19:42:46.720451400 +0100 @@ -147,7 +147,7 @@ usage() " --stream, or -S Use stream socket (TCP and unix-stream)\n" " --dgram, or -D Use datagram socket (UDP and unix-dgram)\n" " --size, or -s Specify the size of the syslog message\n" - " --interval, or -i <sec> Number of seconds to run the test for\n"); + " --interval, or -I <sec> Number of seconds to run the test for\n"); exit(0); } @@ -176,9 +176,9 @@ main(int argc, char *argv[]) int opt; #if HAVE_GETOPT_LONG - while ((opt = getopt_long(argc, argv, "r:I:ixsSDh", syslog_ng_options, NULL)) != -1) + while ((opt = getopt_long(argc, argv, "r:I:ixs:SDh", syslog_ng_options, NULL)) != -1) #else - while ((opt = getopt(argc, argv, "r:I:ixsSDh")) != -1) + while ((opt = getopt(argc, argv, "r:I:ixs:SDh")) != -1) #endif { switch (opt) @@ -211,6 +211,7 @@ main(int argc, char *argv[]) } if (!unix_socket) { +#if HAVE_GETADDRINFO struct addrinfo hints; struct addrinfo *res, *rp; @@ -250,6 +251,53 @@ main(int argc, char *argv[]) fprintf(stderr, "Error connecting to target server: %s\n", strerror(errno)); return 2; } +#else + struct hostent *host; + struct servent *serv = NULL; + struct sockaddr_in saddr; + + if (argc - optind < 2) + { + fprintf(stderr, "No target server specified\n"); + usage(); + } + + host = gethostbyname (argv[optind]); + if (!host) + { + fprintf(stderr, "Name lookup error"); + return 2; + } + memcpy (&saddr.sin_addr, host->h_addr_list[0], host->h_length); + serv = getservbyname (argv[optind + 1], + sock_type == SOCK_STREAM ? "tcp" : "udp"); + if (!serv) + { + char *c; + + saddr.sin_port = htons (strtol (argv[optind + 1], &c, 10)); + if (c == argv[optind + 1]) + { + fprintf(stderr, "Service lookup error"); + return 2; + } + } + else + saddr.sin_port = serv->s_port; + saddr.sin_family = AF_INET; + sock = socket (AF_INET, sock_type, 0); + if (sock != -1 + && connect(sock, (struct sockaddr *) &saddr, sizeof saddr) == -1) + { + close (sock); + sock = -1; + } + if (sock == -1) + { + fprintf(stderr, "Error connecting to target server: %s\n", strerror(errno)); + return 2; + } +#endif } else { -- Corinna Vinschen Cygwin Project Co-Leader Red Hat

2 15

Bug in 2.0.7's use_dns(persist_only) and large host files
by Steve Bernacki 27 Jan '08

27 Jan '08

Environment: CentOS 5 and Syslog-ng 2.0.7 (current snapshot) Relevant syslog-ng.conf snippet: === options { use_dns(persist_only); dns_cache_hosts("/etc/hosts"); }; === If you point syslog-ng to an /etc/hosts file that has more than 1007 entries, upon receiving its first message, syslog-ng will throw the following assertion: # ./syslog-ng -Fed syslog-ng starting up; version='2.0.7' Incoming log entry; line='<38>well hello' file dnscache.c: line 99 (dns_cache_key_hash): should not be reached Aborted Workaround: In syslog-ng.conf, set dns_cache_size() to a value greater than the number of entries in your /etc/hosts file. Fix: See the attached patch; it should apply cleanly to any version of 2.0.7, and perhaps older versions as well. Steve -- Steve Bernacki, Jr To date, the Pan-Massachusetts Challenge has raised 204 million dollars for cancer research. Get involved! http://www.pmc.org/ diff -ur syslog-ng-2.0.7/src/dnscache.c syslog-ng-2.0.7-patch/src/dnscache.c --- syslog-ng-2.0.7/src/dnscache.c 2008-01-03 06:59:18.000000000 -0500 +++ syslog-ng-2.0.7-patch/src/dnscache.c 2008-01-27 09:24:52.000000000 -0500 @@ -271,7 +271,7 @@ } g_hash_table_replace(cache, &entry->key, entry); - if (g_hash_table_size(cache) > dns_cache_size) + if (!persistent && g_hash_table_size(cache) > dns_cache_size) { /* remove oldest element */ g_hash_table_remove(cache, &cache_first.next->key);

2 1

Fail safe test fot syslog-ng
by dstuxo 27 Jan '08

27 Jan '08

Hi everyone, I am using syslog-ng in a system which is supposed to be fail safe to avoid data loss. However I encountered an unexpected delay and messages are lost.. To test it I am proceeding like this: syslog-ng-config: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); } ... destination d_cron { file("/var/log/cron"); }; ... logger -t tester -p cron.info "power failure test" & tail -f /var/log/cron Waiting aprox. 2 seconds then I power down. The previous command shows the message, but after reboot it is lost. I supposed that it's happening due the fact that the buffers are not flushed to disk. So I tried strace on syslog-ng process. No read/write syscall for the message is made.. Another case is with a cycle of messages: for i in `seq 1 1000`;do logger -t tester -p cron.info "power failure test: $i";echo ">> $i";done After 3-4 seconds all messages are written. But not before that time... I know about the fsync(yes) option and I tried it, but with high load of messages It doesn't work proper (syslog-ng.config becomes corrupted and messages are also lost) There is something I am missing here? Configuration option, maybe? Regards

4 9

2.0.7 contrib/rhel-packaging/syslog-ng.conf typos
by Joe Fegan 26 Jan '08

26 Jan '08

Hi, The example syslog-ng.conf in contrib/rhel-packaging of 2.0.7 has some syntax errors and this patch fixes them. I also noticed that this syslog-ng.conf configures syslog-ng to take over logging of messages from remote machines, but not logging of local messages. I think this mix is odd; it should either take over from syslogd altogether or not at all. This patch resolves that mix by commenting out the remote config. The idea is that installing syslog-ng will not interfere with syslogd up front, but will give you hints on how to do it later if you want. Please consider part or all of this patch for upstream. Joe. _________________________________________________________________ Windows Vista + Windows Live. Open up your digital life. Get Windows Live free. http://get.live.com

4 5

integration and release
by Balazs Scheidler 26 Jan '08

26 Jan '08

hi, Sorry for having been able to dedicate only a small portion of my time to syslog-ng, things have been rough recently. I'm going to integrate pending patches in the coming days and think about the more complex issues (time_sleep, disk flush time etc). Thanks for the patience. -- Bazsi

1 0

Syslog-ng too many open files
by Pedro Rafael Alves Simoes 26 Jan '08

26 Jan '08

Hello, I'm receiving many messages on my syslog-ng like this: syslog-ng[20419]: Error opening file for writing; filename="...", error='Too many open files (24)' syslog-ng[20419]: Error accepting new connection; error='Too many open files (24)' In lsof output, the syslog-ng daemon has many files opened and when I try to login from other console or change to root account, it gets blocked. I've increased the sysctl fs.file-max to 300000 but the problem persists. Any ideas what's going wrong?

4 3

Re: [syslog-ng] Syslog-ng too many open files
by Pedro Rafael Alves Simoes 25 Jan '08

25 Jan '08

> ------------------------------ > > Message: 3 > Date: Thu, 24 Jan 2008 12:41:36 +0000 > From: Joe Fegan <joe_fegan(a)hotmail.com> > Subject: Re: [syslog-ng] Syslog-ng too many open files > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <BAY111-W20A021959661E71A0B78C39B380(a)phx.gbl> > Content-Type: text/plain; charset="windows-1256" > > > What version of syslog-ng is this? Is it 2.0.7? > > Could it be the same issue recently reported as "syslog-ng 2.0.7 major problem?" > in the list archives, where syslog-ng 2.0.7 does not close old files after SIGHUP? > 300,000 is a lot of files... > I'm using Syslog-ng 2.0.0-1 in a Debian Etch Linux. > ------------------------------ > > Message: 4 > Date: Thu, 24 Jan 2008 12:46:53 -0000 > From: "Geller, Sandor $IT$" <Sandor.Geller(a)morganstanley.com> > Subject: Re: [syslog-ng] Syslog-ng too many open files > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <14F0A35F6E466D48BF11108F4E09E68C0970D239(a)LNWEXMB58.msad.ms.com> > Content-Type: text/plain; charset="us-ascii" > > Hi, > > > Hello, > > > > I'm receiving many messages on my syslog-ng like this: > > > > syslog-ng[20419]: Error opening file for writing; filename="...", > > error='Too many open files (24)' > > syslog-ng[20419]: Error accepting new connection; error='Too many open > > files (24)' > > Try raising the per-process number of open files. Please refer to > ulimit. > > > In lsof output, the syslog-ng daemon has many files opened and when I > > try to login from other console or change to root account, it gets > > blocked. > > > > I've increased the sysctl fs.file-max to 300000 but the > > problem persists. > > This raises the per-system limit only. per-process limits still > apply as well as the maximum (maybe 4096, I'm not sure) is hardcoded > into the kernel, so without recompiling you r kernel you can't exceed > this limitation. When you said that's hardcoded, you mean that I have to change the source code or it's configurable via make config? > > > Any ideas what's going wrong? > > Without seeing more details (syslog-ng version number, configuration, > approximate number of clients, client configs) we could only guess. > > Regards, > > Sandor > -------------------------------------------------------- > > NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. > >

2 1

increasing number of TCP connections from same number of remote hosts
by Nagy Daniel 25 Jan '08

25 Jan '08

Hi, It's syslog-ng v2.0.7 on RHEL4, compiled from source. The system is a log server, it receives logs via TCP from various clients. After restarting syslog-ng, netstat -t shows that each remote host open only one TCP connection to the server, which is normal. The problem is, that the number of established TCP connections is increasing constantly, but the number of clients is the same. For example netstat -t --numeric-ports | grep fmx23 now shows: tcp 0 0 barapp1:514 fmx23.freemail.privat:52391 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:50852 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:50172 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:59367 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:50979 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:55828 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:53013 ESTABLISHED tcp 0 0 barapp1:514 fmx23.freemail.privat:50038 ESTABLISHED Why is that? AFAIK there should be only one established connection per client. Is it a server or client problem? Thanks, Daniel

2 4