syslog-ng 2.0.3 is released
Dear syslog-ng users, I've uploaded syslog-ng 2.0.3, the latest stable release in the syslog-ng 2.0 branch to our webserver. The list of changes is summarized below. The source is available at the usual places at http://www.balabit.com/products/syslog-ng/ 2.0.3 Mon, 26 Mar 2007 14:10:00 +0200 Bugfixes: * Fixed a possible segmentation fault in the usertty() destination driver when it was referenced on a log path with disabled flow-control. * Fixed various usertty() functionality problems. * Removed the 1024 byte limitation of internally generated messages. * Clarified the text of various internal messages, consistency updates for message severities. * Increased the minimum value for log_fifo_size to 1000 elements. * HP-UX portability fixes. * Some minor documentation fixes. Features: * Added spoof-source support for IPv4 & IPv6 to match 1.6.x functionality. * Added support for remembering the last posistion of file sources across restarts. * Added tcp-wrapper support. * Added support for more than 9 regexp matches. -- Bazsi
Under linux (redhat AS 4) if I statically link without spoofsource I can the following build error % ./configure --enable-static-linking --enable-spoof-source=no % make [snip] /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../libwrap.a(hosts_access.o)(.text+0xc08): In function `host_match': : undefined reference to `yp_get_default_domain' collect2: ld returned 1 exit status glibc is a little old on this machine glibc-2.3.4-2.25 glibc-headers-2.3.4-2.25 glibc-devel-2.3.4-2.25 but that is what Redhas ships, so I would rather not update it. The question is "Why does a static link require a yellow pages routine?" Evan Rempel. Balazs Scheidler wrote:
Dear syslog-ng users,
I've uploaded syslog-ng 2.0.3, the latest stable release in the syslog-ng 2.0 branch to our webserver. The list of changes is summarized below.
The source is available at the usual places at http://www.balabit.com/products/syslog-ng/
With the following configuration source local { unix-stream("/dev/log" max-connections(200)); file("/proc/kmsg" log_prefix("kernel: ")); internal(); }; template standard_file { template("$ISODATE $FULLHOST $FACILITY.$LEVEL $PRI $MESSAGE\n"); template_escape(no); }; destination auth.log { file("/var/syslog/auth.log.$YEAR$MONTH$DAY.000000" perm(0640) template(standard_file) ); }; destination unknown.log { file("/var/syslog/unknown.log.$YEAR$MONTH$DAY.000000" perm(0640) template(standard_file) ); }; destination test.log { file("/var/syslog/test.log.$YEAR$MONTH$DAY.000000" perm(0640) template(standard_file) ); }; filter f_auth { facility(auth); }; filter f_local1 { facility(local1); }; filter f_ldap { program(^slapd); }; # ****** PROBLEM LOG LINE **** log { source(local); filter(f_local1); filter(f_ldap); destination(test.log); }; # ****** PROBLEM LOG LINE **** log { source(local); filter(f_auth); destination(auth.log); }; # safegaurd to catch anything missed by other filters log { source(local); destination(unknown.log); flags(fallback); }; all of the log messages with the facility of auth go into the unknown.log file. If I remove the problematic log line (to test.log) or change the order of its filters or change the first filter to be other than f_local1 (even changing the f_local1 filter to match a different facility) then everything works as expected. All of these work fine # ----------------------------------------------------- filter f_auth { facility(auth); }; filter f_local1 { facility(local1); }; filter f_ldap { program(slapd); }; log { source(local); filter(f_auth); filter(f_ldap); destination(test.log); }; # ----------------------------------------------------- filter f_auth { facility(auth); }; filter f_local1 { facility(local1); }; filter f_ldap { program(slapd); }; # log { source(local); filter(f_local1); filter(f_ldap); destination(test.log); }; # ----------------------------------------------------- filter f_auth { facility(auth); }; filter f_local1 { facility(local0); }; filter f_ldap { program(^slapd); }; log { source(local); filter(f_local1); filter(f_ldap); destination(test.log); }; # ----------------------------------------------------- filter f_auth { facility(auth); }; filter f_local1 { facility(local1); }; filter f_ldap { program(^slapd); }; filter f_test { facility(local1) and program(^sldapd); }; log { source(local); filter(f_test); destination(test.log); }; So, how is the first example going wrong? Evan.
We were running syslog-ng with default setting for time_sleep (which is 0) and were consuming approx 50% of a single CPU with syslog-ng. We were concerned about scalability because syslog-ng is single threaded, so we started tuning with time_sleep. We set it to 5 and adjusted related parameters (log_fetch_limit(250), log_fifo_size(1000000)) and now we are seeing a sawtooth CPU usage with a small IO wait (2%) constant on top of the "user" cpu sawtooth. Tooth cpu duration 1 1-10% 25 minutes 2 2-10% 10 minutes 3 3-10% 10 minutes 4 4-10% 10 minutes This pattern repeats. Any ideas on why the sawtooth pattern started appearing only after we adjusted these settings? Just for the curious Redhad 3 - build from source.
On Tue, 2007-11-13 at 13:04 -0800, Evan Rempel wrote:
We were running syslog-ng with default setting for time_sleep (which is 0) and were consuming approx 50% of a single CPU with syslog-ng. We were concerned about scalability because syslog-ng is single threaded, so we started tuning with time_sleep. We set it to 5 and adjusted related parameters (log_fetch_limit(250), log_fifo_size(1000000)) and now we are seeing a sawtooth CPU usage with a small IO wait (2%) constant on top of the "user" cpu sawtooth.
Tooth cpu duration
1 1-10% 25 minutes 2 2-10% 10 minutes 3 3-10% 10 minutes 4 4-10% 10 minutes
This pattern repeats.
Any ideas on why the sawtooth pattern started appearing only after we adjusted these settings?
Could you post an mrtg image? I guess you were gathering this input from an image of some kind, and it's very difficult to understand without actually seeing the picture. You know a picture is worth thousand words :) which applies to this case very nicely. You can send it to me in private, if you think it contains sensitive information. -- Bazsi
Balazs Scheidler wrote:
On Tue, 2007-11-13 at 13:04 -0800, Evan Rempel wrote:
We were running syslog-ng with default setting for time_sleep (which is 0) and were consuming approx 50% of a single CPU with syslog-ng. We were concerned about scalability because syslog-ng is single threaded, so we started tuning with time_sleep. We set it to 5 and adjusted related parameters (log_fetch_limit(250), log_fifo_size(1000000)) and now we are seeing a sawtooth CPU usage with a small IO wait (2%) constant on top of the "user" cpu sawtooth.
Tooth cpu duration
1 1-10% 25 minutes 2 2-10% 10 minutes 3 3-10% 10 minutes 4 4-10% 10 minutes
This pattern repeats.
Any ideas on why the sawtooth pattern started appearing only after we adjusted these settings?
Could you post an mrtg image? I guess you were gathering this input from an image of some kind, and it's very difficult to understand without actually seeing the picture.
You know a picture is worth thousand words :) which applies to this case very nicely.
You can send it to me in private, if you think it contains sensitive information.
Here are two images, one is of the CPU on the system, and the other is the count of the syslog messages per minute. Don't spend too much time on it because we have another system that is getting the same copy of syslog messages and it does not show this pattern. The challenge for me is that the other system is a lot faster, so it basically runs at 1% usage, and it is running Redhat 4. Additionally, the system I am asking the question about is routing message streams to program destinations that may be doing some kind of buffering and processing that could explain the cpu pattern. Watching "top" when this is going on shows that it is the primary syslog-ng process (the one accepting messages from the network) that is using the cpu, but I'm sill not 100% sure that there isn't some other relationship. The question is more of "have you seen this before?" or "is this expected behavior?". Evan.
On Wed, 2007-03-28 at 12:01 -0700, Evan Rempel wrote:
Under linux (redhat AS 4) if I statically link without spoofsource I can the following build error
% ./configure --enable-static-linking --enable-spoof-source=no % make [snip] /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../libwrap.a(hosts_access.o)(.text+0xc08): In function `host_match': : undefined reference to `yp_get_default_domain' collect2: ld returned 1 exit status
glibc is a little old on this machine glibc-2.3.4-2.25 glibc-headers-2.3.4-2.25 glibc-devel-2.3.4-2.25 but that is what Redhas ships, so I would rather not update it.
The question is "Why does a static link require a yellow pages routine?"
because libwrap depends on it. disable tcp wrapper support. -- Bazsi
Balazs Scheidler wrote:
On Wed, 2007-03-28 at 12:01 -0700, Evan Rempel wrote:
Under linux (redhat AS 4) if I statically link without spoofsource I can the following build error
% ./configure --enable-static-linking --enable-spoof-source=no % make [snip] /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../libwrap.a(hosts_access.o)(.text+0xc08): In function `host_match': : undefined reference to `yp_get_default_domain' collect2: ld returned 1 exit status
glibc is a little old on this machine glibc-2.3.4-2.25 glibc-headers-2.3.4-2.25 glibc-devel-2.3.4-2.25 but that is what Redhas ships, so I would rather not update it.
The question is "Why does a static link require a yellow pages routine?"
because libwrap depends on it. disable tcp wrapper support.
How doest that explain that linking dynamically does not produce this error? -- Evan Rempel erempel@uvic.ca Senior Programmer Analyst 250.721.7691 Computing Services University of Victoria
On Thu, 29 Mar 2007 08:37:00 PDT, Evan Rempel said:
The question is "Why does a static link require a yellow pages routine?"
because libwrap depends on it. disable tcp wrapper support.
How doest that explain that linking dynamically does not produce this error?
Well, if you don't have a usable static-library version of the yellow pages routines, you'll hit that sort of issue...
participants (3)
-
Balazs Scheidler
-
Evan Rempel
-
Valdis.Kletnieks＠vt.edu