May 2006 - syslog-ng - lists.balabit.hu

Starting syslog-ng as root
by Muath Al Khalaf 26 May '06

26 May '06

Hi, I am trying to start syslog-ng as root but failed. The startup script is using daemon. I have tried also by issuing /sbin/syslog-ng but got nothing. I know it is better to run it with user privileges but this is to be able to log kernel messages file("proc/kmsg");

2 1

1.9.11 patch for src/stats.c
by Albert Chin 26 May '06

26 May '06

src/stats.c has the following: gchar *tag_names[SC_TYPE_MAX] = { [SC_TYPE_DROPPED] = "dropped", [SC_TYPE_PROCESSED] = "processed", }; "[SC_TYPE_DROPPED]" is a GCC thing. I guess GCC allows you to index items in an array. But, the vendor compilers from Sun, HP, IBM, etc. don't. -- albert chin (china(a)thewrittenword.com) -- snip snip Index: src/stats.c =================================================================== --- src/stats.c.orig 2006-02-11 04:57:29.000000000 -0600 +++ src/stats.c 2006-05-25 17:14:38.681857000 -0500 @@ -143,8 +143,8 @@ StatsCounterType type; gchar *tag_names[SC_TYPE_MAX] = { - [SC_TYPE_DROPPED] = "dropped", - [SC_TYPE_PROCESSED] = "processed", + "dropped", + "processed" }; e = msg_event_create(EVT_PRI_NOTICE, "Log statistics", NULL);

2 1

Facility / Priority
by Jeremy M. Guthrie 26 May '06

26 May '06

'lo I am the main author of a GPL packaged called Syslog Management Tool(SMT) - smt.dangermen.com. I am currently using Modular Syslog as the syslog package to import from syslog to PostgreSQL. I have found it has issues with about anymore than 70 messages per second. Modular Syslog also looks largely abandoned. Syslog-ng appears to be able to replace the functionality Modular Syslog via the use of pipes. The issue I have found with Syslog-NG is that it does convert the facility & priority/severity to full names when writing. Today I have Modular Syslog writing facility and severity as individual decimal values rather than the aggregate 0-255 decimal(see the table layout below). I was wondering if someone knew of a way to get Syslog-NG to write out the facility/priority values separately? .ie via the syslog-ng.conf filtering/manipulation kung fu? Or does this look like a modification that I will have to make in the .c code? TSyslog=# \d tsyslog Table "public.tsyslog" Column | Type | Modifiers ------------+------------------------+----------------------------------------------------------------- tsyslog_id | bigint | not null default nextval('public.tsyslog_tsyslog_id_seq'::text) facility | integer | severity | integer | date | date | time | time without time zone | host | character varying(128) | message | text | Indexes: "tsyslog_pkey" PRIMARY KEY, btree (tsyslog_id) "host_idx" btree (host) "tsyslhostid_idx" btree (tsyslog_id, host) "tsyslogdatetime_idx" btree (date, "time") -- -------------------------------------------------- Jeremy M. Guthrie jeremy.guthrie(a)berbee.com Senior Network Engineer Phone: 608-298-1061 Berbee Fax: 608-288-3007 5520 Research Park Drive NOC: 608-298-1102 Madison, WI 53711

3 12

RE: [syslog-ng] Syslog-ng dropped packets.
by Vaibhav Goel 26 May '06

26 May '06

We are using 1.6.5 (soon to upgrade to 1.6.11) in a production environment. Is 1.9.11 production ready? I recall a message from you saying that you are looking to release 2.0.0 within a month. I will install 1.9.11 on my QA server and start testing it once we deploy 1.6.11 to prod. I am actually not using /dev/xconsole... I have 2 production loghosts and one of them is more heavily used then the other. I am seeing dropped packets on that one during peak hours. That leads me to believe that syslog-ng. Is there a way to increase the incoming buffer in another way? -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: May 25, 2006 3:51 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng dropped packets. On Wed, 2006-05-24 at 16:35 -0700, Vaibhav Goel wrote: > Recently, I started seeing dropped packets in syslog-ng. I increased > the log_fifo_size to 2000 (it was 1000) and it abated the problem > somewhat. But I am still seeing some dropped packets. Could I > increase the log_fifo_size? Are there any other tips? Unfortunately, > we are going to be adding more logging load to this loghost and I am > worried that this problem might increase. Any ideas? The most important part in addressing this problem is finding out which destination drops packets. syslog-ng 1.6.x only gave a global counter. You might have a slow destination (like /dev/xconsole) whereas your normal log flow works fine. If your primary log output is slow, you could switch to 1.9.11 which includes log message flow control. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

log line processed after flags(final)
by Kevin Lee Phair II 25 May '06

25 May '06

Are there any known situations where the use of flags(final) would still allow the processing of a matched line by log statements which come later in the config? In this case there are three log statements total which a particular line could possibly match; I want it to be processed by the first two statements but not the third. After adding flags(final) to the second log statement, all statements are still being matched.

2 3

Syslog-ng dropped packets.
by Vaibhav Goel 25 May '06

25 May '06

Recently, I started seeing dropped packets in syslog-ng. I increased the log_fifo_size to 2000 (it was 1000) and it abated the problem somewhat. But I am still seeing some dropped packets. Could I increase the log_fifo_size? Are there any other tips? Unfortunately, we are going to be adding more logging load to this loghost and I am worried that this problem might increase. Any ideas? Thanks!

2 1

syslog-ng 1.9.11 released
by Balazs Scheidler 23 May '06

23 May '06

Hi, Syslog-ng 1.9.11 has been released and uploaded to the usual places at http://www.balabit.com/products/syslog-ng/ with the following list of user-visible changes: 1.9.11 Tue, 23 May 2006 19:45:21 +0200 Some minor new features: * Added IPv6 support. * Added multicast support. Bugfixes: * Avoid changing owner/group/permission information for files that do not exist, to avoid clobbering the error value. * Added close-on-exec flag to avoid inheriting file descriptors for programs started by syslog-ng. * Fixed an off-by-one in flush_lines() calculation. * Fixed a possible memory leak. * Fixed a CPU spinning possibility when a non-existing filter is referenced on the log path the processes internal() messages. * Fixed hanging on reading /proc/kmsg on Linux. * Fixed sun-streams() fetching problem on Solaris. Other changes: * Documentation updates. I would like to release 2.0.0 real soon now, for this to happen I would need some positive feedback on this release. Anyone using the 1.9.x series and especially 1.9.11, please speak up. People on non-Linux platforms, now is the time to submit bugreports on portability issues. If nothing major shows up, I am going to release 2.0.0 in about a months time and obsolete 1.6.x once and for all. Thanks in advance, -- Bazsi

1 0

Syslog-ng + Freebsd
by Stefan Lind 23 May '06

23 May '06

I have a problem with syslog-ng It hangs every now and then, When it hangs I have to "kill -9" and then start again. I know this is a bad description do you know what I can do to find the problem? Mvh Stefan Lind 0520-500525 Leissner Data AB

2 1

filter set but still logged to mySQL
by Dennis Kavadas 23 May '06

23 May '06

Hi all I have a question, or rather a problem I've written a filter to trap entries from our BIND host that sends the output to /de/null (see below) I can send the match to any file or to /dev/null but syslog-ng is still seeing the entry from the host and is logging it to "destination d_mysql " I just need to filter out the match so it never reaches "destination d_mysql" Can anyone offer any advice ? my syslog-ng file below.... -------------------------------------- options { long_hostnames(off); sync(0); stats(43200); dns_cache(yes); use_fqdn(no); keep_hostname(yes); use_dns(yes); }; source shell { internal(); unix-stream("/dev/log"); udp(ip( my.syslog-ng.host.com)) }; destination err-update-dns { file("/dev/null"); }; filter err-update-dns { match("(59.167.232.229)"); }; log {source(shell); filter(err-update-dns); destination(err-update-dns); }; destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(shell); destination(d_mysql); };

2 2

syslog-ng stalls system
by Mario Aeby 19 May '06

19 May '06

Hi Guys Wow, thanks for all your replies! So at least I'm not the only one having problems. - I'm running Debian testing with (self-compiled *boah*) Kernel 2.4.25 - I replaced syslog-ng with syslog until the issues are fixed - dpkg says I had the following syslog-ng installed: rc syslog-ng 1.9.9-1 Next generation logging daemon Really confusing, since Balazs is talking about 1.6.11 ... Do Debian maintainers follow a different numbering scheme? - My configuration is available as a download at http://mad4you.homeip.net/downloads/syslog-ng.conf . Note: It cannot be caused by this, since I soon replaced syslog-ng.conf with the one the original Debian package provided (to rule out exactly such a cause) Many thanks! Kind regards Mario -- Mario Aeby Stritenstrasse 47 CH-3176 Neuenegg Mobile 078 690 50 11 Fax 031 741 28 93 http://www.eMeidi.com/

2 1