March 2004 - syslog-ng - lists.balabit.hu

template() buggy for program()?
by Loic Minier 22 Mar '04

22 Mar '04

Hi, My Perl program launched by syslog-ng with a program() destination receives logs in this form: "<182>Mar 19 14:35:00 met-ixb postfix/smtpd[3234]: connect from l ocalhost[127.0.0.1]" (ie the log starts with '<', '1', '8', '2', '>'.) The *same* log line appears as: "Mar 19 14:35:00 hostname postfix/smtpd[3234]: connect from localhost[127.0.0.1]" in a file destination. My config looks like: destination dp_parse { program("/path/to/perl/program" template("$DATE $MSG\n")); }; destination df_tech { file("/path/to/file/tech.log" template("$DATE $HOST $MSG\n")); }; I am confused about this behavior, is it normal for a program() output to receive a syslog header? -- Loïc Minier <lool(a)dooz.org>

2 4

Encrypted document
by entity73＠earthlink.net 21 Mar '04

21 Mar '04

1 0

destination sockets
by Porkchop 20 Mar '04

20 Mar '04

I saw an earlier thread about syslog-ng not creating/bind()ing/socket()ing sockets when it was generating messages... it never seemed quite resolved. I am attempting to create an outbound socket for syslog-ng to write to and that multiple clients (servers?) can read from. I am attempting to: destination d-plog { unix-stream("/tmp/d-plog.sock" template("$HOST $HOUR $MIN $SEC $FACILITY $PRIORITY $PROGRAM $MSG\n") template_escape(no) ); }; I'd then have multiple clients reading /tmp/d-plog.sock. Can't use a pipe (multiple clients). Can't use a file (log storage is done in SQL). Of course... root@lostcause:/home/porkchop# syslog-ng -v binding fd 5, unixaddr: /dev/log binding fd 6, inetaddr: 10.10.10.1, port: 514 binding fd 7, inetaddr: 0.0.0.0, port: 514 connecting fd 8, unixaddr: /tmp/d-porklog.sock Error connecting to AF_UNIX(/tmp/d-porklog.sock): Bad file descriptor Error initializing configuration, exiting. root@lostcause:/home/porkchop# Am I missing something? -porkchop

2 1

File permissions not being honored
by Chris Josephes 20 Mar '04

20 Mar '04

I have the following destination directive: destination d_user { file("/var/log/user" perm(0644)); }; But, syslog-ng sets the file permissions to 0600. I'm using version 1.4.17 Is my directive formatted incorrectly? -------------------- Christopher Josephes cpj1(a)visi.com

2 1

Massive lossage with syslog-ng
by donw＠examen.com 20 Mar '04

20 Mar '04

My apologies if this has been discussed, as I haven't been able to find anything useful via google or the FM... Syoposis: Syslog-ng drops something on the order of 90% of the logs remotely flung at it. Detail: I'm using syslog-ng 1.5.15 from the Debian stable package archive. I've been tasked with setting up two remote log servers for my employer; both log servers have fairly beefy IDE raids (IOZone gives me an unbuffered write speed of about 40M/sec), and as far as I can tell with vmstat(8) and Our Friend top(1), syslog-ng isn't running into any I/O bottlenecks. The systems have insane CPUs (Athlon XP 2000) and 512M RAM a pop, which considering their intended tasks (syslog and serial console server), should be more than adequate. I've set up several systems to push their logs onto the log server; a Solaris 8 (with native syslog) box, a Debian Linux box (native syslog again), and a Debian Linux box using syslog-ng. Yet syslog-ng seems to dump between 75% and 90% of all the logs handed to it on the floor. Several of the systems we wish to have logging to syslog will throw out about a thousand lines within a second or two, and this is where most of our problems come in. I've tried both TCP and UDP; TCP barely seems to work at all -- I can throw a thousand lines in on one end, see them in the logfile, and see one line of output out on the other side. I've futzed a bit with the FIFO size and the gc_idle/gc_busy numbers, but the latter two are more-or-less undocumented in the manual, so I have no idea what they really do (no units or anything), and the code regarding them is a bit too complex for me to read. I've been googling and chomping upon the syslog-ng documentation for about the past two weeks, and have yet to find a solution. Any suggestions? An FM for me to read? A USENET article I missed? Thanks-in-advance! -- Don Werve <donw(a)examen.com> (Unix System Administrator) Yorn desh born, der ritt de gitt der gue, Orn desh, dee born desh, de umn bork! bork! bork!

2 1

Auditd
by Coward, Robert (Contractor) 19 Mar '04

19 Mar '04

Is it possible to log auditd information using syslog-ng?

1 0

Re: sysnlog-ng lock whole machine ...
by Pawel Dziekonski 18 Mar '04

18 Mar '04

On śro, 17 mar 2004 at 08:25:57 -0800, Nate Campi wrote: > On Wed, Mar 17, 2004 at 02:32:49PM +0100, Pawel Dziekonski wrote: > > > > 1st: please confirm how do I log kernel messages (assuming klogd is NOT > > running): > > I use this pipe("/proc/kmsg"); more answeres? also, should it be: source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); }; or: source src { unix-dgram("/dev/log"); internal(); pipe("/proc/kmsg"); }; stream or dgram ??? I'm asking those questions, bacause different documentation sources say different things fir linux, or debian specyfically. > > 2nd: the machine "hangs" from time to time - cron is hanging, login is > > not spawning shell, sshd too. i have pinpointed it so i'm pretty sure > > that all those processes hang on socket connection to /dev/log. killing > > syslog-ng cures that. > > I am also a debian user, I don't know if this problem is specific to > debian or not, but I used to have lockups all the time when two things > would happen at once: syslog log rotation and postfix reloads. The HUP > of syslog-ng when rotating logs and postfix writing to /dev/log would > hang up the whole machine, cron processes would pile up and eventually > you couldn't even log into the machine, presumably because syscalls for > logging would hang. If I already had a shell and killed postfix most of > the processes would un-freeze, but cron would never recover. I always > had to reboot or power cycle if init got screwed. this is it!! please, see also (old bug, what the same problem): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=95480 regrads, P -- Pawel Dziekonski <pawel.dziekonski|(a)|pwr.wroc.pl>, KDM WCSS avatar:0:0: Wroclaw Networking & Supercomputing Center, HPC Department -> See message headers for privacy policy and S/MIME info.

2 1

sysnlog-ng lock whole machine ...
by Pawel Dziekonski 17 Mar '04

17 Mar '04

hi, the machine is Linux 2.4.25 on i686, debian woody. it's a log-host for about 30 other machines also. 1st: please confirm how do I log kernel messages (assuming klogd is NOT running): with this: source ksrc { file("/proc/kmsg"); }; or this: source ksrc { pipe("/proc/kmsg"); }; 2nd: the machine "hangs" from time to time - cron is hanging, login is not spawning shell, sshd too. i have pinpointed it so i'm pretty sure that all those processes hang on socket connection to /dev/log. killing syslog-ng cures that. what can I do? thanks in advance, P PS. my config: options { long_hostnames(off); sync(0)}; source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); }; source net { udp(); }; filter f_auth { facility(auth, authpriv) and not match("cron"); }; filter f_NOauth { not facility(auth, authpriv); }; filter f_cron { facility(cron) or match("cron"); }; filter f_NOcron { not facility(cron) and not match("cron"); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mailnews { facility(mail, news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter p_debug { level(debug); }; filter p_info { level(info); }; filter p_notice { level(notice); }; filter p_warn { level(warn); }; filter p_err { level(err); }; filter p_crit { level(crit); }; filter p_alert { level(alert); }; filter p_emergency { level(emerg); }; filter f_all { level(debug..emerg); }; filter f_iptables { facility(kern) and match("IN=[A-Za-z0-9]* OUT=[A-Za-z0-9]*"); }; filter f_NOiptables { not match("IN=[A-Za-z0-9]* OUT=[A-Za-z0-9]*"); }; filter f_named { level(debug..emerg) and match("named"); }; filter f_NOnamed { not match("named"); }; filter f_dhcpd { level(debug..emerg) and match("dhcpd"); }; filter f_NOdhcpd { not match("dhcpd"); }; destination authlog { file("/var/log/syslog/auth.log"); }; destination cron { file("/var/log/syslog/cron.log"); }; destination syslog { file("/var/log/syslog/syslog.log"); }; destination iptables { file("/var/log/syslog/iptables.log"); }; destination named { file("/var/log/syslog/named.log"); }; destination dhcpd { file("/var/log/syslog/dhcpd.log"); }; destination tty11 { file("/dev/tty11"); }; destination tty12 { file("/dev/tty12"); }; log { source(src); filter(f_auth); destination(authlog); }; log { source(src); filter(f_auth); destination(tty11); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_all); filter(f_NOcron); filter(f_NOauth); filter(f_NOiptables); filter(f_NOnamed); filter(f_NOdhcpd); destination(syslog); }; log { source(src); filter(f_all); filter(f_NOcron); filter(f_NOauth); filter(f_NOiptables); filter(f_NOnamed); filter(f_NOdhcpd); destination(tty12); }; log { source(src); filter(f_iptables); destination(iptables); }; log { source(src); filter(f_named); destination(named); }; log { source(src); filter(f_dhcpd); destination(dhcpd); }; filter f_pix { host("pix"); }; destination pix { file("/var/log/syslog/pix.log"); }; log { source(net); filter(f_pix); destination(pix); }; filter f_lit { host("lit"); }; destination lit { file("/var/log/syslog/lit.log"); }; log { source(net); filter(f_lit); destination(lit); }; filter f_beryl { host("beryl"); }; destination beryl { file("/var/log/syslog/beryl.log"); }; log { source(net); filter(f_beryl); destination(beryl); }; filter f_bor { host("bor"); }; destination bor { file("/var/log/syslog/bor.log"); }; log { source(net); filter(f_bor); destination(bor); }; filter f_wegiel { host("wegiel"); }; destination wegiel { file("/var/log/syslog/wegiel.log"); }; log { source(net); filter(f_wegiel); destination(wegiel); }; filter f_azot { host("azot"); }; destination azot { file("/var/log/syslog/azot.log"); }; log { source(net); filter(f_azot); destination(azot); }; filter f_tlen { host("tlen"); }; destination tlen { file("/var/log/syslog/tlen.log"); }; log { source(net); filter(f_tlen); destination(tlen); }; filter f_fluor { host("fluor"); }; destination fluor { file("/var/log/syslog/fluor.log"); }; log { source(net); filter(f_fluor); destination(fluor); }; filter f_neon { host("neon"); }; destination neon { file("/var/log/syslog/neon.log"); }; log { source(net); filter(f_neon); destination(neon); }; filter f_hel { host("hel"); }; destination hel { file("/var/log/syslog/hel.log"); }; log { source(net); filter(f_hel); destination(hel); }; filter f_sod { host("sod"); }; destination sod { file("/var/log/syslog/sod.log"); }; log { source(net); filter(f_sod); destination(sod); }; filter f_magnez { host("magnez"); }; destination magnez { file("/var/log/syslog/magnez.log"); }; log { source(net); filter(f_magnez); destination(magnez); }; filter f_glin { host("glin"); }; destination glin { file("/var/log/syslog/glin.log"); }; log { source(net); filter(f_glin); destination(glin); }; filter f_krzem { host("krzem"); }; destination krzem { file("/var/log/syslog/krzem.log"); }; log { source(net); filter(f_krzem); destination(krzem); }; filter f_fosfor { host("fosfor"); }; destination fosfor { file("/var/log/syslog/fosfor.log"); }; log { source(net); filter(f_fosfor); destination(fosfor); }; filter f_siarka { host("siarka"); }; destination siarka { file("/var/log/syslog/siarka.log"); }; log { source(net); filter(f_siarka); destination(siarka); }; filter f_chlor { host("chlor"); }; destination chlor { file("/var/log/syslog/chlor.log"); }; log { source(net); filter(f_chlor); destination(chlor); }; filter f_argon { host("argon"); }; destination argon { file("/var/log/syslog/argon.log"); }; log { source(net); filter(f_argon); destination(argon); }; filter f_potas { host("potas"); }; destination potas { file("/var/log/syslog/potas.log"); }; log { source(net); filter(f_potas); destination(potas); }; filter f_wapn { host("wapn"); }; destination wapn { file("/var/log/syslog/wapn.log"); }; log { source(net); filter(f_wapn); destination(wapn); }; filter f_gal { host("gal"); }; destination gal { file("/var/log/syslog/gal.log"); }; log { source(net); filter(f_gal); destination(gal); }; filter f_german { host("german"); }; destination german { file("/var/log/syslog/german.log"); }; log { source(net); filter(f_german); destination(german); }; filter f_arsen { host("arsen"); }; destination arsen { file("/var/log/syslog/arsen.log"); }; log { source(net); filter(f_arsen); destination(arsen); }; filter f_selen { host("selen"); }; destination selen { file("/var/log/syslog/selen.log"); }; log { source(net); filter(f_selen); destination(selen); }; filter f_brom { host("brom"); }; destination brom { file("/var/log/syslog/brom.log"); }; log { source(net); filter(f_brom); destination(brom); }; filter f_krypton { host("krypton"); }; destination krypton { file("/var/log/syslog/krypton.log"); }; log { source(net); filter(f_krypton); destination(krypton); }; filter f_rubid { host("rubid"); }; destination rubid { file("/var/log/syslog/rubid.log"); }; log { source(net); filter(f_rubid); destination(rubid); }; filter f_stront { host("stront"); }; destination stront { file("/var/log/syslog/stront.log"); }; log { source(net); filter(f_stront); destination(stront); }; filter f_ind { host("ind"); }; destination ind { file("/var/log/syslog/ind.log"); }; log { source(net); filter(f_ind); destination(ind); }; filter f_cyna { host("cyna"); }; destination cyna { file("/var/log/syslog/cyna.log"); }; log { source(net); filter(f_cyna); destination(cyna); }; filter f_antymon { host("antymon"); }; destination antymon { file("/var/log/syslog/antymon.log"); }; log { source(net); filter(f_antymon); destination(antymon); }; filter f_tellur { host("tellur"); }; destination tellur { file("/var/log/syslog/tellur.log"); }; log { source(net); filter(f_tellur); destination(tellur); }; -- Pawel Dziekonski <pawel.dziekonski|(a)|pwr.wroc.pl>, KDM WCSS avatar:0:0: Wroclaw Networking & Supercomputing Center, HPC Department -> See message headers for privacy policy and S/MIME info.

2 1

libol: configure.in and make_class
by José Pedro Oliveira 16 Mar '04

16 Mar '04

I have a couple of comments/questions about libol. 1) configure ignores the CFLAGS enviroment variable. This affects packagers, in particular, the ones making specfiles (%configure defines CFLAGS as "$RPM_OPT_FLAGS"). I believe the correct solution is to patch configure.in. Something like this ---------- @@ -106,7 +106,7 @@ if test $enable_debug = "yes"; then CFLAGS="-Wall -g" else - CFLAGS="-Wall -O2" + CFLAGS="$CFLAGS -Wall" fi ---------- appears to work (but I'm not an autotools expert). 2) the libol makefile install the file make_class in the directory /usr/bin. Is this file required for building syslog-ng? The main reason I asking this is that make_file is installed with no interpreter. The first three lines are: ---------- #! \ -e main -s !# ---------- The original file - make_class.in - refers to something like "_SCSH_" but this gets replaced by an empty string. Building system: Red Hat 9 Regards, jpo -- José Pedro Oliveira * mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo * * gpg fingerprint = F9B6 8D87 859D 1C94 48F0 84C0 9749 9EB5 91BD 851B * diff -ruN libol-0.3.13-orig/configure.in libol-0.3.13-modified/configure.in --- libol-0.3.13-orig/configure.in 2004-01-19 13:20:13.000000000 +0000 +++ libol-0.3.13-modified/configure.in 2004-03-16 01:51:56.000000000 +0000 @@ -106,7 +106,7 @@ if test $enable_debug = "yes"; then CFLAGS="-Wall -g" else - CFLAGS="-Wall -O2" + CFLAGS="$CFLAGS -Wall" fi MAKE_CLASS="$srcdir/utils/make_class"

1 0

mysql pipe not working!
by TIM MOORE 11 Mar '04

11 Mar '04

I setup syslog-ng to pipe to a fifo (/tmp/mysql.pipe). I copied and pasted the configs exactly as was given and no data is written to the pipe. It always remains "0" in size. I have since tried the config in many different ways, but still cannot get the data to pipe to the file. Is there something wrong with the config? Has anyone else run into this problem? Here is a copy of my config. root@syslog1 ~ # cat /etc/syslog-ng/syslog-ng.conf # # Syslog-ng configuration for for ODJFS-NOC Syslog # # Copyright (c) 1999 anonymous # Copyright (c) 1999 Balazs Scheidler # $Id: syslog-ng.conf.sample,v 1.2 1999/11/15 12:30:41 bazsi Exp $ # # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # options { use_fqdn(yes); use_dns(yes); keep_hostname(yes); long_hostnames(off); use_time_recvd(yes); sync(0); }; ###################################### #Source Section #source for local data source src { unix-dgram("/etc/log/log"); internal(); }; #source src { pipe("/proc/kmsg" log_prefix("kernel: ")); # unix-stream("/dev/log"); # internal(); #}; #source for PIX firewalls #source pix { # udp(ip("10.3.250.5") port (514)); # udp(ip("10.x.x.x") port (514)); # udp(ip("10.x.x.x") port (514)); # udp(ip("10.x.x.x") port (514)); # udp(ip("10.x.x.x") port (514)); #}; #source for network devices source net { udp(); }; ##################################### #Destination Section #Standard destination destination std { file("/var/log/$HOST/$FACILITY/$DAY" remove_if_older(172800) owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; ## Log syslog-ng to mysql database ## destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; #log for src log { source(src); destination(std); destination(d_mysql); }; #log for net log { source(net); destination(std); destination(d_mysql); }; ########END OF FILE######## Any ideas? Tim M. ODJFS

1 0