May 2001 - syslog-ng - lists.balabit.hu

Not remotely logging...
by Matt Mencel 24 May '01

24 May '01

Hi all. I'm hoping someone can help me with this because I'm stumped. I'm running 1.4.11 and am trying to send my logs to another machine. It is working (kind of) for the messages log and some of the other logs though sometimes it is extremely slow. Sometimes taking an hour or more to get logs to the other machine. My main problem though is that my ftp and http logs are not getting there at all. My FTP server is set to log to the LOCAL7 facility and is logging correctly to the machine it is on. My remote logger is not recieving anything however. Same goes for my HTTP logs. One logs to LOCAL6 facility, and another is logging to a named_pipe file that syslog-ng is "listening" to. Both HTTP logs are working on the host box but I get nothing at the remote logger. I've included my .conf files from both machines. Any help at all would be appreciated. Thanks. Matt Mencel matt(a)cmitech.com ************ MAIN SERVER CONF FILE ************ # # Syslog-ng example configuration for for Debian GNU/Linux # # Copyright (c) 1999 anonymous # Copyright (c) 1999 Balazs Scheidler # $Id: syslog-ng.conf.sample,v 1.2 1999/11/15 12:30:41 bazsi Exp $ # # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # options { long_hostnames(off); sync(0); log_fifo_size(1000); gc_busy_threshold(5000); }; source src { unix-dgram("/var/run/log"); internal(); file("/dev/klog"); }; source httpd { pipe("/var/run/httpd-access"); }; destination authlog { file("/var/log/auth.log"); }; destination syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron"); }; destination daemon { file("/var/log/daemonlog"); }; destination kern { file("/var/log/kern.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/userlog"); }; destination uucp { file("/var/log/uucp.log"); }; #destination ppp { file("/var/log/ppp.log"); }; destination mail { file("/var/log/maillog"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; #destination newscrit { file("/var/log/news/news.crit"); }; #destination newserr { file("/var/log/news/news.err"); }; #destination newsnotice { file("/var/log/news/news.notice"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; # Added/modified by Matt destination ftpd { file("/var/log/ftp.log"); }; destination loghost { tcp("192.168.0.9" port(514)); }; destination httpd { file("/var/log/httpd-access.log"); }; destination httpderr { file("/var/log/httpd-error.log"); }; #destination xconsole { pipe("/dev/xconsole"); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(authpriv, mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(cron); }; #filter f_ppp { facility(ppp); }; #filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; # Added/modified by Matt filter f_ftpd { facility(local7); }; filter f_httpderr { facility(local6); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; #log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; #log { source(src); filter(f_news); filter(f_err); destination(newserr); }; #log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; #log { source(src); filter(f_ppp); destination(ppp); }; log { source(src); destination(console_all); }; # Added/modified by Matt log { source(src); destination(loghost); }; log { source(src); filter(f_ftpd); destination(ftpd); }; log { source(httpd); destination(httpd); destination(loghost); }; log { source(src); filter(f_httpderr); destination(httpderr); }; ************ REMOTE LOGGING SERVER CONF FILE ************ # # Syslog-ng example configuration for for Debian GNU/Linux # # Copyright (c) 1999 anonymous # Copyright (c) 1999 Balazs Scheidler # $Id: syslog-ng.conf.sample,v 1.2 1999/11/15 12:30:41 bazsi Exp $ # # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # options { long_hostnames(off); sync(0); log_fifo_size(1000); gc_busy_threshold(5000); }; source src { unix-dgram("/var/run/log"); tcp(ip(192.168.0.9) port(514)); internal(); file("/dev/klog"); }; destination authlog { file("/var/log/auth.log"); }; destination syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron"); }; destination daemon { file("/var/log/daemonlog"); }; destination kern { file("/var/log/kern.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/userlog"); }; destination uucp { file("/var/log/uucp.log"); }; #destination ppp { file("/var/log/ppp.log"); }; destination mail { file("/var/log/maillog"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; #destination newscrit { file("/var/log/news/news.crit"); }; #destination newserr { file("/var/log/news/news.err"); }; #destination newsnotice { file("/var/log/news/news.notice"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; #destination xconsole { pipe("/dev/xconsole"); }; # Added/modified by Matt destination ftpd { file("/var/log/ftp.log"); }; destination httpd { file("/var/log/httpd-access.log"); }; destination httpderr { file("/var/log/httpd-error.log"); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(authpriv, mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(cron); }; #filter f_ppp { facility(ppp); }; #filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; # Added/modified by Matt filter f_ftpd { facility(local7); }; filter f_httpderr { facility(local6); }; filter f_httpd { match("apache:"); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; #log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; #log { source(src); filter(f_news); filter(f_err); destination(newserr); }; #log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; #log { source(src); filter(f_ppp); destination(ppp); }; log { source(src); destination(console_all); }; # Added/modified by Matt log { source(src); filter(f_ftpd); destination(ftpd); }; log { source(src); filter(f_httpderr); destination(httpderr); }; log { source(src); filter(f_httpd); destination(httpd); };

1 0

RE: [syslog-ng]filter question: already matched
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 21 May '01

21 May '01

James, Well, I'm not sure if you are still making an error in your cut and paste, but the destination and filter you put in the message don't match your log line. You mention that they are set correctly but you didn't post the correct one. Not sure we can help without the correct line. The idea you put forth is sound and should work with what you have. What exactly is happening? Is it not logging? As a first look take off the hostname part of the filter to see what you are getting in the DEFAULT filter. I do this on occasion and sometimes will surprise you that you aren't getting what you expect. Regards, Drew -----Original Message----- From: James Hamilton [mailto:jamesh@swcp.com] Sent: Friday, May 18, 2001 8:08 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]filter question: already matched And yes the destination and filter is set correctly in my rules. I just did a poor copy and paste job in my email :-) On Fri, May 18, 2001 at 11:42:16AM -0600, James Hamilton wrote: > > Hi, I'm having some trouble setting up a filter. In plain english the rule would read something like below, any suggestions? > > Match everything for this host except things that have already been matched for this host then drop them into a messages file. > > ## > ## hosts messages log > ## > destination d_messages { > file("/var/log/$MONTH/$HOST/$HOST_messages.$MONTH-$DAY-$YEAR" > owner(root) > group(staff) > perm(0640) > dir_perm(0750) > create_dirs(yes)); > }; > > filter f_messages { > (filter(DEFAULT) and host("somehostname")); > }; > > log { > source(root); > filter(f_cron); ^^^^^^^^^^^^^^^^^^^^ > destination(d_cron); ^^^^^^^^^^^^^^^^^^^^ > }; > > > -- > > James Hamilton > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng -- James Hamilton _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

filter question: already matched
by James Hamilton 18 May '01

18 May '01

Hi, I'm having some trouble setting up a filter. In plain english the rule would read something like below, any suggestions? Match everything for this host except things that have already been matched for this host then drop them into a messages file. ## ## hosts messages log ## destination d_messages { file("/var/log/$MONTH/$HOST/$HOST_messages.$MONTH-$DAY-$YEAR" owner(root) group(staff) perm(0640) dir_perm(0750) create_dirs(yes)); }; filter f_messages { (filter(DEFAULT) and host("somehostname")); }; log { source(root); filter(f_cron); destination(d_cron); }; -- James Hamilton

1 1

syslog-ng and stunnel
by Eric Parusel 16 May '01

16 May '01

Hello, I've been able to find anything in the archives pertaining to this.... I've successfully set up syslog-ng to tunnel through stunnel. I'm having one problem though, all messages come through with a hostname of "localhost", presumably since stunnel is coming from localhost on the syslog server.... Obviously it's hard to tell which server is generating the syslog message if they're all showing up as "localhost"... :) Does anyone have any workarounds for this problem? Thanks, Eric Parusel

3 3

-u and -g switches on syslog-ng 1.5.6
by MURVAI-BUZOGANY Laszlo 15 May '01

15 May '01

Hi! I tried to run my newly-compiled syslog-ng-1.5.6 with different user and group values than usually (root). As trying it, got segfault several times, so after a short time I discovered some silly coding in src/main.c. In order to make it work, only _really_ simple changes have to be made, I also made a diff-file of the release and my (working) source. I attach it. Hope I got the problem right, and didn't get messed up :) sci -- Murvai-Buzogány László Laszlo.Murvai-Buzogany(a)gt-systems.hu

2 1

TCP reliability
by Chris Sibbitt 15 May '01

15 May '01

Greetings. I'm running syslog-ng 1.4.11 and am having trouble getting the tcp destination to work reliably. My major concern is with what happens when the receiving end of the tcp link is reset. In my scenario I have one syslog-ng machine forwarding messages to another syslog-ng machine using a tcp() destination on one end and a tcp() source on the other. All works fine until I interrupt the socket on the receiving machine. Rebooting, HUP'ing syslog-ng, etc all cause the logs to stop coming in. I assume that the sender is simply trying to write the messages to an existing TCP socket and failing. What happens to these messages (are they queued anywhere for later delivery?), and is there anything that I can do to ensure that messages continue to flow when the receiver comes back up? Losing messages while the receiving end if offline is an acceptable loss, but when it comes back up, I would really like it to continue to receive. In the current scenario I have to restart every syslog-ng machine that is forwarding messages to my central logger when I change its configuration, this is definitely not sane. Anyone with hints/tips for setting up TCP based messaging, please contact me. Chris Sibbitt JetNet Internetworking Services Inc. Providing Secure Internet Communications Phone 613-271-6220x228 FAX 613-271-6229 Email csibbitt(a)jetnet.ca Web www.jetnet.ca <http://www.jetnet.ca/>

2 1

RE: [syslog-ng]networking problems
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 15 May '01

15 May '01

Justin, It's hard to really tell what's going on without the entire config file, but it looks like your remote machine isn't logging to the log host machine. There is no log statement with the remote machine as the destination. Also in your loghost machine there is no log statement with the remote source as the source. Understand that the source, destination and filter statements only define your log environment. The log statement does the work. You need a log statement like: log { source(external_source); filter(f_whatever); destination(d_remote); }; or something similar on the loghost. On the client you need: log { source(local_source); filter(f_remote_whatever); destination(remote_john); }; That should work for you. Regards, Drew -----Original Message----- From: Justin Scheiber (98030) [mailto:jscheibe@css.tayloru.edu] Sent: Monday, May 14, 2001 11:09 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]networking problems I checked the archives, but no entries exactly described my problem. It seems as though syslog-ng does not log properly over the network (at least with my configuration). The configuration is supposed to send the data across the network to a centralized logging machine. However, it seems as though the data gets logged only out of random chance. version: syslog-ng-1.4.10 Here is a listing of the logs off the centralized logging machine: SIZE DATE LOG FILES 0 Apr 12 16:02 machine1.automation.log 1760 Apr 18 19:21 machine1.security.log 0 Apr 17 00:01 machine2.automation.log 899 Apr 18 19:13 machine2.security.log 0 Apr 10 17:01 machine3.automation.log 3608 Apr 19 15:20 machine3.security.log 0 Apr 12 15:01 machine4.automation.log 903 Apr 23 04:59 machine4.error.log 14073 Apr 23 10:54 machine4.security.log 4618 Apr 18 19:48 machine5.security.log 0 Apr 10 17:01 machine5.automation.log 0 Apr 12 16:01 machine6.automation.log 1804 Apr 18 19:09 machine6.security.log Here is the configuration file for the log host: options { long_hostnames(off); sync(10); }; ## sources ## source local_src { unix-stream("/dev/log"); internal(); }; ## external source (log messages from other machcines) ## #source external_src { tcp(); }; source external_src { udp(ip(10.0.1.4) port(1800)); }; ## destinations ## destination kernel_dest { file("/var/log/kernel.log"); }; destination automation_dest { file("/var/log/automation.log"); }; destination security_dest { file("/var/log/security.log"); }; destination error_dest { file("/var/log/error.log"); }; destination eternal_peril_dest { file("/dev/null"); }; ## destination for external logs ## destination external_security_dest { file("/export/hda3/log/$HOST.security.log"); }; destination external_automation_dest { file("/export/hda3/log/$HOST.automation.log"); }; destination external_error_dest { file("/export/hda3/log/$HOST.error.log"); }; ## filters ## filter kernel_filter { facility(kern); }; filter security_filter { match("connect") or program("PAM_pwdb") or match("pam") or program("sshd") or program("login") or facility(security) and not program("sendmail"); }; filter error_filter { level(crit..panic); }; filter automation_filter { program("runfixes"); }; filter eternal_peril_filter { program("gpm") or program("talkd") or program("gnome-name-server") or program("automount") or program("sendmail") or program("CMD") and level(debug..error); }; ## log the messages ## log { source(local_src); filter(kernel_filter); destination(kernel_dest); }; log { source(local_src); filter(security_filter); destination(security_dest); }; log { source(local_src); filter(automation_filter); destination(automation_dest); }; log { source(local_src); filter(error_filter); filter(DEFAULT); destination(error_dest); }; log { source(local_src); filter(eternal_peril_filter); destination(eternal_peril_dest); }; The log-client configuration is almost identical, except it for a destination line: and the log lines: destination remote_john { udp("10.0.1.4" port(1800)); }; log { source(local_src); filter(kernel_filter); destination(kernel_dest); } Any help would be appreciated. -justin _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

networking problems
by Justin Scheiber (98030) 14 May '01

14 May '01

I checked the archives, but no entries exactly described my problem. It seems as though syslog-ng does not log properly over the network (at least with my configuration). The configuration is supposed to send the data across the network to a centralized logging machine. However, it seems as though the data gets logged only out of random chance. version: syslog-ng-1.4.10 Here is a listing of the logs off the centralized logging machine: SIZE DATE LOG FILES 0 Apr 12 16:02 machine1.automation.log 1760 Apr 18 19:21 machine1.security.log 0 Apr 17 00:01 machine2.automation.log 899 Apr 18 19:13 machine2.security.log 0 Apr 10 17:01 machine3.automation.log 3608 Apr 19 15:20 machine3.security.log 0 Apr 12 15:01 machine4.automation.log 903 Apr 23 04:59 machine4.error.log 14073 Apr 23 10:54 machine4.security.log 4618 Apr 18 19:48 machine5.security.log 0 Apr 10 17:01 machine5.automation.log 0 Apr 12 16:01 machine6.automation.log 1804 Apr 18 19:09 machine6.security.log Here is the configuration file for the log host: options { long_hostnames(off); sync(10); }; ## sources ## source local_src { unix-stream("/dev/log"); internal(); }; ## external source (log messages from other machcines) ## #source external_src { tcp(); }; source external_src { udp(ip(10.0.1.4) port(1800)); }; ## destinations ## destination kernel_dest { file("/var/log/kernel.log"); }; destination automation_dest { file("/var/log/automation.log"); }; destination security_dest { file("/var/log/security.log"); }; destination error_dest { file("/var/log/error.log"); }; destination eternal_peril_dest { file("/dev/null"); }; ## destination for external logs ## destination external_security_dest { file("/export/hda3/log/$HOST.security.log"); }; destination external_automation_dest { file("/export/hda3/log/$HOST.automation.log"); }; destination external_error_dest { file("/export/hda3/log/$HOST.error.log"); }; ## filters ## filter kernel_filter { facility(kern); }; filter security_filter { match("connect") or program("PAM_pwdb") or match("pam") or program("sshd") or program("login") or facility(security) and not program("sendmail"); }; filter error_filter { level(crit..panic); }; filter automation_filter { program("runfixes"); }; filter eternal_peril_filter { program("gpm") or program("talkd") or program("gnome-name-server") or program("automount") or program("sendmail") or program("CMD") and level(debug..error); }; ## log the messages ## log { source(local_src); filter(kernel_filter); destination(kernel_dest); }; log { source(local_src); filter(security_filter); destination(security_dest); }; log { source(local_src); filter(automation_filter); destination(automation_dest); }; log { source(local_src); filter(error_filter); filter(DEFAULT); destination(error_dest); }; log { source(local_src); filter(eternal_peril_filter); destination(eternal_peril_dest); }; The log-client configuration is almost identical, except it for a destination line: and the log lines: destination remote_john { udp("10.0.1.4" port(1800)); }; log { source(local_src); filter(kernel_filter); destination(kernel_dest); } Any help would be appreciated. -justin

1 0

Syslog-ng and Linux iptables
by Ken Armstrong 14 May '01

14 May '01

Hello, I am having problems getting syslog-ng to work with the iptables logging facility under linux. The iptables rules allow logging and it works fine with the standard klogd/syslogd loggers with RedHat 7.1. I believe that the default kernel messeges use kern.warn as the facility.priority. When I replace syslogd with syslog-ng (and keep using the default klogd) I no longer receive any of the kernel messages. I have tried both of the following filter lines filter f_warn { level(warn); }; and filter f_kern { facility(kern); }; I can get all of the other looging to work, I am only having problems with the kernel messages? Has anyone else had this problem? Thanks Ken

2 2

Aborted
by Brian E. Seppanen 11 May '01

11 May '01

I recently tried instituting $YEAR-$MONTH-$DAY directory creation into my syslogging, due to problems with logrotation. An hour after I got it to work it dropped a core file and died, indicating that it was aborted. Can anyone see if there is a reason from the following info. PIII 500, 512MB, Redhat Linux 7.0. Syslog-ng 1.4.11, libol-0.2.21 Any help appreciated. Thanks, [root@host ~]# gdb syslog-ng core GNU gdb 5.0 Copyright 2000 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux"... Core was generated by `/usr/local/sbin/syslog-ng -p /var/run/syslog-ng.pid -f /usr/local/etc/syslog-ng'. Program terminated with signal 6, Aborted. Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 #0 0x40068dd1 in __kill () from /lib/libc.so.6 (gdb) bt #0 0x40068dd1 in __kill () from /lib/libc.so.6 #1 0x40068baa in raise (sig=6) at ../sysdeps/posix/raise.c:27 #2 0x4006a352 in abort () at ../sysdeps/generic/abort.c:88 #3 0x400627fa in __assert_fail () at assert.c:60 #4 0x80510d4 in io_iter () #5 0x8049bf1 in main_loop () #6 0x8049f2d in main () #7 0x40057f31 in __libc_start_main (main=0x8049d24 <main>, argc=6, ubp_av=0xbffff9d4, init=0x8049204 <_init>, fini=0x80535bc <_fini>, rtld_fini=0x4000e274 <_dl_fini>, stack_end=0xbffff9cc) at ../sysdeps/generic/libc-start.c:129 Brian Seppanen Charter Communications seppy(a)chartermi.net Advanced Services Engineer 906-228-4226 ext 23 Therefore, since brevity is the soul of wit and tediousness the limbs and outward flourishes, I will be brief. - Polonius

1 0