- syslog-ng - lists.balabit.hu

syslog-ng and MySQL
by Fleet, James 13 Jan '03

13 Jan '03

> Hello, > > I have been trying to create a syslog server with MySQL and with several > attempts it does not work. I have attached my configs for the system which > runs on Linux RH 7.3. > > > This is my syslog-ng.conf file : > > # > # Syslog-ng example configuration file showing all the existing/future > # features. > # > # Copyright (c) 1999 Balazs Scheidler > # $Id: syslog-ng.conf.demo,v 1.2 1999/11/15 12:30:41 bazsi Exp $ > # > # Note that some of the keywords do _nothing_, this includes: > # mac, authentication, encryption, compress > # > > #options { sync(0); time_reopen(10); log_fifo_size(100); }; > ## Log syslog-ng to mysql database > ## > destination d_mysql { > pipe("/tmp/mysql.pipe" > template ("INSERT INTO logs (host, facility, priority, level, tag, date, > time, program, msg) VALUES ( '$HOST', '$FACILITY', > '$PRIORITY','$LEVEL','$TAG','$YEAR-$MONTH-$DAY','$HOUR:$MIN:$SEC','$PROGRAM' > ,'$MSG' );\n") template-escape(yes)); > }; > log { source(net); destination(d_mysql); > > }; > #source src { > # unix-stream("/dev/log"); > # udp(); > # tcp(ip("127.0.0.1") port(1000) mac(allow) authentication(required) > encrypt(allow)); > # tcp(); > # sun-streams("/dev/log" door("/etc/.syslog_door")); > # internal(); > #}; > source src { unix-dgram("/etc/log/log"); internal(); }; > source net { udp(); }; > > #log { source(src); destination(everything); }; > > > I also have this script running which says that /tmp/msql.pipe is my fifo > directory. > > But I am still not receiving logs into the database but when I switch it > will log to messages. > > > -James Fleet - > > Systems Engineer > Union Planters Bank > > > > > > > > > > > > > -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

Re: Lost CR-LF ?
by Hypo 13 Jan '03

13 Jan '03

I'm using syslog-ng v1.5.24 with libol 0.2.23 To generate log messages from a remote machine I use Kiwi SyslogGen v2.0.2 http://www.kiwisyslog.com/software_downloads.htm ) When used in burst mode ( 100 msg/sec ), in UDP I just lost some messages, but in TCP starts omitting CR-LF ( while in continuos mode it works well ). I don't know if is a KIWI problem while generatig lot of messages, or a syslog-ng one. Another little thing I like to ask, is why not all facility ( as RFC ) are defined, and why there's no mapping with numeric values...there's any reason for that choice ? Thx for support Amodiovalerio [Hypo] Verde > > which version of syslog-ng are you using? > > -- Bazsi

1 0

RE: [syslog-ng]problem with multiply-defined source/filter/destin ation combinations?
by Hamilton, Andrew 13 Jan '03

13 Jan '03

Well. That was a hefty message. Anyway, syslog-ng will typically log messages only once unless you tell it otherwise. This is the default behavior. You may think that by adding another log line that sends the same message to a different destination you are telling it to do just that. That is incorrect. Once a message is logged it is then forgotten in a sense. The way to do this is to use multiple destinations on the same line. (i.e., log { source(src); filter(filter1); destination(d1); destination(d2); }; ). This works well for me, I have been doing it for nearly 4 years. Syslog-ng will do exactly what you want to do you just have to tweak it a little. I have never done this before but in theory, and I guess someone else from the list can correct me if I'm wrong, you could write a log line from multiple sources as well. (i.e., log { source(s1); source(s2); filter(f1); destination(d1); destination(d2); }; ) etc... I think you can probably get the picture. That might help you clean up your config file a little. I hope this helps. Regards, Drew -----Original Message----- From: Jay Goldberg [mailto:jg@open.ch] Sent: Monday, January 13, 2003 4:34 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]problem with multiply-defined source/filter/destination combinations? Hi all I've just encountered a very strange problem with syslog-ng after modifying our standard syslog-ng.conf file, and hopefully someone on the group has seen this problem. Background: We've created a fairly complex system for managing logs on hosts in various locations. We have log collectors all running syslog-ng, and we're now using an Event-to-Syslog package that we have configured to send Windows Event Log messages to our log collectors. The log collectors use an external Perl script which filters out 'uninteresting' messages and uses our own reliable encrypted transport mechanism to send the logs to our in-house management systems. The Windows messages come in on local3. The external Perl script sends messages on local6. The intention of the configuration is: - local6 gets written to its own logfile so that our system doesn't get into any message loops if it encounters problems. - everything else is written to its appropriate logfile on disk AND sent through this external Perl program. Before modifying our syslog-ng.conf file to send messages to the external program, we saw all messages written to their appropriate logfiles on disk. After modifying the syslog-ng.conf, we see that the external script is working just fine, but the messages received on UDP are no longer being written to disk; they are only being sent to this external Perl script. I've implemented a workaroud, but I don't know if I've done something wrong, or if there's perhaps a bug in syslog-ng (we're running 1.5.22 on Solaris 8). Here's the original syslog-ng.conf: ================== start syslog-ng.conf, ORIGINAL ===================== options { sync (0); owner(syslogd); group(syslogd); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { sun-streams ("/dev/log" door("/etc/.syslog_door")); internal(); }; source s_udp { udp(); }; destination d_cons { file("/dev/console"); }; destination d_smsg { file("/dev/sysmsg"); }; destination d_udp { udp("loghost" port(514)); }; destination d_mlop { usertty("operator"); }; destination d_mlrt { usertty("root"); }; # # the following are scripts using cronolog to control logfile rotation # only so we can maintain symlinks to the 'traditional' logfile locations # destination d_mesg { program("/opt/OSAGslog/bin/c_messages"); }; destination d_sysl { program("/opt/OSAGslog/bin/c_syslog"); }; destination d_mail { program("/opt/OSAGslog/bin/c_maillog"); }; destination d_auth { program("/opt/OSAGslog/bin/c_authlog"); }; destination d_clog { program("/opt/OSAGslog/bin/cronolog"); }; filter f_auth { facility(auth); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; filter f_mesg { facility(cron) or facility(daemon) or facility(kern) or facility(lpr) or facility(news) or facility(user) or facility(uucp) or facility(local0) or facility(local1) or facility(local2) or facility(local3) or facility(local4) or facility(local5) or facility(local7); }; # # Log Directives # # We log AUTH to authlog, MAIL to syslog, everything # else to messages. # log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_udp); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_sysl); }; log { source(s_udp); filter(f_mail); destination(d_sysl); }; log { source(s_sys); filter(f_mesg); destination(d_mesg); }; log { source(s_udp); filter(f_mesg); destination(d_mesg); }; ================== end syslog-ng.conf, ORIGINAL ===================== ...here is the addition that we made to the end of the syslog-ng.conf to send logs to our Perl script called 'gumafilter': ================== start additions for 'gumafilter' ===================== source sg_global { sun-streams ("/dev/log" door("/etc/.syslog_door")); internal(); udp(); }; # # c_gumalog just uses cronolog to control logfile rotation for # messages generated by the 'gumafilter' script. # destination dg_filt { program("/opt/OSAGguma/bin/gumafilter"); }; destination dg_log { program("/opt/OSAGslog/bin/c_gumalog"); }; filter fg_local6 { facility(local6); }; filter fg_all { facility(auth) or facility(mail) or facility(cron) or facility(daemon) or facility(kern) or facility(syslog) or facility(lpr) or facility(news) or facility(user) or facility(uucp) or facility(local0) or facility(local1) or facility(local2) or facility(local3) or facility(local4) or facility(local5) or facility(local7); }; # This guma sensor accepts remote messages (from guma.conf) log { source(sg_global); filter(fg_all); destination(dg_filt); }; log { source(sg_global); filter(fg_local6); destination(dg_log); }; ================== end additions for 'gumafilter' ===================== The workaround is to add the following line at the bottom of the syslog-ng.conf file, and I don't know why it's needed! log { source(sg_global); filter(f_mesg); destination(d_mesg); }; Without this line, the Perl script 'gumafilter' works correctly, but none of the logs received on UDP get written to disk in the messages file, which I would have expected based on the lines: log { source(s_sys); filter(f_mesg); destination(d_mesg); }; log { source(s_udp); filter(f_mesg); destination(d_mesg); }; Any ideas? ...sorry for the long message! Thanks in advance, Jay -- jay goldberg . open systems ag raeffelstrasse 29 jg(a)open.ch _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

(no subject)
by syslogng-bipsen＠nerdnet.dk 13 Jan '03

13 Jan '03

From: <big(a)boss.com> To: <syslogng-bipsen(a)nerdnet.dk> Date: Mon, 13 Jan 2003 12:37:17 +0100 Importance: Normal Subject: Segmentation fault ? Hi! When using the supplied init.d script for RedDat in /contrib I get a segmentation fault is I do a: /etc/init.d/syslog-ng status Any idea why ?? /Brian

1 0

Segmentation fault ?
by syslogng-bipsen＠nerdnet.dk 13 Jan '03

13 Jan '03

Hi! When using the supplied init.d script for RedHat in /contrib I get a segmentation fault is I do a: /etc/init.d/syslog-ng status Any idea why ?? I'm using version 1.4.17 of syslog-ng /Brian

1 0

problem with multiply-defined source/filter/destination combinations?
by Jay Goldberg 13 Jan '03

13 Jan '03

Hi all I've just encountered a very strange problem with syslog-ng after modifying our standard syslog-ng.conf file, and hopefully someone on the group has seen this problem. Background: We've created a fairly complex system for managing logs on hosts in various locations. We have log collectors all running syslog-ng, and we're now using an Event-to-Syslog package that we have configured to send Windows Event Log messages to our log collectors. The log collectors use an external Perl script which filters out 'uninteresting' messages and uses our own reliable encrypted transport mechanism to send the logs to our in-house management systems. The Windows messages come in on local3. The external Perl script sends messages on local6. The intention of the configuration is: - local6 gets written to its own logfile so that our system doesn't get into any message loops if it encounters problems. - everything else is written to its appropriate logfile on disk AND sent through this external Perl program. Before modifying our syslog-ng.conf file to send messages to the external program, we saw all messages written to their appropriate logfiles on disk. After modifying the syslog-ng.conf, we see that the external script is working just fine, but the messages received on UDP are no longer being written to disk; they are only being sent to this external Perl script. I've implemented a workaroud, but I don't know if I've done something wrong, or if there's perhaps a bug in syslog-ng (we're running 1.5.22 on Solaris 8). Here's the original syslog-ng.conf: ================== start syslog-ng.conf, ORIGINAL ===================== options { sync (0); owner(syslogd); group(syslogd); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { sun-streams ("/dev/log" door("/etc/.syslog_door")); internal(); }; source s_udp { udp(); }; destination d_cons { file("/dev/console"); }; destination d_smsg { file("/dev/sysmsg"); }; destination d_udp { udp("loghost" port(514)); }; destination d_mlop { usertty("operator"); }; destination d_mlrt { usertty("root"); }; # # the following are scripts using cronolog to control logfile rotation # only so we can maintain symlinks to the 'traditional' logfile locations # destination d_mesg { program("/opt/OSAGslog/bin/c_messages"); }; destination d_sysl { program("/opt/OSAGslog/bin/c_syslog"); }; destination d_mail { program("/opt/OSAGslog/bin/c_maillog"); }; destination d_auth { program("/opt/OSAGslog/bin/c_authlog"); }; destination d_clog { program("/opt/OSAGslog/bin/cronolog"); }; filter f_auth { facility(auth); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; filter f_mesg { facility(cron) or facility(daemon) or facility(kern) or facility(lpr) or facility(news) or facility(user) or facility(uucp) or facility(local0) or facility(local1) or facility(local2) or facility(local3) or facility(local4) or facility(local5) or facility(local7); }; # # Log Directives # # We log AUTH to authlog, MAIL to syslog, everything # else to messages. # log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_udp); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_sysl); }; log { source(s_udp); filter(f_mail); destination(d_sysl); }; log { source(s_sys); filter(f_mesg); destination(d_mesg); }; log { source(s_udp); filter(f_mesg); destination(d_mesg); }; ================== end syslog-ng.conf, ORIGINAL ===================== ...here is the addition that we made to the end of the syslog-ng.conf to send logs to our Perl script called 'gumafilter': ================== start additions for 'gumafilter' ===================== source sg_global { sun-streams ("/dev/log" door("/etc/.syslog_door")); internal(); udp(); }; # # c_gumalog just uses cronolog to control logfile rotation for # messages generated by the 'gumafilter' script. # destination dg_filt { program("/opt/OSAGguma/bin/gumafilter"); }; destination dg_log { program("/opt/OSAGslog/bin/c_gumalog"); }; filter fg_local6 { facility(local6); }; filter fg_all { facility(auth) or facility(mail) or facility(cron) or facility(daemon) or facility(kern) or facility(syslog) or facility(lpr) or facility(news) or facility(user) or facility(uucp) or facility(local0) or facility(local1) or facility(local2) or facility(local3) or facility(local4) or facility(local5) or facility(local7); }; # This guma sensor accepts remote messages (from guma.conf) log { source(sg_global); filter(fg_all); destination(dg_filt); }; log { source(sg_global); filter(fg_local6); destination(dg_log); }; ================== end additions for 'gumafilter' ===================== The workaround is to add the following line at the bottom of the syslog-ng.conf file, and I don't know why it's needed! log { source(sg_global); filter(f_mesg); destination(d_mesg); }; Without this line, the Perl script 'gumafilter' works correctly, but none of the logs received on UDP get written to disk in the messages file, which I would have expected based on the lines: log { source(s_sys); filter(f_mesg); destination(d_mesg); }; log { source(s_udp); filter(f_mesg); destination(d_mesg); }; Any ideas? ...sorry for the long message! Thanks in advance, Jay -- jay goldberg . open systems ag raeffelstrasse 29 jg(a)open.ch

1 0

postfix logs mangled
by Ken Blinco 13 Jan '03

13 Jan '03

Hi, product: syslog-ng-1.5.24 platform: linux 2.4.18 (redhat dist) I'm having a problem with postfix logs sent across the network through to syslog-ng which in turn pipes output through to a mysql server using the following directive: destination d_mysql pipe("/tmp/mysql.pipe" template("INSERT INTO syslogs (host, facilit etc etc... The output from postfix has the 'to' and 'from' address dropped: [snip] postfix/smtp[30704]: 157A6703B8: to=, relay=mail.domain.com.... [end snip] It should look like this: [snip] postfix/smtp[30704]: 157A6703B8: to=<user(a)domain.com>, relay=mail.domain.com.... [end snip] I'm guessing it's something to do with the '<' & '>' characters. I'll have to go through the source to see where it's going wrong, but was hoping that someone else out there had any similar sorts of problems and advice on how to fix. Thanks Ken ------------------------------------------------------------- Ken Blinco Bridge Point Communications Tel: +61 7 3839 0999 497 Adelaide Street Mobile: 0407 275 766 Brisbane QLD 4000, Australia Fax: +61 7 3839 0888 This email and any attachments are subject to copyright. They may also contain confidential information. This email and any attachments may not be distributed, reproduced, copied, stored or transmitted in any form or by any means, without the prior written consent of Bridge Point Communications Pty Ltd ABN 29 083 424 668. Any personal information in this email must be handled in accordance with the Privacy Act 1988 (Cth). Emails may be interfered with, may contain computer viruses or other defects and may not be successfully replicated on other systems. Bridge Point Communications Pty Ltd gives no warranties in relation to these matters. If you have any doubts about the authenticity of an email purportedly sent by us, please contact Bridge Point Communications Pty Ltd immediately.

2 2

Syslog-ng-1.5.24 build instructions for cygwin
by John P. Rouillard 13 Jan '03

13 Jan '03

Hi all: Thanks go to whoever fixed the bug in cygwin1.dll that caused syslog to fail as soon as a udp packet was sent to it. The bug was introduced sometime around 1.3.12, but is fixed now. So here is my recipe for building syslog-ng-1.5.24. Download syslog-ng-1.5.24 and libol-0.3.6. Patch the syslog source with the patch at the end of this message. The patch adds a -N flag to stop the syslog server from forking so that windows services mechanism and cygrunsrv will properly detect that the program/service is running. The man page is updated with this flag, but the html/sgml isn't patched. The patch also stops resolv.h from being included under __CYGWIN__, and defined the typdef uint32_t as unsigned. After applying the patch, touch src/main.c.x src/filter.c.x, then configure and make the two packages. Then run cygrunsrv as: cygrunsrv -I syslog -a '-N' -p /path/to/syslog -c /var/log and cygrunsrv -S syslog I use the syslog server with backlog to send the event log on my windows 2000 SP3 system to a file. I did most of my testing without tcp wrappers enabled, but I did finally get it to compile with the wrappers. To do this, change the lines that read: old_LDFLAGS=$LDFLAGS LDFLAGS=-lwrap to read old_LIBS=$LIBS LIBS=-lwrap and you will get checking for hosts_access in -lwrap... yes rather than checking for hosts_access in -lwrap... no Hopefully the syslog-ng people will include this patch in newer versions. -- rouilj John Rouillard =========================================================================== My employers don't acknowledge my existence much less my opinions. diff -ur syslog-ng-1.5.24/doc/syslog-ng.8 syslog-ng-1.5.24.l/doc/syslog-ng.8 --- syslog-ng-1.5.24/doc/syslog-ng.8 1999-07-10 11:58:32.000000000 -0400 +++ syslog-ng-1.5.24.l/doc/syslog-ng.8 2003-01-12 11:30:30.000000000 -0500 @@ -7,7 +7,7 @@ .B syslog-ng [ -.B \-dvV +.B \-NdvV ] [ .B \-f <\fIconfig-filename\fP> @@ -76,6 +76,15 @@ .B \-p \0<filename> Write the current PID information to the specified file. .TP +.B \-N +Non-daemon mode. Adding \fB-N\fp to the command line will make syslog +run in the foreground. This is used when starting syslog from service +monitor programs in windows platforms using cygwin's cygrunsrv, but +may also be useful for a startup/monitor wrapper that will restart +syslog if it dies for some unusual reason. This does not cause any +additional messages to be printed, so it will not fill up +log files as \fB-d\fP or \fB-v\fP can. +.TP .B \-v Enable verbose mode. Process will not become a daemon. Prints out fewer messages, compared to \fB-d\fP. diff -ur syslog-ng-1.5.24/src/filters.c syslog-ng-1.5.24.l/src/filters.c --- syslog-ng-1.5.24/src/filters.c 2002-02-04 11:07:50.000000000 -0500 +++ syslog-ng-1.5.24.l/src/filters.c 2003-01-12 11:21:57.000000000 -0500 @@ -259,6 +259,9 @@ struct log_filter *rule UNUSED, struct log_info *log) { +#if defined(__CYGWIN__) + typedef unsigned uint32_t; +#endif CAST(filter_expr_netmask, self, c); if (log->saddr) { diff -ur syslog-ng-1.5.24/src/main.c syslog-ng-1.5.24.l/src/main.c --- syslog-ng-1.5.24/src/main.c 2002-10-30 14:36:03.000000000 -0500 +++ syslog-ng-1.5.24.l/src/main.c 2003-01-12 11:31:31.000000000 -0500 @@ -46,7 +46,9 @@ #include <arpa/nameser.h> #endif -#include <resolv.h> +#if ! defined(__CYGWIN__) + #include <resolv.h> +#endif #include "main.c.x" @@ -425,7 +427,7 @@ gc_idle_threshold = 100; gc_busy_threshold = 3000; - while ((opt = getopt_long(argc, argv, "sFf:p:dvhyVC:u:g:", syslog_ng_options, NULL)) != -1) { + while ((opt = getopt_long(argc, argv, "sFf:p:dvhyVC:u:g:N", syslog_ng_options, NULL)) != -1) { switch (opt) { case 'f': strncpy(cfgfilename, optarg, sizeof(cfgfilename)); @@ -464,6 +466,9 @@ yydebug = 1; break; #endif + case 'N': + do_fork = 0; + break; case 'h': default: usage();

1 0

New user of syslog-ng
by Zulazman Zakaria 11 Jan '03

11 Jan '03

Hi Bazsi I'm Zulazman Zakaria from Telekom Malaysia. I'm working on additional syslog daemon You have shown us the replacement of default syslog daemon which is called syslog-ng. I have succesfully installed the syslog-ng Solaris 7 server, but I have some question to ask. I'm a new person which using syslog-ng. 1) Can we use current syslog daemon simultaneously with syslog-ng? I mean the current syslog daemon still can log up to 8 machines and syslog-ng shall log up the other 5 machines? 2) If no, do we have to log all the syslog including default syslog using syslog-ng? 3) How to write the script (syslog-ng) in order to get the all the machine's logs? 4) How can we specify the all machines in syslog-ng script? I mean on which column we shall specify the machines whether using the facility or IP address because we have aound 14 machines. For your information, the machines we are using is a Broadband Access Server. Please provide us some guideline. Thank you, Zulazman Zakaria Asst. Manager Telekom Malaysia

2 1

Re: Document
by big＠boss.com 11 Jan '03

11 Jan '03

Attached file:

1 0