March 2004 - syslog-ng - lists.balabit.hu

What's wrong with my syntax?
by Adam 05 Mar '04

05 Mar '04

Using Syslog 1.4.17. Can't work out why this is incorrect syntax. destination testlog { file("/var/log/testlog" template("$MONTH-$DAY-$YEAR $HOUR:$MIN:$SECOND $HOST $LEVEL $MSG") ); }; Commenting out the template line causes it to run fine, yet this looks exactly like the syntax I see in the samples and documentation?

2 1

Syslog-ng timed sync
by Hugh Clark 04 Mar '04

04 Mar '04

Is there any way to be able to do a timed sync? What I want to do is be able to have syslog-ng buffer about 10 lines, but if nothing is logged within, say, 10 seconds, syslog-ng will automatically flush the buffer to disk. This way, if the logging is bursty, the system doesn't get strangled by the I/O writes during the heavy times and I don't get caught waiting long times for more logging to fill the buffer during the quiet times. I'm concerned about using sync=0 because during the times when the logging is heavy, it take a great deal of I/O to write to the logs. Any thoughts? Thanks, -Hugh

1 0

Get priority and facility with an syslog-ng 1.4.17
by delman 04 Mar '04

04 Mar '04

Hi all, I've got a central logserver running on a SuSe i386 which is running an old version of syslog-ng that I'm unwilling to upgrade unless I'm forced to: # syslog-ng --version syslog-ng 1.4.17 Now, I'd like to force some log entry into a db, including facility/priority pair. I'm forcing logs into a named pipe, and I use a custom perl script to make some clustering and insert the results in the db. The problem is that syslog-ng doesn't write the tags <xxx> in front of the lines and so I miss facility/priority part of information. Moreover, using an old version, I have not the template facility to play with. Is there a way to go around that? -------------------------------------- Protect yourself from spam, use http://sneakemail.com

1 0

Getting syslog-ng 1.6.2 running under IRIX
by Leslie Robert Lait 04 Mar '04

04 Mar '04

Hi, Our research group has started using syslog-ng (Version 1.6.2), which compiles and runs fine on many of our machines. However, we do have some SGI boxes running IRIX 6.5, and these presented some problems. I was able to get the software working on those machines after making a few minor changes to the source. (We compile with the native SGI C compiler, by the way, not gcc.) I am sending out this message with a description of each problem we encountered, and what we did to fix or work around the problem. This is not a request for support (although suggestions for better workarounds are welcome, of course). I simply hope that the information may prove useful for others getting syslog-ng running under IRIX, or perhaps for future releases of syslog-ng. Looking through the mailing list archives, I see that some have asked about IRIX in the past. Issue 1: -------- The configuration script correctly detected that IRIX uses the sun-streams() source for reading /dev/log (without the Solaris "door" facility). However, syslog-ng running under IRIX was not getting log messages from /dev/log successfully. Fortunately, I found that the syslog-ng was in fact reading the log messages from the getmsg() call in afstreams.c, but the messages were being discarded because no terminating characters were being found by the memchr() calls. The current code in afstreams.c assumes that each message is terminated by a newline or a null character, and IRIX apparently does not terminate the message string with anything. Apparently, SGI users are expected to use the returned string length alone. Our fix involves testing the returned string for a newline or a null in its last character position (at line[n-1]). If neither is found, then a null terminating character is added, and the length n is incremented. In order to avoid the possibility of overwriting the end of the buffer, of course, the maximum number of characters to read into the buffer is set to one less than the length of the buffer. ----------------------- begin patch -------------------------------------- --- afstreams.c.O Mon Feb 23 13:25:04 2004 +++ afstreams.c Mon Mar 1 11:36:43 2004 @@ -96,7 +96,8 @@ ctl.maxlen = ctl.len = sizeof(lc); ctl.buf = (char *) &lc; - data.maxlen = length; + data.maxlen = length - 1; /* the -1 is to allow for a possible null + terminator at the end */ data.len = 0; data.buf = buf; flags = 0; @@ -133,6 +134,17 @@ UINT32 length; struct log_info *li; + /* Note: some systems do not terminate the message with anything. + This null termination lets the memchr call below work + in these cases. We are not writing past the end of the + buffer here, because we told the getmsg call to read in + a maximum of (bufferlength - 1) characters, to guarantee + room for this null terminator. */ + if ( n > 0 && line[n-1] != '\0' && line[n-1] != '\n' ) { + line[n] = '\0'; + n++; + } + bol = line; eol = memchr(bol, '\n', n); if (!eol) eol = memchr(bol, '\0', n); ----------------------- end patch -------------------------------------- Issue 2: -------- IRIX does not offer the date format construct "%z", so the ISODATE macro leaves a stray "%z" in the output of each output log line. We tried avoiding using $ISODATE and getting the same effect with "$YEAR-$MONTH=$DAYT$HOUR$MIN$SEC" instead, but of course the "T" turns DAY into the nonexistent DAYT macro, and we could not find a way to separate them. Instead, we introduced a C macro, NO_TZOFFSET, into macros.c. If defined, it omits the %z construct from the ISODATE definition. When we run configure, we make sure to define the CPPFLAGS environment variable beforehand to include "-DNO_TZOFFSET". In fact, we do this on all machines, not just IRIX, so that our logs have a consistent, architecture-independent format. ----------------------- begin patch -------------------------------------- --- macros.c.O Tue Jan 13 13:08:02 2004 +++ macros.c Thu Feb 26 11:41:28 2004 @@ -320,7 +320,11 @@ case M_ISODATE: case M_ISODATE_RECVD: case M_ISODATE_STAMP: +#ifndef NO_TZOFFSET length = strftime(*dest, *left - 1, "%Y-%m-%dT%H:%M:%S%z", tm); +#else + length = strftime(*dest, *left - 1, "%Y-%m-%dT%H:%M:%S", tm); +#endif break; case M_FULLDATE: case M_FULLDATE_RECVD: ----------------------- end patch -------------------------------------- Issue 3: -------- IRIX defines the AUDIT syslog facility (also called "sat", for some reason) that syslog-ng does not know about. (IRIX also defines LOG_LFMT, but since it does not appear in the facilitynames[] list of strings in /usr/include/sys/syslog.h, I stayed away from it.) Linux defines the "AUTHPRIV" facility, as well, which is different from "AUTH". We added conditional definitions of these to syslog-names.c. ----------------------- begin patch -------------------------------------- --- syslog-names.c.O Fri Oct 18 08:31:08 2002 +++ syslog-names.c Thu Feb 26 13:37:53 2004 @@ -43,6 +43,9 @@ }; struct sl_name sl_facilities[] = { +#ifdef LOG_AUDIT + { "audit", LOG_AUDIT }, +#endif { "auth", LOG_AUTH }, #ifdef LOG_AUTHPRIV { "authpriv", LOG_AUTHPRIV }, @@ -56,6 +59,9 @@ { "lpr", LOG_LPR }, { "mail", LOG_MAIL }, { "news", LOG_NEWS }, +#ifdef LOG_AUDIT + { "sat", LOG_AUDIT }, +#endif { "security", LOG_AUTH }, /* DEPRECATED */ { "syslog", LOG_SYSLOG }, { "user", LOG_USER }, ----------------------- end patch -------------------------------------- Issue 4: -------- When we tried to compile and install on our Linux machines (running a recently-patched version of RedHat 7.3), with tcp_wrappers enabled, the compile fails trying to link src/tests/test_nscache.c, because allow_severity and deny_severity are not defined. tcp wrappers doesn't seem to be used here, but test_nscache is linked with the same flags (including libraries) as other things, and the missing symbols trigger an error before the linker realizes that we don't actually need the tcp_wrapper library routines that require them. Our solution was to add the snippet of code from afinet.c: ----------------------- begin patch -------------------------------------- --- tests/test_nscache.c.O Fri Sep 27 05:12:08 2002 +++ tests/test_nscache.c Mon Mar 1 16:15:38 2004 @@ -2,6 +2,12 @@ #include "nscache.h" #include <stdio.h> +#if ENABLE_TCP_WRAPPER +#include <tcpd.h> +int allow_severity = 0; +int deny_severity = 0; +#endif + int main() { struct in_addr in; ----------------------- end patch -------------------------------------- Issue 5: -------- There is a bit of code in the configure script that sets CFLAGS to include the "-Wall" parameter. While this is OK for gcc, the native SGI cc compiler does not recognize "-Wall". The code is in configure.in (i.e., it is not generated by autoconf), so it's a question of dropping the "-Wall" from the "else" clause around lines 274 through 277 in configure.in. Of course, the enable_debug compiler options are likely to be installation-dependent as well. Perhaps pulling these out into a separate definition (say, "CFLAGS_DEBUG") and getting autoconf to generate them, would work better. Anyway, for now, we have inserted a few lines in configure to test for IRIX and run CFLAGS through sed to remove "-Wall". It's ugly, but it works. [Oh, and I am probably required to say something along these lines: I do not represent NASA or my employer SSAI in any official capacity. Neither NASA nor SSAI officially endorse syslog-ng or take any responsibility for any adverse consequences that may result from using this advice or these patches. Use at your own risk. Your mileage may vary. "SGI", "IRIX", "Solaris", "Linux", and "RedHat" are all trademarks registered by their respective owners. Blah blah blah....] -- Leslie Robert Lait | I do not speak for NASA or for my lrlait(a)code916.gsfc.nasa.gov | employer, SSAI---at least, not until I | have achieved my goal of complete world | domination.

2 1

RE: [syslog-ng]$HOST $YEAR, etc not working correctly..
by Doug Peterson 04 Mar '04

04 Mar '04

More Info.. Linux Kernel 2.4.20-30.9smp 1.6.0rc4 produces the same results 1.6.0rc2 ignores $HOST/$YEAR/$MONTH completely (even in the directory structure), so therefore dumps all files in /var/log/hosts with messages_24 ($DAY still works) __________________________________ Do you Yahoo!? Yahoo! Mail SpamGuard - Read only the mail you want. http://antispam.yahoo.com/tools

3 7

RE: [syslog-ng]regex and priority(local7) question
by Rule, Ted 03 Mar '04

03 Mar '04

all levels > -----Original Message----- > From: syslog-ng-admin(a)lists.balabit.hu > [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of cdowns > Sent: Wednesday 03 March 2004 14:38 > To: syslog-ng(a)lists.balabit.hu > Subject: RE: [syslog-ng]regex and priority(local7) question > > > I do have one quick question on the > > filter f_local7 { facility(local7) and level(debug..emerg); }; > > does this mean it is grabbing all level between debug - > emerg, or just those 2 ? > > thanks again. > > ~!>D > > > > > This config snippet works for me .... my PIXies are set to > log via TCP1468 > rather than UDP/514. > Real IP Addresses replaced with fakes. As with your > situation, I have other > network > gear running on local7 as well. Syslog-ng's wonderful > filtration allows for > a nice separation > of all the classes of switch/router/firewall/VPN-bricks. > > > Ted > > ...... > source local { > unix_stream("/dev/log" max-connections(200) keep-alive(yes) ); > > # UDP listeners for Internal syslog-ng interconnections > udp(ip(127.0.0.1) port(514)); > udp(ip(1.2.3.4) port(514)); > > }; > > source pixtcp { > tcp(ip(1.2.3.4) port(1468) > max-connections(10) keep-alive(yes)); > }; > > filter f_local7 { facility(local7) and level(debug..emerg); }; > filter f_pixmsg { match("%PIX"); }; > filter f_misclocal7 { filter(f_local7) > and not filter(f_pixmsg); }; > > destination d_local7 { file("/var/log/local7.debug" sync(0) ); }; > destination d_pixlog { file("/var/log/pixlog.debug" sync(0) ); }; > > log { source(pixtcp); > filter(f_local7); filter(f_pixmsg); destination(d_pixlog); }; > log { source(local); filter(f_misclocal7); destination(d_local7); }; > > ...... > > > > >> -----Original Message----- > >> From: syslog-ng-admin(a)lists.balabit.hu > >> [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of cdowns > >> Sent: Monday 01 March 2004 17:11 > >> To: syslog-ng(a)lists.balabit.hu > >> Subject: [syslog-ng]regex and priority(local7) question > >> > >> > >> Good Morning, > >> Im new to the list and have a couple questions. > >> > >> I have a PIX firewalls logging on local7.info, how can I > >> match this > >> correctly ? I also have switches logging on local7.info but I > >> need this > >> one pix removed and placed into its own destination. > >> > >> Where can i get a list of the regex syntax used in > >> syslog-ng ? like > >> perl / awk / grep etc.. . > >> > >> Here is what I currently have: > >> > >> ------ Snip ------- > >> ## Regex > >> filter f_fw01 { > >> host("x.x.x.x") and match("PIX"); > >> }; > >> filter f_fw02 { > >> host("x.x.x.x") and match("PIX"); > >> }; > >> filter f_fw03 { > >> host("x.x.x.x") and match("PIX"); > >> }; > >> filter f_fw04 { > >> host("x.x.x.x") and match("PIX"); > >> }; > >> > >> #!!! not working yet.. . > >> filter f_pix{ > >> facility(local7) and match("%PIX*"); > >> }; > >> > >> filter f_iss { > >> match("issDaemon"); > >> }; > >> filter f_snmp { > >> match("ucd-snmp"); > >> }; > >> > >> ## everything else incoming > >> filter f_local7 { facility(local7); }; > >> > >> > >> ----- snip ------ > >> > >> TIA. > >> > >> ~!>D > >> _______________________________________________ > >> syslog-ng maillist - syslog-ng(a)lists.balabit.hu > >> https://lists.balabit.hu/mailman/listinfo/syslog-ng > >> Frequently asked questions at > http://www.campin.net/syslog-ng/faq.html > >> > >> > > > > > > > > ************************************************************** > ********************************** > > This E-mail message, including any attachments, is intended > only for the > person > or entity to which it is addressed, and may contain confidential > information. > If you are not the intended recipient, any review, retransmission, > disclosure, > copying, modification or other use of this E-mail message or > attachments is > strictly forbidden. > If you have received this E-mail message in error, please contact the > author and > delete the message and any attachments from your computer. > You are also advised that the views and opinions expressed in > this E-mail > message and any attachments are the author's own, and may not > reflect the > views > and opinions of FLEXTECH Television Limited. > ************************************************************** > ********************************** > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > > > > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > ************************************************************************************************ This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. If you are not the intended recipient, any review, retransmission, disclosure, copying, modification or other use of this E-mail message or attachments is strictly forbidden. If you have received this E-mail message in error, please contact the author and delete the message and any attachments from your computer. You are also advised that the views and opinions expressed in this E-mail message and any attachments are the author's own, and may not reflect the views and opinions of FLEXTECH Television Limited. ************************************************************************************************

2 2

RE: [syslog-ng]regex and priority(local7) question
by cdowns 03 Mar '04

03 Mar '04

I do have one quick question on the filter f_local7 { facility(local7) and level(debug..emerg); }; does this mean it is grabbing all level between debug - emerg, or just those 2 ? thanks again. ~!>D This config snippet works for me .... my PIXies are set to log via TCP1468 rather than UDP/514. Real IP Addresses replaced with fakes. As with your situation, I have other network gear running on local7 as well. Syslog-ng's wonderful filtration allows for a nice separation of all the classes of switch/router/firewall/VPN-bricks. Ted ...... source local { unix_stream("/dev/log" max-connections(200) keep-alive(yes) ); # UDP listeners for Internal syslog-ng interconnections udp(ip(127.0.0.1) port(514)); udp(ip(1.2.3.4) port(514)); }; source pixtcp { tcp(ip(1.2.3.4) port(1468) max-connections(10) keep-alive(yes)); }; filter f_local7 { facility(local7) and level(debug..emerg); }; filter f_pixmsg { match("%PIX"); }; filter f_misclocal7 { filter(f_local7) and not filter(f_pixmsg); }; destination d_local7 { file("/var/log/local7.debug" sync(0) ); }; destination d_pixlog { file("/var/log/pixlog.debug" sync(0) ); }; log { source(pixtcp); filter(f_local7); filter(f_pixmsg); destination(d_pixlog); }; log { source(local); filter(f_misclocal7); destination(d_local7); }; ...... >> -----Original Message----- >> From: syslog-ng-admin(a)lists.balabit.hu >> [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of cdowns >> Sent: Monday 01 March 2004 17:11 >> To: syslog-ng(a)lists.balabit.hu >> Subject: [syslog-ng]regex and priority(local7) question >> >> >> Good Morning, >> Im new to the list and have a couple questions. >> >> I have a PIX firewalls logging on local7.info, how can I >> match this >> correctly ? I also have switches logging on local7.info but I >> need this >> one pix removed and placed into its own destination. >> >> Where can i get a list of the regex syntax used in >> syslog-ng ? like >> perl / awk / grep etc.. . >> >> Here is what I currently have: >> >> ------ Snip ------- >> ## Regex >> filter f_fw01 { >> host("x.x.x.x") and match("PIX"); >> }; >> filter f_fw02 { >> host("x.x.x.x") and match("PIX"); >> }; >> filter f_fw03 { >> host("x.x.x.x") and match("PIX"); >> }; >> filter f_fw04 { >> host("x.x.x.x") and match("PIX"); >> }; >> >> #!!! not working yet.. . >> filter f_pix{ >> facility(local7) and match("%PIX*"); >> }; >> >> filter f_iss { >> match("issDaemon"); >> }; >> filter f_snmp { >> match("ucd-snmp"); >> }; >> >> ## everything else incoming >> filter f_local7 { facility(local7); }; >> >> >> ----- snip ------ >> >> TIA. >> >> ~!>D >> _______________________________________________ >> syslog-ng maillist - syslog-ng(a)lists.balabit.hu >> https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html >> >> > > ************************************************************************************************ This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. If you are not the intended recipient, any review, retransmission, disclosure, copying, modification or other use of this E-mail message or attachments is strictly forbidden. If you have received this E-mail message in error, please contact the author and delete the message and any attachments from your computer. You are also advised that the views and opinions expressed in this E-mail message and any attachments are the author's own, and may not reflect the views and opinions of FLEXTECH Television Limited. ************************************************************************************************ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

cycling filenames every n-secs?
by email lists 03 Mar '04

03 Mar '04

Hi All, Is it possible to have syslog-ng cycle filename every n-seconds (eg - every 5, 10 or 15 secs) instead of every minute or every second? Tia, Darren

1 0

Re:[syslog-ng]Log prefix under Solaris
by Amodiovalerio Verde 03 Mar '04

03 Mar '04

You can disable solaris message ID generation editing the file /kernel/drv/log.conf Just set msgid=0 instead of msgid=1. That way message IDs will not be generated. Amodiovalerio [Hypo] Verde ---------- Initial Header ----------- From : syslog-ng-admin(a)lists.balabit.hu To : "Syslog-NG" syslog-ng(a)lists.balabit.hu Cc : Date : Tue, 2 Mar 2004 11:34:52 +0100 Subject : [syslog-ng]Log prefix under Solaris > Hey again, > > All log messages that syslog-ng reads from sun-streams("/dev/log" > door("/etc/.syslog_door")); are prefixed with the following string: > "[ID 702911 local3.alert] " > > If I understand correctly, it seems that any program using the standard > syslog interface under Solaris will see there messages prefixed with an > ID and the facility they used. This is despite any formatting option I > can tell, this is simply the content of $MSG. > > This, however, doesn't happen with UDP messages. > > I wonder if there's a way to remove this part, either at on syslog-ng > side or in the calls to the syslog interface of Solaris? > > Regards, > > -- > Loïc Minier <lool(a)dooz.org> > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > >

2 1

sparc64 hostname weirdness with FreeBSD and NetBSD
by thomas+syslogng＠stromberg.org 03 Mar '04

03 Mar '04

I'm having problems with syslog-ng under both FreeBSD 5.2 and NetBSD 1.6.2 on the sparc64 architecture. For some reason, with my configuration, it thinks all log messages from remote hosts are actually coming from the current host. Even in chained hostname mode, all the messages came in with the hostname of the loghost itself. This seems to affect: FreeBSD 5.2 SPARC64 with libol 0.3.13 and syslog-ng 1.6.2 NetBSD 1.6.2 SPARC64 with libol 0.3.13 and syslog-ng 1.6.2 It seems to work perfectly fine with: FreeBSD 5.2 x86 with libol 0.3.12 and syslog-ng 1.6.1 OpenBSD 3.4 SPARC32 with libol ? and syslog-ng 1.5.24 While I went through many configurations, this is the last test case I used to demonstrate the issue: options { use_dns(yes);long_hostnames(off); chain_hostnames(on); owner(daemon); group(logreader); perm(0640); dir_perm(0750); dir_owner(daemon); dir_group(logreader); create_dirs(yes); }; source remote { tcp(); udp(); }; destination logremote { file("/var/log/ng/ALL.log"); }; log { source(remote); destination(logremote); }; Can anyone reflect a bit more on this? ----------------------------------------------------------- Thomas Stromberg <thomas(a)stromberg.org> Coordinator of UNIX Systems <tstrombe(a)indiana.edu> Department of Chemistry work://812-855-8450 Indiana University, Bloomington. cell://812-391-2843

1 0