February 2006 - syslog-ng - lists.balabit.hu

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

ignore src{}; contents at startup?
by Brian A. Seklecki 25 Oct '06

25 Oct '06

All: On FreeBSD, /dev/klog provides for output from the kernel (normally redirected to /dev/console before syslog starts). At startup, /etc/rc.d/dmesg copies the kernel boot hardware paramters to /var/run/dmesg.boot. If I have my src(); set to: source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ...at starup, syslog-ng drains the contents of /dev/klog. By default it tsends to them to user.notice. I like to log{} these to an SMS pager (so I know if a RAID is failing or other hardware problem). However, I'd like to avoid duplication of the efforts of /etc/rc.d/dmesg on behalf of syslog-ng(8) ...especially since I route user.notice to my pager, and kernel boot dmesg(8) can grow over 50 lines. /dev/klog really isn't a socket. It's a special char device. Is there anyway to tell syslog-ng(8) to ignore it's contents at startup? If it was tail(1), i'd say "tail -0 -f /file". Perhaps clear out the contents of /dev/klog somehow? TIA, ~BAS

2 2

Profiling syslog-ng
by John Morrissey 27 Mar '06

27 Mar '06

I'm trying to profile syslog-ng on a set of servers that generate high levels of logging traffic (a few GB/day). Currently, syslog-ng is consuming anywhere from 40-80% of a CPU on a dual Pentium IV Xeon machine. gprof(1) generates a profile, except for time consumed - it says "no time accumulated" even after running a profile-enabled syslog-ng for several minutes. For example: Flat profile: Each sample counts as 0.01 seconds. no time accumulated % cumulative self self total time seconds seconds calls Ts/call Ts/call name 0.00 0.00 0.00 667 0.00 0.00 xalloc 0.00 0.00 0.00 593 0.00 0.00 prepare 0.00 0.00 0.00 434 0.00 0.00 gc_mark 0.00 0.00 0.00 372 0.00 0.00 ol_string_free 0.00 0.00 0.00 313 0.00 0.00 do_prepare_write I've found similar complaints in the binutils list archives: http://sourceware.org/ml/binutils/2005-07/msg00364.html http://sourceware.org/ml/binutils/2002-01/msg00121.html http://sourceware.org/ml/binutils/2001-10/msg00315.html Personally, we're using glibc 2.3.2.ds1-22 and building the Debian syslog-ng 1.6.9-1 package using gcc 3.3.5-3 or 3.4.3-13. I've also tried oprofile, but it only(?) seems to report call counts, not time consumed. For example: samples % symbol name 102829 67.7349 io_iter 16560 10.9083 prepare 2975 1.9597 parse_log_msg 2518 1.6586 expand_macro I've read io_iter() and it doesn't seem to contain any smoking guns that would account for 68% of total syslog-ng CPU usage. Any suggestions on how to profile syslog-ng? I can post my configuration if it'd be helpful, but we've twisted many knobs (gc idle/busy thresholds, buffering via different sync() levels, disabling remote syslogging via UDP, etc.) without a major change in CPU consumption. john -- John Morrissey _o /\ ---- __o jwm(a)horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__

5 23

"MSG_DONTWAIT undeclared"-error during compiling libol-0.3.17 on HP-UX 11.00/11.11
by Tob_Sch＠gmx.de 22 Mar '06

22 Mar '06

Hi, compiling libol-0.3.16 (and syslog-ng-1.6.8) on HP-UX 11.00/11.11 worked without any problems. Today, I tried to compile libol-0.3.17 (and syslog-ng-1.6.9). I got following errors during 'make': Making all in src make all-am source='io.c' object='io.lo' libtool=yes \ depfile='.deps/io.Plo' tmpdepfile='.deps/io.TPlo' \ depmode=hp /bin/sh /tmp/libol-0.3.17/depcomp \ /bin/sh ../libtool --mode=compile cc -DHAVE_CONFIG_H -I. -I/tmp/libol-0.3.17/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c -o io.lo `test -f 'io.c' || echo '/tmp/libol-0.3.17/src/'`io.c cc -DHAVE_CONFIG_H -I. -I/tmp/libol-0.3.17/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c io.c -Wp,-M.deps/io.TPlo +Z -DPIC -o .libs/io.o cc: "io.c", line 493: error 1588: "MSG_DONTWAIT" undefined. cc: "io.c", line 493: warning 563: Argument #4 is not the correct type. *** Error exit code 1 Stop. *** Error exit code 1 Stop. *** Error exit code 1 Stop. It seems, that there is no MSG_DONTWAIT variable defined in the socket.h file on HP-UX like for example on SUSE 10.0. It seems, that the reference to MSG_DONTWAIT came new with libol-0.3.17. -- Lust, ein paar Euro nebenbei zu verdienen? Ohne Kosten, ohne Risiko! Satte Provisionen f�r GMX Partner: http://www.gmx.net/de/go/partner

4 8

remote logging not reliable
by stucky 13 Mar '06

13 Mar '06

guys I've been pulling my hair out here. I gonna go insane.... Here's the story. I first setup syslog-ng-1.6.6 for remote syslogging a year ago. I seemed to work out great - all my stuff appeared to be logged remotely and everything was dandy. Then I decided to come up with some sort of check that would tell me if a machine was not logging remotely anymore cause of network problems or whatever. I have a perl logchecker which I basically told to check for the cron.hourly log message and if it finds it to grep for the source host and add this host to a file. I was then gonna check this list against a pre-defined array of hosts that should be logging blablaba.... This is how I discovered that not everything is really logged. All my clients have this conf file: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); sync(1); stats(0); create_dirs(yes); owner(root); group(root); perm(0644); dir_perm(0755); use_dns(yes); dns_cache(yes); }; source src { internal(); unix-stream("/dev/log"); file("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; destination auth { file("/var/log/auth"); }; destination cron { file("/var/log/cron"); }; destination mail { file("/var/log/mail"); }; destination loghost { tcp("**********" port(5000)); }; filter auth { facility(auth, authpriv); }; filter cron { facility(cron); }; filter mail { facility(mail); }; filter nagios { not ( match("Accepted publickey for nagios from ********") or match("COMMAND=/usr/local/nagios/home/check_duplex") or match("session opened for user nagios") or match("session closed for user nagios")); }; log { source(src); filter(cron); destination(cron); destination(loghost); flags(final); }; log { source(src); filter(mail); destination(mail); destination(loghost); flags(final); }; log { source(src); filter(auth); filter(nagios); destination(auth); destination(loghost); flags(final); }; log { source(src); filter(nagios); destination(messages); destination(loghost); }; If I check the cronlog locally on a client I can see that all the hourly logs are there: Feb 27 12:01:01 sbeta crond[31964]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 13:01:01 sbeta crond[9232]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 14:01:01 sbeta crond[18969]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 15:01:01 sbeta crond[28630]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 16:01:01 sbeta crond[5311]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 17:01:01 sbeta crond[14977]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 18:01:01 sbeta crond[22335]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 19:01:01 sbeta crond[31995]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 20:01:01 sbeta crond[9336]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 21:01:01 sbeta crond[19002]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 22:01:01 sbeta crond[28663]: (root) CMD (run-parts /etc/cron.hourly) Feb 27 23:01:01 sbeta crond[5928]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 00:01:01 sbeta crond[15495]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 01:01:01 sbeta crond[25160]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 02:01:01 sbeta crond[2394]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 03:01:01 sbeta crond[12087]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 04:01:01 sbeta crond[21753]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 05:01:01 sbeta crond[31573]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 06:01:01 sbeta crond[8840]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 07:01:01 sbeta crond[18506]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 08:01:01 sbeta crond[28166]: (root) CMD (run-parts /etc/cron.hourly) Feb 28 09:01:01 sbeta crond[5433]: (root) CMD (run-parts /etc/cron.hourly) However, checking the remote cron logs for that machine for that whole of February shows this: Feb 21 12:01:01 sbeta crond[23754]: (root) CMD (run-parts /etc/cron.hourly) Feb 24 05:01:01 sbeta crond[5012]: (root) CMD (run-parts /etc/cron.hourly) Feb 24 17:01:01 sbeta crond[26525]: (root) CMD (run-parts /etc/cron.hourly) Feb 25 03:01:01 sbeta crond[28212]: (root) CMD (run-parts /etc/cron.hourly) Feb 25 13:01:01 sbeta crond[30890]: (root) CMD (run-parts /etc/cron.hourly) Feb 26 01:01:01 sbeta crond[20620]: (root) CMD (run-parts /etc/cron.hourly) Feb 16 19:01:01 sbeta crond[6299]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 00:01:01 sbeta crond[22138]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 04:01:01 sbeta crond[28292]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 08:01:01 sbeta crond[1917]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 12:01:01 sbeta crond[7812]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 16:01:01 sbeta crond[14136]: (root) CMD (run-parts /etc/cron.hourly) Feb 17 19:01:01 sbeta crond[10669]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 00:01:01 sbeta crond[26489]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 04:01:01 sbeta crond[32742]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 08:01:01 sbeta crond[6852]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 12:01:01 sbeta crond[13106]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 15:01:01 sbeta crond[9768]: (root) CMD (run-parts /etc/cron.hourly) Feb 18 20:01:01 sbeta crond[25588]: (root) CMD (run-parts /etc/cron.hourly) Feb 12 08:01:01 sbeta crond[22554]: (root) CMD (run-parts /etc/cron.hourly) Feb 12 12:01:01 sbeta crond[28808]: (root) CMD (run-parts /etc/cron.hourly) Feb 12 16:01:01 sbeta crond[2735]: (root) CMD (run-parts /etc/cron.hourly) Feb 12 20:01:01 sbeta crond[9063]: (root) CMD (run-parts /etc/cron.hourly) Feb 13 00:01:01 sbeta crond[15268]: (root) CMD (run-parts /etc/cron.hourly) Feb 14 17:01:01 sbeta crond[22845]: (root) CMD (run-parts /etc/cron.hourly) Feb 14 21:01:02 sbeta crond[29177]: (root) CMD (run-parts /etc/cron.hourly) Feb 15 22:01:01 sbeta crond[12945]: (root) CMD (run-parts /etc/cron.hourly) Feb 16 05:01:01 sbeta crond[17917]: (root) CMD (run-parts /etc/cron.hourly) Feb 16 16:01:01 sbeta crond[29758]: (root) CMD (run-parts /etc/cron.hourly) Feb 16 18:01:01 sbeta crond[16687]: (root) CMD (run-parts /etc/cron.hourly) As you can see it seemingly randomly logs this stuff now and then. So this machine hasn't been sending logs remotely since Feb 21. ? As I said all clients run the same version of syslog-ng and the exact same config - yet some of then seem to log cron stuff every hour and others don't. I get a log ot those 'Error connecting to remote host AF_INET' messages BUT i get those for EVERY host so why is there no consistency ? I know what everbody is gonna this - your network is hosed but here is the problem with that theory. I can log on to that box and I will immediately see this on the loghost in /var/log/auth Feb 28 10:43:06 sbeta sshd[21815]: Accepted publickey for stucky from ******** port 47427 ssh2 Feb 28 10:43:06 sbeta sshd(pam_unix)[21819]: session opened for user stucky by (uid=0) This will work every time for all other machines as well. I'd understand if it didn't log at all or with a slight delay like say 1 minute. Besides it appears cron stuff really gets lost but then why doesn't any of the auth/authpriv stuff get lost ? I can't help feeling it has something to do with logging cron but looking at the config I clearly told it to log to the local file first and then again to the remote host just like auth/authpriv stuff. Here is my logservers config: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); use_fqdn(no); use_dns(yes); dns_cache(yes); keep_hostname(yes); long_hostnames(off); sync(1); stats(0); create_dirs(yes); perm(0640); dir_perm(0750); }; source src { internal(); unix-stream("/dev/log"); file("/proc/kmsg"); udp (ip("*********")); tcp (ip("*********") port(5000) max-connections(1000) keep-alive(yes)); }; destination messages { file("/var/log/messages"); }; destination auth { file("/var/log/auth"); }; destination cron { file("/var/log/cron"); }; destination mail { file("/var/log/mail"); }; destination arch { file("/usr/local/var/log_archive/$HOST/$YEAR/$MONTH/$DAY/archive"); }; filter auth { facility(auth, authpriv); }; filter cron { facility(cron); }; filter mail { facility(mail); }; filter messages { not facility(auth, authpriv, cron, mail); }; filter nagios { not ( match("Accepted publickey for nagios from *******") or match("COMMAND=/usr/local/nagios/home/check_duplex") or match("session opened for user nagios") or match("session closed for user nagios")); }; filter junk { not ( match("Accepted publickey for oracle from ***********") or match("Accepted publickey for oracle from ***********") or match("Accepted publickey for stucky from ***********") or match("COMMAND=/usr/local/nagios/home/check_duplex") or match(".+reconnecting to LDAP server.+sleeping") or match("session opened for user root") or match(".+AF_INET client connected from .+") or match(".+AF_INET client dropped connection from.+") or match(".+Connection broken to AF_INET.+")); }; log { source(src); filter(cron); destination(cron); destination(arch); flags(final); }; log { source(src); filter(mail); destination(mail); destination(arch); flags(final); }; log { source(src); filter(auth); filter(nagios); destination(auth); destination(arch); flags(final); }; log { source(src); filter(junk); filter(nagios); destination(messages); destination(arch); }; And here's the really messed up thing. EVERY time I set up a tcpdump on the loghost at the full hour to see whether the box actually sends stuff to it - it does ! Yet, if I wait again for a while and check the logs I can see again that a log of hourly cronlogs are skipped. Can anybody please take a look at my configs and at least tell me that they are not completely wrong ? I'm running out of ideas. -- stucky

4 7

how can I change the priority/facility of a log entry?
by Dave Augustus 02 Mar '06

02 Mar '06

Hello All, I have spent hours looking for this. Is there a way, using syslog-ng to re-write an incoming log entry's priority/facility. We have some windows boxen that we are logging but the log entries need a little massaging. Any ideas would be helpful. TIA -- Dave Augustus <davea(a)support.kcm.org>

2 1

RE: [syslog-ng] Make Error messages
by Stuart Boyle 28 Feb '06

28 Feb '06

Removed 1.9.8 and installed 1.9.9 and it went very smoothly. Many thanks for the assistance. Stuart -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Tuesday, February 28, 2006 2:59 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Make Error messages On Tue, 2006-02-28 at 14:06 -0500, Stuart Boyle wrote: > The file cfg-lex.c is empty !! Now, that's what I suspected. This is a problem. Try removing the file or restore it from the tarball and recompile. (please note that I've release 1.9.9 which should be better than 1.9.8 wsa) -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] Make Error messages
by Stuart Boyle 28 Feb '06

28 Feb '06

Output from ./configure :/usr/local/syslog-ng-1.9.8 # ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for gcc option to accept ANSI C... none needed checking for bison... bison -y checking for flex... flex checking for yywrap in -lfl... yes checking lex output file root... lex.yy checking whether yytext is a pointer... yes checking whether make sets $(MAKE)... (cached) yes checking for ranlib... ranlib checking how to enable static linking for certain libraries... GNU or Solaris checking how to run the C preprocessor... gcc -E checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking dmalloc.h usability... no checking dmalloc.h presence... no checking for dmalloc.h... no checking for strings.h... (cached) yes checking openssl/ssl.h usability... no checking openssl/ssl.h presence... no checking for openssl/ssl.h... no checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking stropts.h usability... yes checking stropts.h presence... yes checking for stropts.h... yes checking sys/strlog.h usability... no checking sys/strlog.h presence... no checking for sys/strlog.h... no checking door.h usability... no checking door.h presence... no checking for door.h... no checking whether struct tm is in sys/time.h or time.h... time.h checking for struct tm.tm_gmtoff... yes checking for I_CONSLOG... no checking for door_create in -ldoor... no checking for socket in -lsocket... no checking for gethostbyname in -lnsl... yes checking for regexec in -lregex... no checking for strdup... yes checking for strtol... yes checking for inet_aton... yes checking for inet_ntoa... yes checking for getopt_long... yes checking whether to enable Sun STREAMS support... no checking whether to enable Sun door support... no checking for pkg-config... /usr/local/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for GLIB_CFLAGS... -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include checking for GLIB_LIBS... -L/usr/local/lib -lglib-2.0 checking for EVTLOG_CFLAGS... -I/usr/local/include/eventlog checking for EVTLOG_LIBS... -L/usr/local/lib -levtlog checking for static GLib libraries... yes configure: creating ./config.status config.status: creating dist.conf config.status: creating Makefile config.status: creating src/Makefile config.status: creating doc/Makefile config.status: creating doc/docvars.xml config.status: creating contrib/Makefile config.status: creating tests/Makefile config.status: creating tests/unit/Makefile config.status: creating tests/functional/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands Output from MAKE :/usr/local/syslog-ng-1.9.8 # make make all-recursive make[1]: Entering directory `/usr/local/syslog-ng-1.9.8' Making all in src make[2]: Entering directory `/usr/local/syslog-ng-1.9.8/src' gcc -g -O2 -Wall -g -o syslog-ng main.o libsyslog-ng.a -lnsl -Wl,-Bstatic -lfl -L/usr/local/lib -lglib-2.0 -L/usr/local/lib -levtlog -Wl,-Bdynamic libsyslog-ng.a(cfg.o)(.text+0x5d6): In function `cfg_new': /usr/local/syslog-ng-1.9.8/src/cfg.c:247: undefined reference to `lex_init' libsyslog-ng.a(cfg-grammar.o)(.text+0x17): In function `yyerror': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:829: undefined reference to `linenum' libsyslog-ng.a(cfg-grammar.o)(.text+0x18ea): In function `yyparse': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.c:1689: undefined reference to `yylex' collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/usr/local/syslog-ng-1.9.8/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/syslog-ng-1.9.8' make: *** [all] Error 2 The file cfg-lex.l contains the following: /usr/local/syslog-ng-1.9.8/src # more cfg-lex.l /*************************************************************************** * * Copyright (c) 1999 Balázs Scheidler * Copyright (c) 1999-2001 BalaBit IT Ltd. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. * * Inspired by nsyslog, originally written by Darren Reed. * * $Id: cfg-lex.l,v 1.8 2003/01/22 11:11:18 bazsi Exp $ * ***************************************************************************/ %{ #include "syslog-ng.h" #include "filter.h" #include "cfg-grammar.h" #include <string.h> #include <strings.h> struct keyword { char *kw_name; int kw_token; }; static struct keyword keywords[] = { /* statements */ { "source", KW_SOURCE }, { "destination", KW_DESTINATION }, { "log", KW_LOG }, { "options", KW_OPTIONS }, { "filter", KW_FILTER }, /* source or destination items */ { "file", KW_FILE }, { "fifo", KW_PIPE }, { "pipe", KW_PIPE }, { "internal", KW_INTERNAL }, { "unix_dgram", KW_UNIX_DGRAM }, { "unix_stream", KW_UNIX_STREAM }, { "udp", KW_UDP }, { "tcp", KW_TCP }, { "usertty", KW_USER }, { "door", KW_DOOR }, { "sun_stream", KW_SUN_STREAMS }, { "sun_streams", KW_SUN_STREAMS }, { "program", KW_PROGRAM }, /* option items */ { "flags", KW_FLAGS }, { "catchall", KW_CATCHALL }, { "fallback", KW_FALLBACK }, { "final", KW_FINAL }, { "flow_control", KW_FLOW_CONTROL }, { "no_parse", KW_NO_PARSE }, { "pad_size", KW_PAD_SIZE }, { "mark_freq", KW_MARK_FREQ }, { "stats_freq", KW_STATS_FREQ }, { "flush_lines", KW_FLUSH_LINES }, { "flush_timeout", KW_FLUSH_TIMEOUT }, /* obsolete */ { "sync_freq", KW_FLUSH_LINES }, { "sync", KW_FLUSH_LINES }, { "fsync", KW_FSYNC }, { "long_hostnames", KW_CHAIN_HOSTNAMES }, { "chain_hostnames", KW_CHAIN_HOSTNAMES }, { "keep_hostname", KW_KEEP_HOSTNAME }, { "keep_timestamp", KW_KEEP_TIMESTAMP }, { "ts_format", KW_TS_FORMAT }, { "time_zone", KW_TIME_ZONE }, { "recv_time_zone", KW_RECV_TIME_ZONE }, { "send_time_zone", KW_SEND_TIME_ZONE }, { "use_time_recvd", KW_USE_TIME_RECVD }, { "use_fqdn", KW_USE_FQDN }, { "use_dns", KW_USE_DNS }, { "gc_threshold", KW_GC_BUSY_THRESHOLD }, { "gc_busy_threshold", KW_GC_BUSY_THRESHOLD }, { "gc_idle_threshold", KW_GC_IDLE_THRESHOLD }, { "time_reopen", KW_TIME_REOPEN }, { "time_reap", KW_TIME_REAP }, { "follow_freq", KW_FOLLOW_FREQ }, { "file_template", KW_FILE_TEMPLATE }, { "proto_template", KW_PROTO_TEMPLATE }, { "log_fifo_size", KW_LOG_FIFO_SIZE }, { "log_fetch_limit", KW_LOG_FETCH_LIMIT }, { "log_iw_size", KW_LOG_IW_SIZE }, { "log_msg_size", KW_LOG_MSG_SIZE }, { "log_prefix", KW_LOG_PREFIX }, { "create_dirs", KW_CREATE_DIRS }, { "localip", KW_LOCALIP }, { "ip", KW_IP }, { "localport", KW_LOCALPORT }, { "port", KW_PORT }, { "destport", KW_DESTPORT }, { "owner", KW_OWNER }, { "group", KW_GROUP }, { "perm", KW_PERM }, { "dir_owner", KW_DIR_OWNER }, { "dir_group", KW_DIR_GROUP }, { "dir_perm", KW_DIR_PERM }, { "template", KW_TEMPLATE }, { "template_escape", KW_TEMPLATE_ESCAPE }, { "keep_alive", KW_KEEP_ALIVE }, { "max_connections", KW_MAX_CONNECTIONS }, { "mac", KW_MAC }, { "authentication", KW_AUTH }, { "encrypt", KW_ENCRYPT }, { "required", KW_REQUIRED }, { "allow", KW_ALLOW }, { "deny", KW_DENY }, { "compress", KW_COMPRESS }, { "dns_cache", KW_DNS_CACHE }, { "dns_cache_size", KW_DNS_CACHE_SIZE }, { "dns_cache_expire", KW_DNS_CACHE_EXPIRE }, { "dns_cache_expire_failed", KW_DNS_CACHE_EXPIRE_FAILED }, /* filter items */ { "or", KW_OR }, { "and", KW_AND }, { "not", KW_NOT }, { "level", KW_LEVEL }, { "priority", KW_LEVEL }, { "facility", KW_FACILITY }, { "program", KW_PROGRAM }, { "host", KW_HOST }, { "match", KW_MATCH }, /* on/off switches */ { "yes", KW_YES }, { "on", KW_YES }, { "no", KW_NO }, { "off", KW_NO } }; #define MAX_REGEXP_LEN 1024 int linenum = 1; int lex_filter_params = 0; char buf[MAX_REGEXP_LEN]; char *str; static int check_reserved_words(char *token); static void append_string(int length, char *str); static void append_char(char c); %} %option nounput white [ \t] digit [0-9] alpha [a-zA-Z] alphanum [a-zA-Z0-9] word [^ \#'"\{\}\\;\n\t,|\.] %x string %x qstring %% \#.*$ ; \n { linenum++; } {white}+ ; \.\. { return DOTDOT; } 0x{digit}+ { yylval.num = strtol(yytext, NULL, 16); return NUMBER; } 0{digit}+ { yylval.num = strtol(yytext, NULL, 8); return NUMBER; } {digit}+ { yylval.num = atoi(yytext); return NUMBER; } ({word}+(\.)?)*{word}+ { return check_reserved_words(yytext); } $ { return '('; } $ { return ')'; } \; { return ';'; } \{ { return '{'; } \} { return '}'; } \| { return '|'; } \, ; \" { str = buf; /* yy_push_state(string);*/ BEGIN(string); } \' { str = buf; BEGIN(qstring); } <string>\\a { append_char(7); } <string>\\n { append_char(10); } <string>\\r { append_char(13); } <string>\\t { append_char(9); } <string>\\v { append_char(11); } <string>\\[^anrtv] { append_string(1, yytext + 1); } <string>\" { BEGIN(INITIAL); /* yy_pop_state();*/ yylval.cptr = strdup(buf); return STRING; } <string>[^"\\]+ { append_string(strlen(yytext), yytext); } <qstring>[^']+ { append_string(strlen(yytext), yytext); } <qstring>\' { BEGIN(INITIAL); yylval.cptr = strdup(buf); return STRING; } %% int lex_init(FILE *file) { yyrestart(file); linenum = 1; return 0; } int check_reserved_words(char *token) { int i, j; for (i = 0; i < (sizeof(keywords) / sizeof(struct keyword)); i++) { for (j = 0; token[j] && keywords[i].kw_name[j]; j++) { if (token[j] == '-' || token[j] == '_') { if (keywords[i].kw_name[j] != '_') break; } else if (token[j] != keywords[i].kw_name[j]) break; } if (token[j] == 0 && keywords[i].kw_name[j] == 0) return keywords[i].kw_token; } yylval.cptr = strdup(token); return IDENTIFIER; } static void append_string(int length, char *s) { int to_copy = MIN(MAX_REGEXP_LEN - (str - buf) - 1, length); memcpy(str, s, to_copy); str += to_copy; *str = 0; } static void append_char(char c) { *str = c; str++; *str = 0; } The file cfg-lex.c is empty !! -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Tuesday, February 28, 2006 1:52 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Make Error messages On Mon, 2006-02-27 at 16:08 -0500, Stuart Boyle wrote: > Im getting the following errors when I run make, ./configure runs > fine, no errors. > > I have bison and flex installed. > > cyblops2a:/usr/local/syslog-ng-1.9.8 # make make all-recursive > make[1]: Entering directory `/usr/local/syslog-ng-1.9.8' > Making all in src > make[2]: Entering directory `/usr/local/syslog-ng-1.9.8/src' > gcc -g -O2 -Wall -g -o syslog-ng main.o libsyslog-ng.a -lnsl > -Wl,-Bstatic -lfl -L/usr/local/lib -lglib-2.0 -L/usr/local/lib > -levtlog -Wl,-Bdynamic > > libsyslog-ng.a(cfg.o)(.text+0x5d6): In function `cfg_new': > /usr/local/syslog-ng-1.9.8/src/cfg.c:247: undefined reference to > `lex_init' > libsyslog-ng.a(cfg-grammar.o)(.text+0x17): In function `yyerror': > /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:829: > undefined reference to `linenum' > libsyslog-ng.a(cfg-grammar.o)(.text+0x18ea): In function `yyparse': > /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.c:1689: > undefined reference to `yylex' > collect2: ld returned 1 exit status Now as I look at it again, it is not necessarily the flex version that is not correct, syslog-ng ships a generated cfg-lex.c file, and "linenum" variable is not something that -lfl exports but it is defined by cfg-lex.l (which is used to generate cfg-lex.c), as it seems you might have regenerated cfg-lex.c for some reason but it might be empty? Please try to include the make output that generated cfg-lex.c and the contents of your cfg-lex.c is also interesting. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

Segfault under RHEL4U2
by Paul Krizak 28 Feb '06

28 Feb '06

I just downloaded 1.9.9 and I'm getting a segfault for the following syslog-ng.conf: source s_external_syslog { udp(port(514)); }; destination d_re_mce_aggregator { program( "/bin/true" ); }; log { source(s_external_syslog); destination(d_re_mce_aggregator); }; I think that the problem is when programs exit unexpectedly and/or don't wait for input. I haven't run many more test cases to see exactly what's going on. The segfault drops two cores, 1MB and 256KB. If they would be helpful, please let me know and I can send them somewhere. -- Paul Krizak 5900 E. Ben White Blvd. MS 625 Advanced Micro Devices Austin, TX 78741 Linux/Unix Systems Engineering Phone: (512) 602-8775 Microprocessor Solutions Sector Cell: (512) 791-0686

3 3

Make Error messages
by Stuart Boyle 28 Feb '06

28 Feb '06

Im getting the following errors when I run make, ./configure runs fine, no errors. I have bison and flex installed. cyblops2a:/usr/local/syslog-ng-1.9.8 # make make all-recursive make[1]: Entering directory `/usr/local/syslog-ng-1.9.8' Making all in src make[2]: Entering directory `/usr/local/syslog-ng-1.9.8/src' gcc -g -O2 -Wall -g -o syslog-ng main.o libsyslog-ng.a -lnsl -Wl,-Bstatic -lfl -L/usr/local/lib -lglib-2.0 -L/usr/local/lib -levtlog -Wl,-Bdynamic libsyslog-ng.a(cfg.o)(.text+0x5d6): In function `cfg_new': /usr/local/syslog-ng-1.9.8/src/cfg.c:247: undefined reference to `lex_init' libsyslog-ng.a(cfg-grammar.o)(.text+0x17): In function `yyerror': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:829: undefined reference to `linenum' libsyslog-ng.a(cfg-grammar.o)(.text+0x18ea): In function `yyparse': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.c:1689: undefined reference to `yylex' collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/usr/local/syslog-ng-1.9.8/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/syslog-ng-1.9.8' make: *** [all] Error 2 Can anyone advise on a fix. Stuart

2 2