November 2009 - syslog-ng - lists.balabit.hu

Question about flags(ignore-case) option
by PAUL WILLIAMSON 30 Nov '09

30 Nov '09

I have the following filter set up: filter f_wireless_devices { host("system") flags(ignore-case); }; and I'm getting a syntax error. If I remove the flags(ignore-case), filter f_wireless_devices { host("system"); }; It is fine. On page 58 of the 3.0 sys admin guide, I have the syntax correct. Is there any other way to do this? Maybe put an ignore_case(yes) in the options sections at the top of the configuration file? Thanks, Paul ************************************ This email may contain privileged and/or confidential information that is intended solely for the use of the addressee. If you are not the intended recipient or entity, you are strictly prohibited from disclosing, copying, distributing or using any of the information contained in the transmission. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act and the Sarbanes-Oxley Act. You may not directly or indirectly reuse or disclose such information for any purpose other than to provide the services for which you are receiving the information. There are risks associated with the use of electronic transmission. The sender of this information does not control the method of transmittal or service providers and assumes no duty or obligation for the security, receipt, or third party interception of this transmission. ************************************

7 21

Troubles with the pattern database
by Guillaume Rousse 30 Nov '09

30 Nov '09

Hello list. I'm the mandriva maintainer for syslog-ng. I'm trying to play with the pattern database, with syslog-ng 3.0.4. I rebuild the package with the attached patch, so as to use /usr/share/syslog-ng a database location (a bit more FHS-compliant than /var), and extracted the files downloaded from http://www.balabit.com/downloads/files/patterndb there. However, loading them fails with this message: Error parsing pattern database file; filename='/usr/share/syslog-ng/patterndb.xml', error='Unexpected <rule> element' Error reloading pattern database, no pattern recognition will be done; It looks like some DB format issue. According to http://www.balabit.com/dl/html/syslog-ng-v3.0-guide-admin-en.html/ch08s06.h… the supported format is v1 until syslog-ng 3.0.2, and the NEWS file doesn't list any change here, while the patterndb file is already using v2. Am I correct ? And in this case, is there any way to easily convert the base to the old format ? -- BOFH excuse #252: Our ISP is having {switching,routing,SMDS,frame relay} problems diff -Naur -x '*~' syslog-ng-3.0.4/configure.in syslog-ng-3.0.4-fix-pattern-database-location/configure.in --- syslog-ng-3.0.4/configure.in 2009-04-30 18:32:19.000000000 +0200 +++ syslog-ng-3.0.4-fix-pattern-database-location/configure.in 2009-10-12 15:20:11.000000000 +0200 @@ -722,6 +722,7 @@ AC_DEFINE_UNQUOTED(PATH_SYSCONFDIR, "`patheval $sysconfdir`", [sysconfdir]) AC_DEFINE_UNQUOTED(PATH_LOCALSTATEDIR, "`patheval $localstatedir`", [local state directory]) AC_DEFINE_UNQUOTED(PATH_PIDFILEDIR, "`patheval $pidfiledir`", [local state directory]) +AC_DEFINE_UNQUOTED(PATH_DATADIR, "`patheval $datadir`", [datadir]) if test -n "$timezonedir"; then AC_DEFINE_UNQUOTED(PATH_TIMEZONEDIR, "`patheval $timezonedir`", [timezone base directory]) fi diff -Naur -x '*~' syslog-ng-3.0.4/src/syslog-ng.h syslog-ng-3.0.4-fix-pattern-database-location/src/syslog-ng.h --- syslog-ng-3.0.4/src/syslog-ng.h 2009-04-22 13:06:57.000000000 +0200 +++ syslog-ng-3.0.4-fix-pattern-database-location/src/syslog-ng.h 2009-10-12 15:14:10.000000000 +0200 @@ -52,7 +52,7 @@ #define PATH_PERSIST_CONFIG PATH_LOCALSTATEDIR "/syslog-ng.persist" #define PATH_QDISK PATH_LOCALSTATEDIR -#define PATH_PATTERNDB_FILE PATH_LOCALSTATEDIR "/patterndb.xml" +#define PATH_PATTERNDB_FILE PATH_DATADIR "/syslog-ng/patterndb.xml" #define LOG_PRIORITY_LISTEN 0 #define LOG_PRIORITY_READER 0

3 3

Beginner's questions for deploying syslog-ng
by Hailu Meng 29 Nov '09

29 Nov '09

Hi All, I'm planning create a centralized logging server for our network. We have 100 cisco routers and switches. Not too big organization. Suppose every cisco must have log to be sent to centralized logging server. I'm considering to deploy syslog-ng in CentOS 5. As I understand, I also need a web front end for user to search the log easily. Someone recommend splunk to do this front end job. I checked it out and found the free version of splunk has a 500MB/day for the indexing size. I'm not sure how this limit will affect on our network. Or is there any other good web front end for use with syslog-ng? Any help is appreciated. Lou

1 0

filter logger tags from syslog
by Jose Sanchez 28 Nov '09

28 Nov '09

Hello, I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie: "Nov 25 21:25:37 server1 logger:" I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file. Thanks in advance.

3 5

problems with syslog-ng-3.0.4 and -C option
by Chris Fabri 26 Nov '09

26 Nov '09

Greetings- I believe I've discovered an issue in syslog-ng 3.0.4. I've compiled this version on RH 3 (which might be the problem), and when you try for example: syslog-ng -C /var/log -u syslog-ng you get: Error opening configuration file; filename='/usr/local/etc/syslog-ng.conf', error='No such file or directory (2)' even specifying "-f /usr/local/etc/syslog-ng.conf" causes the same problem. Placing syslog-ng.conf in the directory specified by -C works, so perhaps the -C option has some issue with using the actual path instead of the chroot'ed path. I didn't see anything in the last several months of emails from the list, and I'm surprised this wouldn't have bitten more people, so it could be an RH3-ism, but wanted to make sure I reported it, in case it was more of a problem. chris

2 1

Some issues with syntax checking
by Guillaume Rousse 21 Nov '09

21 Nov '09

Hello list. Unresolved object errors seems not to be catched by 'syslog-ng --syntax-only', whereas they are fatal errors. For instance, with this configuration file: @version:3.0 log { source(s_foo); destination(d_foo); }; [root@oberkampf guillaume]# syslog-ng --syntax-only [root@oberkampf guillaume]# syslog-ng Error in configuration, unresolved source reference; source='s_foo' Moreover, the error message is sometimes a bit misleading. with this configuration file: @version:3.0 source s_foo { unix-stream ("/dev/log" max-connections(100)); }; log { source(s_foo); destination(d_foo); }; [root@oberkampf guillaume]# syslog-ng --syntax-only [root@oberkampf guillaume]# syslog-ng Error in configuration, unresolved destination reference; filter='d_foo' What is missing is a destination, not a filter. -- BOFH excuse #327: The POP server is out of Coke

2 1

log action or discard ?
by Jan Schaumann 18 Nov '09

18 Nov '09

Hello, I have a syslog server that receives syslog messages and will forward a selection of them to a central host via TLS. It will log all other messages to a local file. The current configuration looks something like this: destination forward { syslog(<options here>); }; destination all { file("/var/log/all"); }; filter forward { match("mumble"); }; log { all-my-sources filter(forward); destination(forward); flags(final); }; log { all-my-sources destination(all); }; Now if my remote server to which I'm forwarding messages to goes down or stops accepting connections for any reason, _all_ my logging will come to a grinding halt. That is, nothing gets appended to /var/log/all. What I want is for my forwarding to be a "try this or discard" kind of scenario. That is, if I can't forward the messages, just drop them, but continue to log all other messages unphased. Is that possible? If so, how? Thanks in advance, -Jan

1 0

Re: [syslog-ng] syslog-ng on solaris locks up after a while
by Balazs Scheidler 18 Nov '09

18 Nov '09

Thanks. It indeeed seems to be the same as Author: Balazs Scheidler <bazsi(a)balabit.hu> 2009-08-30 11:41:24 Committer: Balazs Scheidler <bazsi(a)balabit.hu> 2009-08-30 11:41:24 Parent: 1ad4da07d5305ba0140ac385d661ab6de25fc5f3 ([patterndb] estring parser length calculation must include ending quote) Child: c2e8aa58763a89cab58d05fb7a2b2a18021413b4 ([logmsg] added support for ASA timestamps) Branches: master, remotes/balabit/master, remotes/origin/master Follows: v3.0.4 Precedes: [afinter] don't block on the internal_msg_queue even in the threaded case (fixes: pub#48) A hang was reported in bugzilla ticket #48 which seems to have been caused by MARK messages interfering with local messages: * if the MARK is due in the same poll iteration as a local message * the MARK timeout is checked and the internal source is marked as having input available * then the local message comes in pushing the mark timeout further ahead in time * then the internal() dispatch callback checks the mark timeout again, but at this time it is already in the future -> * the dispatch callback falls back to fetching the internal message from internal_msg_queue, assuming it was that which caused the dispatch callback to be scheduled * this blocks indefinitely. The solution is very simple: use g_async_queue_try_pop() instead of g_async_queue_pop(), the dispatch code already takes care about a NULL message value. Thanks for the helpful reporters to hunt down the issue. Reported-By: Arkadiusz Miśkiewicz, Elan Ruusamäe A recent snapshot (and the git tree) should contain the fix. On Thu, 2009-11-12 at 11:10 -0500, Manassypov, Igor wrote: > Here is the information: > > (gdb) info thread > 2 process 153634 0xfed4b000 in _door_return () from /lib/libc.so.1 > * 1 process 88098 0xfed46df0 in __lwp_park () from /lib/libc.so.1 > (gdb) thread 1 > [Switching to thread 1 (process 88098 )]#0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 > (gdb) bt > #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 > #1 0xfed40c44 in cond_sleep_queue () from /lib/libc.so.1 > #2 0xfed40e08 in cond_wait_queue () from /lib/libc.so.1 > #3 0xfed41350 in cond_wait () from /lib/libc.so.1 > #4 0xfed4138c in pthread_cond_wait () from /lib/libc.so.1 > #5 0xff119d80 in g_async_queue_pop_intern_unlocked (queue=0x757e0, try=0, end_time=0x75618) at gasyncqueue.c:359 > #6 0xff119e80 in g_async_queue_pop (queue=0x757e0) at gasyncqueue.c:398 > #7 0x0003e984 in afinter_source_dispatch (source=0x8d260, callback=0x3e9dc <afinter_source_dispatch_msg>, user_data=0x8d1e0) > at afinter.c:112 > #8 0xff143564 in g_main_context_dispatch (context=0x8d158) at gmain.c:2144 > #9 0xff1459a4 in g_main_context_iterate (context=0x8d158, block=1, dispatch=1, self=0x76030) at gmain.c:2778 > #10 0xff146050 in g_main_context_iteration (context=0x8d158, may_block=1) at gmain.c:2841 > #11 0x0001bc20 in main_loop_run (cfg=0xffbffbc8) at main.c:149 > #12 0x0001c260 in main (argc=1, argv=0xffbffd44) at main.c:394 > (gdb) thread 2 > [Switching to thread 2 (process 153634 )]#0 0xfed4b000 in _door_return () from /lib/libc.so.1 > (gdb) bt > #0 0xfed4b000 in _door_return () from /lib/libc.so.1 > #1 0xff370bdc in door_return () from /lib/libdoor.so.1 > #2 0xff370c38 in door_create_func () from /lib/libdoor.so.1 > #3 0xfed46d54 in _lwp_start () from /lib/libc.so.1 > #4 0xfed46d54 in _lwp_start () from /lib/libc.so.1 > Backtrace stopped: previous frame identical to this frame (corrupt stack?) > (gdb) > > > > > ----------------------------------------------------------------------------------- > Igor R. Manassypov, M.Eng., P.Eng., CCIE 23032, CCVP > Network Architect, CI Investments > 416.795.3147 > > -----Original Message----- > From: Balazs Scheidler [mailto:bazsi@balabit.hu] > Sent: Thursday, November 12, 2009 11:02 AM > To: imanassypov(a)rogers.com > Cc: Syslog-ng users' and developers' mailing list; Pallagi Zoltán; Network > Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a while > > Hi, > > not really, are there multiple threads in the same core file? > > e.g. what is the response for "info threads"? > > It would be nice to have the backtrace for all threads, like this: > > (gdb) thread 1 > (gdb) bt > (gdb) thread 2 > (gdb) bt > > and so on, for each threadid that "info thread" lists. > > > On Fri, 2009-11-06 at 11:41 -0800, Igor Manassypov wrote: > > Would this one make more sense? > > > > > > > > bash-3.00# ps -eaf | grep syslog > > root 22562 22561 0 Nov 04 ? > > 0:30 /usr/local/sbin/syslog-ng > > root 22561 1 0 Nov 04 ? > > 0:00 /usr/local/sbin/syslog-ng > > > > bash-3.00# truss -f -p 22562 > > 22562/2: door_return(0x00000000, 0, 0x00000000, 0) > > (sleeping...) > > 22562/1: lwp_park(0x00000000, 0) (sleeping....) > > 22562/1: Received signal #11, SIGSEGV, in lwp_park() > > [default] > > 22562/1: siginfo: SIGSEGV pid=12717 uid=0 > > 22562/1: lwp_park(0x00000000, 0) Err#4 > > EINTR > > > > Core was generated by `/usr/local/sbin/syslog-ng'. > > Program terminated with signal 11, Segmentation fault. > > [New process 88098 ] > > [New process 153634 ] > > #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 #0 0xfed46df0 in > > __lwp_park () from /lib/libc.so.1 > > > > bash-3.00# gdb syslog-ng core > > > > Core was generated by `/usr/local/sbin/syslog-ng'. > > Program terminated with signal 11, Segmentation fault. > > [New process 88098 ] > > [New process 153634 ] > > #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 > > (gdb) > > > > --- On Tue, 11/3/09, Balazs Scheidler <bazsi(a)balabit.hu> wrote: > > > > From: Balazs Scheidler <bazsi@balabit..hu> > > Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a > > while > > To: imanassypov(a)rogers.com, "Syslog-ng users' and developers' > > mailing list" <syslog-ng(a)lists.balabit.hu> > > Cc: "Pallagi Zoltán" <pzolee(a)balabit.hu>, network(a)ci.com > > Date: Tuesday, November 3, 2009, 2:11 PM > > > > Hi, > > > > The problem is that you killed the supervisor process, which > > restarts > > syslog-ng in case it crashes.. However the hang is not in this > > part, but > > in its child. > > > > So by looking at the ps output, I'd say that in this situation > > you > > should have trussed 13621 and not its parent. > > > > On Tue, 2009-11-03 at 08:54 -0800, Igor Manassypov wrote: > > > Hi Zoltan, > > > > > > > > > Here are the traces: > > > > > > bash-3.00# ps -eaf | grep syslog > > > root 12694 12616 0 11:37:07 pts/1 0:00 grep > > syslog > > > root 13012 1 0 Oct 21 ? 0:00 syslog-ng > > -v > > > root 13013 13012 0 Oct 21 ? 0:41 syslog-ng > > -v > > > root 13620 1 0 Oct 08 ? > > > 0:00 /usr/local/sbin/syslog-ng > > > root 13621 13620 0 Oct 08 ? > > > 6:16 /usr/local/sbin/syslog-ng > > > bash-3.00# truss -f -p "13620" > > > 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED) > > > (sleeping...) > > > > > > 13620: Received signal #11, SIGSEGV, in waitid() > > [default] > > > 13620: siginfo: SIGSEGV pid=12717 uid=0 > > > 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED) > > Err#4 EINTR > > > > > > Core was generated by `/usr/local/sbin/syslog-ng'. > > > Program terminated with signal 11, Segmentation fault. > > > [New process 79156 ] > > > #0 0xfed4ad80 in _waitid () from /lib/libc.so.1 > > > (gdb) bt full > > > #0 0xfed4ad80 in _waitid () from /lib/libc.so.1 > > > No symbol table info available. > > > #1 0xfecee038 in _waitpid () from /lib/libc.so.1 > > > No symbol table info available. > > > #2 0xfed3a70c in waitpid () from /lib/libc.so.1 > > > No symbol table info available. > > > #3 0x0003017c in g_process_start () at gprocess.c:1042 > > > rc = 0 > > > deadlock = 0 > > > pid = 13621 > > > __PRETTY_FUNCTION__ = "g_process_start" > > > #4 0x0001c214 in main (argc=1, argv=0xffbffd14) at > > main.c:371 > > > cfg = (GlobalConfig *) 0x10034 > > > rc = 310272 > > > ctx = (GOptionContext *) 0x76030 > > > error = (GError *) 0x0 > > > > > > Please let me know if I can provide you with more > > information, > > > > > > Thanks! > > > > > > --- On Tue, 11/3/09, Pallagi Zoltán <pzolee(a)balabit.hu> > > wrote: > > > > > > From: Pallagi Zoltán <pzolee(a)balabit.hu> > > > Subject: Re: [syslog-ng] syslog-ng on solaris locks > > up after a > > > while > > > To: imanassypov(a)rogers.com, "Syslog-ng users' and > > developers' > > > mailing list" <syslog-ng(a)lists.balabit.hu> > > > Received: Tuesday, November 3, 2009, 11:10 AM > > > > > > Hi Igor, > > > > > > Can you show me truss output or backtrace of the > > stuck > > > syslog-ng?: > > > truss: > > > > > > truss -f -p "syslog-ng pid" > > > > > > backtrace: > > > > > > kill -11 "syslog-ng pid" (syslog-ng will drop a core > > file) > > > gdb syslog-ng core > > > bt full > > > > > > Igor Manassypov írta: > > > > Hello, > > > > > > > > > > > > I am having an issue with a solaris installation > > of the > > > > syslog-ng. It is configured such that all the logs > > are > > > > stored different per-ip folders. This is my > > centralized > > > > logging device, so it is fairly heavily loaded > > with > > > > receiving logs from a few dozen hosts. The > > syslog-ng process > > > > locks up every two to three weeks, with no > > messages logging > > > > to any of the files. The only way of getting it > > back is kill > > > > -9 the process and restart it. > > > > > > > > Is there any known issue of same sorts and is > > there any > > > > other way around it other than recycling the > > daemon every > > > > night? > > > > > > > > > > > > here is the version info: > > > > > > > > bash-3.00# syslog-ng --version > > > > syslog-ng 3.0.4 > > > > Revision: ssh > > > > > > +git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10 > > > > Compile-Date: Aug 11 2009 10:44:17 > > > > Enable-Threads: on > > > > Enable-Debug: off > > > > Enable-GProf: off > > > > Enable-Memtrace: off > > > > Enable-Sun-STREAMS: on > > > > Enable-Sun-Door: on > > > > Enable-IPv6: off > > > > Enable-Spoof-Source: on > > > > Enable-TCP-Wrapper: off > > > > Enable-SSL: on > > > > Enable-SQL: on > > > > Enable-Linux-Caps: off > > > > Enable-Pcre: on > > > > > > > > bash-3.00# uname -a > > > > SunOS prelude 5.10 Generic_137137-09 sun4v sparc > > SUNW,T5240 > > > > Thanks! > > > > > > > > -igor > > > > > > > > Igor Manassypov., M.Eng, P.Eng, CCIE 23032, CCVP > > Network > > > > Architect > > > > > > > > > > ____________________________________________________________ > > > > > > > > > > ______________________________________________________________________________ > > > > Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > -- > > Bazsi > > > > > > > -- > Bazsi > > ################################################################################### > This communication is confidential and may be > privileged. If you received it in error, please destroy > without copying and advise the sender. > > By submitting personal information to CI Investments, > you agree to the collection, use and disclosure of > such personal information for the purposes described > in our Privacy Policy available at www.ci.com. > > > Cette communication est confidentielle et pourrait être > privilégiée. Si vous la recevez par erreur, veuillez > l'éliminer sans en faire une copie et aviser l'expéditeur. > > Lorsque vous soumettez des renseignements > personnels à Placements CI, vous nous permettez de > conserver, utiliser et divulguer ces renseignements > personnels aux fins décrites dans nos Principes > directeurs en matière de protection des renseignements personnels > qui sont disponibles au www.ci.com. > > ################################################################ > -- Bazsi

1 0

udp6 source declaration causes syslog+tls to fail
by Jan Schaumann 17 Nov '09

17 Nov '09

Hello, I'm seeing what appears to be an odd interaction between the declaration of an udp6 source and trying to forward messages via tls to another host. I have the following configuration: # UDP will use the regular port 514 as per /etc/services. source s_udp { udp(log_fetch_limit(400000000)); # UDP6 definition requires an explicit port statement. # See upstream bug # https://bugzilla.balabit.com/show_bug.cgi?id=58 # udp6( # port(514) # log_fetch_limit(400000000) # ); }; destination d_amslog { syslog("<hostname>" port(6514) suppress(30) transport("tls") tls( ca_dir("<dir>") cert_file("<cert>") key_file("<key>") trusted_keys("<fp>") ) ); }; log { source(s_udp); destination(d_amslog); flags(final); }; Now this works just fine with the "udp6" part commented out as above. However, if I uncomment the udp6 part, I get the following error: ERROR:gsockaddr.h:86:g_sockaddr_inet_get_sa: assertion failed: (g_sockaddr_inet_check(s)) failed! Note that if I remove the destination from the log directive (ie comment out the line "destination(d_amslog);", I still get the same error. If I comment out the entire syslog directive and replace it with a file destination, there is no problem (even though the udp6 source definition is there). It seems as if -- and I'm guessing here -- the udp6 declaration causes syslog-ng to try to treat the subsequent "syslog" directive to be an IPv6 type connection, which fails since the host does not (currently) have IPv6 configured. Ie, I believe syslog-ng assumes that just by the existence of an IPv6 source all connections will be IPv6 (which, if correct, seems like a bug to me). The version of syslog-ng here is: syslog-ng 3.0.3 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#08c9bf9d98e4e021756adc12c42605840140ba8b Compile-Date: Sep 12 2009 02:43:20 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on Any suggestion on how to fix this would be appreciated. Thanks in advance, -Jan

1 1

Dear syslog-ng@lists.balabit.hu SALE 80% OFF on Pfizer
by Pfizer Inc 17 Nov '09

17 Nov '09

1 0