- syslog-ng - lists.balabit.hu

unsubscribe
by Roy G Davis 08 Sep '04

08 Sep '04

-----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Horváth Ákos Sent: Wednesday, September 08, 2004 6:12 AM To: syslog-ng(a)lists.balabit.hu Cc: Iulian Topliceanu Subject: Re: [syslog-ng]filtering question Hi, AFAIK you do not need a _matching_ filter, but to set up a filter on the local5 logging _facility_. bye MaXX On Wednesday 08 September 2004 12.03, Iulian Topliceanu wrote: > Hi, > > I have to following task. > > I have some messages coming from a host from local5. > > The messages look like: > > "Sep 6 17:07:21 herakles local5:meimlo.std.me herakles.tomcat8310 > 2004-09-06 17:07:21,157 [WARN] [main] [] [] > voicemail.mboxp.MboxpServlet: MboxpServlet not initialized" > > The thing I have to is to make a filter that matches the things > between 'local5:' and the first ' '. In this case the filter is > 'meimlo.std.me' but this 'meimlo.std.me' could be anything like: > 'blah.blu.admin'. > > Is there a possibility to create a filter that if that string is > different, syslog-ng will automate create a new file? Like it does > when a new host is logging and it creates the dirs/files. > > Does syslog-ng support such a function? > > Regards, > Iulian Topliceanu _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

filtering question
by Iulian Topliceanu 08 Sep '04

08 Sep '04

Hi, I have to following task. I have some messages coming from a host from local5. The messages look like: "Sep 6 17:07:21 herakles local5:meimlo.std.me herakles.tomcat8310 2004-09-06 17:07:21,157 [WARN] [main] [] [] voicemail.mboxp.MboxpServlet: MboxpServlet not initialized" The thing I have to is to make a filter that matches the things between 'local5:' and the first ' '. In this case the filter is 'meimlo.std.me' but this 'meimlo.std.me' could be anything like: 'blah.blu.admin'. Is there a possibility to create a filter that if that string is different, syslog-ng will automate create a new file? Like it does when a new host is logging and it creates the dirs/files. Does syslog-ng support such a function? Regards, Iulian Topliceanu

2 1

Missing pieces of information when destination = program
by mrugan 06 Sep '04

06 Sep '04

Hy list, My goal is to log to a linux host (slackware 9) running syslog-ng 1.6.5, libol 0.3.14, libnet 1.1.2.1 information broadcasted by multiple windows hosts running MBM (mother board monitor) via syslog logging mechanism that MBM has build in. MBM is set up to broadcast via udp 514 at 300 seconds interval readings from 12 sensors. Syslog-ng message gathering setup via filters was required options { chain_hostnames(no); create_dirs (yes); dir_perm(0755); dns_cache(yes); keep_hostname(yes); log_fifo_size(2048); log_msg_size(16348); long_hostnames(on); perm(0644); stats(3600); sync(0); time_reopen (10); use_dns(no); use_fqdn(no); }; # filter f_mbm { facility(daemon) and match("MBM"); }; # test destination to pipe destination mbm_pipe { pipe("/tmp/mbm-pipe" template("$WEEKDAY $DATE $YEAR $HOST $MESSAGE\n")); }; # wanted to be the real-time analyzer destination mbm_pipe { program("/var/rrd/test.pl" template("$WEEKDAY $DATE $YEAR $HOST $MESSAGE\n")); }; # # log setup log { source(s_udp); source(s_internal); filter(f_mbm); destination(mbm_pipe); }; # however i encountered these problems: on the typical setup destination to a fifo pipe (mkfifo ...) every 300 seconds arrive 12 lines looking like: Sat Sep 4 14:53:54 2004 TAQWA MBM[CaseTemp]: C=39 LA=5 HA=70 L=32 H=42 A=38 Sat Sep 4 14:53:54 2004 TAQWA MBM[CPUTemp]: C=35 LA=5 HA=70 L=31 H=45 A=35 Sat Sep 4 14:53:54 2004 TAQWA MBM[VCore]: C=1.60 LA=1.90 HA=2.10 L=1.57 H=1.60 A=1.59 Sat Sep 4 14:53:54 2004 TAQWA MBM[VAGP]: C=1.60 LA=1.90 HA=2.10 L=1.60 H=1.60 A=1.60 Sat Sep 4 14:53:54 2004 TAQWA MBM[VDimm]: C=2.50 LA=3.13 HA=3.46 L=2.50 H=2.50 A=2.50 Sat Sep 4 14:53:54 2004 TAQWA MBM[Vio]: C=3.26 LA=4.75 HA=5.25 L=3.23 H=3.28 A=3.26 Sat Sep 4 14:53:54 2004 TAQWA MBM[5V]: C=5.11 LA=11.40 HA=12.60 L=5.00 H=5.11 A=5.09 Sat Sep 4 14:53:54 2004 TAQWA MBM[12V]: C=12.22 LA=-11.40 HA=-12.60 L=12.10 H=12.59 A=12.27 Sat Sep 4 14:53:54 2004 TAQWA MBM[5VSB]: C=5.43 LA=4.95 HA=5.05 L=5.43 H=5.46 A=5.43 Sat Sep 4 14:53:54 2004 TAQWA MBM[VBat]: C=3.15 LA=4.75 HA=5.25 L=3.15 H=3.15 A=3.15 Sat Sep 4 14:53:54 2004 TAQWA MBM[CPUFan]: C=3013 LA=2600 HA=99999 L=2909 H=3068 A=2974 Sat Sep 4 14:53:54 2004 TAQWA MBM[CPUMHz]: C=1760 LA=0 HA=100 L=1760 H=1760 A=1760 nothing to worry, processing goes on with sucessfull results. beeing greedy, a simple perl script was written to illustrate capability of syslog-ng to log via program. open(FILE, ">> /tmp/mbm-file") or die "Can't open FILE : $!\n"; while (<>) { print FILE $_; } close (FILE); this time i noticed pieces of info missing, random lines beeing truncated, no special pattern could be distinguished, writing stops randomly while MBM logs whole 12 lines. ex: Sat Sep 4 14:53:54 2004 TAQWA MBM[CPUFan] or even Sat Sep 4 ... and so on... please advise. thanks a log. :) mrugan |_at_| softhome.net

2 1

No new snapshots since 2004-08-05
by José Pedro Oliveira 06 Sep '04

06 Sep '04

Hi, No new snapshots of libol and syslog-ng have been created since 2004-08-05 (release date of syslog-ng 1.6.5 and libol-0.3.14). Would it possible to correct this? Thanks in advance, jpo Snapshots directory: http://www.balabit.com/downloads/syslog-ng/1.6/src-snapshot/ -- José Pedro Oliveira * mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo * * gpg fingerprint = F9B6 8D87 859D 1C94 48F0 84C0 9749 9EB5 91BD 851B *

2 3

syslog-ng performance problems
by west＠x-dot.ch 05 Sep '04

05 Sep '04

Hallo I have replaced the original syslog with syslog-ng. everything works fine. the only problem I have is that syslog-ng uses between 50% and 70% of the cpu. I think that it is a configuration problem because the old syslogd never uses more than 10% of the cpu. thank you for your help stefan here are my configuration files: traditional syslog: # /etc/syslog.conf syslog configuration file.################################### # output to local file "messages" for automatic log file analysis *.err;auth,daemon,mark,kern.debug;mail,user.notice /var/adm/messages # display emergencies on all terminals (uses WALL) *.emerg * #print time on console every 20mins (not needed if you have contool) #mark.* /dev/console kern.info ifdef(`LOGHOST', /var/log/kernlog, @loghost) user.info ifdef(`LOGHOST', /var/log/userlog, @loghost) mail.info ifdef(`LOGHOST', /var/log/maillog, @loghost) daemon.info ifdef(`LOGHOST', /var/log/daemonlog, @loghost) auth.info ifdef(`LOGHOST', /var/log/authlog, @loghost) lpr.info ifdef(`LOGHOST', /var/log/lprlog, @loghost) news,uucp.info ifdef(`LOGHOST', /var/log/newslog, @loghost) cron.info ifdef(`LOGHOST', /var/log/cronlog, @loghost) ## other "local" messages not yet used local0,local1.info ifdef(`LOGHOST', /var/log/local0log, @loghost) local2,local3,local4.info ifdef(`LOGHOST', /var/log/local2log, @loghost) local5.info ifdef(`LOGHOST', /var/log/local5log, @loghost) local6.info ifdef(`LOGHOST', /var/log/local6log, @loghost) local7.info ifdef(`LOGHOST', /var/log/local7log, @loghost) # Put all alerts (& higher) into a seperate log: *.err ifdef(`LOGHOST', /var/log/alertlog, @loghost) ########################################################################### syslog-ng: # # Syslog-ng configuration for SUN Solaris # # Copyright (c) 1999 anonymous # Copyright (c) 1999 Balazs Scheidler # Copyleft 2004 Stefan Wenger # $Id: syslog-ng.conf.sample,v 1.2 1999/11/15 12:30:41 bazsi Exp $ # # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # options { keep_hostname(yes); time_reopen (1); time_reap(300); use_dns(yes); use_fqdn(no); use_time_recvd(yes); dns_cache(yes); dns_cache_expire(3600); dns_cache_expire_failed(10); sync(4); gc_idle_threshold(300); gc_busy_threshold(1000); log_fifo_size(16777216); log_msg_size(8192); chain_hostnames(no); owner(root); group(nobody); perm(0644); dir_perm(0755); create_dirs(yes); }; source src { sun_streams("/dev/log" door("/etc/.syslog_door")); internal (); }; source net { udp(); }; destination alertlog { file("/var/log/alertlog"); }; destination messages { file("/var/adm/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; destination kernlog { file("/var/log/kernlog"); }; destination userlog { file("/var/log/userlog"); }; destination maillog { file("/var/log/maillog"); }; destination daemonlog { file("/var/log/daemonlog"); }; destination authlog { file("/var/log/authlog"); }; destination lprlog { file("/var/log/lprlog"); }; destination newslog { file("/var/log/newslog"); }; destination cronlog { file("/var/log/cronlog"); }; destination local0log { file("/var/log/local0log"); }; destination local2log { file("/var/log/local2log"); }; destination local5log { file("/var/log/local5log"); }; destination local6log { file("/var/log/local6log"); }; destination local7log { file("/var/log/local7log"); }; destination fallbacklog { file("/var/log/fallbacklog"); }; destination loghost { udp("loghost"); }; #destination xconsole { pipe("/dev/xconsole"); }; destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date,time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG','$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; filter f_emerg { level(emerg); }; filter f_kern_info {facility(kern) and not priority(debug); }; filter f_user_info {facility(user) and not priority(debug); }; filter f_mail_info {facility(mail) and not priority(debug); }; filter f_daemon_info {facility(daemon) and not priority(debug); }; filter f_auth_info {facility(auth) and not priority(debug); }; filter f_lpr_info {facility(lpr) and not priority(debug); }; filter f_news_info {(facility(news) or facility(uucp)) and not priority(debug); }; filter f_cron_info {facility(cron) and not priority(debug); }; filter f_local0 {facility(local0, local1) and not priority(debug); }; filter f_local2 {facility(local2, local3, local4) and not priority(debug); }; filter f_local5 {facility(local5) and not priority(debug); }; filter f_local6 {facility(local6) and not priority(debug); }; filter f_local7 {facility(local7) and not priority(debug); }; filter f_err {priority(err..emerg); }; filter f_messages {priority(err..emerg) or facility(auth, daemon, kern) or (facility(mail, user) and priority(notice..emerg)); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emerg); destination(console); }; log { source(src); filter(f_kern_info); destination(kernlog); destination(loghost); }; log { source(src); filter(f_user_info); destination(userlog); destination(loghost); }; log { source(src); filter(f_mail_info); destination(maillog); destination(loghost); }; log { source(src); filter(f_daemon_info); destination(daemonlog); destination(loghost); }; log { source(src); filter(f_auth_info); destination(authlog); destination(loghost); }; log { source(src); filter(f_lpr_info); destination(lprlog); destination(loghost); }; log { source(src); filter(f_news_info); destination(newslog); destination(loghost); }; log { source(src); filter(f_cron_info); destination(cronlog); destination(loghost); }; log { source(src); filter(f_local0); destination(local0log); destination(loghost); }; log { source(src); filter(f_local2); destination(local2log); destination(loghost); }; log { source(src); filter(f_local5); destination(local5log); destination(loghost); }; log { source(src); filter(f_local6); destination(local6log); destination(loghost); }; log { source(src); filter(f_local7); destination(local7log); destination(loghost); }; log { source(src); filter(f_err); destination(alertlog); destination(loghost); }; log { source(src); destination(fallbacklog); flags(fallback); }; log { source(net); filter(f_messages); destination(messages); }; #log { source(net); filter(f_emerg); destination(console); }; log { source(net); filter(f_kern_info); destination(kernlog); destination(loghost); }; log { source(net); filter(f_user_info); destination(userlog); destination(loghost); }; log { source(net); filter(f_mail_info); destination(maillog); destination(loghost); }; log { source(net); filter(f_daemon_info); destination(daemonlog); destination(loghost); }; log { source(net); filter(f_auth_info); destination(authlog); destination(loghost); }; log { source(net); filter(f_lpr_info); destination(lprlog); destination(loghost); }; log { source(net); filter(f_news_info); destination(newslog); destination(loghost); }; log { source(net); filter(f_local0); destination(local0log); destination(loghost); }; log { source(net); filter(f_local2); destination(local2log); destination(loghost); }; log { source(net); filter(f_local5); destination(local5log); destination(loghost); }; log { source(net); filter(f_local6); destination(local6log); destination(loghost); }; log { source(net); filter(f_local7); destination(local7log); destination(loghost); }; log { source(net); filter(f_err); destination(alertlog); destination(loghost); }; log { source(net); destination(fallbacklog); flags(fallback); }; ## MYSQL #log { source(src); destination(d_mysql); }; #log { source(net); destination(d_mysql); };

3 2

setting (or changin) facility / level feature request
by Dave Johnson 03 Sep '04

03 Sep '04

Syslog-ng is working relatively well; however one performance item: We have a named pipe configured for non syslog messages and want to add another named pipe from a separate message source (again non-syslog based). Messages are aggregated and sent centrally. Centrally messages are broken out /$DATE/$HOST/$FACILITY. ---- Initially messages were just "routing" based on the above information (date/host/facility), creating a nice-straight-forward higher performance configuration. Now the central server and many of our client machines have to do "extra" items (stamp messages, filter based on certain text), etc... We are talking about 15-20 gigabytes a day of logs (today), and much more tomorrow. Going through an external program or msg filtering adds a decent amount of CPU on all our machines, especially centrally. It would be very helpful to have a "set _ facility/level" or (change_facility, change_level) in the destination() or src () or log() section of syslog-ng. This would allow our client machines to be able to src from several files and put data in local0-7. - Dave Johnson

2 1

logging to a named pipe?
by Michael Rash 03 Sep '04

03 Sep '04

Hi - I would like to use syslog-ng to log priority info kernel messages to a named pipe, and I seem to have an issue with defining a correct filter. This config works: destination psadpipe { pipe("/var/lib/psad/psadfifo"); }; filter f_kerninfo { level(info); }; log { source(src); filter(f_kerninfo); destination(psadpipe); }; We can see that it works because syslog-ng has opened the named pipe under the output of lsof: # lsof |grep psadfifo syslog-ng 22675 root 8u FIFO 3,3 699187 /var/lib/psad/psadfifo However, the above config should really use the filter: filter f_kerninfo { facility(kern) and level(info); }; But, the addition of the kern facility in the filter seems to cause syslog-ng to not open the named pipe (lsof returns nothing). Not being intimately familiar with syslog-ng, can anyone give me any pointers? I'm running syslog-ng 1.6.4 on Gentoo Linux. Thanks, --Mike Michael Rash http://www.cipherdyne.org/ Key fingerprint = 53EA 13EA 472E 3771 894F AC69 95D8 5D6B A742 839F

3 3

RFE: enable tagging of sources
by Enrico Scholz 03 Sep '04

03 Sep '04

Hello, does there exists a way to tag messages from certain sources? E.g. I want to use syslog-ng in linux-vserver[1] enabled hosts. There, the syslog-ng daemon is running on the hosts, and I have lots of logging-sources at /vservers/*/dev/log. Theses logging services are behaving like normal hosts and have own hostnames. For example, the syslog-ng daemon on host 'host' is configured with | source s_vserver_www { unix-stream ("/vservers/www/dev/log"); }; This source is located in a vserver with the hostname 'www'. But when a logmessage is issued there, I get | DATE host: MSG but I would like to get | DATE www: MSG The destination options are supporting 'template()' already and I tried something like | destination d_vserver_www { | tcp("127.0.0.1" port(520) template("<$TAG>$DATE www $MSG\n")); | }; | | log { source(s_vserver_www); destination(d_vserver_www); }; Unfortunately, this does not really work because: * $TAG is hex-encoded; but the other side expects decimal-encoding * every destination spawns a new stunnel-connection; because I have >30 vservers which would result in 30 destinations, this will become very expensive. Probably the best solution would be, when the hostname can be overridden on a per-source base. E.g. something like | source s_vserver_www { unix-stream ("/vservers/www/dev/log" hostname("www")); }; Enrico Footnotes: [1] http://linux-vserver.org

2 1

SIGHUP: messages logged in the wrong order
by José Pedro Oliveira 02 Sep '04

02 Sep '04

When syslog-ng receives the SIGHUP signal it logs the following two messages out of order: ... Sep 2 21:21:51 pateta syslog-ng[13490]: new configuration initialized Sep 2 21:21:50 pateta syslog-ng[13490]: SIGHUP received, restarting syslog-ng ... Shouldn't the first message have been logged after the one reporting the reception of the SIGHUP signal (check the timestamps)? Tested syslog-ng versions: Syslog-ng 1.6.5 in a Fedora Core 2 system Syslog-ng 1.6.2 in a Red Hat 7.3 system Regards, jpo -- José Pedro Oliveira * mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo * * gpg fingerprint = F9B6 8D87 859D 1C94 48F0 84C0 9749 9EB5 91BD 851B *

1 0

Logging to syslog without facility mail
by Laurent CARON 31 Aug '04

31 Aug '04

Hello I'm trying to exclude certain messages from my syslog (facility mail which are inferior to warning) I tried filter f_syslog { not facility(auth, authpriv) and not facility(mail); }; It removes all the mail facility from syslog and then filter f_syslog { not facility(auth, authpriv) or facility(mail) and level(warn .. emerg); }; which is not better at all. Does anyone have a clue about it? Thanks Laurent

2 1