- syslog-ng - lists.balabit.hu

Re: [syslog-ng] syslog-ng Digest, Vol 39, Issue 20
by Anurag Agarwal 12 Aug '08

12 Aug '08

Hello Bazsi, I did some more observation and I have following more precise inputs 1) Same config file is working correctly in my lab Solaris v240 machine. Filtered syslog sent to destined user only. 2) In production the destined user is receiving logs twice ( 2 copies). It means one copy for itself because it is destination user and 2nd copy is because all users are copied. 3) If you telnet to server and don't enter login/password even, still you start receiving a copy of message. ï»¿ Please help ! Anurag aggarwal -----Original Message----- From: Anurag Agarwal Sent: Thursday, July 24, 2008 4:54 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: RE: syslog-ng Digest, Vol 39, Issue 20 Thanks Bazsi for reply. Relevant portion is below ************************************************************** destination console { usertty("wimaxnoc"); }; filter ps_nt_re { match("PS NOT REACHABLE"); }; log { source(network); filter(ps_nt_re); destination(console); }; ************************************************************** Please help. Anurag aggarwal -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request(a)lists.balabit.hu Sent: Thursday, July 24, 2008 3:30 PM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 39, Issue 20 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: Re : Re : Re : Syslogd + Syslog-ng (Christopher Cashell) 2. Re: Re : Re : Re : Syslogd + Syslog-ng (Leandro Ferreira da Silva) 3. Re: Using regexp in match() (Balazs Scheidler) 4. Re: Usertty sending messages to all user and without login also (Balazs Scheidler) ---------------------------------------------------------------------- Message: 1 Date: Wed, 23 Jul 2008 10:04:34 -0500 From: Christopher Cashell <ChristopherCashell(a)solutionary.com> Subject: Re: [syslog-ng] Re : Re : Re : Syslogd + Syslog-ng To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <48874882.8070904(a)solutionary.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Leandro Ferreira da Silva did thus speak on 7/23/2008 7:14 AM: > The configuration is the standart, I only do the follow changes. > > # sources > source src { unix-dgram("/var/run/log"); > unix-dgram("/var/run/logpriv" perm(0600)); > udp(); internal(); file("/dev/klog"); }; > > I add this, > source r_src { udp(ip("*client.domain*") port(514)); }; Here's your problem. For src, you define 'udp()' as one of your log sources, without including any specific options for it. This will cause syslog-ng to go with its defaults for udp(), which is bind to *all* IP addresses assigned to the box, on port 514. Then, for r_src, you are telling syslog-ng to bind to a specific IP address on the box and port 514. When syslog-ng attempts to start, it will fail with the error message you received because that ip/port is already in use (by the source src, which is bound to all IP's). Unless you have a need to bind to a specific IP address on the box, I'd recommend removing 'udp();' from src, and replacing the current 'udp(ip("*client.domain*") port(514));' in r_src with 'udp();'. -- Christopher Cashell ------------------------------ Message: 2 Date: Wed, 23 Jul 2008 14:39:03 -0300 From: Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> Subject: Re: [syslog-ng] Re : Re : Re : Syslogd + Syslog-ng To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <48876CB7.7080909(a)iqm.unicamp.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Christopher Cashell wrote: > Leandro Ferreira da Silva did thus speak on 7/23/2008 7:14 AM: >> The configuration is the standart, I only do the follow changes. >> >> # sources >> source src { unix-dgram("/var/run/log"); >> unix-dgram("/var/run/logpriv" perm(0600)); >> udp(); internal(); file("/dev/klog"); }; >> >> I add this, >> source r_src { udp(ip("*client.domain*") port(514)); }; > > Here's your problem. For src, you define 'udp()' as one of your log > sources, without including any specific options for it. This will > cause syslog-ng to go with its defaults for udp(), which is bind to > *all* IP addresses assigned to the box, on port 514. > > Then, for r_src, you are telling syslog-ng to bind to a specific IP > address on the box and port 514. When syslog-ng attempts to start, it > will fail with the error message you received because that ip/port is > already in use (by the source src, which is bound to all IP's). > > Unless you have a need to bind to a specific IP address on the box, > I'd recommend removing 'udp();' from src, and replacing the current > 'udp(ip("*client.domain*") port(514));' in r_src with 'udp();'. > It worked!! The real problem was udp () at src. Thank you very much for all the help... Now I'll go complete my rules.. See you!! =P ------------------------------ Message: 3 Date: Thu, 24 Jul 2008 10:16:27 +0200 From: Balazs Scheidler <bazsi(a)balabit.hu> Subject: Re: [syslog-ng] Using regexp in match() To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Cc: documentation(a)balabit.com Message-ID: <1216887387.8130.5.camel(a)bzorp.balabit> Content-Type: text/plain; charset=UTF-8 On Tue, 2008-07-22 at 09:45 +0200, Jan Kreps wrote: > > ------------ P?vodn? zpr?va ------------ > > Od: Jan Kreps <krepsj(a)seznam.cz> > > P?edm?t: Re: [syslog-ng] Using regexp in match() > > Datum: 10.7.2008 10:29:35 > > ---------------------------------------- > > > > > > I'm trying to setup central syslog-ng server for my Exchange > > > > servers. On > > > windows servers I use Epilog agent (brother of Snare) forwarding > > > tracking logs to central syslog. > > > Obviously I have used bad regexp. Exchange tracking log uses tabs as delimiters. > > But when I saved $MSG string to text log, tabs was changed to '\011'. > > > > So now I changed my regexp to use tabs as delimiters: > > > > filter f_parsing { > > match("([^\t]*)\t([^\t]*)\t"); > > }; > > > > This works like charm and saves first two tab delimited fields (date > > and time in this case) to $1 and $2. > > > > Some more remarks to subject. > > Syslog-ng Administrator guide 1.1.1 Edition from May 2008 says on page 111: > > "The regular expressions can use up to 255 regexp matches ($1 ... $255)." > > and on page 112: > > "Regarding braces around macro names, the following two formats are equivalent "$MSG" and "${MSG}"." > > Fact is that for regexp matches greater than 9 it MUST be in ${} format. If you use for instance $12 it resolves as content of $1 + character '2'. I found that I have to use ${} syntax in ChangeLog remark for patch 137. I guess that should be stated in Admin Guide more clearly. In this respect, statement "The regular expressions can use up to 255 regexp matches ($1 ... $255)" is not true and should be corrected. > right, this should be fixed in the documentation. > And I have one question about file creation. > > When something is logged a file is created according to destination(). But when I delete the file, it's not created anymore, until restart of syslog-ng. Is that correct/expected behaviour? > > I run Debian Etch and syslog-ng 2.0.9-1 from unstable branch (in > stable is still version 2.0.0) Yes, you need to send the HUP signal to syslog-ng in order to reopen log files. This is the way UNIX works. (the application does not know about the deletion of a logfile) -- Bazsi ------------------------------ Message: 4 Date: Thu, 24 Jul 2008 10:28:06 +0200 From: Balazs Scheidler <bazsi(a)balabit.hu> Subject: Re: [syslog-ng] Usertty sending messages to all user and without login also To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <1216888086.8130.10.camel(a)bzorp.balabit> Content-Type: text/plain On Wed, 2008-07-23 at 13:39 +0530, Anurag Agarwal wrote: > Hello everybuddy, > > I am using syslog-ng and used usertty in destination. I am using a user > 'test' as destination. But other users can also see the message on their > console e.g root and other users. Please help as this is creating a > problem and NG not working as desired. > OS: Solaris 10 > It does work for me. Can you show us the relevant portions of your configuration file? I tried with this: source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; destination all { file("/var/log/messages"); }; log { source(local); destination(all); }; destination all_users { usertty("bazsi"); }; filter alert_emerg { level(err..emerg); }; log { source(local); filter(alert_emerg); destination(all_users); }; It correctly printed err..emerg messages on my tty, and not on others. The relevant portion of the code is here: #if HAVE_MODERN_UTMP if (ut->ut_type == USER_PROCESS && ((self->username->len == 1 && self->username->str[0] == '*') || (self->username->len <= sizeof(ut->ut_user) && memcmp(self->username->str, ut->ut_user, self->username->len) == 0))) #else if ((self->username->len == 1 && self->username->str[0] == '*') || (self->username->len <= sizeof(ut->ut_name) && memcmp(self->username->str, ut->ut_name, self->username->len) == 0)) #endif I don't see how this condition could match for everyone. Even if the wtmp format is different and ut->ut_name would contain a bogus value, the expected breakage would be to not display anywhere, not to display everywhere. -- Bazsi ------------------------------ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng End of syslog-ng Digest, Vol 39, Issue 20 *****************************************

1 0

destination drivers
by hi ro 12 Aug '08

12 Aug '08

Hi To listen on the specified UDP port for incoming messages over IPv6, must I use "udp6()" destination driver? Can I use "udp()" destination driver? Thanks _________________________________________________________________ あなたも総裁候補に？世界に一つだけのオリジナルブログパーツを作ろう！ http://questionbox.jp.msn.com/soudantou/

2 1

individual files
by Ryan Eldridge 11 Aug '08

11 Aug '08

Hello all. After playing around with syslog-ng all afternoon, needless to say I'm a little confused and frustrated. I have managed to get it running on most of my Linux boxes and it to accept data from a couple of them still running syslog which is what I want. The issue I'm facing is the config file I found do to the wonders of Google is splitting the information based on host but not into separate files like I'm use to with syslog aka cron, messages, syslog etc.. what i'm looking for output is the following: ls -shl /var/log/HOSTS/peter.domain.com/2008/08 4.0K -rw------- 1 root root 866 2008-08-10 20:46 auth.log 0 -rw-r--r-- 1 root root 0 2008-08-10 17:22 cron.log 0 -rw-r--r-- 1 root root 0 2008-08-10 17:22 daemon.log 4.0K -rw------- 1 root root 2.8K 2008-08-10 18:26 debug 8.0K -rw-r--r-- 1 root root 6.6K 2008-08-10 14:36 dmesg 0 -rw-r----- 1 root root 0 2002-04-06 19:13 maillog 12K -rw------- 1 root root 8.2K 2008-08-10 20:05 messages 0 -rw-r--r-- 1 root root 0 2008-08-10 18:22 ntpd 0 -rw-r----- 1 root root 0 1994-05-09 03:06 secure 0 -rw-r----- 1 root root 0 2002-03-09 00:29 spooler 12K -rw------- 1 root root 8.6K 2008-08-10 20:05 syslog Currently i'm getting the following: ls -shl /var/log/HOSTS/peter.domain.com/2008/08 12K -rw------- 1 root root 11K 2008-08-10 21:31 10 which contains all the results. I'll post the file I'm using below any help would be useful ############################################################### # First, set some global options. options { use_fqdn(yes); use_dns(yes); dns_cache(yes); keep_hostname(yes); long_hostnames(off); sync(1); log_fifo_size(1024); }; ############################################################### # # Logs may come from unix stream, and UDP:514 # source src { pipe("/proc/kmsg"); unix-stream("/dev/log"); internal(); udp(ip("192.168.2.3") port(514)); tcp(ip("192.168.2.3") port(514) keep-alive(yes)); }; ############################################################### # First some standard logfile # destination authlog { file("/var/log/auth.log"); }; destination syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination user { file("/var/log/user.log"); }; #destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; ########################################## # Here's the filter options. With this rules, we can set which # message go where. filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(auth, authpriv) and not facility(mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; #filter f_mail { facility(mail); }; filter f_user { facility(user); }; #filter f_news { facility(news); }; #filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; ############################################################### # # log statements actually send logs somewhere, to a file, across the network, etc # log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_authpriv); destination(std); }; log { source(src); filter(f_syslog); destination(std); }; log { source(src); filter(f_cron); destination(std); }; log { source(src); filter(f_daemon); destination(std); }; log { source(src); filter(f_daemon); destination(std); }; log { source(src); filter(f_kern); destination(std); }; log { source(src); filter(f_user); destination(std); }; #log { source(src); filter(f_debug); destination(std); }; log { source(src); filter(f_messages); destination(std); }; ## set up logging to loghost #destination loghost { # tcp("10.0.0.1" port(514)); #}; # send everything to loghost, too #log { # source(src); # destination(loghost); #}; # # automatic host sorting (usually used on a loghost) # # set it up destination std { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$FACILITY_$HOST_$YEAR_$MONTH_$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes) ); }; # log it log { source(src); destination(std); };

2 1

Logrotate signal instead of HUP
by Jonas Eriksson 11 Aug '08

11 Aug '08

Hello The thing that i miss the most in modern syslog daemons is the ability to only release the local files without also releasing sockets for syslog servers. I also saw that this was mentioned earlier on this list[1] but no one seems to have written the patches for this until now. I have however created patches for this, that may be found at [2]. There are however one problem: In order to figure out which LogPipes that should be closed is now done by comparing logdriver->super.init with the affile_dd_init-function pointer. If it's a match, the LogDriver is a AFFileDestDriver. A more elegant solution could be to require some larger changes. My suggestion is to include a gboolean "logrotate_release" in the LogDriver, or similar. I have not done any of this, since this would be best to discuss first. I am also not sure what to do with write_hash in AFFileDestDriver. Should these be reinited as well? If that is the case, i can write the patch required for that as well. I have spent alot of time reading the code and trying to make the patch fit in with the style of the rest of the code. I hope I succeeded at this. I also have a git repos with the changes available at [3] and patches for the Ubuntu syslog-ng packages for 6.06 (dapper) and 7.04 (hardy) if it's of interest to anyone. I have been testing these patches by throwing alot of both USR1 and HUP signals on the running deamon and it seems to behave very well. I'm thinking about rolling them out in production next week. Your opinions are welcome. With regards, Jonas, HPC2N [1] https://lists.balabit.hu/pipermail/syslog-ng/2002-November/004063.html [2] http://www.acc.umu.se/~zqad/syslog-ng-2.0-git.diff [3] /afs/hpc2n.umu.se/home/z/zqad/Public/repos/syslog-ng-2.0/ ------------------------------------- - Jonas Eriksson, zqad(a)hpc2n.umu.se - - sysadmin @ {acc,hpc2n}.umu.se - -------------------------------------

2 1

OpenBSD + Syslog-NG 2.x Users ?
by Brian A. Seklecki 01 Aug '08

01 Aug '08

Please ping me on or off-list. ~BAS

1 1

Stripping numerals in the destination
by Cliff Fogle 01 Aug '08

01 Aug '08

I have several server types, like web001-web100 and thumb001-thumb010 or db001-db004 I want each server type to log to the same file and I can't for the life of me figure out a regex for $HOST in the destination statement to strip numerals. Where the first $HOST is below I want to do something like a sed "s/[0-9]*$//" destination d_host { file("/log1/syslog/$HOST/$R_YEAR/$R_MONTH/$R_YEAR-$R_MONTH-$R_DAY" template("$ISODATE <$FACILITY.$PRIORITY> $HOST $MSG\n") template_escape(no) ); }; Thanks for any help you can provide. I've been banging my head against google for hours now...

2 1

AI_ADDRCONFIG missing from netdb.h on OpenBSD
by Brian A. Seklecki 31 Jul '08

31 Jul '08

obsd43# grep -ir AI_ADDRCONFIG /usr/include/* obsd43# fbsd63$ grep -ir AI_ADDRCONFIG /usr/include/* /usr/include/lwres/netdb.h:#define AI_ADDRCONFIG 0x00000020 Seems to be fine if you just cram it into: w-syslog-ng-2.0.9/syslog-ng-2.0.9/tests/loggen/loggen.c I suppose that we'll need to maintain it as a local patch. l8* -lava (Brian A. Seklecki - Pittsburgh, PA, USA) http://www.spiritual-machines.org/ "Guilty? Yeah. But he knows it. I mean, you're guilty. You just don't know it. So who's really in jail?" ~Maynard James Keenan

2 1

Example 3.4. Script to feed a growing logfile into syslog-ng
by Chris Pratt 31 Jul '08

31 Jul '08

Followed http://www.syslog.org/syslog-ng/v2/#id2535001 and created some tail logger combos to feed some growing logfiles into syslog-ng. Issue that I am need to get around is that while I haven't missed capturing any messages this way the time stamps in syslog-ng seem to be when logger actually gets the line into syslog rather than that of the tailed files line. I have a very quick-filling file (logged 143 messages yesterday all at 14:15:18) and when I view syslog-ng these are stamped up to 14:15:36 and no longer in any order. Does anyone have any suggestions? Thanks.

4 5

The best Front end interface to syslog-ng 2.0.9
by VPN Cluster 30 Jul '08

30 Jul '08

What is the best and easy to install web interface to syslog-ng 2.0.9 open source.

1 0

Re: [syslog-ng] Example 3.4. Script to feed a growing logfileinto syslog-ng
by Alan Edmonds 25 Jul '08

25 Jul '08

This is how I capture garbage collections with tomcat. On the tomcat host in the startup.sh script. Create a named pipe, catalinia.gc Run logger to read named pipe and forward to syslog-ng host. I use a "tag" (or program) to identify it as a gc.out. I created an $INSTANCE for the different tomcats running on a single host. .. rm -f "$CATALINA_BASE"/logs/catalina.gc mkfifo "$CATALINA_BASE"/logs/catalina.gc ... "$_RUNJAVA" $JAVA_OPTS $CATALINA_OPTS \ -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \ -Dcatalina.base="$CATALINA_BASE" \ -Dcatalina.home="$CATALINA_HOME" \ -Djava.io.tmpdir="$CATALINA_TMPDIR" \ -Xloggc:"$CATALINA_BASE"/logs/catalina.gc -XX:-DisableExplicitGC \ org.apache.catalina.startup.Bootstrap "$@" start 2>&1 | logger -p local2.info -t tomcat$INSTANCE & if [ ! -z "$CATALINA_PID" ]; then echo $! > $CATALINA_PID fi fi logger -p local3.info -t GC$INSTANCE < "$CATALINA_BASE"/logs/catalina.gc & .. On the syslog-ng server, it sorts it out like this: filter f_gc { facility(local3); }; destination host-program-out { file("/logs/portal/$HOST/$PROGRAM.out" template ("$MSG\n") template_escape(no) ); }; log { source(local); filter(f_gc); destination(host-program-out); }; you get a GC$INSTANCE.out file in $HOST/ on the syslog-ng server. But, you have to strip the GC$INSTANCE: from each line of the .out file before running it through HPJtune, etc. you get the idea. Alan Edmonds -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: 25 July 2008 08:35 To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Example 3.4. Script to feed a growing logfileinto syslog-ng On Thu, 2008-07-24 at 16:10 -0400, Chris Pratt wrote: > Thanks for your suggestion. This is for Tomcat. Unfortunately I am > stuck reading from the log because the syslog appender in the log4j > version on our app does not capture stack traces while the file > appenders do. In other cases we want to capture garbage collections > which are only written to stdout. I tried sending my file appends to > /dev/log but that didn't work so I am stuck trying to read from files. > Only my central log server runs syslog-ng. All of my Tomcat hosts > just use syslogd and send *.* @mylogserver. Otherwise I'd probably > try # source s_tail { file("log.log" follow_freq(1) flags(no-parse)); > }; to see if it kept it in order any better. > > What I could really use is a logger that has the ability to read and > forward to syslog the existing timestamp... syslog-ng could that if it was running on the client hosts too. but otherwise what about this dirty trick: tail -f <logfile> | sed -e 's/^/<133>/' | nc syslog-server 514 If the timestamps are otherwise in syslog-ng processable format, the above scriptlet would work. The sed sets local6.notice as a priority. However the difficult part is in doing the "tail -f" part robust enough, but I guess you achieved that already. -- Bazsi ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html T-Mobile International UK Limited Company Registered Number: 3951860 Registered Office Address: Hatfield Business Park, Hatfield, Hertfordshire, AL10 9BW Registered in England and Wales NOTICE AND DISCLAIMER This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose

1 0