August 2007 - syslog-ng - lists.balabit.hu

Custom Syslog-ng logs and remote syslog-ng
by Corey Bobb 22 Aug '07

22 Aug '07

Greetings, I am trying to accomplish something and am requiring some assistance. I have custom logs that write needed output to this directory: /production/server/20070822/xxxx.log This log file rotates every day so the next date will be 20070823 and so on. How can I direct a custom destination file to log remotely that log file to the central server and have it pick it up everyday? So for example . . . I have a central syslog server that writes all its input to /remote/logs/<date>/<HOST> On the server sending the remote logs I have this parameter: destination { file("/production/server/$YYYY$MM$DD/xxxx.log"); }; For some reason I think its not working properly due to the $DATE variables, I tried also using % to no avail. All the necessary logs do log remotely as I wish them to. Is this the proper way to configure this scenario? What I want to do is log the custom logging @ xxxx.log to the correct server $hostname on the central server This would be the idea: Custom.log -> local syslog-ng -> central-syslog-ng -> /remote/logs/<date>/<HOST> I am currently logging all the servers logging locally and remotely, but I cannot get syslog-ng to log that custome log file to the remote server. I hope this makes sense! Thanks. Regards, Corey M. Bobb Data Center Manager 850 South Concourse Suite 150 Maitland, FL 32751 D: 321.445-2150 M: 321-262-2845 F: 407.333.7701 cbobb(a)cygnus.com www.cygnus.com

1 0

Multiple woes with sles10sp1 / syslog-ng-1.6.8-20.18
by Robin P. Blanchard 22 Aug '07

22 Aug '07

# syslog-ng -d -f /etc/syslog-ng/syslog-ng.conf [snip] Cannot open file /export/syslog/10.10.0.2102007/08/21/messages for writing (Permission denied) # ls -ald /export/ ; ls -ald /export/syslog/ drwxr-xr-x 5 root root 4096 Aug 21 10:16 /export/ drwxrwsr-x 2 root root 4096 Aug 21 09:41 /export/syslog/ # fgrep "\$HOST" /etc/syslog-ng/syslog-ng.conf |fgrep messages destination localmessages { file("/export/syslog/$HOST/$YEAR/$MONTH/$DAY/localmessages"); }; destination messages { file("/export/syslog/$HOST/$YEAR/$MONTH/$DAY/messages"); }; #destination allmessages { file("/export/syslog/$HOST/$YEAR/$MONTH/$DAY/allmessages"); }; So...it would apppear that: 1) macro expansion is not working ? (I can hand hack-in an extra "/" to force that path to look correct) 2) syslog-ng cannot write to its output dir (regardless if I hack in that extra "/" Ideas ? # uname -a Linux auth 2.6.16.46-0.14-default #1 Thu May 17 14:00:09 UTC 2007 i686 i686 i386 GNU/Linux # rpm -qa |fgrep -i syslog syslog-ng-1.6.8-20.18 --------------------------------------- Robin P. Blanchard Systems Integration Specialist The University of Georgia Center for Continuing Education Conference Center & Hotel fon: 706.542.2404 < > fax: 706.542.1587 ---------------------------------------

3 3

REGEX rewrites on packet body possible?
by Eli Stair 21 Aug '07

21 Aug '07

I've got a problem with some network devices that is leading me to need to find some way to do regex rewriting of portions of the message body of syslog messages. Problem: I've got a bunch of Foundry devices that put their hostname followed by a comma in the body of the message (and some that do not). Some of these look like this: { 2007-08-16:2007-08-16T09:50:16-07:00 hostname [hostname.local7.notice] hostname, Linecard Module 13 temperature 50.0 C degrees is normal } # template("$R_ISODATE $HOST [$PROGRAM.$FACILITY.$PRIORITY] $MSG\n") As you can see from the template, the second hostname reported with the comma is part of the MSG body. For reasons of properly searching/indexing this data I need to strip this out. I've seen mention of a tool called 'syslog-mailer' that sounds like it would do the job somewhat. Additionally, I've seen blog chatter about potentially adding full regex rewrite capability to syslog-ng in the recent past. I can't find evidence of either of these methods however. My first try at solving this using an external program showed that when passing data OUT of syslog-ng to a defined program, only the message body is sent and before application of a template, the other information is dropped. Thus it's not possible to do processing of the whole payload externally, re-import the data via a socket and finish writing because the facility and HOST information is all gone! I'm looking at writing a log proxy using Net::Dev::Tools::Syslog in perl to handle listening, rewriting if necessary, and forwarding full messages on to syslog-ng after. I'd just like to know if there are any better suggestions, or if this has been done before successfully in another way? Cheers, and thanks for any insight. /eli

2 2

syslog-ng feature request
by Andreas Kurz 17 Aug '07

17 Aug '07

Hello, I am currently integrating syslog-ng as an HA cluster resource, because the remote syslog logging (MySQL error logfile on shared filesystem) for one resource should be moved around the cluster with the resource. I encounterd one syslog-ng feature that would be nice to have: A configureable path for the persistent state file (like the config file or pid file) to allow several instances of syslog-ng per node and to place a persistant state file also on a shared file system. Regards, Andreas

1 0

syslog-ng 2.0.5 does not remember last sent log line position for file() source
by Andreas Kurz 17 Aug '07

17 Aug '07

Hello all, I've installed syslog-ng 2.0.5 (compiled from src) on a CentOS 4.5 server and wanted to include a MySQL logfile via a file() source directive: ... source s_db_com { file("/var/lib/mysql/community/community.err" log_prefix("mysql-community: ") follow_freq(1) flags(no-parse)); }; filter f_db_com { match("mysql-community"); }; log { source(s_db_com); filter(f_db_com); destination(ha_local6); destination(ha_tcp); flags(final); }; ... But when I restart syslog-ng the complete MySQL error file is read and logged again by syslog-ng. Is there a special option that has to be set to get this feature working? Where does syslog-ng store its persistant data to get the information after a restart? Regards, Andreas

2 2

Re: [syslog-ng] Logging Third party application logs to Syslog-NG
by Wilson Lai 17 Aug '07

17 Aug '07

Dear Bill, If there is a Scalix application which generate the log file in "/var/opt/Scalix/ml/s/logs/fatal", where "fatal" is the log file. How could the source be defined to be listened by Syslog-NG? Thanks. Regards, Wilson Lai System Engineer IT Dept., SJM Office ( : (853)2978585 Mobile ( : (853)66506709 Email +: : wilsonlai(a)macausjm.com -----Original Message----- From: syslog-ng-request(a)lists.balabit.hu [mailto:syslog-ng-request@lists.balabit.hu] Sent: Thursday, August 16, 2007 6:00 PM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 28, Issue 20 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. GLib version problem? (Burns Andrew) 2. Re: GLib version problem? (Geller, Sandor (IT)) 3. Re: GLib version problem? (Valdis.Kletnieks(a)vt.edu) 4. Logging Third party application logs to Syslog-NG in Linux! (Wilson Lai) 5. Re: Logging Third party application logs to Syslog-NG in Linux! (Bill Nash) 6. Troubles with SE-Linux Syslog-ng and ntpd (Martin Voelker) ---------------------------------------------------------------------- Message: 1 Date: Wed, 15 Aug 2007 08:02:38 -0500 From: "Burns Andrew" <aburns(a)snyderdrug.com> Subject: [syslog-ng] GLib version problem? To: <syslog-ng(a)lists.balabit.hu> Message-ID: <699402D08697B94EAB9429B5083FAA1C042C2290(a)exchange_2k.snyderdrug.co m> Content-Type: text/plain; charset="us-ascii" Hello, I'm trying to install syslog-NG onto a Cent OS 5 server, and running into a problem. The problem appears to be more of a RedHat/Cent problem (or even Glib problem) rather than Syslog-NG problem, but I thought I'd check with the community before I harass Red Hat. I have the RPM verion of glib2.12 installed, however the libraries seemed to be labled 2.0.0. When I configure syslog-ng, I run across the following error: checking pkg-config is at least version 0.9.0... yes checking for GLIB... no configure: error: Cannot find GLib library version >= 2.2: is pkg-config in path? [root@syslog syslog-ng-2.0.5]# I have the rpm installed: [root@syslog syslog-ng-2.0.5]# yum list installed glib2 Loading "installonlyn" plugin Loading "fastestmirror" plugin Installed Packages glib2.i386 2.12.3-2.fc6 installed [root@syslog syslog-ng-2.0.5]# However the libraries seem to be mislabeled: [root@syslog syslog-ng-2.0.5]# locate glib | grep "/lib/libg" /lib/libglib-2.0.so.0 /lib/libglib-2.0.so.0.1200.3 [root@syslog syslog-ng-2.0.5]# Am I just missing something simple, or is this a problem I should be trying to speak to Cent/Red Hat about? Thanks! Andrew

2 1

Buffer Overflow : Insufficient buffer space for retrieving STREAMS log message; res='2'
by Nicolas de Marqué - Fromentin 17 Aug '07

17 Aug '07

Hello, During a big flow from a local server to a central server, we have a "buffer overflow" on the local server. After this message, syslog-ng is crash and don't be capable to send any messages. The local server is a solaris 10 product, the central linux a redhat EL4. Do you know a rule to stop this problem Best regard, Nicolas

3 2

new syslog-ng white paper published
by Balazs Scheidler 16 Aug '07

16 Aug '07

Dear all, We have published a new, syslog-ng related white paper on our website titled: "Distributed syslog architectures with syslog-ng" It basically contains the various deployment scenarios, their pros and cons we've encountered in our experience. It does contain information on syslog-ng Premium Edition, but still we tried to concentrate on the technical content, and some of it equally applies to syslog-ng OSE. We'd appreciate feedback at documentation(a)balabit.com. Thanks. -- Bazsi

1 0

Troubles with SE-Linux Syslog-ng and ntpd
by Martin Voelker 16 Aug '07

16 Aug '07

Hi, I have some troubles using syslog-ng on RHEL4 with SE Linux. There comes following message: audit(1187252020.581:514): avc: denied { write } for pid=2646 comm="ntpd" name="log" dev=tmpfs ino=14840162 scontext=root:system_r:ntpd_t tcontext=root:object_r:device_t tclass=sock_file What can I do??? Thanks Martin Westdeutsche Lotterie GmbH & Co. OHG | Sitz: Münster Registergericht: Amtsgericht Münster Handelsregister: Münster HRA 4379 Geschäftsführer: Dr. Winfried Wortmann Vorsitzender des Beirates: Ernst Gerlach Gesellschafter: Nordwestlotto in Nordrhein-Westfalen GmbH | Sitz: Münster Registergericht: Amtsgericht Münster Handelsregister: HRB 3840 Geschäftsführer: Dr. Winfried Wortmann NRW.BANK | Sitz: Düsseldorf und Münster Rechtsform: Anstalt des öffentlichen Rechts Registergerichte: Amtsgerichte Düsseldorf/Münster Handelsregister: Düsseldorf HRA 15277/Münster HRA 5300

1 0

Logging Third party application logs to Syslog-NG in Linux!
by Wilson Lai 16 Aug '07

16 Aug '07

Dear ALL, I am now using the Syslog-NG OSE for centralized logging system. How could I get the third party application (Linux Client) logs logging to the Syslog-NG server? These third party application logs are not managed by the Linux syslog daemon. Would there be someone gives some information or suggestion! Thanks a lot!!! Regards, Wilson Lai System Engineer IT Dept., SJM Office ( : (853)2978585 Mobile ( : (853)66506709 Email +: : wilsonlai(a)macausjm.com

2 1