- syslog-ng - lists.balabit.hu

syslog-ng plugins
by Balazs Scheidler 06 Mar '10

06 Mar '10

Hi, This is a repost of my blog article which I've found important enough to post here as well. The original article is at: http://bazsi.blogs.balabit.com/2010/03/plugins-branch-updated.html It is about a git branch in the syslog-ng OSE 3.1 git repository, as such it is available at: git://git.balabit.hu/bazsi/syslog-ng-3.1.git Plugins branch updated ---------------------- Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called "afsocket". The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory. By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL. And the same plugin framework will enable me to create a wide variety of plugins. My ideas: * Plugins for all syslog-ng components (source, destination, filter, rewrite, parser) * Python scriptability (a simple correllation engine in Python?) * macro transformation functions, for example: $(stripslashes $macro), usable anywhere in templates and stripslashes a plugin that is invoked whenever such an expansion occurs * Hooks for transforming the log message as it enters syslog-ng (to fix parsing errors for example), Do you have other ideas? Please post them as comments or as emails to the mailing list. Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :) -- Bazsi

1 0

problem with filtering in syslog-ng
by Evan Baer 06 Mar '10

06 Mar '10

Hello, I'm trying to exclude ntpd update messages from /var/log/messages. I have created a filter like so: filter not_ntpd { not program("ntpd*"); }; And implemented it in my log lines: log { source(src); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; Yet the lines for ntpd still seem to pass through to the logfile. Feb 23 09:20:48 magpie-shn1e1324 ntpd[37397]: synchronized to 10.2.253.22, stratum 1 Feb 23 09:20:49 magpie-shn1f1318 ntpd[729]: kernel time sync status change 2001 Any thoughts? -- Evan

6 11

what is the 'supervising syslog-ng' process?
by Tim Edwards 06 Mar '10

06 Mar '10

We have syslog-ng-client-3.0.5-1.rhel5 installed on several machines. However on 2 of those Nagios has flagged that there's an extra process with the command 'supervising syslog-ng'. Both machine1 and machine2 are Scientific Linux 5.4 (a RHEL clone) x86_64 Xen virtual machines with the exact same syslog-ng.conf: [root@machine1 ~]# ps -ef | grep syslog-ng root 24891 1 0 09:26 ? 00:00:00 supervising syslog-ng root 24892 24891 0 09:26 ? 00:00:00 /opt/syslog-ng/sbin/syslog-ng --no-caps [root@machine2 ~]# ps -ef | grep syslog root 1718 1695 0 09:38 pts/0 00:00:00 grep syslog root 2137 1 0 Feb01 ? 00:00:09 /opt/syslog-ng/sbin/syslog-ng --no-caps Can someone explain what this 'supervising' process is? Thanks Tim Edwards

3 3

composing file names with system varible
by peceka 06 Mar '10

06 Mar '10

Hi, i've on my machine file with some name, for example: # cat /var/hostname smeagol now i want to have log files like this: /var/log/smeagol-all.log if i change content of /var/hostname and then restart syslog-ng, syslog-ng shoud start logging to files with new name. is it possible? mabye no directly but with some workaround? regards, p.

3 4

RFC3339 timestamp parsing
by Ian Wienand 06 Mar '10

06 Mar '10

Hi, I notice that in the syslog-ng from my Debian install (2.0.9) log_msg_parse_date has p = strptime(self->date.str, "%Y-%m-%dT%H:%M:%S", &tm); and in 3.1beta2 I see p = (guchar *) strptime((gchar *) date, "%Y-%m-%d T%H:%M:%S", &tm); Is the extra space intended? I'm also wondering about a seeming ambiguity in RFC3339 [1] In 5.6 it says NOTE: ISO 8601 defines date and time separated by "T". Applications using this syntax may choose, for the sake of readability, to specify a full-date and full-time separated by (say) a space character. However in Appendix A it says ISO 8601 states that the "T" may be omitted under some circumstances. This grammar requires the "T" to avoid ambiguity. So is the T required or not? Should syslog-ng be able to parse either format? -i [1] http://www.ietf.org/rfc/rfc3339.txt

2 1

pcre matches
by Patrick H. 06 Mar '10

06 Mar '10

How do you use pcre named pattern matches with store-matches? I've been trying to do things like message("IP: (?<ip>\S+)" flags(store-matches) type("pcre")) and then insert into a database with things like values("$ip") or values("$+{ip}") and nothing works. I cant seem to find any examples of usage through google either.

2 2

Using amazon s3 as logging backend
by Rory Toma 23 Feb '10

23 Feb '10

Has anyone used amazon s3 as logging back end yet?

1 0

Re: [syslog-ng] syslog-ng gets started twice when system reboots
by Xuan Zhong 23 Feb '10

23 Feb '10

I looked both /etc/rc.d/rc3.d and /etc/rc.d/rc5.d, and other rcx.d directories. Only one "S" script for syslog-ng and it's symbolic link to /etc/init.d/syslog-ng. All files under rcx.d directory are symbolic links. Do you know if the first occurance of "Starting syslog-ng:" or "Stopping syslog-ng:" comes from syslog-ng binary file? Thanks Xuan > Date: Thu, 18 Feb 2010 11:09:55 -0600 > From: wes.hardin(a)maxim-ic.com > To: syslog-ng(a)lists.balabit.hu > Subject: Re: [syslog-ng] syslog-ng gets started twice when system reboots > > On 02/17/2010 09:59 AM, Xuan Zhong wrote: > > > > I use the stable version syslog-ng-3.0.5-1.rhel5.x86_64.rpm downloaded > > from your website. It starts/stops twice only when the system reboots. > > The init script is only generated once. The second "Starting syslog-ng: > > [ok]" was generated by init script because I modified the init script > > text display and only the second start changed. Not sure where the first > > start comes from. > > > > Starting syslog-ng: Starting syslog-ng: [ok] > > [ok] > > > > > > When the system shuts down, I saw the following: > > Stopping syslog-ng: Stopping syslog-ng: [ok] > > [ok] > > > > > > Similarly, only the second "Stopping syslog-ng:" comes from the > > syslog-ng service script. > > Look in /etc/rc.d/rc{3,5}.d for multiple "S" scripts for syslog-ng. Everything > in the rc?.d directories is typically a symlink to a similarly named script in > /etc/rc.d/init.d (usually a relative link like ../init.d/script). > > If there is a file that isn't a symlink or doesn't link to an initscript in > /etc/rc.d/init.d, that should be considered suspicious. > -- > /* Wes Hardin */ _________________________________________________________________ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469226/direct/01/

2 2

Problem with windows client
by Heinze, René 22 Feb '10

22 Feb '10

Good morning, I´m using syslog-ng on a sles 10 server acting as a syslog-ng server and a sles 10 server acting as a syslog-ng client. The clients logs successfully everything to the server. I added a Windows XP Professional SP2 client and installed the syslog-ng client for windows. But unfortunately the windows syslog-ng client is not able to connect to the server. The only thing i configured is the ip of the syslog-ng server. I attached the debug log to this email. What am I doing wrong? The is no firewall causing the problem. On the syslog-ng server i see incoming packages from the windows host using tcpdump. Best regards, René Heinze Besuchen Sie die Bundesdruckerei auf der CeBIT 2010 vom 2.-6.3.2010, Halle 9, Stand D80 Visit Bundesdruckerei at CeBIT, exhibition centre, hall 9 / stand D80 weitere Informationen unter: http://www.bundesdruckerei.de/de/unternehmen/untern_cebit2010/index.html find more information here http://www.bundesdruckerei.de/en/company/comp_cebit2010/index.html

2 3

Syslog-ng 3.x and kernel messages
by Phusion 22 Feb '10

22 Feb '10

I've used syslog-ng 1.x and 2.x, but am new to using 3.x. I'm using syslog-ng 3.0.3 on a FreeBSD 8.0 server. I'm having trouble getting kernel messages. I've made a very small config file, and this isn't working either. @version:3.0 source local { internal(); file("/dev/klog"); unix-dgram("/var/run/log"); }; destination all { file("/var/log/all.log"); }; log { source(local); destination(all); }; This picks up other things, but not the kernel messages. I assume it's a small configuration issue. Let me know. Thanks. Phusion

3 3

syslog-ng performance tuning
by Rémi BUISSON 22 Feb '10

22 Feb '10

Hi everybody, I'have an issue with syslog-ng configuration. I would like to centralize my logs on one server. I've a lot of logs to send. I don't know how many but I can estimate it to 500GB per day from decades of servers. But, it writes only 25 GB per day. For some reasons I work on a debian etchnhalf environnement. So, I'm working with syslog-ng 2.0.0. I wrote a perl program which spawn p "logger -p local5.info" processes and send n lines of m characters. I'have tested with: p: 1 000 n: 1 000 m: 1 000 Instead of having 1 000 000 lines in my logs I have nearly 10 000 lines ! But my test was not revelant because normal logs where not stopped. So, maybe normal. I've googled my issue and tried to play on log_fifo_size(), max_connections() and sync() on both server and client but nothing to do ... Here is my client configuration file: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(adm); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); bad_hostname("^gconfd$"); }; source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); }; destination dn_fslog { tcp("xxx.xxx.xxx.xxx" port(514)); }; filter f_local5 { facility(local5); }; log { source(s_all); filter(f_local5); destination(dn_fslog); }; # for all my apache vhosts destination df_access_www.foo.com { file("/var/www/www.foo.com/logs/access.log"); }; filter f_local0_access_www.foo.com { level(info) and facility(local0) and program("www\.foo\.com"); }; log { source(s_all); filter(f_local0_access_www.foo.com); destination(df_access_www.foo.com); destination(dn_fslog); }; destination df_error_www.foo.com { file("/var/www/www.foo.com/logs/error.log"); }; filter f_local0_error_www.foo.com { level(error) and facility(local0) and program("www\.foo\.com"); }; log { source(s_all); filter(f_local0_error_www.foo.com); destination(df_error_www.foo.com); destination(dn_fslog); }; etc with www.foo2.com Here is my server configuration file: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(adm); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); bad_hostname("^gconfd$"); }; source s_net { tcp(ip(xxx.xxx.xxx.xxx) port(514)); }; destination df_net_access_aaa.aaa.aaa.aaa-www.foo.com { file("/logs/www.foo.com/aaa.aaa.aaa.aaa-access.log"); }; filter f_local0_access_aaa.aaa.aaa.aaa-www.foo.com { level(info) and facility(local0) and program("www\.foo\.com") and host("aaa\.aaa\.aaa\.aaa"); }; log { source(s_net); filter(f_local0_access_aaa.aaa.aaa.aaa-www.foo.com); destination(df_net_access_aaa.aaa.aaa.aaa-www.foo.com); }; destination df_net_error_aaa.aaa.aaa.aaa-www.foo.com { file("/logs/www.foo.com/aaa.aaa.aaa.aaa-error.log"); }; filter f_local0_error_aaa.aaa.aaa.aaa-www.foo.com { level(error) and facility(local0) and program("www\.foo\.com") and host("aaa\.aaa\.aaa\.aaa"); }; log { source(s_net); filter(f_local0_error_aaa.aaa.aaa.aaa-www.foo.com); destination(df_net_error_aaa.aaa.aaa.aaa-www.foo.com); }; etc with aaa.aaa.aaa.bbb and www.foo2.com Just to give you a number I'm speaking about 20 web servers with 20 load balanced websites on each. Servers are on a Gigabit networks with bonded Gigabit interfaces. It is running on Linux 2.6.24-etchnhalf.1-686-bigmem. Maybe syslog-ng cannot handle 400 different filter rules but it would suprize me. If it's the case, in there any better way to separate, for instance, apache error and access log for in each server, in a file name /logs/<project = progname>/<ip>-access.log and /logs/<project = progname>/<ip>-error.log ? I don't think my hardware is in cause because cpu average 98% idle and no io wait. Can you tell me what I misunderstood or simply what's wrong in my configuration files. Or, it may work better with a 3.x version compiled with some options ? Thanks in advance. Regards, Rémi

3 6

Issue is resolved
by dwy1008 19 Feb '10

19 Feb '10

The main problem is that 'eventlog'.Because there are problems compiling,this results in the syslog-ng non start. I would like to thank you for helping me.

1 0

syslog-ng gets started twice when system reboots
by Xuan Zhong 18 Feb '10

18 Feb '10

I am using syslog-ng-3.0.5 RHEL5 x86. After the installation and rebooting the system: Starting syslog-ng: Starting syslog-ng: [ OK ] [ OK ] Does anyone know why syslog-ng gets started twice after reboot? Thanks Xuan _________________________________________________________________ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. http://clk.atdmt.com/GBL/go/201469229/direct/01/

4 6

syslog-ng with logrotate
by Tom Simons 16 Feb '10

16 Feb '10

We're replacing sysklogd on our RHEL4 & 5 servers with syslog-ng-premium-edition 3.0.4-1. Has anyone configured logrotate to work with syslog-ng? Can we just build /etc/logrotate.d/syslog-ng from /etc/logrotate.d/syslog?

3 2

UDP source performance improvement sugestions
by Jean F. Mousinho 15 Feb '10

15 Feb '10

Hello, I'm doing some performance tests between syslog-ng and rs####g (yes I guess I'm not the first to think/do something like this). I've noticed that syslog-ng is much slower in the case of a single UDP source and destination to file due to the nature of the message reception code. I then decided to play a bit with the buffering parameters that syslog-ng allows me to set to try improve its performance in this specific test. I expect that when I set time_sleep to 0, it even avoids calling nanosleep from what I saw, so it should continuously read from the UDP socket. I then set a big log_fifo_size 10000 (without knowing exactly if its useful or not in this case, destination is file) and on the source udp I set log_fetch_limit also something like 5000, log_iw_size to 10000. These steps didn't improved the performance that much, maybe I could be doing something wrong.. >From what I understood, this will make syslog-ng read 5000 messages from the socket (if they are available) and only after that it queues them to the fifo? fifo waits for flush_lines number of lines or flush_timeout to write the data to the file? Is there anything else that you know I could do to improve? flush_lines > 0? .. or maybe some resource/doc I could/should read. Thanks for your time. Jean Mousinho

2 2

Re: [syslog-ng] Syslog-NG Premium Edition installation on SuSE Linux Enterprise Server 10
by Heinze, René 15 Feb '10

15 Feb '10

>On Mon, Feb 15, 2010 at 05:05:21PM +0100, Balazs Scheidler wrote: >> On Mon, 2010-02-15 at 16:38 +0100, Heinze, René wrote: >> > Hello, >> > >> > I´m having trouble installing the Syslog-NG Premium Edition on SLES >> > 10. I´m installing the RPM package 3.04 x86 for OpenSuSE 10.1. The RPM >> > installes just fine, but I´m not able to run the .run installer. The >> > installation of the rpm package is completely unattended. How can I >> > start the .run installer? >> > >> >> That's right, the rpm package is not interactive. You can start the .run >> installer by removing the .rpm package first and starting the .run file >> as a command: >> >> # ./syslog-ng-premium-edition-3.0.4-linux-i386.run >Please make sure, that the .run file has the executable attribute set >before you attempt to run it: > ># chmod +x syslog-ng-premium-edition-3.0.4-linux-i386.run ># ./syslog-ng-premium-edition-3.0.4-linux-i386.run > >or force the shell to execute it: > ># sh syslog-ng-premium-edition-3.0.4-linux-i386.run Thanks for you support! I have overlooked the generic linux version on the Balabit webpage. What´s the best method installing Syslog-NG Premium on a SLES 10 System, generic or the rpm version? On the rpm version, how do i attach the license file? If the rpm version is not interactive then there is a error in the admin manual. The manual says on page 86 in the first line: "Answer the configuration questions of syslog-ng. These are described in detail in Section 4.1, “Installing syslogng using the .run installer” (p. 78)." Best Regards, René H. Besuchen Sie die Bundesdruckerei auf der CeBIT 2010 vom 2.-6.3.2010, Halle 9, Stand D80 Visit Bundesdruckerei at CeBIT, exhibition centre, hall 9 / stand D80 weitere Informationen unter: http://www.bundesdruckerei.de/de/unternehmen/untern_cebit2010/index.html find more information here http://www.bundesdruckerei.de/en/company/comp_cebit2010/index.html

1 0

Syslog-NG Premium Edition installation on SuSE Linux Enterprise Server 10
by Heinze, René 15 Feb '10

15 Feb '10

Hello, I´m having trouble installing the Syslog-NG Premium Edition on SLES 10. I´m installing the RPM package 3.04 x86 for OpenSuSE 10.1. The RPM installes just fine, but I´m not able to run the .run installer. The installation of the rpm package is completely unattended. How can I start the .run installer? Thanks! Best Regards, René Heinze ________________________________ Besuchen Sie die Bundesdruckerei auf der CeBIT 2010 vom 2.-6.3.2010, Halle 9, Stand D80 Visit Bundesdruckerei at CeBIT, exhibition centre, hall 9 / stand D80 weitere Informationen unter: http://www.bundesdruckerei.de/de/unternehmen/untern_cebit2010/index.html find more information here http://www.bundesdruckerei.de/en/company/comp_cebit2010/index.html

3 2

Getting a SQL insert to match what's being written to file
by Jeremy M. Guthrie 15 Feb '10

15 Feb '10

I have an issue where what is being written to file is different than what I have being written to disk. I don't know if the macro is broken, my understanding is wrong, or I am just using the wrong macro altogether. My understanding is roughly as follows: PROGRAM The name of the program sending the message. PID The PID of the program sending the message. MSG or MESSAGE Message contents including the program name and pid. MSGONLY Message contents without the program name. 1. The first thing I ran into: Example messages on file: Jan 25 08:34:15 XXXXXXXXXX sshd[7209]: service name is :: sshd Same example message in SQL: Jan 25 08:34:15 XXXXXXXXXX service name is :: sshd template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$MSG');\n" ) 2. Then what I tried to get it closer.... Example messages on file: Jan 25 08:40:18 XXXXXXXXXX sshd[7209]: service name is :: sshd Same example message in SQL: Jan 25 08:40:18 XXXXXXXXXX sshd: service name is :: sshd template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$PROGRAM: $MSG');\n" ) 3. Then what I tried to get even closer: Example messages on file: Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd Same example message in SQL: Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$PROGRAM[$PID]: $MSG');\n" ) However, this broke messages coming from Cisco devices... Jan 25 08:55:10 FIREWALL %FWSM-3-710003[]: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445 The message above should be: Jan 25 08:55:10 FIREWALL %FWSM-3-710003: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445 Notice the above '[]' after the FWSM code? I am running 3.0.5 syslog-ng. Ideally I'd get both: Jan 25 08:55:10 FIREWALL %FWSM-3-710003: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445 AND Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd Is the MSG/MESSAGE macro broken? What am I missing? -- -------------------------------------------------- Jeremy M. Guthrie jeremy.guthrie(a)cdw.com Managed Cisco Security Services Hosting and Managed Services Technical Architect Phone: 608-298-1061 CDW Fax: 608-288-3007 5520 Research Park Drive NOC: 608-298-1102 Madison, WI 53711 NOC Email: hmshelp(a)cdw.com

2 2

Syslog restarting
by Henric Larsson 11 Feb '10

11 Feb '10

Hi, We're using syslog-ng PE with disk buffering enabled. The syslog clients gets disconnected all the time, and when they restart I get these log messages: 2010-02-11T14:19:37+00:00 bruatosd003 syslog-ng[4686]: I/O error occurred while writing; fd='8', error='Connection reset by peer (104)' 2010-02-11T14:19:37+00:00 bruatosd003 syslog-ng[4686]: Syslog connection broken; fd='8', server='AF_INET(172.27.12.22:1998)', time_reopen='60' 2010-02-11T14:19:37+00:00 bruatosd003 syslog-ng[4686]: Closing log transport fd; fd='8' 2010-02-11T14:20:41+00:00 bruatosd003 syslog-ng[4686]: Syslog connection established; fd='8', server='AF_INET(172.27.12.22:1998)', local='AF_INET(0.0.0.0:0)' 2010-02-11T14:20:54+00:00 bruatosd003 syslog-ng[28626]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/var/ig/logs/orderserver/os-gc.1.log' 2010-02-11T14:20:54+00:00 bruatosd003 syslog-ng[28626]: Disk-buffer state saved; filename='/var/ig/logs/orderserver/syslog-ng-db/syslog-ng-00004.qf', qout_length='0', qbacklog_length='0', qoverflow_length='0', qdisk_length='0' 2010-02-11T14:20:54+00:00 bruatosd003 syslog-ng[28626]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/var/ig/logs/orderserver/os-gc.2.log' After that syslog starts to send the contents of the file from the start of the file again, after a while it restart this process. In a seemingly endless loop. As you can see the pid changes between the disconnect and the message about mismatching size/inode. I'm trying to work out what causes it and how to fix it, any suggestions? Best, Henric ________________________________ The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc.) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Index Ltd is a company registered in England and Wales under number 01190902. VAT registration number 761 2978 07. Registered Office: Friars House, 157-168 Blackfriars Road, London SE1 8EZ. Authorised and regulated by the Financial Services Authority. FSA Register number 114059.

1 0

[Re: handling errors with sql inserts]
by Juhasz Viktor 11 Feb '10

11 Feb '10

1 0

handling errors with sql inserts
by Bob Belnap 10 Feb '10

10 Feb '10

Hello, I am parsing some of my log messages into different database columns using the sql() function, and I have noticed that when there is an insert error into the database no further logs for that host make it to the db. In fact, syslog-ng keeps attempting the insert, and logs the insert error indefinitely. I couldn't find a way to gracefully handle these sorts of errors in the docs. I'd like to be able to specify an action on a database insert failure (log message to a different log and stop trying to insert, for example). But I do not want a single insert failure to stop all subsequent inserts for that host. What are my options here? I'm currently using OSE 3.0.4. Thanks for your help. --Bob

2 2

Re: [syslog-ng] need help with a match filter
by chris packham 10 Feb '10

10 Feb '10

Someone else on the list may correct me but I think you'll find that the destination is evaluated at the time syslog is started (or restarted) not when a log message is processed. I'm pretty sure there was a discussion on the list about something similar a while back (although I couldn't find it in my emails) So short answer is that you can't use a file desination. You may however get some mileage out of writing a custom program destination and do the last bit of processing in your program. >>> Rory Toma <rory(a)ooma.com> 01/25/10 5:12 PM >>> Looking more at the docs, it looks likes I also need a value for 3.0 syntax. So I am trying something like below. However, the value of $foo_hostdir is not being set correctly. It is being set to "foo_hostdir". How do I get the value to be the value of the match? filter foo_filter { match('host("foo_[0-9A-F]\{9\}$")' value("foo_hostdir") type("string") ); }; # foo destinations destination foo_dest { file("/logs/$foo_hostdir/$R_YEAR$R_MONTH$R_DAY/$HOST-$R_YEAR$R_MONTH$R_DAY.log" owner(root) group(root) perm(0444) template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC $MSG\n") template_escape(no)); }; On 1/25/10 12:02 PM, Rory Toma wrote: > I'm trying to dynamically create lognames and directories form the host > name that is sent. My hosts' names are of the form > > foo_<mac address> > > I want to create a destination that would be > > /logs/foo_<first 9 digits of mac address>//foo_.log > > Here is what I tried so far, and it does not work, but is close. Can > someone suggest a config that might work? Thx. > > I can do this individually, but having 100+ filters and 100+ log lines > in the config really slows things down. > > #filter for foo > filter foo_filter { match('host("foo_[0-9A-F]\{9\}$")' > flags("store-matches")); }; > > # foo destinations > destination foo_dest { > > file("/logs/$0/$R_YEAR$R_MONTH$R_DAY/$HOST-$R_YEAR$R_MONTH$R_DAY.log" > owner(root) group(root) perm(0444) > template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC $MSG\n") > template_escape(no)); > }; > > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

3 7

--fd-limit
by Rory Toma 10 Feb '10

10 Feb '10

My understanding (now) is that you need to set --fd-limit *and* up the ulimit. Simply updating ulimit is not sufficient for syslog-ng to be able to handle more than the default 4k open files. Is this correct?

2 1

Apache + syslog-ng w/o logger?
by Brian Donaldson 09 Feb '10

09 Feb '10

My goal is to enable Apache to log large access_log entries (> 1024 chars) to a syslog-ng server. The vast majority of examples I've seen for connecting Apache to syslog-ng employ the ancient /usr/bin/logger -- but that approach chops up messages longer than 1024 chars. The alternative approach is to use a perl/python script, but I'm nervous about the overhead. Anyone know of an updated logger binary? Thanks for any pointers, -Brian

5 4

two sources one udp address
by Alexei Shilin 06 Feb '10

06 Feb '10

Hi! I have source src { unix-stream("/dev/log" max-connections(512)); internal(); file("/proc/kmsg"); udp(port(514)); }; ... source temp { udp(port(514)); }; and then I have to log rules: one for src and one for temp, no flags are set. Will those two sources play nice with each other and will not interfer? E.g. if one reads from that port, will the log message that was read by the first available for read by the second? -- http://www.fastmail.fm - Email service worth paying for. Try it for free

2 1