- syslog-ng - lists.balabit.hu

pcre matches
by Patrick H. 06 Mar '10

06 Mar '10

How do you use pcre named pattern matches with store-matches? I've been trying to do things like message("IP: (?<ip>\S+)" flags(store-matches) type("pcre")) and then insert into a database with things like values("$ip") or values("$+{ip}") and nothing works. I cant seem to find any examples of usage through google either.

2 2

Using amazon s3 as logging backend
by Rory Toma 24 Feb '10

24 Feb '10

Has anyone used amazon s3 as logging back end yet?

1 0

Re: [syslog-ng] syslog-ng gets started twice when system reboots
by Xuan Zhong 23 Feb '10

23 Feb '10

I looked both /etc/rc.d/rc3.d and /etc/rc.d/rc5.d, and other rcx.d directories. Only one "S" script for syslog-ng and it's symbolic link to /etc/init.d/syslog-ng. All files under rcx.d directory are symbolic links. Do you know if the first occurance of "Starting syslog-ng:" or "Stopping syslog-ng:" comes from syslog-ng binary file? Thanks Xuan > Date: Thu, 18 Feb 2010 11:09:55 -0600 > From: wes.hardin(a)maxim-ic.com > To: syslog-ng(a)lists.balabit.hu > Subject: Re: [syslog-ng] syslog-ng gets started twice when system reboots > > On 02/17/2010 09:59 AM, Xuan Zhong wrote: > > > > I use the stable version syslog-ng-3.0.5-1.rhel5.x86_64.rpm downloaded > > from your website. It starts/stops twice only when the system reboots. > > The init script is only generated once. The second "Starting syslog-ng: > > [ok]" was generated by init script because I modified the init script > > text display and only the second start changed. Not sure where the first > > start comes from. > > > > Starting syslog-ng: Starting syslog-ng: [ok] > > [ok] > > > > > > When the system shuts down, I saw the following: > > Stopping syslog-ng: Stopping syslog-ng: [ok] > > [ok] > > > > > > Similarly, only the second "Stopping syslog-ng:" comes from the > > syslog-ng service script. > > Look in /etc/rc.d/rc{3,5}.d for multiple "S" scripts for syslog-ng. Everything > in the rc?.d directories is typically a symlink to a similarly named script in > /etc/rc.d/init.d (usually a relative link like ../init.d/script). > > If there is a file that isn't a symlink or doesn't link to an initscript in > /etc/rc.d/init.d, that should be considered suspicious. > -- > /* Wes Hardin */ _________________________________________________________________ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469226/direct/01/

2 2

Problem with windows client
by Heinze, René 22 Feb '10

22 Feb '10

Good morning, I´m using syslog-ng on a sles 10 server acting as a syslog-ng server and a sles 10 server acting as a syslog-ng client. The clients logs successfully everything to the server. I added a Windows XP Professional SP2 client and installed the syslog-ng client for windows. But unfortunately the windows syslog-ng client is not able to connect to the server. The only thing i configured is the ip of the syslog-ng server. I attached the debug log to this email. What am I doing wrong? The is no firewall causing the problem. On the syslog-ng server i see incoming packages from the windows host using tcpdump. Best regards, René Heinze Besuchen Sie die Bundesdruckerei auf der CeBIT 2010 vom 2.-6.3.2010, Halle 9, Stand D80 Visit Bundesdruckerei at CeBIT, exhibition centre, hall 9 / stand D80 weitere Informationen unter: http://www.bundesdruckerei.de/de/unternehmen/untern_cebit2010/index.html find more information here http://www.bundesdruckerei.de/en/company/comp_cebit2010/index.html

2 3

Syslog-ng 3.x and kernel messages
by Phusion 22 Feb '10

22 Feb '10

I've used syslog-ng 1.x and 2.x, but am new to using 3.x. I'm using syslog-ng 3.0.3 on a FreeBSD 8.0 server. I'm having trouble getting kernel messages. I've made a very small config file, and this isn't working either. @version:3.0 source local { internal(); file("/dev/klog"); unix-dgram("/var/run/log"); }; destination all { file("/var/log/all.log"); }; log { source(local); destination(all); }; This picks up other things, but not the kernel messages. I assume it's a small configuration issue. Let me know. Thanks. Phusion

3 3

syslog-ng performance tuning
by Rémi BUISSON 22 Feb '10

22 Feb '10

Hi everybody, I'have an issue with syslog-ng configuration. I would like to centralize my logs on one server. I've a lot of logs to send. I don't know how many but I can estimate it to 500GB per day from decades of servers. But, it writes only 25 GB per day. For some reasons I work on a debian etchnhalf environnement. So, I'm working with syslog-ng 2.0.0. I wrote a perl program which spawn p "logger -p local5.info" processes and send n lines of m characters. I'have tested with: p: 1 000 n: 1 000 m: 1 000 Instead of having 1 000 000 lines in my logs I have nearly 10 000 lines ! But my test was not revelant because normal logs where not stopped. So, maybe normal. I've googled my issue and tried to play on log_fifo_size(), max_connections() and sync() on both server and client but nothing to do ... Here is my client configuration file: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(adm); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); bad_hostname("^gconfd$"); }; source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); }; destination dn_fslog { tcp("xxx.xxx.xxx.xxx" port(514)); }; filter f_local5 { facility(local5); }; log { source(s_all); filter(f_local5); destination(dn_fslog); }; # for all my apache vhosts destination df_access_www.foo.com { file("/var/www/www.foo.com/logs/access.log"); }; filter f_local0_access_www.foo.com { level(info) and facility(local0) and program("www\.foo\.com"); }; log { source(s_all); filter(f_local0_access_www.foo.com); destination(df_access_www.foo.com); destination(dn_fslog); }; destination df_error_www.foo.com { file("/var/www/www.foo.com/logs/error.log"); }; filter f_local0_error_www.foo.com { level(error) and facility(local0) and program("www\.foo\.com"); }; log { source(s_all); filter(f_local0_error_www.foo.com); destination(df_error_www.foo.com); destination(dn_fslog); }; etc with www.foo2.com Here is my server configuration file: options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(adm); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); bad_hostname("^gconfd$"); }; source s_net { tcp(ip(xxx.xxx.xxx.xxx) port(514)); }; destination df_net_access_aaa.aaa.aaa.aaa-www.foo.com { file("/logs/www.foo.com/aaa.aaa.aaa.aaa-access.log"); }; filter f_local0_access_aaa.aaa.aaa.aaa-www.foo.com { level(info) and facility(local0) and program("www\.foo\.com") and host("aaa\.aaa\.aaa\.aaa"); }; log { source(s_net); filter(f_local0_access_aaa.aaa.aaa.aaa-www.foo.com); destination(df_net_access_aaa.aaa.aaa.aaa-www.foo.com); }; destination df_net_error_aaa.aaa.aaa.aaa-www.foo.com { file("/logs/www.foo.com/aaa.aaa.aaa.aaa-error.log"); }; filter f_local0_error_aaa.aaa.aaa.aaa-www.foo.com { level(error) and facility(local0) and program("www\.foo\.com") and host("aaa\.aaa\.aaa\.aaa"); }; log { source(s_net); filter(f_local0_error_aaa.aaa.aaa.aaa-www.foo.com); destination(df_net_error_aaa.aaa.aaa.aaa-www.foo.com); }; etc with aaa.aaa.aaa.bbb and www.foo2.com Just to give you a number I'm speaking about 20 web servers with 20 load balanced websites on each. Servers are on a Gigabit networks with bonded Gigabit interfaces. It is running on Linux 2.6.24-etchnhalf.1-686-bigmem. Maybe syslog-ng cannot handle 400 different filter rules but it would suprize me. If it's the case, in there any better way to separate, for instance, apache error and access log for in each server, in a file name /logs/<project = progname>/<ip>-access.log and /logs/<project = progname>/<ip>-error.log ? I don't think my hardware is in cause because cpu average 98% idle and no io wait. Can you tell me what I misunderstood or simply what's wrong in my configuration files. Or, it may work better with a 3.x version compiled with some options ? Thanks in advance. Regards, Rémi

3 6

Issue is resolved
by dwy1008 19 Feb '10

19 Feb '10

The main problem is that 'eventlog'.Because there are problems compiling,this results in the syslog-ng non start. I would like to thank you for helping me.

1 0

syslog-ng gets started twice when system reboots
by Xuan Zhong 18 Feb '10

18 Feb '10

I am using syslog-ng-3.0.5 RHEL5 x86. After the installation and rebooting the system: Starting syslog-ng: Starting syslog-ng: [ OK ] [ OK ] Does anyone know why syslog-ng gets started twice after reboot? Thanks Xuan _________________________________________________________________ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. http://clk.atdmt.com/GBL/go/201469229/direct/01/

4 6

syslog-ng with logrotate
by Tom Simons 16 Feb '10

16 Feb '10

We're replacing sysklogd on our RHEL4 & 5 servers with syslog-ng-premium-edition 3.0.4-1. Has anyone configured logrotate to work with syslog-ng? Can we just build /etc/logrotate.d/syslog-ng from /etc/logrotate.d/syslog?

3 2

UDP source performance improvement sugestions
by Jean F. Mousinho 15 Feb '10

15 Feb '10

Hello, I'm doing some performance tests between syslog-ng and rs####g (yes I guess I'm not the first to think/do something like this). I've noticed that syslog-ng is much slower in the case of a single UDP source and destination to file due to the nature of the message reception code. I then decided to play a bit with the buffering parameters that syslog-ng allows me to set to try improve its performance in this specific test. I expect that when I set time_sleep to 0, it even avoids calling nanosleep from what I saw, so it should continuously read from the UDP socket. I then set a big log_fifo_size 10000 (without knowing exactly if its useful or not in this case, destination is file) and on the source udp I set log_fetch_limit also something like 5000, log_iw_size to 10000. These steps didn't improved the performance that much, maybe I could be doing something wrong.. >From what I understood, this will make syslog-ng read 5000 messages from the socket (if they are available) and only after that it queues them to the fifo? fifo waits for flush_lines number of lines or flush_timeout to write the data to the file? Is there anything else that you know I could do to improve? flush_lines > 0? .. or maybe some resource/doc I could/should read. Thanks for your time. Jean Mousinho

2 2