July 2011 - syslog-ng - lists.balabit.hu

Apache vhost logs
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello again, I try all the weekend to do it but there is no luck on it. I want to send all apache vhosts logs to a remote server. I read this guide: http://www.facebook.com/note.php?note_id=338493890632 But, when I create the named pipes, I cannot restart Apache. I try to send the logs files this way: file("/var/log/apache2/access.log"); file("/var/log/apache2/error.log"); But on the server-side there is any access.log and error.log files. I have lots of vhost on the client-side. How could I configure it? Thanks and best regards.

5 14

reconnect DB session after timeout
by "Marcus Groß" 14 Jul '11

14 Jul '11

Hi, I'm running syslog-ng v. 3.2.4 on SLES 10. There are three defined MySQL DB destinations. Everything is running fine, until the MySQL server resets the connection(s) after a defined idle time if no messages are send to this database. The problem is that syslog-ng doesn't reconnect the disconnected connection ;) I have to restart syslog-ng to bring it back to life. Is there any posssibility to reopen the DB connection instead of disabling the idle timeout on the MySQL side? syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' syslog-ng[4904]: Error checking table, disconnecting from database, trying again shortly; time_reopen='60' syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' Jun 9 09:43:56 snnasmg8421 syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' M.

3 2

[PATCH] Fix: maximum thread number can be 0
by Juhasz Viktor 14 Jul '11

14 Jul '11

[threading] It is possible that the _SC_NPROCESSORS_ONLN isn't defined. In this case the max threads must greater than 0 Signed-off-by: Viktor Juhasz <jviktor(a)balabit.hu> diff --git a/lib/mainloop.c b/lib/mainloop.c index e659fff..b23d055 100644 --- a/lib/mainloop.c +++ b/lib/mainloop.c @@ -702,6 +702,8 @@ main_loop_add_options(GOptionContext *ctx) { #ifdef _SC_NPROCESSORS_ONLN main_loop_io_workers.max_threads = MIN(MAX(2, sysconf(_SC_NPROCESSORS_ONLN)), 64); +#else + main_loop_io_workers.max_threads = 2; #endif g_option_context_add_main_entries(ctx, main_loop_options, NULL);

2 1

check clients
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I want to know if there is a option to know which log-clients are sending data to syslog-ng server. Is it possible to get a list of clients? Thanks and best regards.

4 4

change rsyslog for syslog-ng
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I am learning how to get a syslog server with syslog-ng. I started working on a IT department and I have more than 50 servers, we actually has rsyslog on the client side and syslog-ng on the syslog server. I want to change the client side rsyslog to configure it with syslog-ng. On the server side I read lot of documentation and I know how to configure it (source, destination and filter). I want to get this logs on the client side and send to the syslog server on a TCP+SSL: /var/log/auth.log /var/log/daemon.log /var/log/dmesg /var/log/messages /var/log/syslog I want to get on the server side something like this: /var/log/extern/host1/auth.log /var/log/extern/host1/ daemon.log /var/log/extern/host1/ dmesg /var/log/extern/host1/ messages /var/log/extern/host1/ syslog /var/log/extern/host2/auth.log /var/log/extern/host2/ daemon.log /var/log/extern/host2/ dmesg /var/log/extern/host2/ messages /var/log/extern/host2/ syslog ... /var/log/extern/hostn/auth.log /var/log/extern/hostn/ daemon.log /var/log/extern/hostn/ dmesg /var/log/extern/hostn/ messages /var/log/extern/hostn/ syslog On the past post you help me and I have how to get It on the server side for the TCP+SSL: source s_tcptls { tcp (ip("10.200.42.1") port(10514) tls( peer-verify(require-trusted) ca_dir("/etc/syslog-ng/certs/") key_file("/etc/syslog-ng/certs/server.key") cert_file("/etc/syslog-ng/certs/server.crt") ) ); }; I have some questions: 1. Must I uninstall rsyslog before install syslog-ng? 2. How can I configure the client side to send my logs to the server and to save on the local /var/log/? 3. Is there any client limit on syslog-ng? I will try to read and learn as much as I can, I will appreciate your help. Thanks for your job and best regards.

4 4

Macros expansion in pattern-db actions
by Denis Gasparin 14 Jul '11

14 Jul '11

Hi to all. We're using syslog-ng 3.2.4 and we're having a weird behaviour in using macros with actions values. Here it is a sample pattern-db rule: <?xml version='1.0' encoding='UTF-8'?> <patterndb version='4' pub_date='2011-06-20'> <ruleset name='cron' id='cron-ruleset'> <pattern>/usr/sbin/cron</pattern> <rules> <rule provider="patterndb" id="cron-1" class="system" context-id="sample-context-id"> <patterns> <pattern>(@ESTRING:usracct.username:) @CMD (@ESTRING:details:)@</pattern> </patterns> <actions> <action trigger="match" condition="match('mymatch' value('details'))"> <message> <values> <value name="MESSAGE">[${details}] was found in a cron log message. Rule number [${.classifier.rule_id}]</value> <value name="TRIGGER">yes</value> </values> </message> </action> </actions> </rule> </rules> </ruleset> </patterndb> We tested the rule using pdtool match command and the output was: # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[] was found in a cron log message. Rule number [] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes We instead expected the following output from pdtool match execution : # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[mymatch] was found in a cron log message. Rule number [cron-1 ] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes Macro expansion was not executed in action values but it was in action definition... What are we missing? Thank you in advance for your help Denis Gasparin --- Edistar SRL

2 1

Trying to let my harddrive spin down once in a while
by Lars Stokholm 12 Jul '11

12 Jul '11

Hello On my laptop I run Arch Linux with syslog-ng and cronie (cron) among other programs of course. I'm trying to get my harddrive to spin down automatically once in a while (using hdparm). Me and a lot of other people both in the Arch and Ubuntu community are having our attempts to do this ruined, by what first seemed to be the EXT4 journalling block device (jbd2). After long discussions and finally a bug report (see at the bottom of this mail), I think I've boiled my troubles down to be mostly about syslog-ng (and possibly cronie - but I don't think so). No matter what I try I can't seem to get syslog-ng to stop "waking up my disks" (at least that's what kernel tracing tells me). Killing the crond process almost totally stops these syslog-ng writes to the disk. Unfortunately I've been unable to configure my way out of it using syslog-ng.conf. So my harddrive won't stay spinned down - not even for a minute. This is what tracing the journalling gives me (see the kernel bug report below for an explanaition): http://pastebin.com/raw.php?i=J6BtGVrz As you can probably see, syslog-ng writes every minute at least. I guess that's because my user crontab has a command that gets run every minute. What I don't get is why syslog-ng writes to the disk, even though I'm telling it not to log crond's output. Here is my syslog-ng.conf - as you can see I tried to output the crond messages to /dev/null. I guess it works, because there are no mentions of crond anywhere in /var/log anymore. http://pastebin.com/raw.php?i=H02KKWPV Here are the discussions and the bug report I mentioned: https://bbs.archlinux.org/viewtopic.php?id=113516 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/607560 https://bugzilla.kernel.org/show_bug.cgi?id=39072 I don't know if syslog-ng is causing trouble for only me, but it seems unlikely. Regards, Lars

3 13

Re: [syslog-ng] Syslog-ng log viewer/analyser
by Terry Slattery 08 Jul '11

08 Jul '11

I don't recall where I found it. It may have been at CiscoLive or from something on the Cisco web site, such as an article you wrote. I thought it was a very interesting statistic. -tcs On 7/8/11 2:17 PM, syslog-ng-request(a)lists.balabit.hu wrote: > Date: Fri, 8 Jul 2011 14:16:19 -0400 > From: Clayton Dukes<cdukes(a)gmail.com> > Subject: Re: [syslog-ng] Syslog-ng log viewer/analyser > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <CAANP6Qd=0xnGJ6BkvuyLLTsz7yCQKJMXviwq6=PNV17DnJs8aQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi Terry, > Checked out your blog. > I was wondering where you got the quote about 90 traps and 6000 syslog > messages? > I ask because it's my quote, heh > Just think it's neat how things travel:-) > -- Terry Slattery CCIE# 1026

2 1

Re: [syslog-ng] Syslog-ng log viewer/analyser
by Terry Slattery 08 Jul '11

08 Jul '11

I have a couple of blogs that might be of interest. The first references a couple of other blogs about handling events. The second is about a summary script that I've found useful for summarizing syslog each day. http://www.netcordia.com/community/blogs/terrys_blog/archive/2010/06/06/han… http://www.netcordia.com/community/blogs/terrys_blog/archive/2009/10/28/sys… -tcs -- Terry Slattery CCIE# 1026 On Jul 8, 2011, at 6:00 AM, syslog-ng-request(a)lists.balabit.hu wrote: > > Date: Thu, 07 Jul 2011 15:08:02 +0200 > From: Peter Czanik <czanik(a)balabit.hu> > Subject: Re: [syslog-ng] Syslog-ng log viewer/analyser > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <4E15AFB2.1070505(a)balabit.hu> > Content-Type: text/plain; charset=ISO-8859-1 > > On 07/07/2011 01:49 PM, Pramod Pillai wrote: >> Hi >> >> Is there any Syslog-ng log viewer/analyser which I can use along with >> syslog-ng . > There are many. See my blog at > https://czanik.blogs.balabit.com/2011/06/a-comparison-of-syslog-ng-web-guis/ > Bye, > > -- > Peter Czanik (CzP) <czanik(a)balabit.hu> > BalaBit IT Security / syslog-ng upstream > http://czanik.blogs.balabit.com/

2 1

change from rsyslog to syslog-ng
by Josu Lazkano 08 Jul '11

08 Jul '11

Hello everybody! I finished to configure my syslog system. I am using rsyslog on clients side and syslog-ng on the server side. It works well, but I have a little problem with Apache vhosts logs. I want to send all vhosts logs to my syslog-ng server, but I don't know how to do with rsyslog. Is possible to do with syslog-ng? My Apache logs are configured this way: ErrorLog /var/www/domain1/log/error.log LogLevel warn SetEnvIf Remote_Addr "x\.x\.x\.x" dontlog SetEnvIf Remote_Addr "y\.y\.y\.y" dontlog CustomLog /var/www/domain1/log/access.log common env=!dontlog On the same machine I have lots of vhost logs: /var/www/domain1/log/error.log /var/www/domain1/log/access.log /var/www/domain2/log/error.log /var/www/domain2/log/access.log /var/www/domain3/log/error.log /var/www/domain3/log/access.log ... Is preferable to change all rsyslog to syslog-ng? Thanks for your great help. Best regards.

2 1