October 2006 - syslog-ng - lists.balabit.hu

AIX startup
by Brian Loe 27 Mar '08

27 Mar '08

Okay, my syslog-ng instance died at some point last night and because I don't have it setup properly for respawning... Can anyone running syslog-ng on AIX 5.* give me some tips for this? I'm feeding pipes so i have to start instances of SEC as soon as I start syslog-ng - but the SEC scripts appear to "close" so if they're configured to respawn I'll have 2000 instances of them in an hour.

7 21

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

Analysis of logs and alerting
by Muath Al Khalaf 11 Feb '07

11 Feb '07

Hi, I am implementing a syslog-ng centralized correlation server. The server receives the log entries from some servers, apps and devices. Is there any good product (commercial or open-source) with GUI or web interface to analyze these logs and give real time alerts with strong data mining tools? I need a tool which could be used by both help desk non-technical staff and technical engineers. It should give basic alerting on applications' or devices' failures first for help desk, and then engineers could dig deeper into these alerts to understand the problem. Kind regards

4 3

UI for editing syslog-ng configuration files
by John Svazic 29 Nov '06

29 Nov '06

Hi, Not sure if this is the best forum to ask this question, but I'm not having any luck trolling through Google, Sourceforge and Freshmeat. Does anyone know of any front-end UI (web or GUI) for editing a syslog-ng configuration file? I'm looking for something simple for users who are not familiar with the config file format to take on. Thanks!

5 5

Re: syslog-ng Digest, Vol 15, Issue 12
by dwarnock＠harbrook.net 28 Nov '06

28 Nov '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

5 9

Syslog-ng load testing
by Vaibhav Goel 22 Nov '06

22 Nov '06

Hi all I am planning to do some capacity and load testing our syslog-ng servers over TCP. Towards that end, I will be writing a script to send packets to a syslog-ng host. Has anyone done anything similar? I was just thinking of writing a simple script that will open multiple sockets to the server and send a message (based upon our current calculated average message size). Any other suggestions are appreciated. Thanks,

4 15

RE: [syslog-ng] help- Problem with TCP Based Centralized LogServer using Syslog-ng
by Padmanabhan, Rajeesh (GE Healthcare) 31 Oct '06

31 Oct '06

Hi Bazsi, Im using a Red Hat Enterprise Linux AS 4.0 as Syslog-ng Server ( Centralized Log Server). I have got Syslog-ng 1.6 running on Solaris 8.0 and Syslog-ng 2.0rc1 running on Linux AS 4.0. All the logs from clients ( Both Linux & Solaris ) are transferred to Syslog-ng Server ( Centralized Log Server) on TCP Based communication(Destination). I restarted the Central Log Server after I noticed the error messages in Clients. After I restarted the server, server is able to accept the logs. Thanks a lot for your quick response. -Rajeesh -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Tuesday, October 31, 2006 4:29 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] help- Problem with TCP Based Centralized LogServer using Syslog-ng On Tue, 2006-10-31 at 14:52 +0530, Padmanabhan, Rajeesh (GE Healthcare) wrote: > Hi > > Im trying to setup a centralized, TCP Based Log Server using > Syslog-ng. Syslog-ng server stops accepting logs from clients > intermediatley. > It works again, once i restart the syslog-ng server. Im getting > following error messages from clients. > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Oct 18 10:29:28 System1 syslog-ng[11745]: Connection broken; > time_reopen='60' > Oct 18 15:06:26 RADIUM syslog-ng[4999]: io.c: do_write: write() failed > (errno 32), Broken pipe Oct 18 15:06:26 RADIUM syslog-ng[4999]: > pkt_buffer::do_flush(): Error flushing data Oct 18 15:06:26 RADIUM > syslog-ng[4999]: Connection broken to AF_INET(10.70.201.233:514), > reopening in 10 seconds > syslog-ng[13843]: Connection failed; error='Connection refused > (111)', time_reopen='60 > syslog-ng[302]: Error connecting to remote host > AF_INET(10.70.201.233:514), reattempting in 10 seconds > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Please help me... > I'm sorry but you need to give further details, like OS, architecture syslog-ng version that you used. When did you receive the messages above? after restarting syslog-ng or was this the reason why you restarted syslog-ng in the first place? Because restarting syslog-ng naturally breaks existing connections and there's a small race of opportunity while the port is closed and clients try to connect to it, in this case you get connection refused. So all in all we'd need more information in order to help you. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

logs written twice
by Nick Baronian 31 Oct '06

31 Oct '06

I setup a syslog-ng server to get logs from various boxes and devices. I think I am a little confused about the correct way to set this up. I am seeing a lot of logs getting written twice. For instance I have some old Unix boxes logging to it and some Linux boxes. The old Unix boxes log to a syslog file for most stuff and the Linux boxes log most stuff to the messages file. I have some cisco devices that I wanted to log to particular filename but it is getting written to the custom filename and being duplicated to a syslog file, plus it is writing a messages file which is not an exact duplicate for some reason. I am guessing my reason is because I have stuff like filter f_syslog { not facility(auth, authpriv) and not facility(mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail); }; destination syslog { file("/u01/log/$HOST/syslog"); }; destination messages { file("/u01/log/$HOST/messages"); }; Is there a way I can duplicate the name schema the remote box is using or do I need to say all syslog type events goto blah file and such? My conf looks like options { chain_hostnames(no); create_dirs (yes); dir_perm(0750); dns_cache(no); keep_hostname(no); log_fifo_size(2048); log_msg_size(8192); perm(0640); stats(3600); sync(0); time_reopen (10); use_dns(no); use_fqdn(yes); }; source src { unix-dgram("/dev/log"); internal(); udp(); tcp(); }; destination authlog { file("/u01/log/$HOST/auth.log"); }; destination syslog { file("/u01/log/$HOST/syslog"); }; destination cron { file("/u01/log/$HOST/cron.log"); }; destination daemon { file("/u01/log/$HOST/daemon.log"); }; destination kern { file("/u01/log/$HOST/kern.log"); }; destination lpr { file("/u01/log/$HOST/lpr.log"); }; destination user { file("/u01/log/$HOST/user.log"); }; destination uucp { file("/u01/log/$HOST/uucp.log"); }; destination debug { file("/u01/log/$HOST/debug"); }; destination messages { file("/u01/log/$HOST/messages"); }; destination bootlog { file("/u01/log/$HOST/boot.log"); }; destination mail { file("/u01/log/$HOST/mail.log"); }; destination maillog { file("/u01/log/$HOST/maillog"); }; destination mailinfo { file("/u01/log/$HOST/mail.info"); }; destination mailwarn { file("/u01/log/$HOST/mail.warn"); }; destination mailerr { file("/u01/log/$HOST/mail.err"); }; destination cisco_asa { file("/u01/log/$HOST/cisco_asa.log"); }; destination console { usertty("root"); }; filter cr_asa { facility(local5); }; filter pump_renewal { not program("pumpd") or not level(info,debug) ; }; filter imap_debug { not program("imapd") or not level(info,debug) ; }; filter ipop3_debug { not program("ipop3d") or not level(info,debug); }; filter sshd_debug { not program("sshd") or not level(debug); }; filter f_attack_alert { match("attackalert"); }; filter f_ssh_login_attempt { program("sshd.*") and match("(Failed|Accepted)") and not match("Accepted (hostbased|publickey) for (root|zoneaxfr) from (10.4.3.1)"); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(auth, authpriv) and not facility(mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(cron); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail); }; filter f_emergency { level(emerg); }; filter f_bootlog { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_daemon); destination(messages); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_kern); destination(messages); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(messages); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); destination(maillog); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; log { source(src); filter(cr_asa); destination(cisco_asa); }; Thanks, Nick

1 0

help- Problem with TCP Based Centralized Log Server using Syslog-ng
by Padmanabhan, Rajeesh (GE Healthcare) 31 Oct '06

31 Oct '06

Hi Im trying to setup a centralized, TCP Based Log Server using Syslog-ng. Syslog-ng server stops accepting logs from clients intermediatley. It works again, once i restart the syslog-ng server. Im getting following error messages from clients. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Oct 18 10:29:28 System1 syslog-ng[11745]: Connection broken; time_reopen='60' Oct 18 15:06:26 RADIUM syslog-ng[4999]: io.c: do_write: write() failed (errno 32), Broken pipe Oct 18 15:06:26 RADIUM syslog-ng[4999]: pkt_buffer::do_flush(): Error flushing data Oct 18 15:06:26 RADIUM syslog-ng[4999]: Connection broken to AF_INET(10.70.201.233:514), reopening in 10 seconds syslog-ng[13843]: Connection failed; error='Connection refused (111)', time_reopen='60 syslog-ng[302]: Error connecting to remote host AF_INET(10.70.201.233:514), reattempting in 10 seconds ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Please help me... Thanks in advance. Rajeesh

2 1

which syslog-ng version support default filter rule to uncaught messages?
by Csuti Istvan 31 Oct '06

31 Oct '06

I use syslog-ng 1.6.5-2.2 on debian sarge, and I try DEFAULT filter to manage uncaught messages. log { source(s_sys); filter(DEFAULT); destination(df_facility_other); }; When I restart syslog-ng daemon, I get a following error message: Starting system logging: syslog-ngunresolved reference: DEFAULT Error initializing configuration, exiting. start failed. The Question: Which syslog-ng version support the DEFAULT filter rule? Or, I must compile the pached source version? Istvan Csuti

2 1