- syslog-ng - lists.balabit.hu

Fwd: new to list, and syslog-ng
by Brian Loe 02 Aug '06

02 Aug '06

<I hope you don't get this twice (hadn't confirmed my subscription the first time).> I'm hoping some of you can give me some guidance in my implementation of syslog-ng on an AIX 5.3 system. This is what I have for a config file thus far - will it work? Also, some help on the best way to move from the AIX syslogd to syslog-ng? I'm new to AIX as well... # # Global options. # options { long_hostnames(off); use_dns(no); chain_hostnames(no); sync(0); perm(0640); stats(3600); }; # # 'src' is our main source definition. you can add # more sources driver definitions to it, or define # your own sources, i.e.: # #source my_src { .... }; # source s_local { internal(); unix-stream("/dev/log"); }; source s_udp { udp(ip("0.0.0.0") port(514)); }; # # Filter definitions # filter f_cron { facility(cron); }; # filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7) and not filter(f_cron); }; # filter f_messages { not facility(news, mail); }; filter f_warn { level(warn, err, crit); }; filter f_alert { level(alert); }; # # Cron-messages in one file: # destination cron { file("/var/log/cron"); }; log { source(s_lcoal); filter(f_cron); destination(cron); }; # # local[1-7]: # destination localmessages { file("/var/log/localmessages"); }; log { source(s_local); destination(localmessages); }; # # Network messages: # destination netmessages { file("/var/log/netmessages"); }; log { source(s_udp); destination(netmessages); }; # # Network message pipe: # destination net_pipe { file("/tmp/log/net_pipe"); }; log { source(s_udp); destination(net_pipe); };

1 0

syslog-ng 1.0rc1 stops listening to UDP when zero-length packet received
by Dark Angael 02 Aug '06

02 Aug '06

I have been testing 2.0 rc1 for use in a production environment and have discovered a potential DOS exploit. I discovered this when I stopped receiving logs after a particular application started. It turns out that for some reason this app was sending a zero-length packet to syslog. After receiving this packet, syslog-ng gives the following output in debug mode: EOF occurred while reading; fd='3' Closing log reader fd; fd='3' This is reproducible using packet injection also. It has been shown to happen on both solaris 8 and linux (2.6.15-gentoo-r5). It does not happen on syslog-ng version 1.6.11. _________________________________________________________________ New year, new job there's more than 100,00 jobs at SEEK http://a.ninemsn.com.au/b.aspx?URL=http%3A%2F%2Fninemsn%2Eseek%2Ecom%2Eau&_…

2 3

ISODATE, timezone offset and daylight saving
by Per Jessen 31 Jul '06

31 Jul '06

My local timezone is CET and as we've currently got daylights saving on, we're two hours ahead of UTC. So the offset - when using the ISODATE format - should be +0200. Instead it is +0100. I'm on glibc 2.4. Distro is SUSE 10.1, syslog-ng is 1.6.8. The machines hardware clock is in UTC, localtime time CEST. tournesol:/var/log # date Fri Jul 28 18:19:56 CEST 2006 tournesol:/var/log # date -u Fri Jul 28 16:20:01 UTC 2006 tournesol:/var/log # tournesol:/var/log # date +%z +0200 Am I likely to be doing something wrong? From /var/log/mail: 2006-07-28T18:06:39+0100 tournesol postfix/qmgr[7815]: 3E62737129: removed /Per Jessen, Zürich

2 2

Setting UDP Receive Buffer size, then when increase performance problems
by James 31 Jul '06

31 Jul '06

SUMMARY: Due to the large number of UDP packets (aprox: 6,700 per second) coming into our syslog-ng (version 1.6.4) running on an HP-UX 11.11 server we have had a desire to increase the receive buffer for UDP. I could NOT find a way to do this in syslog-ng, so we changed the default at the OS level (socket_udp_rcvbuf_default). Has it been considered (or possibly already added) to allow this option to be specified in the config file? Increasing this buffer size, seemed to cause a performance problem because syslog-ng (again at least on HP-UX 11.11) has POLLIN set to POLLRDNORM|POLLRDBAND (default when _XOPEN_SOURCE_EXTENDED is set) which must mean the OS walks the entire buffer looking for priority message when poll() is called. Since syslog-ng appears to do a poll(), then 1 read(), followed by another poll(), the OS is walking the entire buffer, and then only reading 1 entry. During the time the OS spends walking this buffer, hundreds of packets have been added. Thus, syslog-ng can reach a point of no return as long as data keeps coming in at this rate. I did a quick change in io.c to override the POLLIN setting HP-UX sets for syslog-ng to the value it would use if _XOPEN_SOURCE_EXTENDED was NOT set (0x0001). This change seems to work for us, but more details and maybe another option is mentioned in my details below. DETAILS: We have an HP-UX system that is regularly receiving 6,700 log entries per second from several systems via UDP. Originally we tried handling this workload with one daemon, but was CPU constrained due to it being single threaded. We then split the work among 3 daemons each on their own virtual IP so we could utilize the other CPUs in the system. We also increased socket_udp_rcvbuf_default (default receive buffer size for UDP) from 65K to 75mb - 150mb. It would be nice if the syslog-ng config file allowed us to set this value instead of impacting ALL UDP listeners on the system. I have edited our copy of io.c to include something like the following. This example is crude, but seems to work for us setting 75Mb buffer (I hard coded the value, but obtaining from the config file would be a better solution): int Buffer = 78643199; socklen_t Len = sizeof(Buffer); if ( (setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &Buffer, Len)) == 0 ) { Buffer=0; getsockopt(fd, SOL_SOCKET, SO_RCVBUF, &Buffer, &Len); debug("io.c: SO_RCVBUF now set to: %i\n", Buffer); } else { werror("io.c: setsockopt() failed setting SO_RCVBUF (errno %i), %z\n", errno, strerror(errno)); } However, this can lead to another problem with syslog-ng (at least on on HP-UX). Once the buffer starts to fill the OS starts spending a LOT of CPU in system time. This appears to be because of the socket options specified due to _INCLUDE_XOPEN_SOURCE_EXTENDED being set during compile. On HP-UX 11.11 poll.h sets POLLIN to (POLLRDNORM|POLLRDBAND) when _INCLUDE_XOPEN_SOURCE_EXTENDED is defined. It is my understanding that this means the OS will walk the entire buffer looking for a priority message when syslog-ng calls poll() to look for the next packet. Once we increase the buffer past the 65K default in HP-UX this walking can take some time and lots of system CPU time (hate to see this if buffer was set the the 2GB maximum). Overriding the POLLIN options to the setting poll.h would use if _INCLUDE_XOPEN_SOURCE_EXTENDED was NOT defined (0x0001) seems to resolve this problem, and actually allows syslog-ng to empty out this large UDP buffer space very quickly. Another solution (which looks like may have been talked about in a post from John Morrissey) may be to change the way syslog-ng loops on the UDP socket. It appears to me that syslog-ng is currently doing: poll() read() Back to poll() This delay maybe could only be encountered once if syslog-ng was to do something like: poll() read() with O_NONBLOCK set if return -1 with EAGAIN, Back to poll() else Back to read() with O_NONBLOCK set I am not sure what impact this would have if syslog-ng was receiving data from more than one UDP socket (which is what we do) as it may spend too much time doing emptying the one buffer while the other data listeners wait. Have concerns like this already been addressed in more current releases? Do these issues I have seen HP-UX concerns for other operating systems, or should I just continue to modify the standard syslog-ng code as there would be little to no benefit for others? Thanks, /James

2 1

solaris package
by Joop Boonen 28 Jul '06

28 Jul '06

Hello All, I've just build the solaris package via solbuild/rules. When i was doing this i was wondering about a few things. These is a nice init.d file in the solbuild directory. solbuild/syslog-ng.init.d After the build it cant be found in the package at all. When i copy the above mentioned init.d script it uses the config file /opt/syslog-ng/etc/syslog-ng/syslog-ng.conf. Would /etc/syslog-ng/syslog-ng.com a better location? I adapted the script for this now. Why is the package in stalled in /opt/syslog-ng in stead of /usr/local? By the way Sun uses /opt/sfw for gnu software. Why are to init.d packages available in the source, being solbuild/syslog-ng.init.d and contrib/init.d.solaris . The first one is realted to /opt/syslog-ng the second one to /usr/local . Regards, Joop Boonen.

2 3

syslog hangup
by Vincent Régnard 28 Jul '06

28 Jul '06

Hi, Since at least a couple of month now, we've been experiencing some strange troubles on some of our routers which we now associate to a syslog-ng (or a syslog-ng internal facility) hangup. The version of syslog ng we use is 1.6.8 on different linux kernel 2.4.31 and 2.6.8. I read some posts on this list (https://lists.balabit.hu/pipermail/syslog-ng/2006-May/008784.html) establishing some similar problems. The conditions in which the problem occur are not clear to us. There are obviously different sircumstances that lead syslog-ng to hangup. We first noticed the problem on a server where some users where not able to login, certainly because of the impossibility to write to the /dev/log socket. Reloading syslog-ng when we had an active shell on the server corrected the problem. For this server we "solved" the problem by adding a cron to check the syslog activity and reload syslog if needed. This is not a nice soltution, but it avoids hard reboot of the server. More recently we realized that some routers where not logging some (iptables firewall) events sent to syslog, when in the same time, log from other daemons where treated correctly, again reloading syslog-ng fixes the problem untill the problem randmly accurs again. I am presently studdying the way these log messages are sent to syslog to understand this trouble better. We are tracking the causes of such an annoying behaviour without succes untill now. First of all we would like to understand what is happening in syslog-ng itselfs, at what level is this hangup ? kernel ? syslog ? is it related to /dev/log socket ? Maybe some experts or syslog devloppers can send us some hints ? Is it related to the kernel environement ? /proc ? udev ? Or is it possible that another daemon is responsible for this syslog hangup. Apparently the problem is also present in newer releases in the 1.6.X branch according to the posts on the list, I checked the branch changelogs without seeing anything on that. Has some work been devoted to fix this kind of trouble in more recent branches (1.9 and 2.0) ? We are planning to develop a daemon to monitor syslog-ng and reload the service in case of hangup. If some of you already performed some work in that direction, we would be glad to share the effort or learn the best and more efficient way proceed. Any hints, comments or suggestions are welcome. Thanks in advance. -- Vincent Régnard vregnard(a)tbs-internet.com TBS-internet.com 027 630 5902

2 1

flow-control and pipes
by Steen Thomsen 27 Jul '06

27 Jul '06

Hello I am currently exploring the possibilitys with flowcontrol from syslog-ng 2.0 on a freeBSD system. I have logs coming in from different sources to a centralized logserver, where I want to save the logs in an archive, and also read the logs into a database. What I seek to accomplish is that in case of database breakdown, logs should still be written to the archive, and when the database is up again, the missing logs should then automatically be written to the database, all in chronological order. The way I plan to do it, with the new possibilities flow-control offer, is that incoming logs are piped to a script that do 2 things. One is to keep count of the linenumbers in the archivefiles, and add to each log its own linenumber, before writting the log to its archivefile. The second task is to pipe a copy of the log (with the line number added) back to syslog-ng. Syslog-ng then pipes the log to another script that reads the linenumber from the log, and compare it to the linenumber from the previous log this script read. If the linenumbers are not consecutive, it means that some logs where lost on the way from the archivescript to this accounterscript. It therefore fetches the missing logs from the archive, identified by the linenumbers, before reading from the pipe again. If I use flow-control on the logs coming in from the sources and piped to the archive script, and no flow-control on the logs that are piped back to syslog-ng and further on to the accounterscript, I accomplish that logs are always written to the archive, even when the database fails, and all logs are later written to database when it comes up again, since there is account for all the logs through their linenumber. The basic idea being that everything is piped under normal conditions, and I only read from the archive in case of databaseproblems. The flow-control works fine for the first part of the process. I can have the archivescript write to the archive with no loss of logs. I can also let the archive script write the same log both to the archive and a pipe, when I place a script at the other end of the pipe to read the pipe, and still not loose logs.. But when I try to write to a pipe that syslog-ng shall read, the flowcontrol doesn't seem to work. Log are lost on the way to the archive script, even though its flowcontrolled. My question is, if there is something I have overlooked that might cause this behaviour. It works when I read the pipe from a script of my own, but not when syslog-ng reads it. The reason I want syslog-ng to read the pipe, and not simply a script of my own, is that syslog-ng will read from the pipe, even if the database is down, where my own script would block on the pipe, and consequently block for incoming logs, since that part ( the other end of the pipe where the archivescript is) is flow-controlled. I read somewhere that syslog-ng opens a pipe for both reading and writing, but I don't quite understand how this comply with the nature of pipes. Can anyone elaborate on this? And could that be a problem in my setup? Kind Regards Steen Thomsen softwaredeveloper <http://www.bridicum.com/> <http://www.bridicum.com/>

1 1

Need help pulling out message info before sending on to loghost from syslog-ng proxy
by Steven Meacham 27 Jul '06

27 Jul '06

Hi, Need help with removing some information before message is sent to the loghost server. I am trying to remove what looks like a priority or facility code date and time....is there any way to remove this out of the message before fowarding on to loghost Thanks Steven -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.394 / Virus Database: 268.10.4/396 - Release Date: 7/24/2006

2 1

Re: syslog-ng works only with one Core-CPU
by Rolf Heitfeld 27 Jul '06

27 Jul '06

>> Hi, >> >> we're using syslog-ng-1.6.2-29.3 (default on SUSE SLES9 Systems). >> >> We saw, that syslog-ng only uses one core of the Dual Core AMD Opteron(tm) Processor 275 of the System. >> >> Is there any possibility to turn this off? >> >> Thanx for any help! >> >> Rolf Heitfeld >Hello, >Can you clarify what is the problem? Syslog-ng isn't a multithreaded >program, so it runs only on one processor, as any singlethreaded program >does. Hello, Sorry perhaps I have expressed myself in a wrong way. The question for me was: Is there any Version of syslog-ng that is multithreaded and we could use instead? And if I unterstood you correctly the answer is "no". Thank you for answering! Regards, Rolf Heitfeld

2 1

RE: [syslog-ng] Hi Every Body - Host Name Rewrite Help!
by Nasir, Meraz 26 Jul '06

26 Jul '06

Hi I don't create any local file to capture logs, everything piped to mysql database so that I can query it using php-syslog-ng. ##################################### options { group(logs); dir_group(logs); perm(0640); dir_perm(0750); create_dirs(yes); use_time_recvd(yes); sync(0); use_fqdn(no); stats (7200); log_fifo_size(4096); keep_hostname(yes); chain_hostnames(no); bad_hostname("gconfd"); gc_idle_threshold(500); gc_busy_threshold(10000); }; ############################################################# Issue: I maintain all the device entry on the hosts file, usually syslog-ng rewrites hostname for the log source. For some reason it is not able to rewrite host names for the logs that is generated from all my APC devices. Thanks, Meraz 718.613.4661 -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Alexander Clouter Sent: Wednesday, July 26, 2006 3:38 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Hi Every Body Hi, Sorry but this is not the greatest of support requests.... Nasir, Meraz <MNasir(a)INTERFAITHMEDICAL.com> [20060726 15:22:31 -0400]: > > Hi > > Bellow is my Environment: > > * Host: Linux version 2.6.9-5.ELsmp (bhcompile(a)decompose.build.redhat.com) (gcc version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)) #1 SMP Wed Jan 5 19:30:39 EST 2005 > usually not relevant > * mysql Ver 14.7 Distrib 4.1.7, for redhat-linux-gnu (i386) > * httpd-2.2.0 > * php-5.1.2 > Not syslog's fault, unless you plan on dumping all your logs to an SQL database, but really it could be any database in practise. > * syslog-ng-1.6.9.orig > Useful bit. > My Issue: > > I am not able to re-write UPS's name, > to me an "UPS's" is a uninterruptable power supply....is that what you mean. As you don't put it in any context (why don't you get it to advertise a different name?) its hard to tell what you are doing. Usually when confronted by a support request with no detail I ask the four magic questions that seem applicable to *any* fault: 1. what is it you are actually trying to do? 2. how are you trying to go about doing it? 3. what are you expecting to happen? 4. what is actually happening? We all need details, examples, configuration extracts, errors being quoted. Whats going into syslog-ng and whats coming out? Explain why your "UPS's" are fixed with their name. We all would love to help but, I at least, don't have the foggiest of an idea in regards to what you are talking about. What little I can extract I think what you could use is the template() system and strap it to some rDNS lookups cached locally with a lightweight DNS server on the local machine. Of course I need to know what it is you are trying to do :( Cheers Alex > This message is for the named person's use only. It may contain > confidential, proprietary or legally privileged information. No > confidentiality or privilege is waived or lost by any mistransmission. > If you receive this message in error, please immediately delete it and > all copies of it from your system, destroy any hard copies of it and > notify the sender. You must not, directly or indirectly, use, disclose, > distribute, print, or copy any part of this message if you are not the > intended recipient. Interfaith Medical Center and any of its > subsidiaries each reserve the right to monitor all e-mail communications > through its networks. > This message is for the named person's use only. It may contain confidential, proprietary or legally privileged > information. o Confidentiality is waved or lost by any mistransmission. If you receive this > message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of > it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, orint, or copy any > part of this message if you are not the intended recipient. Interfaith Medical Center and any of its subsidiaries > each reserve the right to monitor all e-mail communications through its networks. > does this really need to be this long? Why can't people have "disclaimer at http://example.com/legal-mumble-jumble.html"? </rant> > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] Hi Every Body
by Nasir, Meraz 26 Jul '06

26 Jul '06

Hi Bellow is my Environment: * Host: Linux version 2.6.9-5.ELsmp (bhcompile(a)decompose.build.redhat.com) (gcc version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)) #1 SMP Wed Jan 5 19:30:39 EST 2005 * mysql Ver 14.7 Distrib 4.1.7, for redhat-linux-gnu (i386) * httpd-2.2.0 * php-5.1.2 * syslog-ng-1.6.9.orig My Issue: I am not able to re-write UPS's name, Thanks, Meraz 718.613.4661 ________________________________ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Nasir, Meraz Sent: Wednesday, July 26, 2006 3:06 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Hi Every Body Importance: High Hello I need some help on the configuration. I am not sure about the process Regards, Meraz Nasir Manager of Systems & Network Information Technology Services (ITS) Interfaith Medical Center 1545 Atlantic Avenue Brooklyn, NY 11213 718.613.4661 This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Interfaith Medical Center and any of its subsidiaries each reserve the right to monitor all e-mail communications through its networks. This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. o Confidentiality is waved or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, orint, or copy any part of this message if you are not the intended recipient. Interfaith Medical Center and any of its subsidiaries each reserve the right to monitor all e-mail communications through its networks.

2 1

Hi Every Body
by Nasir, Meraz 26 Jul '06

26 Jul '06

Hello I need some help on the configuration. I am not sure about the process Regards, Meraz Nasir Manager of Systems & Network Information Technology Services (ITS) Interfaith Medical Center 1545 Atlantic Avenue Brooklyn, NY 11213 718.613.4661 This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. o Confidentiality is waved or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, orint, or copy any part of this message if you are not the intended recipient. Interfaith Medical Center and any of its subsidiaries each reserve the right to monitor all e-mail communications through its networks.

1 0

Re: Re: [syslog-ng] syslog-ng 2.0rc1 memory usage
by Martin, David M 25 Jul '06

25 Jul '06

Configs included.. Similar behavior noticed on my RHEL4 instance. Both are running 2.0rc1 [Solaris 9 Host] PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 29935 root 1 59 0 221M 220M sleep 16:49 2.38% syslog-ng.2.0rc # cat syslog-ng.conf options { sync(0); time_reopen(10); log_fifo_size(8192); flush_lines(4096); long_hostnames(off); use_dns(yes); use_fqdn(no); create_dirs(no); keep_hostname(yes); owner(syslogng); group(syslogng); }; source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; source network { udp(); tcp(); }; # # standard destinations for local standard system messages # destination authlog { file("/var/log/auth.log"); }; destination syslog { file("/var/log/syslog"); }; destination kern { file("/var/log/kern.log"); }; destination maillog { file("/var/log/maillog"); }; destination messages { file("/var/log/messages" perm(0644)); }; # # console destination # destination console { file("/dev/sysmsg"); }; # # filters for standard local system messages which come # in on non-local facilities # filter f_authpriv { facility(auth) ; }; filter f_syslog { not facility(auth) and not facility(mail); }; filter f_kern { facility(kern); }; filter f_mail { facility(mail); }; # # catch the rest # filter f_messages { level(info .. warn) and not facility(auth); }; # # filters for various emergency level messages # filter f_emergency { level(emerg); }; # # log emergency level messages out to console # log { source(local); filter(f_emergency); destination(console); }; # # log messages from local machine # log { source(local); filter(f_authpriv); destination(authlog); }; log { source(local); filter(f_syslog); destination(syslog); }; log { source(local); filter(f_kern); destination(kern); }; log { source(local); filter(f_mail); destination(maillog); }; # # catch the rest of the messages # log { source(local); source(network); filter(f_messages); destination(messages); flags(flow-control); }; destination hosts { file("/var/log/messages" owner(syslogng) group(syslogng) perm(0644) dir_perm(0700)); }; # # logs all incoming messages from network source to the sorted # destination # log { source(network); destination(hosts); }; destination d_sec { program( "/opt/sec/2.3.3/sec.pl -input=\"-\" -conf=/opt/sec/etc/sec.conf" template("$MONTH$DAY $UNIXTIME $HOST $MSG\n") ); }; filter f_mysql { host("XXXXXXXX") or host("XXXXXXXX"); }; log { source(network); filter(f_mysql); destination(d_sec); }; [RHEL 4 Update 3 Host] PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16414 root 15 0 54064 50m 1068 S 1.3 1.2 3:51.60 syslog-ng # cat syslog-ng.conf ############################################################################### # options - global options { log_fifo_size(8192); create_dirs(no); #default chain_hostnames(no); #default yes normalize_hostnames(yes); #default requires 1.9.9 keep_hostname(yes); #default no owner(root); group(root); perm(0600); use_time_recvd(yes); #default no }; ############################################################################### # sources source abs_udp514 { udp(ip(0.0.0.0) port(514)); }; source abs_localhost { unix-stream("/dev/log"); internal(); }; ############################################################################### # filters filter abf_messages { level(debug) and not (facility(mail,auth,authpriv,cron,user)); }; filter abf_auth { facility(auth,authpriv); }; filter abf_mail { facility(mail); }; filter abf_cron { facility(cron); }; filter abf_spooler { facility(uucp) or (facility(news) and level(crit)); }; filter abf_boot { facility(local7); }; filter abf_daemon { facility(daemon); }; filter abf_user { facility(user); }; filter abf_ignore { not (host("2006")); }; filter abf_postfix { program ("postfix/"); }; ############################################################################### # destinations destination abd_messages { file( "/var/log/messages" group(bb) perm(0640) ); }; destination abd_auth { file("/var/log/auth.log"); }; destination abd_mail { file( "/var/log/mail.log" group(bb) perm(0640) ); }; destination abd_cron { file("/var/log/cron.log"); }; destination abd_spooler { file("/var/log/spooler.log"); }; destination abd_boot { file("/var/log/boot.log"); }; destination abd_daemon { file("/var/log/daemon.log"); }; destination abd_user { file("/var/log/user.log"); }; destination abd_sec { program( "/opt/sec/2.3.3/sec.pl -input=\"-\" -conf=/opt/sec/etc/sec.conf" template("$MONTH$DAY $UNIXTIME $HOST $MSG\n") ); }; destination abd_localhost { file("/var/log/localhost.log"); }; ############################################################################### # logs log { source(abs_udp514); filter(abf_messages); destination(abd_messages); }; log { source(abs_udp514); filter(abf_auth); destination(abd_auth); }; log { source(abs_udp514); filter(abf_mail); destination(abd_mail); }; log { source(abs_udp514); filter(abf_cron); destination(abd_cron); }; log { source(abs_udp514); filter(abf_spooler); destination(abd_spooler); }; log { source(abs_udp514); filter(abf_boot); destination(abd_boot); }; log { source(abs_udp514); filter(abf_daemon); destination(abd_daemon); }; log { source(abs_udp514); filter(abf_user); destination(abd_user); }; log { source(abs_udp514); filter(abf_postfix); destination(abd_sec); }; log { source(abs_localhost); destination(abd_localhost); }; ------------------------------ Message: 2 Date: Fri, 21 Jul 2006 11:16:27 +0200 From: Balazs Scheidler <bazsi(a)balabit.hu> Subject: Re: [syslog-ng] syslog-ng 2.0rc1 memory usage To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <1153473387.6682.6.camel(a)bzorp.balabit> Content-Type: text/plain On Thu, 2006-07-20 at 19:47 -0500, Martin, David M wrote: > Anyone running 2.0rc1 on Solaris 9 (sparc)? > If so, what memory requirements are you seeing for the syslog-ng > process? > > My install handles ~ 700 messages/sec (all UDP) and over the course of > a day syslog-ng will gradually consume nearly all available free > memory (I've seen it up to 1GB). > > (syslog-ng is bounced by logrotate daily, so the process repeats > itself) > I was able to substantially decrease the rate of consumption by adding > the 'flush_lines" global (syslog-ng process now at 91MB and growing > ... slowly). > This definitely seems to be some kind of memory leak. Can you post a configuration file that reproduces the problem? Hopefully it is a leak in syslog-ng itself and not in one of the system libraries. -- Bazsi ------------------------------ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng End of syslog-ng Digest, Vol 15, Issue 15 ***************************************** The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 1

syslog-ng works only with one Core-CPU
by Rolf Heitfeld 25 Jul '06

25 Jul '06

Hi, we're using syslog-ng-1.6.2-29.3 (default on SUSE SLES9 Systems). We saw, that syslog-ng only uses one core of the Dual Core AMD Opteron(tm) Processor 275 of the System. Is there any possibility to turn this off? Thanx for any help! Rolf Heitfeld

2 1

syslog-ng works only with one Core-CPU
by Rolf Heitfeld 25 Jul '06

25 Jul '06

Hi, we're using syslog-ng-1.6.2-29.3 (default on SUSE SLES9 Systems). We saw, that syslog-ng only uses one Core of the Dual Core AMD Opteron(tm) Processor 275 of the System. Is there any possibility to turn this off? Rolf Heitfeld LINUX System Engineer / OPS-Infrastructure www.combots.com <http://www.combots.com/> ComBOTS AG Amalienbadstraße 41 76227 Karlsruhe, Germany Tel. +49 721 5160 9935 Fax. +49 721 5160 Mobile + 49 173 385 02 45 Rolf.Heitfeld(a)combots.com <mailto:Rolf.Heitfeld@combots.com>

1 0

Rick Daoust is out of the office - Rick Daoust est hors du bureau
by Rick.Daoust＠borealc.on.ca 24 Jul '06

24 Jul '06

I will be out of the office starting 24/07/2006 and will not return until 31/07/2006. Je serai hors du bureau à partir de lundi le 24 juillet 2006 et retournerai que lundi le 31 juillet 2006!

1 0

Re: syslog-ng Digest, Vol 15, Issue 18
by dwarnock＠harbrook.net 24 Jul '06

24 Jul '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

1 0

Re: syslog-ng Digest, Vol 15, Issue 17
by dwarnock＠harbrook.net 23 Jul '06

23 Jul '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

1 0

Re: syslog-ng Digest, Vol 15, Issue 16
by dwarnock＠harbrook.net 22 Jul '06

22 Jul '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

1 0

Re: syslog-ng Digest, Vol 15, Issue 14
by dwarnock＠harbrook.net 21 Jul '06

21 Jul '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

1 0

Logrotate Questions And Solicitation For Ideas...
by Vadim Pushkin 20 Jul '06

20 Jul '06

Hello All; I've got a syslog-1.6.9 server running quite well with phpsyslog-ng. I am trying to rotate my logs and maintain just 14 days worth of logs within my MySql DB, an issue that I am having and will hopefully get fixed within the phpSyslog-NG group. With regards to flat file logs, the issue that I am writing here for, I wish to have 30 days worth of data within my log files (flat files under /var/log). The problem that I am having is rotating under syslog-ng, *NOT* MySql. Can anyone please provide me with a good method for rotating logs, where my logs are kept under various subdirs, depending on source address? Attached is my syslog-ng.conf, in case anyone cares to read it to determine what my dir-layouts are, also I am looking for help on why some of my source hosts are getting into two locations, I am sure that it is bacause of the "filter" rules in my config. Does the below make sense? # Trying to make all hosts that start with sw or contain sw1, or sw2, # get placed where filter F_switch should go. filter F_switch { host("sw*") or host("sw1") or host("sw2"); }; # Trying to place logs for all hosts that start with fw to go to where filter F_firewall should go. filter F_firewall { host("^fw*"); }; Please, I am sorry for this question if it is in the FAQ, I am having problem finding this info. Thank you all! .vp

7 12

Feature Request: conf.d-like folders or include statement
by JP Senior 19 Jul '06

19 Jul '06

I'm finding that I'm in need of a recursive conf.d-like configuration folder for syslog-ng. The reason I'm asking this is because I have many hosts, each requiring a separate configuration from the other, and my syslog-ng.conf file is getting pretty large, messy, and unmanageable. If it could iterate/enumerate /etc/syslog-ng/conf.d and add each of those to the 'running' config, that would be amazing. I think this could also work, in a less versatile function, by an 'include' statement in syslog-ng.conf. Ie: include { includefile(/etc/syslog-ng/extra/hostname.conf); }; Am I on the right track, or is there a function to do this? :) -JP Senior

2 1

Re: syslog-ng Digest, Vol 15, Issue 10
by dwarnock＠harbrook.net 19 Jul '06

19 Jul '06

I am on holiday from the 17th August returning on Monday the 31st August. If your query is urgent please contact the office on +44 (0)1423 508060 or Neil Harvey on +44 (0)15395 64734 (nharvey(a)harbrook.net) Regards David Warnock

1 0

why this messages from syslog?
by Maurizio B 18 Jul '06

18 Jul '06

why I get this messages from syslog-ng 2.0rc1? Log statistics; processed='source(s_local)=19', processed='source(s_remote)=0', processed='destination(dsl_console)=0', processed='destination(dsl_maillog)=0', processed='destination(ds_emerg)=0', processed='destination(ds_alert)=0', processed='destination(df_uucp)=0', processed='destination(ds_err)=0', processed='destination(dfs_err)=0', processed='destination(dsl_messages)=16', processed='destination(dfs_crit)=0', is it a misconfiguration? thanks in advance

2 1

Re: Logging to program destinations has odd prefix?
by John Caruso 16 Jul '06

16 Jul '06

At Friday 11:14 AM 9/17/2004, Balazs Scheidler wrote: >On Thu, 2004-09-16 at 19:36, Lars Kellogg-Stedman wrote: >> Text logged to a program via the program() destination has a prefix of the >> form <nnn>, which I'm assuming is the encoding of the syslog level and >> facility. Great! But I don't want it :). >> >> Is there any way to prevent this text from being logged? Sure, I can write >> a "wrapper" program to edit these messages before they reach the intended >> destination, but my template doesn't ask for this information, so I'm not >> sure it should be there. > >I think this was fixed in one of the recent versions. I just ran into this bug while using syslog-ng 1.6.0rc1 with a program destination on a Solaris system. After finding the above discussion from 2004 I upgraded to 1.6.11...but the bug is still there. Here are some samples of lines handed off from syslog-ng to the external program: <6>Jul 7 14:53:50 sys1 kernel: SysRq : Emergency Sync <4>Jul 7 14:53:50 sys1 kernel: Emergency Sync complete <6>Jul 7 14:53:51 sys2 kernel: SELinux: initialized (dev 0:15, type nfs), uses genfs_contexts I'm getting around this by editing out the prefix, but it would be good if this were fixed in syslog-ng itself. - John

2 7