- syslog-ng - lists.balabit.hu

syslog-ng 3.0.3 has been released
by devel＠balabit.hu 03 Jul '09

03 Jul '09

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.3 SUMMARY : new stable release DATE : Jul 3, 2009 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.3) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.3 Thu, 25 Jun 2009 08:20:28 +0200 IMPORTANT CHANGES: * There was an unintended change in the meaning of command line argument -C (--chroot) and -R (--persist-file) between 2.x and 3.x versions of syslog-ng. This release reverts this change back to the original 2.x behaviour. This is a incompatible change compared against 3.0.2. Please review your startup scripts if you use either -C or -R. Bugfixes: * Fixed a compilation problem if IPv6 is disabled. * Fixed a crash problem if a log statement has a filter but no destinations. * The usertty() driver was changed to open the terminal device in nonblocking mode, as open on a serial device might block which cause the syslog-ng process to hang. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Re: [syslog-ng] How to configure mail alerts
by ccie 6862 30 Jun '09

30 Jun '09

I had looked at this page several times, and I didn't realize the perl script on the page referenced by "Use this perl script to strip off the..." was the same as what was in the configuration file. It is working. Thank you very much. --- On Tue, 6/30/09, Siem Korteweg <Siem.Korteweg(a)qnh.nl> wrote: > From: Siem Korteweg <Siem.Korteweg(a)qnh.nl> > Subject: Re: [syslog-ng] How to configure mail alerts > To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> > Date: Tuesday, June 30, 2009, 12:37 AM > Have a look at: http://www.campin.net/newlogcheck.html > > regards, > > Siem Korteweg > > -----Oorspronkelijk bericht----- > Van: syslog-ng-bounces(a)lists.balabit.hu > namens ccie 6862 > Verzonden: ma 29-6-2009 22:25 > Aan: syslog-ng(a)lists.balabit.hu > Onderwerp: [syslog-ng] How to configure mail alerts > > > This is a pretty basic question, but I'm stumped as to what > to install to get > email alerts. The configuration is simple, but in the > sample configs I've run > across, I've seen something similar to the following: > > destination mail-alert-perl { > program("/usr/local/bin/syslog-mail-perl"); }; > > I've installed perl-Parse-Syslog-1.10-1.el4.rf.noarch.rpm > and > perl-Parse-Syslog-Mail-0.14-1.el4.rf.noarch.rpm, but > nothing remotely close > has been installed. Can someone please point me in the > right direction? > > Thanks > > > > _____________________________________________________________________________ > _ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > -----Inline Attachment Follows----- > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > >

1 0

How to configure mail alerts
by ccie 6862 30 Jun '09

30 Jun '09

This is a pretty basic question, but I'm stumped as to what to install to get email alerts. The configuration is simple, but in the sample configs I've run across, I've seen something similar to the following: destination mail-alert-perl { program("/usr/local/bin/syslog-mail-perl"); }; I've installed perl-Parse-Syslog-1.10-1.el4.rf.noarch.rpm and perl-Parse-Syslog-Mail-0.14-1.el4.rf.noarch.rpm, but nothing remotely close has been installed. Can someone please point me in the right direction? Thanks

2 1

change msg text before db inserting
by Henk van Lingen 30 Jun '09

30 Jun '09

Hi, Is is possible to change the message text before it is inserted into a database? I have syslog-ng-3.0.2 running and it does insert the messages directly into a PostgreSQL database, using the new sql(type(pgsql) destination feature. The problem is I have lots of cisco firewall loggings, and in the database I have of gin index for doing full text searching. For this index, the database decomposes the message into tokens. I want to be able to search on IP-numbers. However, in lines like world/123.456.192.46(0) -> some-fwsm/123.456.221.121(0) this won't work because the tokens are world/123.456.192.46 and /123.456.192.46, not 123.456.192.46. To solve the prob it would be nice if there is a way to replace the slashes with spaces before the message goes to the database. Regards, -- Henk van Lingen, (o- -+ Netwerk & Telefonie, ICT Service Center /\ | Universiteit Utrecht, Jenalaan 18a, kamer 1.40 v_/_ http://henk.vanlingen.net/ http://www.tuxtown.net/netiquette/

3 2

Talk on syslog-ng a the 10th Libre Software Meeting
by ILLES, Marton 26 Jun '09

26 Jun '09

Hi, For those of you interested, I will be giving a talk at the 10th Libre Software Meeting at Nantes, France. The talk will be in English and I try to focus on syslog-ng 3.0 and the db-parser feature. http://2009.rmll.info/Syslog-ng-3-a-step-towards-log.html?lang=en If you gonna be there at the conference and want to meet to discuss something syslog-ng related stuff or anything else, just drop me line. cheers, Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0

Inconsistent Command Line Arguments
by Leonid Chaichenets 24 Jun '09

24 Jun '09

Hello List, there seems to be a problem with the command line parameter list of syslog-ng (tested for ver. 3.0.2): from what I understand from the source code command line parameters are defined in gprocess.c (g_process_option_entries[]) and in main.c (syslogng_options[]). Only the latter are described by syslog-ng --help (but both of them are usable). Furthermore, the short version of '--chroot' should be '-C', according to the syslog-ng administration guide (The syslog-ng manual pages -> syslog-ng), but '-C' is defined in gprocess.c as "Set default capability set". According to gprocess.c the correct short version for '--chroot' would be '-R', but it collides and is overridden by the short version of the same name from main.c (there it means "persist-file"). IMHO this is a security relevant bug: one might think syslog-ng is running chrooted while it actually has root powers. PS: This also applies to the git-versions syslog-ng-3.0.git and syslog-ng-3.1.git. -- Best Regards, Leonid Chaichenets.

2 2

apache
by john.li＠fifasci.com.ph 24 Jun '09

24 Jun '09

Hello, How do i enable send apache logs on syslog-ng client? Im using centos5 the default location of logs are /var/log/httpd/access_log and /var/log/httpd/error_log? any ideas. tnx Here's my syslog-ng client configuration source log002 { unix-stream("/dev/log" max_connections(1000)); internal(); }; destination splunk {udp("x.x.x.x" port(514));}; log { source(log002); destination(splunk); };

3 3

Perl FIFO eating CPU
by Clayton Dukes 21 Jun '09

21 Jun '09

Hiya Folks! I know this isn't necessarily the purview of this group but I thought I'd ask anyways since there are so many smart people here :-) I have syslog-ng feeding to a pipe which my perl script reads from, does some filtering/deduplication of messages, and then inserts into a mysql db. For some reason, the perl script is running between 85-100% cpu at all times (mysql cpu is ok). I'm receiving roughly 1-2 messages per second on my test server, but plan to use this for a production box that will receive much more (around 50 mps) Is there some perl magic I can do to lower the cpu utilization? caching, etc? I'm happy to share my script, but a large portion of it depends on variables set from within my program (php-syslog-ng) so it won't run on outside systems (unless, of course, you install my software :-)) Thanks! -- ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 1

Time slice destination files
by Martin Holste 20 Jun '09

20 Jun '09

The indexing system I'm writing will work best if it can read in syslog-ng logs as regular files (as opposed to fifos, sockets or SQL). In order to have multiple indexing workers going at the same time, I'm finding it easiest if the output from syslog-ng can be divided into chunks like this: destination d_file { file("/tmp/slices/$R_UNIXTIME.${.classifier.rule_id}" template(t_db_parsed)); }; The workers index the chunks as syslog-ng spits them into the slices buffer directory. This works pretty well, but I'm limited to tuning only between one second intervals provided by $R_UNIXTIME, or minute, hour, etc. intervals by concatenating other time macros together. Is there some way I could get syslog-ng to spit out something lke 5 second slices? Also, will syslog-ng know to close the file handle on a log slice when the $R_UNIXTIME second has elapsed? Thanks, Martin

2 3

Syslog-ng 3.0.2 statistics
by Aaron Robel 19 Jun '09

19 Jun '09

Hello, My apologies in advance, this is my first posting and I'm quite the rook' when it comes to Linux and Syslog-ng. I keep wondering why this is my project. I have a 4 server syslog deployment with 3 front end "relay" boxes and 1 backend archive box all within a virtualized SLES environment. Recently I noticed that the relay's together are averaging about 2500 messages per second (mps). The majority of the messages are coming from a single relay, about 2000 mps. Yet the archive box is only averaging about 400 mps. Since we are running 3.0.2 I decided to turn up the stats_level to (1). I don't see any drops to the about 150 file destinations that I've built. What does stamp, processed, stored, etc.. mean? I couldn't find any detailed documentation about the different statistics. Why am I getting such a large discrepency between "stamp" and "processed" in the log stats? Finally, since I'm sending the email does anyone see an issue with the way I've got the flow control set up in the global options? Here are my stats in question off my archive box: processed='src.udp(s_network#0)=22020892', stamp='src.udp(s_network#0)=1245249328' Here's the global's off the archive box: options { time_sleep(10); log_fetch_limit(250); log_fifo_size(2000); use_dns(no); keep_timestamp(yes); dns_cache(no); long_hostnames(off); flush_lines(2000); flush_timeout(200); perm(0644); stats_freq(1800); stats_level(1); time_reopen(10); create_dirs(yes); dir_perm(755); }; Thanks!

4 9

rate-limit
by Julien lecubin 19 Jun '09

19 Jun '09

hi list, what's the best way with syslog-ng to control the number of syslog mail notifications coming from a machine ? I'm currently using perl script to send notification but sometime it just floods my mail (some 600 mails in 5 minutes for a single event) I want to rate limit it to only 1 mail in 5 minute per host or message. /etc/syslog-ng/syslog-ng.conf : ----------------------------- [...] destination d_mail { program("perl /usr/local/bin/mail_log_mailsend.pl $MSG "); }; [...] filter f_notify_by_mail {level(emerg,alert,crit); }; [...] log { source(s_network); filter(f_notify_by_mail); destination(d_mail); }; mail_log_sendmail.pl -------------------- use MIME::Lite; while (<>) { # $_ contains the Log my $body = "Mail from Syslog-ng $_ "; my $msg = MIME::Lite->new ( From =>'syslog-ng(a)my_domain.fr', To =>'service.informatique(a)my_domain.fr', Subject =>'[SYSLOG-NG] Avertissement', Type =>'multipart/related'); $msg -> attach (Type => 'text/html', Data => qq {$body}); MIME::Lite->send('smtp','imap', Timeout=>60); $msg -> send or die "Impossible to send mail!"; } Is there's a notification_interval control (as i use in nagios) i can add in my syslog config file ? I didn't found anything in the man page (maybe i'm reading like a frog) Anyone have any clues / examples to show ? tks, julien Lecubin -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Julien LECUBIN Institut de Mecanique Celeste et de Calcul des Ephemerides CNRS UMR 8028 - Observatoire de Paris 77, avenue Denfert Rochereau 75014 PARIS tel : 01.40.51.22.80 fax : 01.46.33.28.34 julien.lecubin(a)imcce.fr | service.informatique(a)imcce.fr - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

2 2

Handle multiple encoding from multiple sources
by David Perret 19 Jun '09

19 Jun '09

Hello, I'm trying to understand how syslog-ng handle charset encoding. I've seen that : - without the "encoding("charset")" keyword in the source it just writes to the output what it gets from the input. - with the "encoding("charset")" keyword in the source it converts what it gets from the output from "charset" to UTF-8. What is the best way to handle logs from different charset? Do I really have to create a different source per charset used ? Is there any way to say that I always want the input's charset to be converted to UTF-8 before writing my logs ? Thanks David

2 1

syslog-ng 3.0 seem to chop some of the long lines off
by Joe Hansen 19 Jun '09

19 Jun '09

We're seeing lines in /var/log/messages that are chopped off. They should look like this: v\:* { BEHAVIOR: url(#default#VML) } o\:* { BEHAVIOR: url(#default#VML) } w\:* { BEHAVIOR: url(#default#VML) } .shape { BEHAVIOR: url(#default#VML) } st1\:* { BEHAVIOR: url(#default#ieooui) } @font-face { font-family: MS Mincho; } @font-face { font-family: Tahoma; } @font-face { font-family: Calibri; } @font-face { font-family: Verdana; } @font-face { font-family: Consolas; } @font-face { font-family: MS Sans Serif; } @font-face { font-family: @MS Mincho; } @page Section1 {size: 595.3pt 841.9pt; margin: 72.0pt 90.0pt 72.0pt 90.0pt; } A:link { mso-style-priority: 99 } SPAN.MSOHYPERLINK { mso-style-priority: 99 } A:visited { mso-style-priority: 99 } SPAN.MSOHYPERLINKFOLLOWED { mso-style-priority: 99 } P.MSOPLAINTEXT { mso-style-priority: 99 } LI.MSOPLAINTEXT { mso-style-priority: 99 } DIV.MSOPLAINTEXT { mso-style-priority: 99 } P { mso-style-priority: 99 } SPAN.PLAINTEXTCHAR { mso-style-priority: 99 } SPAN.PLAINTEXTCHAR0 { mso-style-priority: 99 } SPAN.PLAINTEXTCHAR00 { mso-style-priority: 99 } SPAN.PLAINTEXTCHAR000 { mso-style-priority: 99 } SPAN.PLAINTEXTCHAR0000 { mso-style-priority: 99 } P.MsoNormal { FONT-SIZE: 12pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman" } LI.MsoNormal { FONT-SIZE: 12pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman" } DIV.MsoNormal { FONT-SIZE: 12pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman" } A:link { COLOR: blue; TEXT-DECORATION: underline } SPAN.MsoHyperlink { COLOR: blue; TEXT-DECORATION: underline } A:visited { COLOR: purple; TEXT-DECORATION: underline } SPAN.MsoHyperlinkFollowed { COLOR: purple; TEXT-DECORATION: underline } P.MsoPlainText { FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: Consolas } LI.MsoPlainText { FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: Consolas } DIV.MsoPlainText { FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: Consolas } P { FONT-SIZE: 12pt; MARGIN-LEFT: 0cm; MARGIN-RIGHT: 0cm; FONT-FAMILY: "Times New Roman"; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto } SPAN.plaintextchar { FONT-FAMILY: Consolas } SPAN.plaintextchar0 { FONT-FAMILY: Consolas } SPAN.plaintextchar00 { FONT-FAMILY: Consolas } SPAN.plaintextchar000 { FONT-FAMILY: Consolas } SPAN.plaintextchar0000 { FONT-FAMILY: Consolas } SPAN.EmailStyle24 { COLOR: windowtext; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle25 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle26 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: Tahoma; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle27 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle28 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle29 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle30 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle31 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle32 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle33 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle34 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle35 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle36 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle37 { COLOR: #1f497d; FONT-FAMILY: Calibri; mso-style-type: personal } SPAN.EmailStyle38 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle39 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle40 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: Tahoma; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle41 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle42 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: Tahoma; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle43 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle44 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle45 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle46 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle47 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle48 { FONT-WEIGHT: normal; COLOR: black; FONT-STYLE: normal; FONT-FAMILY: "Courier New"; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle49 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle50 { COLOR: #1f497d; FONT-FAMILY: Calibri; mso-style-type: personal } SPAN.EmailStyle51 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle52 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle53 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle54 { FONT-WEIGHT: normal; COLOR: #262626; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal } SPAN.EmailStyle55 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle56 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle57 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle58 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle59 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle60 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle61 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle62 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle63 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle64 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle65 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle66 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle67 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal } SPAN.EmailStyle68 { COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal-reply } DIV.Section1 { page: Section1 } OL { MARGIN-BOTTOM: 0cm } UL { MARGIN-BOTTOM: 0cm } Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> ,to=To: ""Graeme Ross"" <gross(a)mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858" But they look like this: Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> Anyone seen this? Thanks __________________________________________________________________ Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now http://ca.toolbar.yahoo.com.

2 1

Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off
by Joe Hansen 18 Jun '09

18 Jun '09

More info on this issue: We found that if the source message has "\n\r" this behavior happens. Does anyone know if this is by design or listed in the RFC anywhere? Thanks, Joe --- On Wed, 6/17/09, Joe Hansen <yrmf250(a)yahoo.com> wrote: From: Joe Hansen <yrmf250(a)yahoo.com> Subject: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: syslog-ng(a)lists.balabit.hu Received: Wednesday, June 17, 2009, 7:10 PM We're seeing lines in /var/log/messages that are chopped off. They should look like this: #yiv1493839580 v\:* { } #yiv1493839580 o\:* { } #yiv1493839580 w\:* { } #yiv1493839580 .shape { } #yiv1493839580 st1\:* { } _filtered #yiv1493839580 { font-family:MS Mincho;} _filtered #yiv1493839580 { font-family:Tahoma;} _filtered #yiv1493839580 { font-family:Calibri;} _filtered #yiv1493839580 { font-family:Verdana;} _filtered #yiv1493839580 { font-family:Consolas;} _filtered #yiv1493839580 { font-family:MS Sans Serif;} _filtered #yiv1493839580 { font-family:Mincho;} _filtered #yiv1493839580 {margin:72.0pt 90.0pt 72.0pt 90.0pt;} #yiv1493839580 A:link { } #yiv1493839580 SPAN.MSOHYPERLINK { } #yiv1493839580 A:visited { } #yiv1493839580 SPAN.MSOHYPERLINKFOLLOWED { } #yiv1493839580 P.MSOPLAINTEXT { } #yiv1493839580 LI.MSOPLAINTEXT { } #yiv1493839580 DIV.MSOPLAINTEXT { } #yiv1493839580 P { } #yiv1493839580 SPAN.PLAINTEXTCHAR { } #yiv1493839580 SPAN.PLAINTEXTCHAR0 { } #yiv1493839580 SPAN.PLAINTEXTCHAR00 { } #yiv1493839580 SPAN.PLAINTEXTCHAR000 { } #yiv1493839580 SPAN.PLAINTEXTCHAR0000 { } #yiv1493839580 P.MsoNormal { FONT-SIZE:12pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:"Times New Roman";} #yiv1493839580 LI.MsoNormal { FONT-SIZE:12pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:"Times New Roman";} #yiv1493839580 DIV.MsoNormal { FONT-SIZE:12pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:"Times New Roman";} #yiv1493839580 A:link { COLOR:blue;TEXT-DECORATION:underline;} #yiv1493839580 SPAN.MsoHyperlink { COLOR:blue;TEXT-DECORATION:underline;} #yiv1493839580 A:visited { COLOR:purple;TEXT-DECORATION:underline;} #yiv1493839580 SPAN.MsoHyperlinkFollowed { COLOR:purple;TEXT-DECORATION:underline;} #yiv1493839580 P.MsoPlainText { FONT-SIZE:10.5pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:Consolas;} #yiv1493839580 LI.MsoPlainText { FONT-SIZE:10.5pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:Consolas;} #yiv1493839580 DIV.MsoPlainText { FONT-SIZE:10.5pt;MARGIN:0cm 0cm 0pt;FONT-FAMILY:Consolas;} #yiv1493839580 P { FONT-SIZE:12pt;MARGIN-LEFT:0cm;MARGIN-RIGHT:0cm;FONT-FAMILY:"Times New Roman";} #yiv1493839580 SPAN.plaintextchar { FONT-FAMILY:Consolas;} #yiv1493839580 SPAN.plaintextchar0 { FONT-FAMILY:Consolas;} #yiv1493839580 SPAN.plaintextchar00 { FONT-FAMILY:Consolas;} #yiv1493839580 SPAN.plaintextchar000 { FONT-FAMILY:Consolas;} #yiv1493839580 SPAN.plaintextchar0000 { FONT-FAMILY:Consolas;} #yiv1493839580 SPAN.EmailStyle24 { COLOR:windowtext;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle25 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle26 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:Tahoma;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle27 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle28 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle29 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle30 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle31 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle32 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle33 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle34 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle35 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle36 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle37 { COLOR:#1f497d;FONT-FAMILY:Calibri;} #yiv1493839580 SPAN.EmailStyle38 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle39 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle40 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:Tahoma;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle41 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle42 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:Tahoma;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle43 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle44 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle45 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle46 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle47 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle48 { FONT-WEIGHT:normal;COLOR:black;FONT-STYLE:normal;FONT-FAMILY:"Courier New";TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle49 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle50 { COLOR:#1f497d;FONT-FAMILY:Calibri;} #yiv1493839580 SPAN.EmailStyle51 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle52 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle53 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle54 { FONT-WEIGHT:normal;COLOR:#262626;FONT-STYLE:normal;FONT-FAMILY:Verdana;TEXT-DECORATION:none;} #yiv1493839580 SPAN.EmailStyle55 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle56 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle57 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle58 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle59 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle60 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle61 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle62 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle63 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle64 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle65 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle66 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle67 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 SPAN.EmailStyle68 { COLOR:navy;FONT-FAMILY:Arial;} #yiv1493839580 DIV.Section1 { } #yiv1493839580 OL { MARGIN-BOTTOM:0cm;} #yiv1493839580 UL { MARGIN-BOTTOM:0cm;} Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> ,to=To: ""Graeme Ross"" <gross(a)mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858" But they look like this: Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> Anyone seen this? Thanks The new Internet Explorer® 8 - Faster, safer, easier. Optimized for Yahoo! Get it Now for Free! -----Inline Attachment Follows----- ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html __________________________________________________________________ The new Internet Explorer® 8 - Faster, safer, easier. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/

1 0

syslog-ng OSE 2.1.4 released
by Balazs Scheidler 16 Jun '09

16 Jun '09

Hi, I've released syslog-ng OSE 2.1.4 with the following changes: 2.1.4 Tue, 17 Mar 2009 13:35:00 +0100 Bugfixes: * Fixed a possible DoS condition triggered by a destination port unreachable ICMP packet received from a UDP destination. syslog-ng started eating all available memory and CPU until it crashed if this happened. * Fixed the rate at which files regular were read using the file() source. * Report connection breaks as a write error instead of reporting POLLERR as the write error path reports more sensible information in the logs. The release is available at: https://www.balabit.com/network-security/syslog-ng/opensource-logging-syste… -- Bazsi

4 5

Stripping the original hostname /ip from the syslog message
by Shashank Vinchurkar 12 Jun '09

12 Jun '09

Hi, We have a setup where multiple syslog-ng servers send logs to a central syslog-ng server. Finally this central syslog-ng server sends the consolidated logs to an outside server. The outside server can be any server accepting standard syslog messages. The first group of servers are running in the internal network and don't have any hostname associated with them. Also the ip address is internal and does not make sense to outside world. My requirement is that the outside server should only see the ip address of the syslog-ng server which consolidates the messages from these syslog-ng servers. But I always see the ip address of the syslog-ng server which originated the message. Is there anyway to get rid of this? I tried playing with the keep_hostname, long_hostname, chain_hostname and bad_hostname options but I still see the ip address of the originating server. Thanks in advance for the help. -Shashank

3 5

Backslash before quote
by Pennington, Philip 11 Jun '09

11 Jun '09

Syslog-ng version 2. Can anyone please explain the purpose of the backslash being added before each quote in syslog output ? Is there any way to disable this feature ? Is this flags(noparse) ? Is there any other way to do it (other than stripping it out on an output process)? Regards Philip =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===============================================================================

3 4

I/O error occurred while writing; fd=\'6\', error=\'Connection refused (111)\'
by Clayton Dukes 08 Jun '09

08 Jun '09

Hi Folks, I am the owner of php-syslog-ng... While working on one of my servers, I notice the following error getting repeated (a lot): syslog-ng[23692]: I/O error occurred while writing; fd=\'6\', error=\'Connection refused (111)\' I've done multiple searches but haven't been able to nail down why this error is constantly getting logged into my pipe. I've found google results where Baszi asked the user to try using strace and/or lsof, but I can't gleen anything useful from it (maybe I'm just too dumb :-)) Any help would be greatly appreciated :-) Here's my syslog-ng config: options { long_hostnames(off); # doesn't actually help on Solaris, log(3) truncates at 1024 chars log_msg_size(8192); # buffer just a little for performance sync(1); # memory is cheap, buffer messages unable to write (like to loghost) log_fifo_size(16384); # Hosts we don't want syslog from #bad_hostname("^(ctld.|cmd|tmd|last)$"); bad_hostname("^11.16.254.134$"); # The time to wait before a dead connection is reestablished (seconds) time_reopen(10); #Use DNS so that our good names are used, not hostnames use_dns(yes); dns_cache(yes); #Use the whole DNS name use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); #Read permission for everyone perm(0644); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; destination d_syslogdb { pipe("/var/log/mysql.pipe", template ("'$HOST''$FACILITY''$PRIORITY''$LEVEL''$TAG''$YEAR-$MONTH-$DAY''$HOUR:$MIN:$SEC''$PROGRAM''$MSG'\n") template_escape(yes) ); }; log { source(s_all); destination(d_syslogdb); }; -- ______________________________________________________________ Clayton Dukes ______________________________________________________________

4 12

Syslog-3.0.2 build failure on OS X
by Mark Duling 08 Jun '09

08 Jun '09

Hello, Syslog-ng 2.1.4 compiles fine on OS X, but 3.0.2 does not. Below is the error I get. Any idea what the problem might be? Thanks. Mark syslog-ng Open Source Edition 3.0.2 configured Compiler options: compiler : /usr/bin/gcc-4.0 compiler options : -O2 -Wall -I/opt/local/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/eventlog -I/opt/local/include -DHAVE_SOCKADDR_SA_LEN -DLIBNET_BSDISH_OS -DLIBNET_BSD_BYTE_SWAP -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : -L/opt/local/lib prefix : /opt/local linking mode : dynamic Features: Sun STREAMS support : no Sun Door support : no Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : yes SSL support : yes SQL support : no Linux capability support : no PCRE support : no Env wrapper support : no ---> Building syslog-ng ........ rm -f libsyslog-ng.a ar cru libsyslog-ng.a misc.o utils.o messages.o syslog-names.o cfg.o filter.o logmsg.o logstamp.o logpipe.o logsource.o driver.o sgroup.o dgroup.o center.o templates.o logreader.o logwriter.o affile.o afstreams.o afuser.o afprog.o afsocket.o afunix.o afinet.o afinter.o children.o stats.o gsockaddr.o gsocket.o logtransport.o logproto.o memtrace.o dnscache.o serialize.o apphook.o radix.o logpatterns.o cfg-lex.o cfg-grammar.o timeutils.o alarms.o logparser.o logprocess.o logmpx.o logrewrite.o logmatcher.o gprocess.o globals.o control.o compat.o logqueue.o afsql.o tlscontext.o tlstransport.o ranlib: file: libsyslog-ng.a(afstreams.o) has no symbols ranlib: file: libsyslog-ng.a(compat.o) has no symbols ranlib: file: libsyslog-ng.a(afsql.o) has no symbols ranlib libsyslog-ng.a ranlib: file: libsyslog-ng.a(afstreams.o) has no symbols ranlib: file: libsyslog-ng.a(compat.o) has no symbols ranlib: file: libsyslog-ng.a(afsql.o) has no symbols if /usr/bin/gcc-4.0 -DHAVE_CONFIG_H -I. -I. -I.. -I/opt/local/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/eventlog -I/opt/local/include -DHAVE_SOCKADDR_SA_LEN -DLIBNET_BSDISH_OS -DLIBNET_BSD_BYTE_SWAP -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall -MT main.o -MD -MP -MF ".deps/main.Tpo" -c -o main.o main.c; \ then mv -f ".deps/main.Tpo" ".deps/main.Po"; else rm -f ".deps/main.Tpo"; exit 1; fi /usr/bin/gcc-4.0 -O2 -Wall -L/opt/local/lib -o syslog-ng main.o libsyslog-ng.a -lfl -L/opt/local/lib -lglib-2.0 -lintl -liconv -L/opt/local/lib -levtlog -L/opt/local/lib -lssl -lcrypto -lz -lz -lnet -lwrap -lresolv Undefined symbols: "_configuration", referenced from: _configuration$non_lazy_ptr in libsyslog-ng.a(stats.o) _configuration$non_lazy_ptr in libsyslog-ng.a(cfg.o) _configuration$non_lazy_ptr in libsyslog-ng.a(templates.o) _configuration$non_lazy_ptr in libsyslog-ng.a(cfg-lex.o) _configuration$non_lazy_ptr in libsyslog-ng.a(cfg-grammar.o) _configuration$non_lazy_ptr in libsyslog-ng.a(affile.o) _configuration$non_lazy_ptr in libsyslog-ng.a(logmatcher.o) _configuration$non_lazy_ptr in libsyslog-ng.a(logreader.o) ld: symbol(s) not found collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2

2 1

Critical hang while opening tty
by Jean-Baptiste Quenot 07 Jun '09

07 Jun '09

Hi there, I'm using syslog-ng 2.0.9-4.1 on Ubuntu 9.04. In the configuration file, emergency messages are configured to be logged to all ttys. Sometimes syslog-ng blocks the whole system while opening /dev/ttyS0: # pgrep -f syslog-ng # strace -f -p 3172 Process 3172 attached - interrupt to quit open("/dev/ttyS0", O_WRONLY|O_NOCTTY|O_APPEND I'm surprised because I saw the same bug report, and this was supposed to be fixed in Ubuntu's 2.0.9-1 version as stated in the changelog at See http://changelogs.ubuntu.com/changelogs/pool/universe/s/syslog-ng/syslog-ng… The changelog says: syslog-ng (2.0.9-1) unstable; urgency=low * New upstream version. (Closes: #397650, #447105) This corresponds to this bug: syslog-ng hangs writing to /dev/ttyN http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=397650 I notice the bug is fixed in syslog-ng: http://git.balabit.hu/?p=bazsi/syslog-ng-2.0.git;a=commit;h=c5cc92d54d2e33b… Is there maybe another call to tty open() that was not covered by alarm()? The issue happens on several machines, but I don't know what triggers this. Reloading syslog-ng fixes the problem. Thanks in advance, -- Jean-Baptiste Quenot http://jbq.caraldi.com/

2 2

TLS - Mutual authentication issue
by Hahusseau, Thomas 07 Jun '09

07 Jun '09

Hi, I'm using syslog-ng OSE with encrypted message transport thanks to TLS for few week. Now I try to activate the mutual authentication option. I have several issues with the TLS mutual authentication logs error : Jun 4 16:01:31 desktop syslog-ng[26644]: SSL error while reading stream; tls_error='SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca' Jun 4 16:01:31 desktop syslog-ng[26644]: I/O error occurred while reading; fd='14', error='Connection reset by peer (104)' Jun 4 16:01:31 desktop syslog-ng[26644]: Syslog connection closed; fd='14', client='AF_INET(10.254.1.172:43751)', local='AF_INET( <http://0.0.0.0:9999> 0.0.0.0:9999) This is samples of config file: Server : source s_net_tls { tcp(port(9999) tls(key_file("/etc/pfc/credentials/Server/server.key") cert_file("/etc/pfc/credentials/Server/server.pem") ca_dir("/etc/pfc/credentials/CA/") # peer_verify(optional-untrusted) peer_verify(required-trusted) ) ); }; Client : destination d_remote_server_tls { tcp("10.254.1.141" port(9999) tls(ca_dir("/etc/pfc/credentials/CA") key_file("/etc/pfc/credentials/Client/client.key") cert_file("/etc/pfc/credentials/Client/client.pem") peer_verify(required-trusted) # peer_verify(optional-untrusted) )); }; here is how I generated my CA certificate, server and client certificate : openssl genrsa 1024 > CA/ca.key openssl req -new -x509 -days 365 -key CA/ca.key -out CA/ca.cert cat CA/ca.cert CA/ca.key > CA/ca.pem openssl genrsa 1024 > Client/client.key openssl req -new -key Client/client.key -out Client/client.csr openssl x509 -req -days 365 -in Client/client.csr -CA CA/ca.cert -CAkey CA/ca.key -set_serial 01 -out Client/client.cert cat Client/client.cert Client/client.key > Client/client.pem openssl genrsa 1024 > Server/server.key openssl req -new -key Server/server.key -out Server/server.csr openssl x509 -req -days 365 -in Server/server.csr -CA CA/ca.cert -CAkey CA/ca.key -set_serial 01 -out Server/server.cert cat Server/server.cert Server/server.key > Server/server.pem Of course I done the link with Openssl x509 -noout -hash -in ca.pem Ln -s ca.pem XXXX If anyone can help me, or give a step by step procedure that works. I also tried the procedure described in the "Syslog-nd admin guide" it doesn't work too. Regards Thomas ------------------- Thomas Hahusseau Apprenti ingénieur EADS - DS / ENST Bretagne

2 1

udp drops
by Jan Schaumann 06 Jun '09

06 Jun '09

Hello, I have a FreeBSD 6.2 (amd64) host where I'd like to replace the stock syslogd with syslog-ng (3.0.2). This host receives a lot of syslog messages per second from a large number of clients via UDP. The stock syslogd configuration is trivial: *.* /var/log/all This host currently drops about 2-4% of all UDP packets, syslog takes about 50-65% of one CPU. A drop-in replacement configuration using syslog-ng: options { create_dirs(yes); use_dns(no); }; template t_default { template("${DATE} <${FACILITY}.${PRIORITY}> ${HOST} ${MSG}\n"); }; source s_standard { file("/dev/klog"); internal(); udp(); unix-dgram("/var/run/log"); }; destination d_all { file("/var/log/all" template(t_default) ); }; # *.* /var/log/all log { source(s_standard); destination(d_all); }; syslog-ng uses about 90% of one CPU and drops between 15% and 20% of UDP packets (and, based on traffic patterns and logfile size, concurrent logfile rotation etc. even as high as 30%). I've tried a number of things to improve this performance, including: log_fetch_limit(100); log_iw_size(10000); flush_lines(100000); flush_timeout(10); in the global options and log_fifo_size(100000) in the destination definition with flags(flow-control) in the log definition. The best I was able to get with these numbers was 5-7% of UDP drops (ie still double of what the stock syslogd drops). I also tried adjusting "so_rcvbuf" for UDP with no noticable difference. Now consider that I did not do any sysctl tuning, as those should equally influence the stock syslog and I'm trying to sort out why one performs so significantly better than the other. Leaving aside any of the things I can do with syslog-ng further down the road (such as filtering and intentionally dropping certain messages), what can I do to get syslog-ng up to the same performance as the stock syslogd? Many thanks in advance for any pointers and help. -Jan

4 8

New to syslog-ng, basic setup help reuqested
by lance raymond 05 Jun '09

05 Jun '09

Afternoon, looking to do some real basic logging for our webservers. They are all running ubuntu-server 8.10, and I have a standalone box which will act as the 'log server'. They have webmin installed, so I setup using that as it seemed pretty straight forward. So the server is running and when I start one of the webservers which I configured, I recive the following error; * Starting system logging syslog-ng Error binding socket; addr='AF_UNIX(/var/log/apache2/ws8.mydomain.com-access_log)', error='Address already in use (98)' I thought it might be due to apache running, so stopped that, but still the same error. the conf file is HUGE, and 95% is a stock install, so I will just put what the client / server has that is specific to this item I wish to remote log. Client: destination WFLogger { udp(192.168.2.60); }; source domain_access { unix-stream("/var/log/apache2/ws8.domain.com-access_log"); }; log { source(domain_access); destination(WFLogger); flags(catchall); }; On the server, I don't see anything that stands out that say's "accept connections" so not sure if there is anything special that has to be done, but the error on the client is immediate so that part seems to be a local issue. Thanks

2 2

no compatibility between syslog and apache log priority
by gatfi sami 05 Jun '09

05 Jun '09

Hi i a successfully configured a centralised syslog server using syslog-ng 2.09 here is the structure: syslog-client(read apache error log) -------> *send* to a syslog-relay --------->* forword* to a syslog server ----> centralise in a mysql tabe 'logs' CREATE TABLE `syslogtopnet`.`log` ( `seq` int(10) unsigned NOT NULL auto_increment, `host` varchar(60) NOT NULL, `facility` varchar(40) NOT NULL, `*priority*` varchar(40) NOT NULL, `*level*` varchar(40) NOT NULL, `date_log` date NOT NULL, `time_log` time NOT NULL, `program` varchar(40) NOT NULL, `*message*` text NOT NULL, PRIMARY KEY (`seq`), KEY `index_log` (`host`) ) ENGINE=InnoDB AUTO_INCREMENT=684 DEFAULT CHARSET=latin1; my problème is : i find for exemple priority ='*Notice*' and level='*Notic*e' but in the message i find *[warn]* here is an exemple: INSERT INTO `syslogtopnet`.`log` VALUES (320,'192.168.9.195','user',* 'notice'*,*'notice'*,'2009-05-19','19:33:24','','[Tue May 19 19:33:24 2009] *[warn]* Init: Session Cache is not configured [hint: SSLSessionCache]'); i discovered that apache log level are different then syslog's so !!!!! 1 wired isnt it lease is there a solution to keep the same level in the original priority? 2 beside in more than 20589 line in my table priority and level have the same values. is it normal that the level and priority are the same how can this be? thank you

2 1

New db-parser format and other improvements
by ILLES, Marton 05 Jun '09

05 Jun '09

Hi, I have already mentioned earlier that I am planning to back-port the new db-parser patterndb format to syslog-ng OSE. After Bazsi's plan on the 4.0 roadmap I had no more excuse and did the backporting last week. So anyone interested in the new format can find the code in my git tree. Format 2 is more flexible and hopefully easier to use. Meanwhile I removed support for version 1 format. Converting between the formats is very straight forward, but I also plan to add an XSLT for conversions. The new format has support and flexibility for couple of interesting things: * rules for one program can be separated into multiple rulesets * rulesets can be defined without program pattern where these act as a fail-back rule sets if no program pattern is matching or when message does not have program part * one rule can have any number of pattern attached to it (sometimes the same message has multiple format, just think about multilingual messages) * tags can be attached to rules (IMPORTANT: as of now tags are not considered, but at least the format has place for them!) The new patterndb format looks like this: <patterndb version="2" pub_date="2009-06-04"> <ruleset name="testset" id="4182799f-9d1e-48a2-aa1e-aa4bdac6aff8"> <rules> <rule provider="balabit" id="4a3b027a-3c5c-4c55-a28b-ce8052e8a327" class="system"> <tags> <tag>login</tag> </tags> <patterns> <pattern>HERE COMES THE PATTERN</pattern> </patterns> </rule> </rules> </patterndb> I have added new parsers to the db-parser: * IPv6 to parser IPv6 addresses with all the special notation styles * IPvANY to match either IPv4 or IPv6 addresses * NUMBER parser now handles hexadecimal numbers with the 0x or 0X prefix * DOUBLE parser to match and parse non integer I am trying to find some time in the near future to add the missing tag support and provide some more details on the new format. cheers, Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0