April 2013 - syslog-ng - lists.balabit.hu

syslog-ng performance config calculator on Android
by Zoltán Pallagi 26 Apr '13

26 Apr '13

Hi guys, You know I like syslog-ng but also my hobby is Android programming. Because I wanted to practice (and develop my Android knowledge), I've just written a little interesting Android application related to syslog-ng performance tuning. It can calculate the tuning related settings like log_iw_size(), log_fetch_limit(), log_fifo_size() and so on, based on the given source specific parameters. Furthermore it can generate an example configuration that can be saved or sent via email. So it was absolutely just my hobby project related to Android, but after I showed it to a few people interested in syslog, I received positive feedbacks. After I decided to share it with everybody taking into consideration that I already have an Android developer account. I hope it will be useful, at least you can play with the numbers and results, but of course this is not a serious program just as I said my hobby project. You can download it from Android Market (and you can find more details here): https://play.google.com/store/apps/details?id=pzolee.android.syslogngconfig… Of course it's free and no ads. Have fun!

2 1

Re: [syslog-ng] syslog-ng Digest, Vol 96, Issue 23
by Davide D'Amico 26 Apr '13

26 Apr '13

> ------------------------------ > > Message: 3 > Date: Fri, 26 Apr 2013 11:29:39 +0200 > From: Balazs Scheidler <bazsi77(a)gmail.com> > Subject: Re: [syslog-ng] Configuration for a centralized syslog-ng > server > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu>, eh(a)gmail.com > Message-ID: > < > CAKcfE+bvXksZFdz77SkN8VCrmH15wwbTvUk8xJeWFLEwHpay7w(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > How do you know messages are dropped? Using freq_stats :) Anyway, I found the issue that wasn't related to syslog-ng. I had the syslog zfs partition on 92% so, after removing a lot of old files, everything went fine. -- d.

2 1

Configuration for a centralized syslog-ng server
by Davide D'Amico 26 Apr '13

26 Apr '13

Hi, I'm using syslog-ng OSE 3.3.8 on a FreeBSD 9.1 amd64 box on a centralized server (DELL R610, 32GB ram, ZFS with compressed filesystem) and 150 servers with syslog-ng that logs locally and send remotely their logs stream to the centralized syslog-ng. Sometime I see packets dropped, so I think I'm missing something on its configuration. That's what I'm using: - generic server: @version: 3.3 options { perm(0644); dir_perm(0750); create_dirs(yes); group (logs); dir_group (logs); log_fifo_size(10000); use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); stats_freq(3600); }; source s_local { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; destination d_local { file("/var/log/syslog-ng/$YEAR/$MONTH/$DAY/$FACILITY.log"); }; destination d_remote { tcp("10.0.0.9" port(514)); }; log { source(s_local); destination(d_local); destination(d_remote); }; - centralized log server: @version: 3.3 options { perm(0644); dir_perm(0755); create_dirs(yes); group (logs); dir_group (logs); use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); stats_freq(120); log_fifo_size(10000); frac_digits(3); }; source s_local { unix-dgram("/var/run/log" max_connections(20)); unix-dgram("/var/run/logpriv" perm(0600) max_connections(20)); internal(); file("/dev/klog"); }; filter f_localhost { netmask( "127.0.0.1" ); }; filter f_network6 { netmask( "172.16.6.0/255.255.255.0" ); }; destination d_local { file("/tank/syslog/custom/localhost/$YEAR/$MONTH/$DAY/$FACILITY.log"); }; source s_network { udp(ip("10.0.0.9") so_rcvbuf(16777216)); tcp(ip("10.0.0.9") port(514) log_fetch_limit(100) max_connections(200) log_iw_size(20000) so_rcvbuf(16777216)); }; filter f_www_host { host("www1.domain.lan") or host("www2.domain.lan") or host("www3.domain.lan"); }; [...] filter f_www_fac { facility(local6); }; [...] destination d_www { file("/tank/syslog-ng/custom/www/$YEAR/$MONTH/$DAY/$FACILITY.log"); }; [...] log { source(s_network); filter(f_www_host); filter(f_www_fac); destination(d_www); }; Thanks, -- d.

2 1

Re: [syslog-ng] rfc compliance and ciphersuites
by Balazs Scheidler 26 Apr '13

26 Apr '13

Well Syslog-ng can be compiled with SSL support using the openssl library in which case it supports at least TLS 1.1, maybe TLS 1.2 too but I'm not certain. Cipher suites can be configured using cipher-suite option within the TLS configuration block. We support all the algorithms that openssl supports. "Canter, Jeffrey (US SSA)" <jeffrey.canter(a)baesystems.com> wrote: >______________________________________________________________________________ >Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >FAQ: http://www.balabit.com/wiki/syslog-ng-faq >

1 0

rfc compliance and ciphersuites
by Canter, Jeffrey (US SSA) 25 Apr '13

25 Apr '13

Hi We are using syslog-ng version 3.3. Does it conform to rfc 2246, rfc 4346 or rfc 5246? Also, what cipher suites does it support? Thanks.! Jeffrey Canter Member Technical Staff BAE Systems (W) 973-305-2577 (C) 201-320-9403

1 0

syslog-ng 3.3.9 has been released
by devel＠balabit.hu 23 Apr '13

23 Apr '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.9 SUMMARY : new stable release DATE : Apr 15, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.9) has been released. For latest fixes in the 3.3.x feature branch you are recommended to upgrade to this version. CHANGES: 3.3.9 Mon, 15 April 2013 15:00:00 +0100 Highlights ========== This release is a bug-fix release, correcting a handful of issues discovered since the previous one. Bugfixes ======== * A set/subst rewrite related segmentation fault has been fixed: if a rewrite rule doing either set or subst was referenced from multiple logpath, syslog-ng crashed. Other changes ============= * The systemd unit file now has Restart=on-failure set, to restart syslog-ng when it terminates unexpectedly. [#222] * When syslog-ng-ctl disconnects from syslog-ng, syslog-ng will not log an error due to the EOF, but an info-level message instead. Credits ======= syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler <bazsi(a)balabit.hu> Gergely Nagy <algernon(a)balabit.hu> Johnson, Chris <chris.johnson3(a)hp.com> Lucas, Sascha <Sascha.Lucas(a)gisa.de> Paul Dann <pdgiddie+balabit(a)gmail.com> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

4 3

[Bug 231] New: eventlog: small windows compilation problem
by bugzilla＠bugzilla.balabit.com 23 Apr '13

23 Apr '13

https://bugzilla.balabit.com/show_bug.cgi?id=231 Summary: eventlog: small windows compilation problem Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Windows Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ramon.garcia.f+syslog(a)gmail.com Type of the Report: --- Estimated Hours: 0.0 I am compiling eventlog-0.2.12+20120504+1700 under mingw64. I had a small issue compiling, because there is no alloca.h I'm attaching a small fix. malloc.h and alloca.h are checked by autoconf instead of by __WIN32__ Best regards. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[PATCH (3.5)]: Build system reimagined
by Gergely Nagy 22 Apr '13

22 Apr '13

Hi! The patch mentioned in the subject is far too big to include here (58 files changed, 1407 insertions(+), 890 deletions(-)), please see the feature/3.5/build-system-reorg[1] branch in my github repo. [1]: https://github.com/algernon/syslog-ng/commits/feature/3.5/build-system-reorg commit 7b3afe7a17685817e228b80d3da4cb95a35bec77 Author: Gergely Nagy <algernon(a)balabit.hu> Date: Mon Apr 22 11:31:29 2013 +0200 Build system reimagined This is a huge change, one that cannot be sensibly broken up into smaller pieces, so please bear with me! The purpose of the complete rewrite of the build system is to get rid of recursive make calls, and instead, use a single, non-recursive Makefile built up from multiple included parts, that has all the dependency information, therefore providing a much more robust dependency tracking, which is also significantly faster. For the sake of convenience, the Mk/populater-makefiles.sh script will add wrapper Makefiles to every directory that has a Makefile.am (unless it belongs to a third party library), allowing one to restrict 'make all' and 'make check' to a particular directory. The 'make check' provided by this convenience Makefile does not recurse into subdirectories, however. The system also provides shortcuts to build various parts of syslog-ng: - make syslog-ng - make mod-sql mod-mongodb mod-socket - etc Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> As said in the commit message itself, the main driving force behind the change is far better dependency tracking (which also brings us significant compile speed increase). I believe that while this is a huge change, and does have some drawbacks (the need for Mk/populate-makefiles.sh, for example), not having to play nasty tricks with SUBDIRS is already worth it: I've seen that break far too many times in the not too distant past, and the dependency tracking the new system brings gets rid of those problems pretty much forever. Comments, criticism, review and all that is more than welcome. -- |8]

1 0

[Bug 232] New: FEATURE: patterndb to support multiple correlation contexts
by bugzilla＠bugzilla.balabit.com 22 Apr '13

22 Apr '13

https://bugzilla.balabit.com/show_bug.cgi?id=232 Summary: FEATURE: patterndb to support multiple correlation contexts Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 I have come across a correlation requirement that the patterndb can not handle. The general idea is that I need to correlate three lines, where one line has two pieces of correlation information, each of which correlates to on of the other lines. Example: sm-mta[28270]: r3LLxvol028270: from=<someone(a)uvic.ca>, size=9506, class=0, nrcpts=1, msgid=<110bc310e87975113303806e139628b7.squirrel(a)wm3.uvic.ca>, proto=ESMTP, daemon=MTA, relay=[123.69.98.48] sm-mta[10644]: r3LLxvol028270: to=<someone(a)uvic.ca>, delay=00:00:00, pri=91535, stat=Rejected as outbound quota has been exceeded. squirrelmail: Message sent via webmail: by jdc (uvic.ca) at 142.104.193.193 on 04/21/2013 15:01:27: Message-ID: 110bc310e87975113303806e139628b7.squirrel(a)wm3.uvic.ca The first line has the ESMTPID (r3LLxvol028270) and the mail msgid (110bc310e87975113303806e139628b7.squirrel(a)wm3.uvic.ca) The second line only has the ESMTPID (r3LLxvol028270) The third line only has the mail msgid (110bc310e87975113303806e139628b7.squirrel(a)wm3.uvic.ca) So in this case, the only way to get the authenticated user that had the e-mail "Rejected as outbound quota has been exceeded" is to correlate the third line to the second line by using the two correlation components from the first line. I would recommend that the patterdb be changed to permit multiple correlation contexts <rule id="1" context-timeout="2" context-scope="program" provider="UVic"> <contexts> <context>$HOST:$esmtpid</context> <context>$HOST:$msgid</context> </contexts> <patterns> <pattern>@ESTRING:esmtpid::@ from=@QSTRING:from:<>@, size=@NUMBER@, class=@NUMBER@, nrcpts=@NUMBER@, msgid=@QSTRING:msgid:<>@</pattern> </patterns> <actions> <action trigger="timeout"> <message inherit-properties='TRUE'> <value name="MESSAGE">My message here</value> </message> </action> </actions> </rule> <rule id="2" context-timeout="2" context-scope="program" provider="UVic"> <contexts> <context>$HOST:$esmtpid</context> </contexts> <patterns> <pattern>@ESTRING:esmtpid::@ to=@QSTRING:to:<>@, delay=@ESTRING:delay",@ pri=@NUMBER@, stat=Rejected as outbound quota has been exceeded.</pattern> </patterns> <actions> <action trigger="timeout"> <message inherit-properties='TRUE'> <value name="MESSAGE">My message here</value> </message> </action> </actions> </rule> <rule id="3" context-timeout="2" context-scope="program" provider="UVic"> <contexts> <context>$HOST:$msgid</context> </contexts> <patterns> <pattern>Message sent via webmail: by @ESTRING:login: @@QSTRING:domain:()@ at @IPv4@ on @ESTRING:date:: @Message-ID: @ANYSTRING:msgid@</pattern> </patterns> <actions> <action trigger="timeout"> <message inherit-properties='TRUE'> <value name="MESSAGE">My message here</value> </message> </action> </actions> </rule> Obviously the action could do something a little more useful, but I think the point is clear that all three of these would be correlated together. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

tcp on syslog 2.0.x vs 3.3.x
by Hannes De Bondt 19 Apr '13

19 Apr '13

Best, Is it possible there is a different behaviour in the tcp destination. We see a different behaviour. We had a working setup with 2.0.x and while upgrading to 3.3.x we noticed the newlines in our messages are seen as EOM, causing a part of our logging to be lost. Is there a quick workaround for this? Kind regards Hannes De Bondt

3 2