- syslog-ng - lists.balabit.hu

Using patterndb to rewrite any and all macros
by Evan Rempel 21 Jan '13

21 Jan '13

I am considering using the patterndb to rewrite messages that do not conform to syslog standards. Can the patterndb change all normally defined macros? I need to be able to change MSGHDR MESSAGE PROGRAM PID MSG MSGONLY and I think that is all. I know that macros such as MSGHDR can not be changed by the rewrite command but can it be changed by the patterndb? Thanks for your expertise. -- Evan Rempel Senior Systems Administrator, Data Centre Services University of Victoria 250.721.7691

3 7

time zones - probably revisited.
by Evan Rempel 18 Jan '13

18 Jan '13

We run central log servers for our entire infrastructure.. This includes collecting, storing and archiving logs for hosts and applications that we do *not* administer. On a good day, admins for these other hosts are very cooperative and will change configuration as we instruct them (log line formats, time zone, proper host names etc.) Increasingly vendors are shipping VMware application images. You just boot the image and configure it via a GIU or web front end. There is no real login on the host for fine grained configuration. Even the VMware console is almost to this state. Anyhow, the problem I am running into is that these systems are not logging to our servers with consistent time-zone information. The time-zone info is in the syslog message, but syslog will not convert it by default. Enter the destination driver configuration directive of time-zone("+/-XX:YY") This is almost great :-( This means that I can convert the incoming time stamps to a specific time zone, but not automatically to the time zone that my syslog-ng server runs in. Where I live we observe daylight savings time (-07:00) and right now we are using standard time (-08:00). When the time zone changes (PST to PDT then back) I can not get syslog-ng to write the logs with a time zone that follows the PDT->PST->PDT changes. If there is a way to do this I would appreciate the info. I could't find it in the Guide or FAQ. If there isn't ... yet, does it sound reasonable that the time-zone() option be able to take an argument of "local" to allow converting the time zone information to match that of the host. Thanks.

2 3

syslog-ng 3.4.0rc2 has been released
by devel＠balabit.hu 18 Jan '13

18 Jan '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.0rc2 SUMMARY : new stable release candidate DATE : Jan 17, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new release candidate of syslog-ng Open Source Edition (3.4.0rc2) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. This release candidate is intended to become the new stable release by the end of the month. CHANGES: 3.4.0rc2 Wed, 17 Jan 2013 14:00:00 +0100 Features ======== * The dbparser() parser gained support for the template() option, which allows one to have more control (in an easy, and straightforward way) over what data the parser operates on. Bugfixes ======== * Building the AMQP destination can now be properly disabled. * The bundled ivykis was upgraded from the 0.30 stable branch to 0.36, for platform support and other improvements. * Fix the test suite when --with-embedded-crypto is in use. * The LLADDR parser was fixed, formerly it required the exact number of components to be present, and had parts and count values backwards. [#216] Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler <bazsi(a)balabit.hu> Ben Lentz <blentz(a)cswg.com> Evan Rempel <erempel(a)uvic.ca> Gergely Nagy <algernon(a)balabit.hu> Jose Pedro Oliveira <jpo(a)di.uminho.pt> Peter Czanik <czanik(a)balabit.hu> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

4 3

[Bug 198] New: debug-info fails to extract info from libsyslog-ng-crypto.so
by bugzilla＠bugzilla.balabit.com 18 Jan '13

18 Jan '13

https://bugzilla.balabit.com/show_bug.cgi?id=198 Summary: debug-info fails to extract info from libsyslog-ng- crypto.so Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: svensken(a)jla.rutgers.edu Type of the Report: bug Estimated Hours: 0.0 The syslog-ng package fails to build on Centos6 using mock with version 3.3.6 source code and a syslog-ng-3.2.5-3.el6 spec file. The problem seems to involve the debug info package, provided by rpm-4.8.0-27.el6.x86_64 and rpm-build-4.8.0-27.el6.x86_64 Seemingly, debug info cannot be extracted from libsyslog-ng-crypto.so, whether the .so or the 'debugedit' binary is doing something incorrectly. Here's the relevant part of the build log: --------- ... + /usr/lib/rpm/find-debuginfo.sh --strict-build-id /builddir/build/BUILD/syslog-ng-3.3.6 extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/usr/bin/pdbtool extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/usr/bin/loggen extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/lib64/libsyslog-ng-3.3.6.so extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/lib64/syslog-ng/libcsvparser.so extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/lib64/syslog-ng/libdbparser.so extracting debug info from /builddir/build/BUILDROOT/syslog-ng-3.3.6-2.ru6.x86_64/lib64/syslog-ng/libsyslog-ng-crypto.so error: Bad exit status from /var/tmp/rpm-tmp.BZBLBe (%install) Bad exit status from /var/tmp/rpm-tmp.BZBLBe (%install) RPM build errors: Child returncode was: 1 ... --------- Throwing "%define debug_package %{nil}" into the spec will make the rpm build without errors. So it looks like the script "/usr/lib/rpm/find-debuginfo.sh" is failing to 'extract debug info' from "libsyslog-ng-crypto.so" The lack of an error message is no surprise; here's the function (inside find-debuginfo.sh) that's actually doing the debug info extraction: --------- 202 echo "extracting debug info from $f" 203 id=$(/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \ 204 -i -l "$SOURCEFILE" "$f") || exit 205 if [ -z "$id" ]; then 206 echo >&2 "*** ${strict_error}: No build ID note found in $f" 207 $strict && exit 2 208 fi --------- I tried modifying it, and found that the "exit" function is never even reached, since (as it turns out) /usr/lib/rpm/debugedit returns "Segmentation fault" when run on libsyslog-ng-crypto.so... Running /usr/lib/rpm/debugedit on the libraries that seem to be working (e.g. libcsvparser.so and libdbparser.so) without extra arguments returns no errors, however. I've found that the library itself, libsyslog-ng-crypto.so, seems to have a debug info format different from the other libraries' that succeeded in the build. As I understand it, the following command will display the debug info for a shared library: `readelf -w /path/to/library.so` It seems that the libcsvparser.so and libdbparser.so libraries both use an ".eh_frame section" to store their debug info, while libsyslog-ng-crypto.so seems to put its info into a ".debug_info section" (with hex line numbers of some sort), with an empty ".eh_frame section" section. I noticed that this crypto library was added fairly recently in the change logs, so perhaps it uses a newish format for its debug-info that is not yet supported by the debugedit binary, which is owned by rpm-build-4.8.0-27.el6.x86_64 Could there be a problem on my end, or is this an issue with the library itself? Could it be both? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

syslog-ng Insider - January 2013
by Peter Czanik 17 Jan '13

17 Jan '13

Dear syslog-ng users, This is the 20th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS Video: a short introduction to syslog-ng ---------------------------------------- There is a video series in the work to introduce the basics of syslog-ng. The first video is online now, you can watch it at http://youtu.be/1Ff6tY1SvW8 New videos will be published on the same Youtube channel. You are very welcome to subscribe to receive notifications about further syslog-ng videos and other interesting content. syslog-ng OSE configurator -------------------------- syslog-ng OSE configurator is a web application, which can generate a syslog-ng OSE configuration with only a few mouse clicks. Right now it can only deal with global options, sources, destinations and of course with logpath. There are plans to add filters and templates in the future. For more information and links to sources and the configurator, check the authors blog at http://micek.blogs.balabit.com/en/2013/01/syslog-ng-ose-configurator/ syslog-ng 3.4 RC2 is released ----------------------------- Version 3.4 RC2was released today. Compared to the RC1release, it has some minor bugfixes, mainly related to fixing compile problems of different features and platforms. This also involves an update to the bundled ivykis library from version 0.30 to 0.36. For a summary about what's new in the 3.4 releases, check Bazsi's blog at http://bazsi.blogs.balabit.com/2012/12/syslog-ng-3-4beta1-released/ For a list of changes, check the announcement at https://lists.balabit.hu/pipermail/syslog-ng-announce/2013-January/000153.h… For binary package availability check our 3^rd party binary page at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… At the time of this writing there are RC1 packages for Debian, openSUSE, Ubuntu, FreeBSD ports are updated and there is a Fedora source rpm. These should be updated to RC2 in the coming days. OpenSUSE Factory already features syslog-ng 3.4 RC1 and will be updated to RC2 after openSUSE 12.3beta1 is out. SHORT NEWS: * Right now 3.4 is available only in source form for Fedora. Check how you can compile it at http://czanik.blogs.balabit.com/2013/01/for-the-impatient-syslog-ng-3-4rc1-… * FreeBSD 9.1 is released, news about the new release and syslog-ng are available at http://czanik.blogs.balabit.com/2013/01/freebsd-9-1-and-syslog-ng/ * Development of 3.5 is already started. A rough roadmap is available at http://huboard.com/algernon/syslog-ng/board (GitHub account required) NEW RELEASES: * syslog-ng OSE 3.4 RC2: https://lists.balabit.hu/pipermail/syslog-ng-announce/2013-January/000153.h… * syslog-ng OSE 3.3.8: http://lists.balabit.hu/pipermail/syslog-ng-announce/2013-January/000152.ht… ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

syslog-ng 3.3.8 has been released
by devel＠balabit.hu 17 Jan '13

17 Jan '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.8 SUMMARY : new stable release DATE : Jan 17, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.8) has been released. For latest fixes in the 3.3.x stable branch you are recommended to upgrade to this version. CHANGES: 3.3.8 Thu, 17 Jan 2013 11:00:00 +0100 Highlights ========== This release is a bug-fix release, correcting a handful of issues discovered since the previous one. Bugfixes ======== * The system() source now works correctly accross all supported FreeBSD versions. * The patterndb loader was improved to handle certain semantic errors better: to report the error instead of crashing. * Fixed unsafe use of non-reentrant APIs to resolve IP addresses to names, which caused problems when threaded mode was enabled. [#212] * Support for systems lacking GLOB_NOMAGIC (like AIX) was restored. [#213] * The syslog() destination now forces IPv4, like all the other destinations, thus using it after udp6() would not result in syslog-ng trying to use IPv6 for syslog() aswell. [#61] * The bundled ivykis has been updated, fixing the following issues: * Certain Linux kernel & glibc combinations triggered a scenario that confused ivykis, leading to syslog-ng not starting up at all. Other changes ============= * It is now possible to build syslog-ng with crypto routines directly embedded into the main libsyslog-ng.so library, making libsyslog-ng-crypto.so unnecessary. This is useful in a few different cases, and can be enabled by passing the --with-embedded-crypto to the configure script. Credits ======= syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler <bazsi(a)balabit.hu> Balint Kovacs <blint(a)blint.hu> Ben Lentz <blentz(a)cswg.com> Brian Kroth <bpkroth(a)gmail.com> Daniel Neubacher <daniel.neubacher(a)xing.com> "Dit" Gergely Nagy <algernon(a)balabit.hu> Lennert Buytenhek <buytenh(a)wantstofly.org> Jan Schaumann <jschauma(a)netmeister.org> Peter Czanik <czanik(a)balabit.hu> Viktor Juhasz <jviktor(a)balabit.hu> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[Bug 216] New: 3.4.0rc1 LLADDR parser does not work
by bugzilla＠bugzilla.balabit.com 17 Jan '13

17 Jan '13

https://bugzilla.balabit.com/show_bug.cgi?id=216 Summary: 3.4.0rc1 LLADDR parser does not work Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 IT does not matter if I use @LLADDR@, @LLADDR:mac@, @LLADDR:mac:6@ or @LLADDR::6@ I get the same results On syslog-ng 3.3.7 (with a patch to get the LLADDR parser) I get % /usr/local/bin/pdbtool test --validate ./pattern.xml ./pattern.xml validates Testing message program='dhcpd' message='DHCPDISCOVER from e8:3e:b6:95:b2:38 (BLACKBERRY-27BD) via 142.104.103.253' On syslog-ng 3.4.0rc1 % /usr/local/bin/pdbtool test --validate ~/pattern.xml /home1l/erempel/pattern.xml validates Testing message program='dhcpd' message='DHCPDISCOVER from e8:3e:b6:95:b2:38 (BLACKBERRY-27BD) via 142.104.103.253' Wrong match name='.classifier.rule_id', value='', expected='18c8c7bf-977d-4495-ab73-8692f9bfb0f2' Wrong match name='ip', value='', expected='142.104.103.253' Wrong match name='mac', value='', expected='e8:3e:b6:95:b2:38' Wrong match name='name', value='', expected='BLACKBERRY-27BD' With a minimal pattern.xml <patterndb version="4" pub_date="2009-09-01"> <ruleset name="dhcpd" id="RS-4d2abc9e-ae62-4dff-a87d-501503ed1360"> <pattern>dhcpd</pattern> <rules> <rule id="18c8c7bf-977d-4495-ab73-8692f9bfb0f2" class="dhcpd" provider="UVic"> <patterns> <pattern>DHCPDISCOVER from @LLADDR:mac:6@ @QSTRING:name:()@ via @IPv4:ip@</pattern> </patterns> <tags> <tag>ignore</tag> </tags> <values> <value name="AUTHPROGRAM">dhcpd</value> </values> <examples> <example> <test_message program="dhcpd">DHCPDISCOVER from e8:3e:b6:95:b2:38 (BLACKBERRY-27BD) via 142.104.103.253</test_message> <test_values> <test_value name="ip">142.104.103.253</test_value> <test_value name="mac">e8:3e:b6:95:b2:38</test_value> <test_value name="name">BLACKBERRY-27BD</test_value> </test_values> </example> </examples> </rule> </rules> </ruleset> </patterndb> -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

syslog-ng 3.4.0 parserdb on other than MESSAGE
by Evan Rempel 17 Jan '13

17 Jan '13

OK, from what Balazs writes below I guess it is not possible to use the parserdb functionality on the "$HOST $PROGRAM" contents. Can someone confirm this is the case? If it can be done, a quick pointer please. If it can not be done, but I want to, how can I do it? rewrite { set("$MSG" value("orig.message")); set("$HOST($PROGRAM)" value("MESSAGE")); } parser(pattern_host_program); rewrite { set("$orig.message" value("MESSAGE")); set("" value("orig.message")); } parser(pattern_message); ... all of my filters, log statements etc. Evan. ________________________________________ From: Balazs Scheidler [bazsi77(a)gmail.com] Sent: Saturday, December 22, 2012 6:50 AM To: Syslog-ng users' and developers' mailing list; Evan Rempel Subject: Re: [syslog-ng] syslog-ng 3.4.0beta1 has been released ----- Original message ----- > > I thought that 3.4 was going to include > > - junctions so that results of multiple parsers could be combined it is in. > > - the ability to run a parser against a user defined macro rather than > always parsing the complete MESSAGE marcro. csvparser has a template() option, but dbparser doesn't (though that's trivial to add). what I had in mind (fully n dimensional patterndb) wasn't finished, and I didn't want to wait any longer. > > Did these features fail to make it into 3.4 or is the documentation just > missing these items? > docs is not yet competely updated. contributions would be welcome there too.

4 3

syslog-ng help
by Manimaran kk 17 Jan '13

17 Jan '13

Could some please help me to setup an log collection for the following logging services like telnet, ftp, ssh, smpd using syslog-ng. Is there any blog or article available? Thank you. Regards, Mani

1 0

syslog-ng help
by Manimaran kk 17 Jan '13

17 Jan '13

Hi, Could some please help me to setup and

1 0