- syslog-ng - lists.balabit.hu

program destination
by Thomas Wollner 20 Jul '11

20 Jul '11

Hello, we are using syslog-ng version 3.1.3 (open source edition) on a debian 6 system. Currently we switched from version 2 to version 3 and we noticed the following change: we are using sec.pl as a program destination to correlate and filter our logs. on version 2 syslog-ng was sending a term signal in case of a stop or restart to programs/scripts invoked via program destination. on version 3 the programs didnt receive the signals and still keep running. in the admin / reference guide I see some options for the program destination, but none for signal handling. Is this the intended behaviour of syslog-ng v3? How can I tell syslog-ng v3 to send signals to programs which are started by syslog-ng? many thanks in advance, Tom ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

4 6

Host sending "localhost" as hostname
by Andreas Heinlein 19 Jul '11

19 Jul '11

Hello, we have a setup with Syslog-Ng 3.2 on both a debian server and Ubuntu clients. For several reasons, we set keep_hostnames(yes) on the log server, which works quite well. Except for some clients (3 or 4) which insist on sending "localhost" as their hostname. They do resolve perfectly via DNS in both directions, "hostname" on these clients outputs the correct hostname. I have no idea where this comes from. Configuration is identical on all machines, so I don't understand why it works on most but not all of them. Any Ideas? Thanks, Andreas

2 1

Rpmbuild syslog-ng-3.2.4-1.el6: FAIL: test_nvtable
by JP Vossen 19 Jul '11

19 Jul '11

I'm trying to get the EPEL6 syslog-ng-3.2.4-1.el6.src.rpm [1] to rpmbuild on CentOS-5 (eventlog-0.2.12-1.el6.src.rpm worked fine). I'm trying to do that because the EPEL5 version is old syslog-ng-2.1.4-9.el5.src.rpm, and of the very many syslog-ng.spec files I've looked at, I like the EPEL6 one the best. The BB tarball spec seems to require building as root (for chown and ./install) which I don't like, and the EPEL5 and Fedora spec files have a bunch of stuff that I believe is cruft from a CentOS-5 perspective. (And yes, I know that the EPEL6 spec is just an evolution of the previous ones. It's still a lot cleaner and was fixed by someone how knows more about them than I do. :) First, I had to comment out "BuildRequires: tcp_wrappers-devel" since that does not exist in CentOS-5. I am assuming/hoping that that's simply because the packages were split in RHEL6. I've tried with the above comment change, commenting out "--enable-tcp-wrapper" and changing that to "--disable-tcp-wrapper" on both 32 and 64 bit and it *almost* works, but I always get: [...] Assertion strcmp(name, dyn_name) == 0 failed at line 69 FAIL: test_nvtable [...] ==================== 1 of 19 tests failed ==================== make[3]: *** [check-TESTS] Error 1 make[3]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests/unit' make[2]: *** [check-am] Error 2 make[2]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests/unit' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests' make: *** [check-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.4153 (%check) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.4153 (%check) I can't find anything on that in Google, and when I eyeball test_nvtable.c I see where the error is coming from but I have no clue what's causing it to fail. I could probably disable that test, but they are there for a reason. I'm building on both 32 and 64 bit: CentOS release 5.5 (Final) Linux hostname 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:40 EST 2010 i686 i686 i386 GNU/Linux Linux hostname 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64 x86_64 x86_64 GNU/Linux Any clues anyone? _____________ [1] 'http://download.fedora.redhat.com/pub/epel/6/SRPMS/syslog-ng-3.2.4-1.el6.sr…' 'http://download.fedora.redhat.com/pub/epel/6/SRPMS/eventlog-0.2.12-1.el6.sr…' Thanks, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

3 16

syslog-ng 3.3.0beta1 has been released
by devel＠balabit.hu 19 Jul '11

19 Jul '11

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.0beta1 SUMMARY : new stable release DATE : May 24, 2011 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.0beta1) has been released. For latest fixes in the 3.3.x branch you are recommended to upgrade to this version. CHANGES: 3.3.0beta1 Fri, 13 May 2011 13:17:12 +0200 Highlights: =========== This release marks the feature freeze for the upcoming 3.3 version of syslog-ng, development continues on the newly opened 3.4 branch, while this one only receives bugfixes in the future. db-parser(): ============ * db-parser() has a new option called inject-mode() that can be used to specify where synthetic messages generated by syslog-ng itself should be injected. In 3.2 these appeared in the internal() source, in 3.3 they'll be generated from within db-parser(). * "pdbtool patternize": added --no-parse option sql(): ====== * Make the number of retries runtime configurable in case an INSERT is rejected by the database using the new retry-sql-inserts() option. * Added support for using the "default" value for columns, by specifying "default" as the column value. This can be used for auto-incrementing fields. * Explicit commits were adapted to use the proper BEGIN TRANSACTION command on MS SQL and Oracle. * Oracle doesn't like overly long index names, and since this limit was easily reached by syslog-ng, use md5 to compress the index name. * Removed the defaults for columns(), values() and indexes() as it's almost impossible to come up with a set of defaults that works on all databases. * DBI initialization errors are reported earlier, during startup, to make their detection easier. file(): ======= * The owner(), group(), perm(), dir_owner(), dir_group() and dir_perm() can be specified without an argument which tells syslog-ng to avoid changing file permissions even if the globals are set. mongodb(): ========== * The name of the collection was changed to be a simple string, instead of a template, to make implementing bulk inserts much easier. * Added support for the value-pairs() option, that makes it trivial to add fields to records dynamically. $(format-json): =============== * A new module named tfjson was created which implements a $(format-json) template function, capable of exporting the syslog-ng message model as a properly marked up JSON object. * The plugin supports both json-c and json-glib as backend implementations. Performance changes: ==================== * The LogQueue component was restructured for better scalability. This component is responsible to connect source and destination drivers as they work in separate threads, and as such it plays an important role in influencing the overall performance of syslog-ng. Loggen: ======= * Added support for reading files in multi-threaded mode (each thread sending a copy of that file, instead of mixing lines). * Added support for IPv6. Other changes: ============== * The list of plugins loaded by default became build-time configurable using the --default-modules configure switch. The same command line option can be used to override the same at runtime. * --version now includes information on the list of available modules, and --module-registry displays even more information on every loadable module. * Introduced another plugin possibility, inner-source and inner-dest plugin types allow the extension any kind of source or destination driver. * Use the newly introduced "cap-syslog" capability with kernels post-2.6.38. Build changes: ============== * A new shared library was introduced named libsyslog-ng-crypto.so that contains all crypto related code shared between plugins. * Added pkg-config files to be used in -dev packages in order to make it possible to build external modules. * "make uninstall" now properly uninstalls config files and SCL. Note that it removes everything without checking that they were changed locally. Bugfixes: ========= * The program destination caused a segmentation fault during startup, this was fixed. * Fixed syslog() style framing over TLS or tcp, which caused the syslog-ng server to shut connections down prematurely. * Fixed syslog() client code, which may cause invalid framing to be emitted on contended network connections. * Fixed a possibly failed assert in the flow control code in reaction to a broken source connection. * Fixed processing flush_timeout() for destinations, in earlier alpha versions flush_timeout() never expired. * Some platforms return EINVAL for writev() calls with zero I/O buffers, causing file output to fail. Make sure syslog-ng never generates such calls. * Fixed read behaviour for special files (like /proc/kmsg on Linux, /dev/klog for FreeBSD), as epoll/kqueue is unable to handle these. * Fixed write behaviour for special files (like /dev/console) as epoll/kqueue must be used even though syslog-ng doesn't use those for regular files. * Fixed a premature connection closure when the destination is unable to accept data (e.g. EAGAIN is returned from send()). This was easily reproduced by enabling SSL, as SSL is reporting the same condition when the negotiation is running and the client is expecting the response from a server. * Fixed a 100% CPU usage case in the SSL server case. * Fixed a possible segmentation fault during runtime (use-after-free problem which could cause segfaults). * Fixed a segmentation fault at exit when the tcp() destination couldn't connect to the server. * Don't attempt to remember the file position for files that are read using follow-freq(0), e.g. /dev/klog, /proc/kmsg and similar files. * Fixed a db-parser() correllation bug, sometimes timers were not expired as they should have. [3.2 port] * Accept three forms of the catch-all log statement flag: catch-all, catchall and catch_all since the documentation was wrong and it differed from the actual implementation for ages. * PCRE: fixed a potential resource hog triggered by PCRE 8.12 and "global" replacements. * loggen: make sure loggen sends out all data by calling shutdown() on the output socket. * loggen: handle SIGPIPE signals which caused loggen to exit prematurely when connections broke. * loggen: fixed a potential division by zero when reporting the message rate for unsuccessful connections. * Fixed a number of compilation warnings triggered by gcc 4.6 Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: * Martin Holste * Balazs Scheidler (BalaBit) * Gergely Nagy (BalaBit) * Andy Ruch * Peter Czanik (BalaBit) * Viktor Juhasz (BalaBit) * Attila Nagy (BalaBit) * Andras Miczki (BalaBit) * György Fischhof (BalaBit) * Tamás Pál (BalaBit) * Bálint Kovács (BalaBit) * Hendrik Visage * Peter Gyöngyösi (BalaBit) * Mishou Michael (IRS) * Matthew Hall * Sándor Gellér (Morgan Stanley) * Micah Anderson * Jose Oliveira (Fedora) * Serge Hallyn (Ubuntu) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

2 1

syslog-ng 3.3.0beta & ESX crashes
by Hendrik Visage 18 Jul '11

18 Jul '11

HI there, I've have a syslog-ng 3.3.0beta1 (at least the tag I get from the git build) that keeps falling over as it receives logs from ESX servers, this is on a Red Hat Enterprise Linux Server release 6.0 (Santiago) inside a VM. The dmesg output gives "interesting" responses: syslog-ng[1371]: segfault at 4e1fe858 ip 000000004e1fe858 sp 00007fff8d7b7958 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2054]: segfault at 4e1fe992 ip 000000004e1fe992 sp 00007fff8d7b7958 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2080] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2196]: segfault at 4e1feb26 ip 000000004e1feb26 sp 00007fff8d7b7a18 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2281]: segfault at 1d2a930 ip 0000000001d2a930 sp 00007fff8d7b7958 error 15 syslog-ng[2303]: segfault at 4e200944 ip 000000004e200944 sp 00007fff8d7b7a18 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2346] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2354] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2366] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2388]: segfault at 1e350c8 ip 0000000001e350c8 sp 00007fff8d7b7958 error 15 syslog-ng[2397] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2418]: segfault at 34312e35ba ip 00007f42ab7e921f sp 00007fff8d7b7900 error 4 in libsyslog-ng.so.0.0.0[7f42ab7ac000+91000] syslog-ng[2426] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2458]: segfault at 1d2cac8 ip 0000000001d2cac8 sp 00007fff8d7b7a38 error 15 syslog-ng[2487]: segfault at bb ip 00000000000000bb sp 00007fff8d7b7a18 error 14 in syslog-ng[400000+2000] [visagehe@tsysl01 ~]$ Not always, but several times I also see these in the internal syslog-ng log output: Jul 15 11:17:56 tsysl01 syslog-ng[2066]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:19:49 10.122.122.14 syslog-ng[2066]: Error processing log message: </memoryAllocation></obj> Jul 15 11:20:08 tsysl01 syslog-ng[2080]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:21:49 10.122.122.14 syslog-ng[2080]: Error processing log message: </memoryAllocation></obj> Jul 15 11:22:25 tsysl01 syslog-ng[2196]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:24:58 10.122.122.14 syslog-ng[2196]: Error processing log message: </memoryAllocation></obj> Jul 15 11:25:28 tsysl01 syslog-ng[2274]: syslog-ng starting up; version='3.3.0beta1' It appears that the ESX servers are sending big streams of console usage information to the syslog-ng (on UDP) which appears to obviously not always fit in standard syslog-ng packet formats :( Any advice on fixing this, or perhaps going back to a previous version of syslog-ng?? Thank you Hendrik

2 5

AUTO: James B Horwath/TheGuardian is out of the office. (returning 07/22/2011)
by James B Horwath 16 Jul '11

16 Jul '11

I am out of the office until 07/22/2011. I am out of the office until Monday, July 25, 2011 with limited access to e-mail. If there are Nitro issues please contact Dante Rodino at 212-919-2819 or Sunil Gupta at 610-807-6710. Any other issues please contact Christopher Clarke manager of Security Operations Thanks, Jim Note: This is an automated response to your message "syslog-ng Digest, Vol 75, Issue 22" sent on 7/16/11 6:00:01. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

Cross-compile problem with path in 3.2.4
by Ulrich Prinz 14 Jul '11

14 Jul '11

Hi! I cross compiled syslog-ng 3.2.4 for my ARM board. I called ./configure with the usual --host, --prefix and --exec-prefix options. Then I ran make with DESTDIR to my cross target directory. I moved everything to my ARM target and syslog-ng starts up. While it locates and loads the configuration files properly, there is a problem with the modules: ~ # syslog-ng Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error parsing source, source plugin system not found in /etc/syslog-ng/syslog-ng.conf at line 10, column 2: Checking the directory shows: /etc2/syslog-ng # ll /lib/syslog-ng/ drwxr-xr-x 2 root root 1024 Jun 14 16:01 ./ drwxr-xr-x 7 root root 3072 Jun 14 16:01 ../ -rw-r--r-- 1 root root 139122 Jun 14 16:01 libaffile.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libaffile.la* -rw-r--r-- 1 root root 110150 Jun 14 16:01 libafprog.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafprog.la* -rw-r--r-- 1 root root 227144 Jun 14 16:01 libafsocket-notls.a -rwxr-xr-x 1 root root 1313 Jun 14 16:01 libafsocket-notls.la* -rw-r--r-- 1 root root 484708 Jun 14 16:01 libafsocket-tls.a -rwxr-xr-x 1 root root 1326 Jun 14 16:01 libafsocket-tls.la* lrwxrwxrwx 1 root root 18 Jun 14 16:03 libafsocket.so -> libafsocket-tls.so@ -rw-r--r-- 1 root root 89662 Jun 14 16:01 libafuser.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafuser.la* -rw-r--r-- 1 root root 26398 Jun 14 16:01 libbasicfuncs.a -rwxr-xr-x 1 root root 1301 Jun 14 16:01 libbasicfuncs.la* -rw-r--r-- 1 root root 15366 Jun 14 16:01 libconfgen.a -rwxr-xr-x 1 root root 1292 Jun 14 16:01 libconfgen.la* -rw-r--r-- 1 root root 18030 Jun 14 16:01 libconvertfuncs.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libconvertfuncs.la* -rw-r--r-- 1 root root 96416 Jun 14 16:01 libcsvparser.a -rwxr-xr-x 1 root root 1298 Jun 14 16:01 libcsvparser.la* -rw-r--r-- 1 root root 265984 Jun 14 16:01 libdbparser.a -rwxr-xr-x 1 root root 1314 Jun 14 16:01 libdbparser.la* -rw-r--r-- 1 root root 78426 Jun 14 16:01 libdummy.a -rwxr-xr-x 1 root root 1286 Jun 14 16:01 libdummy.la* -rw-r--r-- 1 root root 1662352 Jun 14 16:01 libsyslog-ng.a -rwxr-xr-x 1 root root 1265 Jun 14 16:01 libsyslog-ng.la* -rw-r--r-- 1 root root 60868 Jun 14 16:01 libsyslogformat.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libsyslogformat.la* I tried calling syslog-ng --module-path=/lib/syslog-ng too, but I get the same result, Any idea to get around that would be appreciated :) Best Regards and thanks in advance Ulrich

3 6

[PATCH 0/N]: Various fixes from the syslog-ng team
by Gergely Nagy 14 Jul '11

14 Jul '11

Hi! A few selected patches from the syslog-ng team at BalaBit will follow this e-mail. This here mostly serves as an umberalla mail, under which I can shovel the rest, so they end up like a neat little thread. The patches I am about to send are available on the team/4.1-sprint5/fixes branch of my git tree. Special thanks to Viktor Juhasz <jviktor(a)balabit.hu> of the syslog-ng team for doing the bulk of the work, and assembling the list of patches that can be safely sent to the list. Another batch will be hitting the list a bit later, but unlike this first, which I expect to be safely appliable, the second batch will need a more through review. I'll comment on the whys when those patches are posted. I expect more patches from the team to follow, once the features and fixes are in a good enough shape. (In the meantime, I'll try to browse through their tree to see if any other bugfix can be isolated) -- |8]

3 23

Apache vhost logs
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello again, I try all the weekend to do it but there is no luck on it. I want to send all apache vhosts logs to a remote server. I read this guide: http://www.facebook.com/note.php?note_id=338493890632 But, when I create the named pipes, I cannot restart Apache. I try to send the logs files this way: file("/var/log/apache2/access.log"); file("/var/log/apache2/error.log"); But on the server-side there is any access.log and error.log files. I have lots of vhost on the client-side. How could I configure it? Thanks and best regards.

5 14

reconnect DB session after timeout
by "Marcus Groß" 14 Jul '11

14 Jul '11

Hi, I'm running syslog-ng v. 3.2.4 on SLES 10. There are three defined MySQL DB destinations. Everything is running fine, until the MySQL server resets the connection(s) after a defined idle time if no messages are send to this database. The problem is that syslog-ng doesn't reconnect the disconnected connection ;) I have to restart syslog-ng to bring it back to life. Is there any posssibility to reopen the DB connection instead of disabling the idle timeout on the MySQL side? syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' syslog-ng[4904]: Error checking table, disconnecting from database, trying again shortly; time_reopen='60' syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' Jun 9 09:43:56 snnasmg8421 syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' M.

3 2