- syslog-ng - lists.balabit.hu

program destination
by Thomas Wollner 20 Jul '11

20 Jul '11

Hello, we are using syslog-ng version 3.1.3 (open source edition) on a debian 6 system. Currently we switched from version 2 to version 3 and we noticed the following change: we are using sec.pl as a program destination to correlate and filter our logs. on version 2 syslog-ng was sending a term signal in case of a stop or restart to programs/scripts invoked via program destination. on version 3 the programs didnt receive the signals and still keep running. in the admin / reference guide I see some options for the program destination, but none for signal handling. Is this the intended behaviour of syslog-ng v3? How can I tell syslog-ng v3 to send signals to programs which are started by syslog-ng? many thanks in advance, Tom ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

4 6

Host sending "localhost" as hostname
by Andreas Heinlein 19 Jul '11

19 Jul '11

Hello, we have a setup with Syslog-Ng 3.2 on both a debian server and Ubuntu clients. For several reasons, we set keep_hostnames(yes) on the log server, which works quite well. Except for some clients (3 or 4) which insist on sending "localhost" as their hostname. They do resolve perfectly via DNS in both directions, "hostname" on these clients outputs the correct hostname. I have no idea where this comes from. Configuration is identical on all machines, so I don't understand why it works on most but not all of them. Any Ideas? Thanks, Andreas

2 1

Rpmbuild syslog-ng-3.2.4-1.el6: FAIL: test_nvtable
by JP Vossen 19 Jul '11

19 Jul '11

I'm trying to get the EPEL6 syslog-ng-3.2.4-1.el6.src.rpm [1] to rpmbuild on CentOS-5 (eventlog-0.2.12-1.el6.src.rpm worked fine). I'm trying to do that because the EPEL5 version is old syslog-ng-2.1.4-9.el5.src.rpm, and of the very many syslog-ng.spec files I've looked at, I like the EPEL6 one the best. The BB tarball spec seems to require building as root (for chown and ./install) which I don't like, and the EPEL5 and Fedora spec files have a bunch of stuff that I believe is cruft from a CentOS-5 perspective. (And yes, I know that the EPEL6 spec is just an evolution of the previous ones. It's still a lot cleaner and was fixed by someone how knows more about them than I do. :) First, I had to comment out "BuildRequires: tcp_wrappers-devel" since that does not exist in CentOS-5. I am assuming/hoping that that's simply because the packages were split in RHEL6. I've tried with the above comment change, commenting out "--enable-tcp-wrapper" and changing that to "--disable-tcp-wrapper" on both 32 and 64 bit and it *almost* works, but I always get: [...] Assertion strcmp(name, dyn_name) == 0 failed at line 69 FAIL: test_nvtable [...] ==================== 1 of 19 tests failed ==================== make[3]: *** [check-TESTS] Error 1 make[3]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests/unit' make[2]: *** [check-am] Error 2 make[2]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests/unit' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/home/rpmbuild/BUILD/syslog-ng-3.2.4/tests' make: *** [check-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.4153 (%check) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.4153 (%check) I can't find anything on that in Google, and when I eyeball test_nvtable.c I see where the error is coming from but I have no clue what's causing it to fail. I could probably disable that test, but they are there for a reason. I'm building on both 32 and 64 bit: CentOS release 5.5 (Final) Linux hostname 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:40 EST 2010 i686 i686 i386 GNU/Linux Linux hostname 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64 x86_64 x86_64 GNU/Linux Any clues anyone? _____________ [1] 'http://download.fedora.redhat.com/pub/epel/6/SRPMS/syslog-ng-3.2.4-1.el6.sr…' 'http://download.fedora.redhat.com/pub/epel/6/SRPMS/eventlog-0.2.12-1.el6.sr…' Thanks, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

3 16

syslog-ng 3.3.0beta1 has been released
by devel＠balabit.hu 19 Jul '11

19 Jul '11

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.0beta1 SUMMARY : new stable release DATE : May 24, 2011 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.0beta1) has been released. For latest fixes in the 3.3.x branch you are recommended to upgrade to this version. CHANGES: 3.3.0beta1 Fri, 13 May 2011 13:17:12 +0200 Highlights: =========== This release marks the feature freeze for the upcoming 3.3 version of syslog-ng, development continues on the newly opened 3.4 branch, while this one only receives bugfixes in the future. db-parser(): ============ * db-parser() has a new option called inject-mode() that can be used to specify where synthetic messages generated by syslog-ng itself should be injected. In 3.2 these appeared in the internal() source, in 3.3 they'll be generated from within db-parser(). * "pdbtool patternize": added --no-parse option sql(): ====== * Make the number of retries runtime configurable in case an INSERT is rejected by the database using the new retry-sql-inserts() option. * Added support for using the "default" value for columns, by specifying "default" as the column value. This can be used for auto-incrementing fields. * Explicit commits were adapted to use the proper BEGIN TRANSACTION command on MS SQL and Oracle. * Oracle doesn't like overly long index names, and since this limit was easily reached by syslog-ng, use md5 to compress the index name. * Removed the defaults for columns(), values() and indexes() as it's almost impossible to come up with a set of defaults that works on all databases. * DBI initialization errors are reported earlier, during startup, to make their detection easier. file(): ======= * The owner(), group(), perm(), dir_owner(), dir_group() and dir_perm() can be specified without an argument which tells syslog-ng to avoid changing file permissions even if the globals are set. mongodb(): ========== * The name of the collection was changed to be a simple string, instead of a template, to make implementing bulk inserts much easier. * Added support for the value-pairs() option, that makes it trivial to add fields to records dynamically. $(format-json): =============== * A new module named tfjson was created which implements a $(format-json) template function, capable of exporting the syslog-ng message model as a properly marked up JSON object. * The plugin supports both json-c and json-glib as backend implementations. Performance changes: ==================== * The LogQueue component was restructured for better scalability. This component is responsible to connect source and destination drivers as they work in separate threads, and as such it plays an important role in influencing the overall performance of syslog-ng. Loggen: ======= * Added support for reading files in multi-threaded mode (each thread sending a copy of that file, instead of mixing lines). * Added support for IPv6. Other changes: ============== * The list of plugins loaded by default became build-time configurable using the --default-modules configure switch. The same command line option can be used to override the same at runtime. * --version now includes information on the list of available modules, and --module-registry displays even more information on every loadable module. * Introduced another plugin possibility, inner-source and inner-dest plugin types allow the extension any kind of source or destination driver. * Use the newly introduced "cap-syslog" capability with kernels post-2.6.38. Build changes: ============== * A new shared library was introduced named libsyslog-ng-crypto.so that contains all crypto related code shared between plugins. * Added pkg-config files to be used in -dev packages in order to make it possible to build external modules. * "make uninstall" now properly uninstalls config files and SCL. Note that it removes everything without checking that they were changed locally. Bugfixes: ========= * The program destination caused a segmentation fault during startup, this was fixed. * Fixed syslog() style framing over TLS or tcp, which caused the syslog-ng server to shut connections down prematurely. * Fixed syslog() client code, which may cause invalid framing to be emitted on contended network connections. * Fixed a possibly failed assert in the flow control code in reaction to a broken source connection. * Fixed processing flush_timeout() for destinations, in earlier alpha versions flush_timeout() never expired. * Some platforms return EINVAL for writev() calls with zero I/O buffers, causing file output to fail. Make sure syslog-ng never generates such calls. * Fixed read behaviour for special files (like /proc/kmsg on Linux, /dev/klog for FreeBSD), as epoll/kqueue is unable to handle these. * Fixed write behaviour for special files (like /dev/console) as epoll/kqueue must be used even though syslog-ng doesn't use those for regular files. * Fixed a premature connection closure when the destination is unable to accept data (e.g. EAGAIN is returned from send()). This was easily reproduced by enabling SSL, as SSL is reporting the same condition when the negotiation is running and the client is expecting the response from a server. * Fixed a 100% CPU usage case in the SSL server case. * Fixed a possible segmentation fault during runtime (use-after-free problem which could cause segfaults). * Fixed a segmentation fault at exit when the tcp() destination couldn't connect to the server. * Don't attempt to remember the file position for files that are read using follow-freq(0), e.g. /dev/klog, /proc/kmsg and similar files. * Fixed a db-parser() correllation bug, sometimes timers were not expired as they should have. [3.2 port] * Accept three forms of the catch-all log statement flag: catch-all, catchall and catch_all since the documentation was wrong and it differed from the actual implementation for ages. * PCRE: fixed a potential resource hog triggered by PCRE 8.12 and "global" replacements. * loggen: make sure loggen sends out all data by calling shutdown() on the output socket. * loggen: handle SIGPIPE signals which caused loggen to exit prematurely when connections broke. * loggen: fixed a potential division by zero when reporting the message rate for unsuccessful connections. * Fixed a number of compilation warnings triggered by gcc 4.6 Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: * Martin Holste * Balazs Scheidler (BalaBit) * Gergely Nagy (BalaBit) * Andy Ruch * Peter Czanik (BalaBit) * Viktor Juhasz (BalaBit) * Attila Nagy (BalaBit) * Andras Miczki (BalaBit) * György Fischhof (BalaBit) * Tamás Pál (BalaBit) * Bálint Kovács (BalaBit) * Hendrik Visage * Peter Gyöngyösi (BalaBit) * Mishou Michael (IRS) * Matthew Hall * Sándor Gellér (Morgan Stanley) * Micah Anderson * Jose Oliveira (Fedora) * Serge Hallyn (Ubuntu) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

2 1

syslog-ng 3.3.0beta & ESX crashes
by Hendrik Visage 18 Jul '11

18 Jul '11

HI there, I've have a syslog-ng 3.3.0beta1 (at least the tag I get from the git build) that keeps falling over as it receives logs from ESX servers, this is on a Red Hat Enterprise Linux Server release 6.0 (Santiago) inside a VM. The dmesg output gives "interesting" responses: syslog-ng[1371]: segfault at 4e1fe858 ip 000000004e1fe858 sp 00007fff8d7b7958 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2054]: segfault at 4e1fe992 ip 000000004e1fe992 sp 00007fff8d7b7958 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2080] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2196]: segfault at 4e1feb26 ip 000000004e1feb26 sp 00007fff8d7b7a18 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2281]: segfault at 1d2a930 ip 0000000001d2a930 sp 00007fff8d7b7958 error 15 syslog-ng[2303]: segfault at 4e200944 ip 000000004e200944 sp 00007fff8d7b7a18 error 14 in libselinux.so.1[3841600000+1d000] syslog-ng[2346] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2354] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2366] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2388]: segfault at 1e350c8 ip 0000000001e350c8 sp 00007fff8d7b7958 error 15 syslog-ng[2397] general protection ip:7f42ab38c911 sp:7fff8d7b7a20 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2418]: segfault at 34312e35ba ip 00007f42ab7e921f sp 00007fff8d7b7900 error 4 in libsyslog-ng.so.0.0.0[7f42ab7ac000+91000] syslog-ng[2426] general protection ip:7f42ab38c911 sp:7fff8d7b7960 error:0 in libaffile.so[7f42ab388000+10000] syslog-ng[2458]: segfault at 1d2cac8 ip 0000000001d2cac8 sp 00007fff8d7b7a38 error 15 syslog-ng[2487]: segfault at bb ip 00000000000000bb sp 00007fff8d7b7a18 error 14 in syslog-ng[400000+2000] [visagehe@tsysl01 ~]$ Not always, but several times I also see these in the internal syslog-ng log output: Jul 15 11:17:56 tsysl01 syslog-ng[2066]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:19:49 10.122.122.14 syslog-ng[2066]: Error processing log message: </memoryAllocation></obj> Jul 15 11:20:08 tsysl01 syslog-ng[2080]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:21:49 10.122.122.14 syslog-ng[2080]: Error processing log message: </memoryAllocation></obj> Jul 15 11:22:25 tsysl01 syslog-ng[2196]: syslog-ng starting up; version='3.3.0beta1' Jul 15 11:24:58 10.122.122.14 syslog-ng[2196]: Error processing log message: </memoryAllocation></obj> Jul 15 11:25:28 tsysl01 syslog-ng[2274]: syslog-ng starting up; version='3.3.0beta1' It appears that the ESX servers are sending big streams of console usage information to the syslog-ng (on UDP) which appears to obviously not always fit in standard syslog-ng packet formats :( Any advice on fixing this, or perhaps going back to a previous version of syslog-ng?? Thank you Hendrik

2 5

AUTO: James B Horwath/TheGuardian is out of the office. (returning 07/22/2011)
by James B Horwath 16 Jul '11

16 Jul '11

I am out of the office until 07/22/2011. I am out of the office until Monday, July 25, 2011 with limited access to e-mail. If there are Nitro issues please contact Dante Rodino at 212-919-2819 or Sunil Gupta at 610-807-6710. Any other issues please contact Christopher Clarke manager of Security Operations Thanks, Jim Note: This is an automated response to your message "syslog-ng Digest, Vol 75, Issue 22" sent on 7/16/11 6:00:01. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

Cross-compile problem with path in 3.2.4
by Ulrich Prinz 14 Jul '11

14 Jul '11

Hi! I cross compiled syslog-ng 3.2.4 for my ARM board. I called ./configure with the usual --host, --prefix and --exec-prefix options. Then I ran make with DESTDIR to my cross target directory. I moved everything to my ARM target and syslog-ng starts up. While it locates and loads the configuration files properly, there is a problem with the modules: ~ # syslog-ng Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error parsing source, source plugin system not found in /etc/syslog-ng/syslog-ng.conf at line 10, column 2: Checking the directory shows: /etc2/syslog-ng # ll /lib/syslog-ng/ drwxr-xr-x 2 root root 1024 Jun 14 16:01 ./ drwxr-xr-x 7 root root 3072 Jun 14 16:01 ../ -rw-r--r-- 1 root root 139122 Jun 14 16:01 libaffile.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libaffile.la* -rw-r--r-- 1 root root 110150 Jun 14 16:01 libafprog.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafprog.la* -rw-r--r-- 1 root root 227144 Jun 14 16:01 libafsocket-notls.a -rwxr-xr-x 1 root root 1313 Jun 14 16:01 libafsocket-notls.la* -rw-r--r-- 1 root root 484708 Jun 14 16:01 libafsocket-tls.a -rwxr-xr-x 1 root root 1326 Jun 14 16:01 libafsocket-tls.la* lrwxrwxrwx 1 root root 18 Jun 14 16:03 libafsocket.so -> libafsocket-tls.so@ -rw-r--r-- 1 root root 89662 Jun 14 16:01 libafuser.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafuser.la* -rw-r--r-- 1 root root 26398 Jun 14 16:01 libbasicfuncs.a -rwxr-xr-x 1 root root 1301 Jun 14 16:01 libbasicfuncs.la* -rw-r--r-- 1 root root 15366 Jun 14 16:01 libconfgen.a -rwxr-xr-x 1 root root 1292 Jun 14 16:01 libconfgen.la* -rw-r--r-- 1 root root 18030 Jun 14 16:01 libconvertfuncs.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libconvertfuncs.la* -rw-r--r-- 1 root root 96416 Jun 14 16:01 libcsvparser.a -rwxr-xr-x 1 root root 1298 Jun 14 16:01 libcsvparser.la* -rw-r--r-- 1 root root 265984 Jun 14 16:01 libdbparser.a -rwxr-xr-x 1 root root 1314 Jun 14 16:01 libdbparser.la* -rw-r--r-- 1 root root 78426 Jun 14 16:01 libdummy.a -rwxr-xr-x 1 root root 1286 Jun 14 16:01 libdummy.la* -rw-r--r-- 1 root root 1662352 Jun 14 16:01 libsyslog-ng.a -rwxr-xr-x 1 root root 1265 Jun 14 16:01 libsyslog-ng.la* -rw-r--r-- 1 root root 60868 Jun 14 16:01 libsyslogformat.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libsyslogformat.la* I tried calling syslog-ng --module-path=/lib/syslog-ng too, but I get the same result, Any idea to get around that would be appreciated :) Best Regards and thanks in advance Ulrich

3 6

[PATCH 0/N]: Various fixes from the syslog-ng team
by Gergely Nagy 14 Jul '11

14 Jul '11

Hi! A few selected patches from the syslog-ng team at BalaBit will follow this e-mail. This here mostly serves as an umberalla mail, under which I can shovel the rest, so they end up like a neat little thread. The patches I am about to send are available on the team/4.1-sprint5/fixes branch of my git tree. Special thanks to Viktor Juhasz <jviktor(a)balabit.hu> of the syslog-ng team for doing the bulk of the work, and assembling the list of patches that can be safely sent to the list. Another batch will be hitting the list a bit later, but unlike this first, which I expect to be safely appliable, the second batch will need a more through review. I'll comment on the whys when those patches are posted. I expect more patches from the team to follow, once the features and fixes are in a good enough shape. (In the meantime, I'll try to browse through their tree to see if any other bugfix can be isolated) -- |8]

3 23

Apache vhost logs
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello again, I try all the weekend to do it but there is no luck on it. I want to send all apache vhosts logs to a remote server. I read this guide: http://www.facebook.com/note.php?note_id=338493890632 But, when I create the named pipes, I cannot restart Apache. I try to send the logs files this way: file("/var/log/apache2/access.log"); file("/var/log/apache2/error.log"); But on the server-side there is any access.log and error.log files. I have lots of vhost on the client-side. How could I configure it? Thanks and best regards.

5 14

reconnect DB session after timeout
by "Marcus Groß" 14 Jul '11

14 Jul '11

Hi, I'm running syslog-ng v. 3.2.4 on SLES 10. There are three defined MySQL DB destinations. Everything is running fine, until the MySQL server resets the connection(s) after a defined idle time if no messages are send to this database. The problem is that syslog-ng doesn't reconnect the disconnected connection ;) I have to restart syslog-ng to bring it back to life. Is there any posssibility to reopen the DB connection instead of disabling the idle timeout on the MySQL side? syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' syslog-ng[4904]: Error checking table, disconnecting from database, trying again shortly; time_reopen='60' syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' Jun 9 09:43:56 snnasmg8421 syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' M.

3 2

[PATCH] Fix: maximum thread number can be 0
by Juhasz Viktor 14 Jul '11

14 Jul '11

[threading] It is possible that the _SC_NPROCESSORS_ONLN isn't defined. In this case the max threads must greater than 0 Signed-off-by: Viktor Juhasz <jviktor(a)balabit.hu> diff --git a/lib/mainloop.c b/lib/mainloop.c index e659fff..b23d055 100644 --- a/lib/mainloop.c +++ b/lib/mainloop.c @@ -702,6 +702,8 @@ main_loop_add_options(GOptionContext *ctx) { #ifdef _SC_NPROCESSORS_ONLN main_loop_io_workers.max_threads = MIN(MAX(2, sysconf(_SC_NPROCESSORS_ONLN)), 64); +#else + main_loop_io_workers.max_threads = 2; #endif g_option_context_add_main_entries(ctx, main_loop_options, NULL);

2 1

check clients
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I want to know if there is a option to know which log-clients are sending data to syslog-ng server. Is it possible to get a list of clients? Thanks and best regards.

4 4

change rsyslog for syslog-ng
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I am learning how to get a syslog server with syslog-ng. I started working on a IT department and I have more than 50 servers, we actually has rsyslog on the client side and syslog-ng on the syslog server. I want to change the client side rsyslog to configure it with syslog-ng. On the server side I read lot of documentation and I know how to configure it (source, destination and filter). I want to get this logs on the client side and send to the syslog server on a TCP+SSL: /var/log/auth.log /var/log/daemon.log /var/log/dmesg /var/log/messages /var/log/syslog I want to get on the server side something like this: /var/log/extern/host1/auth.log /var/log/extern/host1/ daemon.log /var/log/extern/host1/ dmesg /var/log/extern/host1/ messages /var/log/extern/host1/ syslog /var/log/extern/host2/auth.log /var/log/extern/host2/ daemon.log /var/log/extern/host2/ dmesg /var/log/extern/host2/ messages /var/log/extern/host2/ syslog ... /var/log/extern/hostn/auth.log /var/log/extern/hostn/ daemon.log /var/log/extern/hostn/ dmesg /var/log/extern/hostn/ messages /var/log/extern/hostn/ syslog On the past post you help me and I have how to get It on the server side for the TCP+SSL: source s_tcptls { tcp (ip("10.200.42.1") port(10514) tls( peer-verify(require-trusted) ca_dir("/etc/syslog-ng/certs/") key_file("/etc/syslog-ng/certs/server.key") cert_file("/etc/syslog-ng/certs/server.crt") ) ); }; I have some questions: 1. Must I uninstall rsyslog before install syslog-ng? 2. How can I configure the client side to send my logs to the server and to save on the local /var/log/? 3. Is there any client limit on syslog-ng? I will try to read and learn as much as I can, I will appreciate your help. Thanks for your job and best regards.

4 4

Macros expansion in pattern-db actions
by Denis Gasparin 14 Jul '11

14 Jul '11

Hi to all. We're using syslog-ng 3.2.4 and we're having a weird behaviour in using macros with actions values. Here it is a sample pattern-db rule: <?xml version='1.0' encoding='UTF-8'?> <patterndb version='4' pub_date='2011-06-20'> <ruleset name='cron' id='cron-ruleset'> <pattern>/usr/sbin/cron</pattern> <rules> <rule provider="patterndb" id="cron-1" class="system" context-id="sample-context-id"> <patterns> <pattern>(@ESTRING:usracct.username:) @CMD (@ESTRING:details:)@</pattern> </patterns> <actions> <action trigger="match" condition="match('mymatch' value('details'))"> <message> <values> <value name="MESSAGE">[${details}] was found in a cron log message. Rule number [${.classifier.rule_id}]</value> <value name="TRIGGER">yes</value> </values> </message> </action> </actions> </rule> </rules> </ruleset> </patterndb> We tested the rule using pdtool match command and the output was: # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[] was found in a cron log message. Rule number [] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes We instead expected the following output from pdtool match execution : # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[mymatch] was found in a cron log message. Rule number [cron-1 ] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes Macro expansion was not executed in action values but it was in action definition... What are we missing? Thank you in advance for your help Denis Gasparin --- Edistar SRL

2 1

Trying to let my harddrive spin down once in a while
by Lars Stokholm 12 Jul '11

12 Jul '11

Hello On my laptop I run Arch Linux with syslog-ng and cronie (cron) among other programs of course. I'm trying to get my harddrive to spin down automatically once in a while (using hdparm). Me and a lot of other people both in the Arch and Ubuntu community are having our attempts to do this ruined, by what first seemed to be the EXT4 journalling block device (jbd2). After long discussions and finally a bug report (see at the bottom of this mail), I think I've boiled my troubles down to be mostly about syslog-ng (and possibly cronie - but I don't think so). No matter what I try I can't seem to get syslog-ng to stop "waking up my disks" (at least that's what kernel tracing tells me). Killing the crond process almost totally stops these syslog-ng writes to the disk. Unfortunately I've been unable to configure my way out of it using syslog-ng.conf. So my harddrive won't stay spinned down - not even for a minute. This is what tracing the journalling gives me (see the kernel bug report below for an explanaition): http://pastebin.com/raw.php?i=J6BtGVrz As you can probably see, syslog-ng writes every minute at least. I guess that's because my user crontab has a command that gets run every minute. What I don't get is why syslog-ng writes to the disk, even though I'm telling it not to log crond's output. Here is my syslog-ng.conf - as you can see I tried to output the crond messages to /dev/null. I guess it works, because there are no mentions of crond anywhere in /var/log anymore. http://pastebin.com/raw.php?i=H02KKWPV Here are the discussions and the bug report I mentioned: https://bbs.archlinux.org/viewtopic.php?id=113516 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/607560 https://bugzilla.kernel.org/show_bug.cgi?id=39072 I don't know if syslog-ng is causing trouble for only me, but it seems unlikely. Regards, Lars

3 13

Re: [syslog-ng] Syslog-ng log viewer/analyser
by Terry Slattery 08 Jul '11

08 Jul '11

I don't recall where I found it. It may have been at CiscoLive or from something on the Cisco web site, such as an article you wrote. I thought it was a very interesting statistic. -tcs On 7/8/11 2:17 PM, syslog-ng-request(a)lists.balabit.hu wrote: > Date: Fri, 8 Jul 2011 14:16:19 -0400 > From: Clayton Dukes<cdukes(a)gmail.com> > Subject: Re: [syslog-ng] Syslog-ng log viewer/analyser > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <CAANP6Qd=0xnGJ6BkvuyLLTsz7yCQKJMXviwq6=PNV17DnJs8aQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi Terry, > Checked out your blog. > I was wondering where you got the quote about 90 traps and 6000 syslog > messages? > I ask because it's my quote, heh > Just think it's neat how things travel:-) > -- Terry Slattery CCIE# 1026

2 1

Re: [syslog-ng] Syslog-ng log viewer/analyser
by Terry Slattery 08 Jul '11

08 Jul '11

I have a couple of blogs that might be of interest. The first references a couple of other blogs about handling events. The second is about a summary script that I've found useful for summarizing syslog each day. http://www.netcordia.com/community/blogs/terrys_blog/archive/2010/06/06/han… http://www.netcordia.com/community/blogs/terrys_blog/archive/2009/10/28/sys… -tcs -- Terry Slattery CCIE# 1026 On Jul 8, 2011, at 6:00 AM, syslog-ng-request(a)lists.balabit.hu wrote: > > Date: Thu, 07 Jul 2011 15:08:02 +0200 > From: Peter Czanik <czanik(a)balabit.hu> > Subject: Re: [syslog-ng] Syslog-ng log viewer/analyser > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <4E15AFB2.1070505(a)balabit.hu> > Content-Type: text/plain; charset=ISO-8859-1 > > On 07/07/2011 01:49 PM, Pramod Pillai wrote: >> Hi >> >> Is there any Syslog-ng log viewer/analyser which I can use along with >> syslog-ng . > There are many. See my blog at > https://czanik.blogs.balabit.com/2011/06/a-comparison-of-syslog-ng-web-guis/ > Bye, > > -- > Peter Czanik (CzP) <czanik(a)balabit.hu> > BalaBit IT Security / syslog-ng upstream > http://czanik.blogs.balabit.com/

2 1

change from rsyslog to syslog-ng
by Josu Lazkano 08 Jul '11

08 Jul '11

Hello everybody! I finished to configure my syslog system. I am using rsyslog on clients side and syslog-ng on the server side. It works well, but I have a little problem with Apache vhosts logs. I want to send all vhosts logs to my syslog-ng server, but I don't know how to do with rsyslog. Is possible to do with syslog-ng? My Apache logs are configured this way: ErrorLog /var/www/domain1/log/error.log LogLevel warn SetEnvIf Remote_Addr "x\.x\.x\.x" dontlog SetEnvIf Remote_Addr "y\.y\.y\.y" dontlog CustomLog /var/www/domain1/log/access.log common env=!dontlog On the same machine I have lots of vhost logs: /var/www/domain1/log/error.log /var/www/domain1/log/access.log /var/www/domain2/log/error.log /var/www/domain2/log/access.log /var/www/domain3/log/error.log /var/www/domain3/log/access.log ... Is preferable to change all rsyslog to syslog-ng? Thanks for your great help. Best regards.

2 1

Syslog-ng log viewer/analyser
by Pramod Pillai 07 Jul '11

07 Jul '11

Hi Is there any Syslog-ng log viewer/analyser which I can use along with syslog-ng . Regards Pramod

5 4

syslog-ng help: how can I use function in double quotes?
by Henry Xu 05 Jul '11

05 Jul '11

Hi All, How can I use a function in double quote? I want to record the IP address of the syslog-ng server. Since there's no available macro that represent the IP of syslog-ng server(only have macros indicate where the message sent to syslog-ng server,like $HOST). So I want to use ip() function and record it to database. The configuration part is: columns( "datetime varchar2(24)", "server_ip varchar2(24)", "host varchar2(32)", ...) values( "${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "ip()", "$HOST", ...) When I started syslog-ng,it gave error message that said expecting double quotes. How can I use a function in double quotes? Or are there any other ways to get the syslog-ng server IP address and record them? Thanks, Henry

3 3

Re: [syslog-ng] Rpmbuild syslog-ng-3.2.4-1.el6: FAIL: test_nvtable - glib2 hash table
by JP Vossen 05 Jul '11

05 Jul '11

Any clues? -------- Original Message -------- Subject: Re: [syslog-ng] Rpmbuild syslog-ng-3.2.4-1.el6: FAIL: test_nvtable - glib2 hash table Date: Sat, 25 Jun 2011 04:49:38 -0400 From: JP Vossen To: syslog-ng@... On 06/24/2011 07:19 AM, Balazs Scheidler wrote: > On Mon, 2011-06-20 at 20:59 +0100, Jose Pedro Oliveira wrote: >> >> There is a problem with the hash table implementation of glib2 >> version 2.12.3-4 (version that ships in RHEL 5.x). >> More details in: >> >> * https://bugzilla.redhat.com/show_bug.cgi?id=714409#c7 > > Thanks for the information, if it's a glib bug then, can I help you in > any way? Or I can "close" the issue? I see that Jose already forwarded the bug, but I wonder; can you tell me what impact the failed test (and thus failed or potentially failed code block) may have on syslog-ng run-time on CentOS-5/RHEL5? Is this hash problem going to cause critical failures? Under what circumstances? Or is it, well, it'd be nice if that hash problem didn't happen, but it's not a big deal... Thanks, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

1 0

Save ALL logs
by Josu Lazkano 05 Jul '11

05 Jul '11

Hello again, I just installed my new log server and I am configuring to save ALL logs on the log server. On the client side I have rsyslog configured this way: $ cat /etc/rsyslog.conf $ModLoad imuxsock $ModLoad imklog $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $RepeatedMsgReduction on $KLogPath /proc/kmsg $WorkDirectory /rsyslog/work $ActionQueueType LinkedList $ActionQueueFileName srvrfwd1 $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@logserver On the server side I have syslog-ng: source logs_externs{ tcp(); }; destination rdf_all { file("/var/log/extern/${HOST}/${YEAR}/${MONTH}/"); }; log { source(logs_externs); destination(rdf_all); }; I want to get all the logs on the log servers and then archive each month. This way it doesn't work, how I must configure it? Thanks for all your help. Kind regards.

2 1

[Bug 127] New: 3.2.4 ose segfaults during startup
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=127 Summary: 3.2.4 ose segfaults during startup Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: balazs.nemeth(a)balabit.com Type of the Report: bug Estimated Hours: 0.0 This bugreport has been opened from an email came to info(a)balabit.com. ------------------------ Core was generated by `syslog-ng -d -p /var/run/syslog-ng.pid -e --enable-core -s'. Program terminated with signal 11, Segmentation fault. #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 (gdb) bt #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 #1 0x00002b71bfe565d5 in cfg_process_flag (handlers=0x2b71c0082230, base=0x228b5f0, flag=0xffffffff <Address 0xffffffff out of bounds>) at cfg-parser.c:249 #2 0x00002b71c04a6ec6 in afsocket_parse (lexer=0x2265440, last_driver=0x7fffe7126608) at afsocket-grammar.y:864 #3 0x00002b71bfe58c67 in cfg_parser_parse (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at cfg-parser.h:82 #4 plugin_parse_config (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at plugin.c:124 #5 0x00002b71bfe570ae in main_parse (lexer=0x2265440, dummy=0x7fffe7128a28) at cfg-grammar.y:539 #6 0x00002b71bfe37a01 in cfg_parser_parse (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg-parser.h:82 #7 cfg_run_parser (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg.c:376 #8 0x00002b71bfe37b03 in cfg_read_config (self=0x22649b0, fname=0x402bb7 "/etc/syslog-ng/syslog-ng.conf", syntax_only=<value optimized out>, preprocess_into=0x0) at cfg.c:398 #9 0x000000000040220c in initial_init () #10 0x000000000040280c in main () (gdb) quit This causes it: source 32001_udp { udp( port(32001) so_rcvbuf(12000000) flags(no_multi_line) ); }; ------------------ ps.: With the given config line snippet it is easy to reproduce. I have cc'd the original author of the email. I hope he doesn't mind it. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 126] New: [systemd] Add installation of systemd unit file via build-sys
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=126 Summary: [systemd] Add installation of systemd unit file via build-sys Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: d(a)falconindy.com Type of the Report: enhancement Estimated Hours: 0.0 I'd like to see syslog-ng offer installation of the systemd unit file via the build system, similar to how other services (e.g., udev or rsyslog) provide this, with the --with-systemdsystemunitdir configure option. autotools is unfortunately not my forte, so the attached patch may not be the most eloquent of solutions (borrowing bits from udev), but it does the job, and does not appear to bother configurations which avoid --enable-systemd or the new --with flag. I felt it was prudent to separate the --enable-systemd flag from this new flag as there's certainly a case for using the --enable flag without the --with flag. I've also provided a one line amendment to the service file itself, which removes the ExecReload. This is discouraged by upstream, as systemd is unable to properly track the behavior of the daemon due to the effects of an asynchronous trigger. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

Transparent TCP forward: what am I doing wrong?
by JP Vossen 30 Jun '11

30 Jun '11

I need to be able to accept TCP/514 and transparently forward it to UDP/514 on the same box. I have a config that almost works, except I'm getting an "extra" time stamp and hostname on the TCP messages. I'm using Netcat to send: echo "<182>loop_test[$$]: LOOP testing 514 UDP, one ping only..." | nc -u -w1 192.168.1.151 514 echo "<182>loop_test[$$]: LOOP testing 514 TCP, one ping only..." | nc 192.168.1.151 514 I'm getting: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>Jun 30 01:48:36 192.168.1.10 loop_test[11061]: LOOP testing 514 TCP, one ping only... I want: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>loop_test[11061]: LOOP testing 514 TCP, one ping only... If I send the messages into syslog-ng or rsyslog, I get the correct data with no "extra" fields in the log file. But unfortunately, I need to send messages into another proprietary syslog listener, and in that one I'm getting the extra fields. I'd blame the whole mess on that, except that when I do a packet dump I do see the 2 extra fields in there. So I *think* that syslog-ng and rsyslog are smart enough to handle them and my other thing isn't. I'm already using keep_timestamp(no) and keep_hostname(no) but they have no effect I've been able to detect. I've tried commented out, set to yes and set to no, but no matter what I get the 2 extra fields. I've also tried syslog-ng 3.0.8 and 3.2.4, same effect. This is the relevant part of the syslog-ng config I'm using: ####### BEGIN: Forward TCP syslog source s_tcpincoming { tcp( ip(0.0.0.0) port(514) max_connections(1000) keep_timestamp(no) keep_hostname(no) ); }; destination d_fe_tcp { udp( "127.0.0.2" port(514) spoof_source(yes) ); }; log { source(s_tcpincoming); # With no "filter" we get everything, which is what we want destination(d_fe_tcp); }; ####### END: Forward TCP syslog As I said, my goal is to receive TCP/514 and **transparently** forward logs with no changes, as if they came in via UDP, to the localhost via UDP/514. In other words, I'm using syslog-ng as a shim to feed syslog over TCP to a listener which only listens on UDP. Why doesn't it work? What totally obvious thing am I missing? Am I doing anything else dumb? TIA, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

4 4