June 2011 - syslog-ng - lists.balabit.hu

Cross-compile problem with path in 3.2.4
by Ulrich Prinz 14 Jul '11

14 Jul '11

Hi! I cross compiled syslog-ng 3.2.4 for my ARM board. I called ./configure with the usual --host, --prefix and --exec-prefix options. Then I ran make with DESTDIR to my cross target directory. I moved everything to my ARM target and syslog-ng starts up. While it locates and loads the configuration files properly, there is a problem with the modules: ~ # syslog-ng Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='syslogformat', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='basicfuncs', error='//lib/syslog-ng/: cannot read file data: Is a directory' Plugin module not found in 'module-path'; module-path='/lib/syslog-ng', module='afsocket' Error opening plugin module; module='affile', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afprog', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='afuser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='dbparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='csvparser', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error opening plugin module; module='confgen', error='//lib/syslog-ng/: cannot read file data: Is a directory' Error parsing source, source plugin system not found in /etc/syslog-ng/syslog-ng.conf at line 10, column 2: Checking the directory shows: /etc2/syslog-ng # ll /lib/syslog-ng/ drwxr-xr-x 2 root root 1024 Jun 14 16:01 ./ drwxr-xr-x 7 root root 3072 Jun 14 16:01 ../ -rw-r--r-- 1 root root 139122 Jun 14 16:01 libaffile.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libaffile.la* -rw-r--r-- 1 root root 110150 Jun 14 16:01 libafprog.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafprog.la* -rw-r--r-- 1 root root 227144 Jun 14 16:01 libafsocket-notls.a -rwxr-xr-x 1 root root 1313 Jun 14 16:01 libafsocket-notls.la* -rw-r--r-- 1 root root 484708 Jun 14 16:01 libafsocket-tls.a -rwxr-xr-x 1 root root 1326 Jun 14 16:01 libafsocket-tls.la* lrwxrwxrwx 1 root root 18 Jun 14 16:03 libafsocket.so -> libafsocket-tls.so@ -rw-r--r-- 1 root root 89662 Jun 14 16:01 libafuser.a -rwxr-xr-x 1 root root 1289 Jun 14 16:01 libafuser.la* -rw-r--r-- 1 root root 26398 Jun 14 16:01 libbasicfuncs.a -rwxr-xr-x 1 root root 1301 Jun 14 16:01 libbasicfuncs.la* -rw-r--r-- 1 root root 15366 Jun 14 16:01 libconfgen.a -rwxr-xr-x 1 root root 1292 Jun 14 16:01 libconfgen.la* -rw-r--r-- 1 root root 18030 Jun 14 16:01 libconvertfuncs.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libconvertfuncs.la* -rw-r--r-- 1 root root 96416 Jun 14 16:01 libcsvparser.a -rwxr-xr-x 1 root root 1298 Jun 14 16:01 libcsvparser.la* -rw-r--r-- 1 root root 265984 Jun 14 16:01 libdbparser.a -rwxr-xr-x 1 root root 1314 Jun 14 16:01 libdbparser.la* -rw-r--r-- 1 root root 78426 Jun 14 16:01 libdummy.a -rwxr-xr-x 1 root root 1286 Jun 14 16:01 libdummy.la* -rw-r--r-- 1 root root 1662352 Jun 14 16:01 libsyslog-ng.a -rwxr-xr-x 1 root root 1265 Jun 14 16:01 libsyslog-ng.la* -rw-r--r-- 1 root root 60868 Jun 14 16:01 libsyslogformat.a -rwxr-xr-x 1 root root 1307 Jun 14 16:01 libsyslogformat.la* I tried calling syslog-ng --module-path=/lib/syslog-ng too, but I get the same result, Any idea to get around that would be appreciated :) Best Regards and thanks in advance Ulrich

3 6

[PATCH 0/N]: Various fixes from the syslog-ng team
by Gergely Nagy 14 Jul '11

14 Jul '11

Hi! A few selected patches from the syslog-ng team at BalaBit will follow this e-mail. This here mostly serves as an umberalla mail, under which I can shovel the rest, so they end up like a neat little thread. The patches I am about to send are available on the team/4.1-sprint5/fixes branch of my git tree. Special thanks to Viktor Juhasz <jviktor(a)balabit.hu> of the syslog-ng team for doing the bulk of the work, and assembling the list of patches that can be safely sent to the list. Another batch will be hitting the list a bit later, but unlike this first, which I expect to be safely appliable, the second batch will need a more through review. I'll comment on the whys when those patches are posted. I expect more patches from the team to follow, once the features and fixes are in a good enough shape. (In the meantime, I'll try to browse through their tree to see if any other bugfix can be isolated) -- |8]

3 23

reconnect DB session after timeout
by "Marcus Groß" 14 Jul '11

14 Jul '11

Hi, I'm running syslog-ng v. 3.2.4 on SLES 10. There are three defined MySQL DB destinations. Everything is running fine, until the MySQL server resets the connection(s) after a defined idle time if no messages are send to this database. The problem is that syslog-ng doesn't reconnect the disconnected connection ;) I have to restart syslog-ng to bring it back to life. Is there any posssibility to reopen the DB connection instead of disabling the idle timeout on the MySQL side? syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' syslog-ng[4904]: Error checking table, disconnecting from database, trying again shortly; time_reopen='60' syslog-ng[4904]: Error running SQL query; type='mysql', host='snsanmg8429', port='', user='syslog', database='SysLog', error='20 06: MySQL server has gone away', query='CREATE TABLE SysLog_Brocade_2011_06_09 (Host varchar(255), Facility varchar(255), Priority varchar(255), Level varchar (45), Date varchar(45), Program varchar(255), Message varchar(700))' Jun 9 09:43:56 snnasmg8421 syslog-ng[4904]: Error creating table, giving up; table='SysLog_Brocade_2011_06_09' M.

3 2

[PATCH] Fix: maximum thread number can be 0
by Juhasz Viktor 14 Jul '11

14 Jul '11

[threading] It is possible that the _SC_NPROCESSORS_ONLN isn't defined. In this case the max threads must greater than 0 Signed-off-by: Viktor Juhasz <jviktor(a)balabit.hu> diff --git a/lib/mainloop.c b/lib/mainloop.c index e659fff..b23d055 100644 --- a/lib/mainloop.c +++ b/lib/mainloop.c @@ -702,6 +702,8 @@ main_loop_add_options(GOptionContext *ctx) { #ifdef _SC_NPROCESSORS_ONLN main_loop_io_workers.max_threads = MIN(MAX(2, sysconf(_SC_NPROCESSORS_ONLN)), 64); +#else + main_loop_io_workers.max_threads = 2; #endif g_option_context_add_main_entries(ctx, main_loop_options, NULL);

2 1

check clients
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I want to know if there is a option to know which log-clients are sending data to syslog-ng server. Is it possible to get a list of clients? Thanks and best regards.

4 4

change rsyslog for syslog-ng
by Josu Lazkano 14 Jul '11

14 Jul '11

Hello list, I am learning how to get a syslog server with syslog-ng. I started working on a IT department and I have more than 50 servers, we actually has rsyslog on the client side and syslog-ng on the syslog server. I want to change the client side rsyslog to configure it with syslog-ng. On the server side I read lot of documentation and I know how to configure it (source, destination and filter). I want to get this logs on the client side and send to the syslog server on a TCP+SSL: /var/log/auth.log /var/log/daemon.log /var/log/dmesg /var/log/messages /var/log/syslog I want to get on the server side something like this: /var/log/extern/host1/auth.log /var/log/extern/host1/ daemon.log /var/log/extern/host1/ dmesg /var/log/extern/host1/ messages /var/log/extern/host1/ syslog /var/log/extern/host2/auth.log /var/log/extern/host2/ daemon.log /var/log/extern/host2/ dmesg /var/log/extern/host2/ messages /var/log/extern/host2/ syslog ... /var/log/extern/hostn/auth.log /var/log/extern/hostn/ daemon.log /var/log/extern/hostn/ dmesg /var/log/extern/hostn/ messages /var/log/extern/hostn/ syslog On the past post you help me and I have how to get It on the server side for the TCP+SSL: source s_tcptls { tcp (ip("10.200.42.1") port(10514) tls( peer-verify(require-trusted) ca_dir("/etc/syslog-ng/certs/") key_file("/etc/syslog-ng/certs/server.key") cert_file("/etc/syslog-ng/certs/server.crt") ) ); }; I have some questions: 1. Must I uninstall rsyslog before install syslog-ng? 2. How can I configure the client side to send my logs to the server and to save on the local /var/log/? 3. Is there any client limit on syslog-ng? I will try to read and learn as much as I can, I will appreciate your help. Thanks for your job and best regards.

4 4

Macros expansion in pattern-db actions
by Denis Gasparin 14 Jul '11

14 Jul '11

Hi to all. We're using syslog-ng 3.2.4 and we're having a weird behaviour in using macros with actions values. Here it is a sample pattern-db rule: <?xml version='1.0' encoding='UTF-8'?> <patterndb version='4' pub_date='2011-06-20'> <ruleset name='cron' id='cron-ruleset'> <pattern>/usr/sbin/cron</pattern> <rules> <rule provider="patterndb" id="cron-1" class="system" context-id="sample-context-id"> <patterns> <pattern>(@ESTRING:usracct.username:) @CMD (@ESTRING:details:)@</pattern> </patterns> <actions> <action trigger="match" condition="match('mymatch' value('details'))"> <message> <values> <value name="MESSAGE">[${details}] was found in a cron log message. Rule number [${.classifier.rule_id}]</value> <value name="TRIGGER">yes</value> </values> </message> </action> </actions> </rule> </rules> </ruleset> </patterndb> We tested the rule using pdtool match command and the output was: # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[] was found in a cron log message. Rule number [] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes We instead expected the following output from pdtool match execution : # pdbtool match -P "/usr/sbin/cron" -M "(root) CMD (mymatch)" MESSAGE=(root) CMD (mymatch) PROGRAM=/usr/sbin/cron .classifier.class=system .classifier.rule_id=cron-1 usracct.username=root details=mymatch HOST= MESSAGE=[mymatch] was found in a cron log message. Rule number [cron-1 ] PROGRAM=/usr/sbin/cron PID= TRIGGER=yes Macro expansion was not executed in action values but it was in action definition... What are we missing? Thank you in advance for your help Denis Gasparin --- Edistar SRL

2 1

[Bug 127] New: 3.2.4 ose segfaults during startup
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=127 Summary: 3.2.4 ose segfaults during startup Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: balazs.nemeth(a)balabit.com Type of the Report: bug Estimated Hours: 0.0 This bugreport has been opened from an email came to info(a)balabit.com. ------------------------ Core was generated by `syslog-ng -d -p /var/run/syslog-ng.pid -e --enable-core -s'. Program terminated with signal 11, Segmentation fault. #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 (gdb) bt #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 #1 0x00002b71bfe565d5 in cfg_process_flag (handlers=0x2b71c0082230, base=0x228b5f0, flag=0xffffffff <Address 0xffffffff out of bounds>) at cfg-parser.c:249 #2 0x00002b71c04a6ec6 in afsocket_parse (lexer=0x2265440, last_driver=0x7fffe7126608) at afsocket-grammar.y:864 #3 0x00002b71bfe58c67 in cfg_parser_parse (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at cfg-parser.h:82 #4 plugin_parse_config (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at plugin.c:124 #5 0x00002b71bfe570ae in main_parse (lexer=0x2265440, dummy=0x7fffe7128a28) at cfg-grammar.y:539 #6 0x00002b71bfe37a01 in cfg_parser_parse (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg-parser.h:82 #7 cfg_run_parser (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg.c:376 #8 0x00002b71bfe37b03 in cfg_read_config (self=0x22649b0, fname=0x402bb7 "/etc/syslog-ng/syslog-ng.conf", syntax_only=<value optimized out>, preprocess_into=0x0) at cfg.c:398 #9 0x000000000040220c in initial_init () #10 0x000000000040280c in main () (gdb) quit This causes it: source 32001_udp { udp( port(32001) so_rcvbuf(12000000) flags(no_multi_line) ); }; ------------------ ps.: With the given config line snippet it is easy to reproduce. I have cc'd the original author of the email. I hope he doesn't mind it. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 126] New: [systemd] Add installation of systemd unit file via build-sys
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=126 Summary: [systemd] Add installation of systemd unit file via build-sys Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: d(a)falconindy.com Type of the Report: enhancement Estimated Hours: 0.0 I'd like to see syslog-ng offer installation of the systemd unit file via the build system, similar to how other services (e.g., udev or rsyslog) provide this, with the --with-systemdsystemunitdir configure option. autotools is unfortunately not my forte, so the attached patch may not be the most eloquent of solutions (borrowing bits from udev), but it does the job, and does not appear to bother configurations which avoid --enable-systemd or the new --with flag. I felt it was prudent to separate the --enable-systemd flag from this new flag as there's certainly a case for using the --enable flag without the --with flag. I've also provided a one line amendment to the service file itself, which removes the ExecReload. This is discouraged by upstream, as systemd is unable to properly track the behavior of the daemon due to the effects of an asynchronous trigger. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

Transparent TCP forward: what am I doing wrong?
by JP Vossen 30 Jun '11

30 Jun '11

I need to be able to accept TCP/514 and transparently forward it to UDP/514 on the same box. I have a config that almost works, except I'm getting an "extra" time stamp and hostname on the TCP messages. I'm using Netcat to send: echo "<182>loop_test[$$]: LOOP testing 514 UDP, one ping only..." | nc -u -w1 192.168.1.151 514 echo "<182>loop_test[$$]: LOOP testing 514 TCP, one ping only..." | nc 192.168.1.151 514 I'm getting: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>Jun 30 01:48:36 192.168.1.10 loop_test[11061]: LOOP testing 514 TCP, one ping only... I want: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>loop_test[11061]: LOOP testing 514 TCP, one ping only... If I send the messages into syslog-ng or rsyslog, I get the correct data with no "extra" fields in the log file. But unfortunately, I need to send messages into another proprietary syslog listener, and in that one I'm getting the extra fields. I'd blame the whole mess on that, except that when I do a packet dump I do see the 2 extra fields in there. So I *think* that syslog-ng and rsyslog are smart enough to handle them and my other thing isn't. I'm already using keep_timestamp(no) and keep_hostname(no) but they have no effect I've been able to detect. I've tried commented out, set to yes and set to no, but no matter what I get the 2 extra fields. I've also tried syslog-ng 3.0.8 and 3.2.4, same effect. This is the relevant part of the syslog-ng config I'm using: ####### BEGIN: Forward TCP syslog source s_tcpincoming { tcp( ip(0.0.0.0) port(514) max_connections(1000) keep_timestamp(no) keep_hostname(no) ); }; destination d_fe_tcp { udp( "127.0.0.2" port(514) spoof_source(yes) ); }; log { source(s_tcpincoming); # With no "filter" we get everything, which is what we want destination(d_fe_tcp); }; ####### END: Forward TCP syslog As I said, my goal is to receive TCP/514 and **transparently** forward logs with no changes, as if they came in via UDP, to the localhost via UDP/514. In other words, I'm using syslog-ng as a shim to feed syslog over TCP to a listener which only listens on UDP. Why doesn't it work? What totally obvious thing am I missing? Am I doing anything else dumb? TIA, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

4 4