July 2013 - syslog-ng - lists.balabit.hu

[Bug 211] New: dbparser pattern order breaks validation
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=211 Summary: dbparser pattern order breaks validation Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 If two log lines share a common leading portion such as; xlog: backup pg_xlog/000000010000014700000076 xlog: backup pg_xlog/000000010000014700000076 failed then the order of the patterns in the pattern database xml file is important. For example, if the two patterns used are; xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed and the shorter pattern is first, it will match BOTH of the above lines because it matches the leading portion of both lines. If the patterns are entered with the longer pattern first, then the longer pattern will fail on the shorter message, allowing the match to fall through to the shorter pattern matcher, but will match the longer message. I expect the behavior of the dbparser to match with the pattern containing the MOST total static content that also matches all of the parsing macros. Some method where the order is not important need to be used. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Syslog-ng & Distributed File Systems??
by Jiann-Ming Su 08 Mar '14

08 Mar '14

Is there a version of syslog-ng that can properly use distributed filesystems like GlusterFS? For example, I have two nodes each running syslog-ng but also sharing a GlusterFS filesystem. Can those independent syslog-ng processes running on different servers write to the same file on the GlusterFS?

7 20

[Bug 186] New: resettable stats
by bugzilla＠bugzilla.balabit.com 21 Dec '13

21 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=186 Summary: resettable stats Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: szaki(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 According to https://lists.balabit.hu/pipermail/syslog-ng/2011-October/017457.html It would be a good idea to be able to reset the statistic, when needed, without full restart, or any magic like altering the config a bit and reload. So I suggest the enable_stats_reset(yes) default option, for sources and destinations (everywhere are statcounters defined), and the syslog-ng-ctl reset-stats command - to reset the statistic where the enable_stats_reset(yes) option allows this. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 226] New: 3.3.8: iv_tls_user_register: called after iv_init
by bugzilla＠bugzilla.balabit.com 21 Nov '13

21 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=226 Summary: 3.3.8: iv_tls_user_register: called after iv_init Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: bug Estimated Hours: 0.0 When building with libivykis library (internal or external) starting syslog-ng 3.3.8 ends up with: # /sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf iv_tls_user_register: called after iv_init System is Linux 3.7.10, glibc 2.17. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

[Bug 224] New: Segmentation fault in 3.3.7 and 3.4.1
by bugzilla＠bugzilla.balabit.com 13 Nov '13

13 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=224 Summary: Segmentation fault in 3.3.7 and 3.4.1 Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Core dump back trace is #0 0x00007f4020000528 in ?? () #1 0x00007f402a10b1a5 in log_msg_refcache_stop () at logmsg.c:1495 #2 0x00007f402a11760c in log_writer_flush (self=0xa0d460, flush_mode=LW_FLUSH_NORMAL) at logwriter.c:1043 #3 0x00007f402a1177ed in log_writer_work_perform (s=0xa0d460) at logwriter.c:129 #4 0x00007f402a117edb in main_loop_io_worker_job_start (self=0xa0d650) at mainloop.c:371 #5 0x00007f402a1341ca in iv_work_thread_do_work (_thr=0xabdd20) at iv_work.c:118 #6 0x00007f402a13352a in iv_run_tasks (st=0x7f400c40a130) at iv_task.c:48 #7 0x00007f402a13574c in iv_main () at iv_main_posix.c:106 #8 0x00007f402a133fe1 in iv_work_thread (_thr=0xabdd20) at iv_work.c:200 #9 0x00007f402a1361b8 in iv_thread_handler (_thr=0xabddf0) at iv_thread_posix.c:142 #10 0x00007f4028d82851 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f4028ad011d in clone () from /lib64/libc.so.6 some times #2 0x00007f402a11760c in log_writer_flush (self=0xa0d460, flush_mode=LW_FLUSH_NORMAL) at logwriter.c:1043 is #2 0x00007f402a11760c in log_writer_flush (self=0x9b3970, flush_mode=LW_FLUSH_BUFFER) at logwriter.c:1043 of course when this happens all buffers are lost. So much for reliable log tansport :-( -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 169] New: Feature Request - reconnect network also resolve dns name
by bugzilla＠bugzilla.balabit.com 10 Oct '13

10 Oct '13

https://bugzilla.balabit.com/show_bug.cgi?id=169 Summary: Feature Request - reconnect network also resolve dns name Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 When a network connection is dropped the destination name should be resolved via DNS prior to the reconnect. Every couple of years to renew the hardware supporting our core syslog service we move the IP name to a new host. The desired process is; 1. Build a new host and ensure that syslog-ng is running correctly. 2. Change the DNS for our central syslog server to the address of the new server. 3. Restart the syslog-ng service on the old syslog server. 4. All of the clients will loose the connection to the old syslog server. 5. All of the clients will reconnect to the *IP Name* in their configuration files. 6. Monitor the old syslog server to determine any hosts that have not made the transition. What really happened in step 5 was that all of the client reconnected to the old syslog server :-( To accomplish the switch over I had to visit 450 servers and manually restart the syslog service on the client. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 241] New: pdbtool test --validate returns success when matching wrong pattern
by bugzilla＠bugzilla.balabit.com 02 Oct '13

02 Oct '13

https://bugzilla.balabit.com/show_bug.cgi?id=241 Summary: pdbtool test --validate returns success when matching wrong pattern Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 pdbtool test --validate pattern.xml only returns a non-zero exit code if test values don't match. If there is a catchall pattern in the pattern.xml database file then any test message that does not match the pattern it belongs to, but does match another pattern will return a zero exit code. The output still shows the wrong .classifier.rule_id Testing message program='mailguard' message='2013-07-03T13:13:38-07:00 local(a)dog.comp.uvic.ca local4.notice mailguard: Could not open /var/mimedefang/sophie.pid' Wrong match name='.classifier.rule_id', value='UN-eb0cfada-af50-48b0-ac1e-de1ffad8fe6c', expected='b826184c-2990-46d4-a777-f87340f1796a' but the return code from the pdbtool is still zero. I would expect any test message that does not match the pattern it belongs to should produce a non-zero return code. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

4 8

syslog-ng leaves a lot of open file handles
by Anton Koldaev 28 Sep '13

28 Sep '13

Hi, I'm using Syslog-NG OSE v.3.3.7-1~mhp1~lucid (Ubuntu Lucid) And I have the following destination file(): file("/u/logs/`app`/${MONTH}${DAY}/${1}/${1}${2}/${LOGSORT.ACCOUNT}.log" Syslog-NG switches to the new file at 23:59:59 every day just fine but for some reason it leaves files for the previous day open: *# date* Wed Jul 24 09:04:19 UTC 2013 *# lsof | grep a/ac/account.log* syslog-ng 30743 root 3351w REG 252,2 72597491 66306075 /u/logs/app/0723/a/ac/account.log (deleted) syslog-ng 30743 root 4896w REG 252,2 17017519 4572052 /u/logs/app/0724/a/ac/account.log And they're being deleted by my rotating script. Reloading syslog-ng using init script or with `kill -HUP` doesn't help - all deleted files are still open by syslog-ng. Global option "time_reap (30);" doesn't seem to help too. Any ideas? -- Best regards, Koldaev Anton

3 6

Problems with rewrite set and template functions...
by Johnson, Chris (HP TippingPoint Roseville) 05 Sep '13

05 Sep '13

Hi all, I've come across a problem when using the rewrite set function with a template function. I've created a custom template function 'audit-TPTI-to-Email' and use it in a rewrite: rewrite r_audit_EMail { set("$(audit-TPTI-to-EMail ${MSG})", value("MSG")); }; Then call it: filter f_audit_pgm{program("AUDIT-*" type("glob"));}; log { source(s_local); filter(f_audit_pgm); log { destination(d_logID_02); }; log { rewrite(r_audit_EMail); rewrite(r_quote_newlines); destination(d_logID_13); }; flags(final); }; Everything work fine. Then if I add another call to rewrite (i.e. add a second email destination): filter f_audit_pgm{program("AUDIT-*" type("glob"));}; log { source(s_local); filter(f_audit_pgm); log { destination(d_logID_02); }; log { rewrite(r_audit_EMail); rewrite(r_quote_newlines); destination(d_logID_13); }; log { rewrite(r_audit_EMail); rewrite(r_quote_newlines); destination(d_logID_14); }; flags(final); }; Syslog-ng crashes with a segfault. I've narrowed in down to any template function (just to make sure *I* wasn't screwing something up in my custom function): rewrite r_echo { set("$(echo $PROGRAM)" value("PROGRAM")); }; destination d_test1{ file("/var/log/test1.log"); }; destination d_test2{ file("/var/log/test2.log"); }; log { source(s_local); log { rewrite(r_echo); destination(d_test1); }; log { rewrite(r_echo); destination(d_test2); }; }; The backtrace: Backtrace: /usr/local/lib/libsyslog-ng-3.3.3.so(plugin_find+0x39)[0x7f3eb76ff019] /usr/local/lib/libsyslog-ng-3.3.3.so(log_template_compile+0x84f)[0x7f3eb7703baf] /usr/local/lib/libsyslog-ng-3.3.3.so(log_rewrite_set_new+0x99)[0x7f3eb76f3349] /usr/local/lib/libsyslog-ng-3.3.3.so[0x7f3eb76f3371] /usr/local/lib/libsyslog-ng-3.3.3.so(log_center_init_pipe_line+0x35d)[0x7f3eb76dfecd] /usr/local/lib/libsyslog-ng-3.3.3.so(log_center_init_pipe_line+0xd2)[0x7f3eb76dfc42] /usr/local/lib/libsyslog-ng-3.3.3.so(log_center_init+0x56)[0x7f3eb76e0226] /usr/local/lib/libsyslog-ng-3.3.3.so(cfg_init+0xb0)[0x7f3eb76e1530] /usr/local/lib/libsyslog-ng-3.3.3.so(main_loop_init+0x11b)[0x7f3eb76f9abb] /usr/local/sbin/syslog-ng(main+0x11f)[0x40168f] /lib/libc.so.6(__libc_start_main+0xe6)[0x7f3eb6240126] /usr/local/sbin/syslog-ng[0x401379] I threw in some debug statements: LogRewrite * log_rewrite_set_new(const gchar *new_value) { fprintf(stderr, "%s('%s'):\n", __FUNCTION__, new_value); Plugin * plugin_find(GlobalConfig *cfg, gint plugin_type, const gchar *plugin_name) { fprintf(stderr, "%s(%p, %d, '%s'): '\n", __FUNCTION__, cfg, plugin_type, plugin_name); Which showed that the 'cfg' pointer is null when rewrite is called the second time: log_rewrite_set_new('$(echo $PROGRAM)'): plugin_find(0x60e210, 13, 'echo'): ' plugin_find: plugin->name = 'sys-to-EMail' plugin_find: plugin->name = 'audit-TPTI-to-EMail' plugin_find: plugin->name = 'quar-TPTI-to-EMail' plugin_find: plugin->name = 'quar-TPTI-to-CEF' plugin_find: plugin->name = 'tab-to-bar' plugin_find: plugin->name = 'tab-to-semicolon' plugin_find: plugin->name = 'tab-to-comma' plugin_find: plugin->name = 'to-upper-case' plugin_find: plugin->name = 'to-lower-case' plugin_find: plugin->name = 'ipv4-to-int' plugin_find: plugin->name = 'log-session-seqnum' plugin_find: plugin->name = 'indent-multi-line' plugin_find: plugin->name = 'if' plugin_find: plugin->name = 'grep' plugin_find: plugin->name = 'echo' plugin_find(0x60e210, 2, 'file'): ' [...] log_rewrite_set_new('$(echo $PROGRAM)'): plugin_find((nil), 13, 'echo'): ' *** Segmentation fault Sooo, my questions are: Is this expected behavior? Has this been patched already? Is there another way I can call a custom function to reformat the message field on a destination-by-destination basis? Thanks, Chris ---------------------------------------- Christopher Johnson chris.johnson3(a)hp.com<mailto:chris.johnson3@hp.com> HP Software - Security Product Group (916) 785-2817 ----------------------------------------

3 5