October 2013 - syslog-ng - lists.balabit.hu

[Bug 211] New: dbparser pattern order breaks validation
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=211 Summary: dbparser pattern order breaks validation Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 If two log lines share a common leading portion such as; xlog: backup pg_xlog/000000010000014700000076 xlog: backup pg_xlog/000000010000014700000076 failed then the order of the patterns in the pattern database xml file is important. For example, if the two patterns used are; xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed and the shorter pattern is first, it will match BOTH of the above lines because it matches the leading portion of both lines. If the patterns are entered with the longer pattern first, then the longer pattern will fail on the shorter message, allowing the match to fall through to the shorter pattern matcher, but will match the longer message. I expect the behavior of the dbparser to match with the pattern containing the MOST total static content that also matches all of the parsing macros. Some method where the order is not important need to be used. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Syslog-ng & Distributed File Systems??
by Jiann-Ming Su 08 Mar '14

08 Mar '14

Is there a version of syslog-ng that can properly use distributed filesystems like GlusterFS? For example, I have two nodes each running syslog-ng but also sharing a GlusterFS filesystem. Can those independent syslog-ng processes running on different servers write to the same file on the GlusterFS?

7 20

[Bug 186] New: resettable stats
by bugzilla＠bugzilla.balabit.com 21 Dec '13

21 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=186 Summary: resettable stats Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: szaki(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 According to https://lists.balabit.hu/pipermail/syslog-ng/2011-October/017457.html It would be a good idea to be able to reset the statistic, when needed, without full restart, or any magic like altering the config a bit and reload. So I suggest the enable_stats_reset(yes) default option, for sources and destinations (everywhere are statcounters defined), and the syslog-ng-ctl reset-stats command - to reset the statistic where the enable_stats_reset(yes) option allows this. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 258] New: syslog-ng 3.4.2 stalls on Ubuntu 12.04 Precise with /var/ log full
by bugzilla＠bugzilla.balabit.com 19 Dec '13

19 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=258 Summary: syslog-ng 3.4.2 stalls on Ubuntu 12.04 Precise with /var/log full Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: carl.chenet-ext(a)cloudwatt.com Type of the Report: bug Estimated Hours: 0.0 Hi, We had several cases of frozen KVM virtual machines running Ubuntu 12.04 Precise with syslog-ng 3.4.2 taken from madhouse Debian repository. Issue: the servers are not reachable any more. You can initiate a ssh/console connection but the connection freezes before displaying the banner. Each time the /var/log lvm file system is full. The following errors occur in /var/log/system.log : Oct 30 07:05:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:06:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:07:18 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:08:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:09:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:10:18 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:11:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:12:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:13:19 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:14:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:15:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:16:19 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:17:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Workaround: We have the mcollective client installed on each server so we were able to kill the syslog-ng process by mcollective. when syslog-ng is killed, the system stops freezing immedialty, the ssh connection is back. I'm not sure to understand how syslog-ng freezes the system but it is worth reporting I guess. Regards, Carl Chenet -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 4

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 253] New: Segfault when reloading configuration
by bugzilla＠bugzilla.balabit.com 21 Nov '13

21 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=253 Summary: Segfault when reloading configuration Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: gonzalo.paniagua+slng1(a)acquia.com Type of the Report: bug Estimated Hours: 0.0 syslog-ng version: 3.4.3 built from sources syslog-ng -V: syslog-ng 3.4.3 Installer-Version: 3.4.3 Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.4#no_branch#64d670f3cbfb90769f3c7f0fdd9c70bb9136ec5b Compile-Date: Sep 11 2013 05:33:29 Available-Modules: confgen,afsocket-tls,csvparser,affile,system-source,syslogformat,afuser,dbparser,afsocket-notls,basicfuncs,cryptofuncs,afsocket,afprog Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: on On rare occasions (I can't reproduce it reliably) syslog-ng segfaults when reloading (syslog-ng-ctl reload). I have a core dump file that gave me this stack trace: #0 0x00007fbd30102131 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fbd30101d76 in strdup () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007fbd2f868b19 in evt_tag_str (tag=0x7fbd309a42f0 "encoding", value=0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>) at evttags.c:76 #3 0x00007fbd3097918c in log_proto_server_options_validate (options=<optimized out>) at logproto-server.c:138 #4 0x00007fbd3097d191 in log_proto_server_validate_options (self=<optimized out>) at logproto-server.h:82 #5 log_reader_init (s=0x14a6a30) at logreader.c:638 #6 0x00007fbd2e9d5b07 in log_pipe_init (cfg=0x0, s=0x14a6a30) at ../../lib/logpipe.h:253 #7 log_pipe_init (cfg=0x0, s=0x14a6a30) at afsocket-source.c:763 #8 afsocket_sc_init (s=0x14a68f0) at afsocket-source.c:149 #9 0x00007fbd2e9d67fd in log_pipe_init (cfg=0x0, s=0x14a68f0) at ../../lib/logpipe.h:253 #10 log_pipe_init (cfg=0x0, s=0x14a68f0) at afsocket-source.c:763 #11 afsocket_sd_init (s=0x1489630) at afsocket-source.c:548 #12 0x00007fbd2e9d9719 in afunix_sd_init (s=<optimized out>) at afunix-source.c:194 #13 0x00007fbd3096d27a in log_pipe_init (cfg=<optimized out>, s=0x1489630) at logpipe.h:253 #14 cfg_tree_start (self=0x1c15c48) at cfg-tree.c:1064 #15 0x00007fbd309685de in cfg_init (cfg=0x1c15ae0) at cfg.c:220 #16 0x00007fbd30984faf in main_loop_reload_config_apply () at mainloop.c:501 #17 0x00007fbd3096d699 in control_connection_reload (self=0x1d09e90, command=<optimized out>) at control.c:150 #18 0x00007fbd3096dae8 in control_connection_io_input (s=0x1d09e90) at control.c:242 #19 control_connection_io_input (s=0x1d09e90) at control.c:172 #20 0x00007fbd309a106c in iv_fd_poll_and_run (st=0x1482580, to=<optimized out>) at iv_fd.c:163 #21 0x00007fbd309a1aac in iv_main () at iv_main_posix.c:117 #22 0x00007fbd309859fb in main_loop_run () at mainloop.c:736 #23 0x000000000040142b in main (argc=1, argv=0x7fffd8212a58) at main.c:267 When at frame #5, I got this: (gdb) print *self->proto->options $19 = {destroy = 0, initialized = -1, encoding = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, convert = 0xffffffffffffffff, max_msg_size = -1, max_buffer_size = 0, init_buffer_size = 0} I could not find anywhere in the code where initialized is set to -1, so I assume this is some kind of memory corruption. Let me know if you need anything else from me. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 16

[Bug 226] New: 3.3.8: iv_tls_user_register: called after iv_init
by bugzilla＠bugzilla.balabit.com 21 Nov '13

21 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=226 Summary: 3.3.8: iv_tls_user_register: called after iv_init Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: bug Estimated Hours: 0.0 When building with libivykis library (internal or external) starting syslog-ng 3.3.8 ends up with: # /sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf iv_tls_user_register: called after iv_init System is Linux 3.7.10, glibc 2.17. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

[Bug 224] New: Segmentation fault in 3.3.7 and 3.4.1
by bugzilla＠bugzilla.balabit.com 13 Nov '13

13 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=224 Summary: Segmentation fault in 3.3.7 and 3.4.1 Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Core dump back trace is #0 0x00007f4020000528 in ?? () #1 0x00007f402a10b1a5 in log_msg_refcache_stop () at logmsg.c:1495 #2 0x00007f402a11760c in log_writer_flush (self=0xa0d460, flush_mode=LW_FLUSH_NORMAL) at logwriter.c:1043 #3 0x00007f402a1177ed in log_writer_work_perform (s=0xa0d460) at logwriter.c:129 #4 0x00007f402a117edb in main_loop_io_worker_job_start (self=0xa0d650) at mainloop.c:371 #5 0x00007f402a1341ca in iv_work_thread_do_work (_thr=0xabdd20) at iv_work.c:118 #6 0x00007f402a13352a in iv_run_tasks (st=0x7f400c40a130) at iv_task.c:48 #7 0x00007f402a13574c in iv_main () at iv_main_posix.c:106 #8 0x00007f402a133fe1 in iv_work_thread (_thr=0xabdd20) at iv_work.c:200 #9 0x00007f402a1361b8 in iv_thread_handler (_thr=0xabddf0) at iv_thread_posix.c:142 #10 0x00007f4028d82851 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f4028ad011d in clone () from /lib64/libc.so.6 some times #2 0x00007f402a11760c in log_writer_flush (self=0xa0d460, flush_mode=LW_FLUSH_NORMAL) at logwriter.c:1043 is #2 0x00007f402a11760c in log_writer_flush (self=0x9b3970, flush_mode=LW_FLUSH_BUFFER) at logwriter.c:1043 of course when this happens all buffers are lost. So much for reliable log tansport :-( -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 244] New: Pattern beginning with a parser never matches
by bugzilla＠bugzilla.balabit.com 13 Nov '13

13 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=244 Summary: Pattern beginning with a parser never matches Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: bug Estimated Hours: 0.0 It would seem the patterns beginning with parsers never match. Example: Consider the following MESSAGE: -- Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist -- The following pattern matches OK: -- <pattern>P@ESTRING:puppet_module: @file @ESTRING:puppet_file: @does not exist</pattern> -- The following pattern does not match: -- <pattern>@ESTRING:puppet_module::: @file @ESTRING:puppet_file: @does not exist</pattern> -- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 257] New: memory leak on reload
by bugzilla＠bugzilla.balabit.com 04 Nov '13

04 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=257 Summary: memory leak on reload Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 We have a complex environment that growns it's memory footprint on every reload. It is completely consistent as long as a reload is not done. A simple configuration does not have this problem. We have: - multiple patterndb uses (4 different databases) - multiple program destinations with different templates - multiple pipe destinations - multiple file destinations Every reload the memory difference is different amount and the difference does not always increase or decrease. For me it is about 13MB every reload. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 17