November 2013 - syslog-ng - lists.balabit.hu

[Bug 211] New: dbparser pattern order breaks validation
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=211 Summary: dbparser pattern order breaks validation Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 If two log lines share a common leading portion such as; xlog: backup pg_xlog/000000010000014700000076 xlog: backup pg_xlog/000000010000014700000076 failed then the order of the patterns in the pattern database xml file is important. For example, if the two patterns used are; xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed and the shorter pattern is first, it will match BOTH of the above lines because it matches the leading portion of both lines. If the patterns are entered with the longer pattern first, then the longer pattern will fail on the shorter message, allowing the match to fall through to the shorter pattern matcher, but will match the longer message. I expect the behavior of the dbparser to match with the pattern containing the MOST total static content that also matches all of the parsing macros. Some method where the order is not important need to be used. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 261] New: DOCBUG: --rekey
by bugzilla＠bugzilla.balabit.com 15 Apr '14

15 Apr '14

https://bugzilla.balabit.com/show_bug.cgi?id=261 Summary: DOCBUG: --rekey Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 In Example 2.3. (Using the rekey() option) from §2.10.1. (Specifying data types in value-pairs) the documentation claims: "The rekey() option can be used with the format-json template-function as well, using the following syntax: $(format-json --key .cee.* --rekey --shift 4 --add-prefix events.)" However, when using this syntax, the config parser fails with the following error: "Error parsing config, Error compiling template (Error parsing value-pairs: expected an equal sign in key=value pair)" The correct syntax (which works, as suggested by algeron on IRC) would be the following: $(format-json --rekey .cee.* --add-prefix events. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

Syslog-ng & Distributed File Systems??
by Jiann-Ming Su 08 Mar '14

08 Mar '14

Is there a version of syslog-ng that can properly use distributed filesystems like GlusterFS? For example, I have two nodes each running syslog-ng but also sharing a GlusterFS filesystem. Can those independent syslog-ng processes running on different servers write to the same file on the GlusterFS?

7 20

[Bug 260] New: exit with code 134
by bugzilla＠bugzilla.balabit.com 06 Mar '14

06 Mar '14

https://bugzilla.balabit.com/show_bug.cgi?id=260 Summary: exit with code 134 Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 I just did a reload on version 3.4.4 (with latest patches) and triggered 2013-11-05T21:24:33-08:00 host daemon.crit supervise/syslog-ng[25895]: Daemon exited due to a deadlock/signal/failure, restarting; exitcode='134' Is it possible that the return code of syslog-ng is somehow linked to the return code of a child program destination? core file sent to Balazs in private e-mail -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 11

patterndb
by Fabien Wernli 03 Mar '14

03 Mar '14

Hi, I'm currently organizing my patterndb files into something easily maintainable, and I ran into the following problem, best explained using an example: sshd and pam_unix. Some of sshd specific patterns reside in a specific ruleset, e.g. <ruleset name='sshd' id='A'> <patterns> <pattern>sshd</pattern </patterns> <rules> <rule> ... </rule> </rules> </ruleset> Some of sshd's messages come from pam_unix, e.g. modules/pam_unix/support.c, and are shared among other daemons: <ruleset name='pam_unix' id='B'> <patterns> <pattern>crond</pattern> <pattern>imap</pattern> <pattern>login</pattern> <pattern>pam</pattern> <pattern>pure-ftpd</pattern> <pattern>proftpd</pattern> <pattern>sshd</pattern> <pattern>su</pattern> <pattern>sudo</pattern> ... <patterns> <rules> <rule> ... </rule> </rules> </ruleset> The problem is, when a sshd message is parsed, even if it doesn't match any rules in ruleset 'A', ruleset 'B' isn't even considered for matching. So how do I properly organize my pdb tree, without duplicating rules in multiple source files? Thanks in advance

3 4

[Bug 186] New: resettable stats
by bugzilla＠bugzilla.balabit.com 21 Dec '13

21 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=186 Summary: resettable stats Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: szaki(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 According to https://lists.balabit.hu/pipermail/syslog-ng/2011-October/017457.html It would be a good idea to be able to reset the statistic, when needed, without full restart, or any magic like altering the config a bit and reload. So I suggest the enable_stats_reset(yes) default option, for sources and destinations (everywhere are statcounters defined), and the syslog-ng-ctl reset-stats command - to reset the statistic where the enable_stats_reset(yes) option allows this. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 262] New: patterndb: Computed <values> are not expanded in context-id
by bugzilla＠bugzilla.balabit.com 20 Dec '13

20 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=262 Summary: patterndb: Computed <values> are not expanded in context-id Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=87) --> (https://bugzilla.balabit.com/attachment.cgi?id=87) patterndb ruleset It seems the values are not being expanded as expected in the context-id: -- pdbtool test /tmp/test.pdb -vd Module loaded and initialized successfully; module='syslogformat' Module loaded and initialized successfully; module='basicfuncs' Testing message program='' message='the big "brown" fox' patterndb rule matches; rule_id='A' Advancing patterndb current time because of an incoming message; utc='-1' Correllation context lookup failure, starting a new context; rule='A', context='brown.', context_timeout='0', context_expiration='-1' Match name='.classifier.rule_id', value='A', expected='A' Match name='matched.var', value='brown', expected='brown' -- as you can see, context-id='brown.' instead of 'brown.brown' as expected by expanding '${matched.var}.${computed.var}' -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 263] New: patterndb: missing "program" attribute in test_message silently ignored
by bugzilla＠bugzilla.balabit.com 20 Dec '13

20 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=263 Summary: patterndb: missing "program" attribute in test_message silently ignored Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 Although the attached pdb validates, the test messages are not run, because "program" attribute is missing. There should be a warning, or something: -- pdbtool --validate test /tmp/test2.pdb -dv Module loaded and initialized successfully; module='syslogformat' Module loaded and initialized successfully; module='basicfuncs' /tmp/test2.pdb validates -- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 258] New: syslog-ng 3.4.2 stalls on Ubuntu 12.04 Precise with /var/ log full
by bugzilla＠bugzilla.balabit.com 19 Dec '13

19 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=258 Summary: syslog-ng 3.4.2 stalls on Ubuntu 12.04 Precise with /var/log full Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: carl.chenet-ext(a)cloudwatt.com Type of the Report: bug Estimated Hours: 0.0 Hi, We had several cases of frozen KVM virtual machines running Ubuntu 12.04 Precise with syslog-ng 3.4.2 taken from madhouse Debian repository. Issue: the servers are not reachable any more. You can initiate a ssh/console connection but the connection freezes before displaying the banner. Each time the /var/log lvm file system is full. The following errors occur in /var/log/system.log : Oct 30 07:05:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:06:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:07:18 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:08:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:09:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:10:18 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:11:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:12:18 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:13:19 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:14:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:15:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Oct 30 07:16:19 s-metcld-0002 syslog-ng[8467]: internal() messages are looping back, preventing loop by suppressing all internal messages until the current message is processed; trigger-msg='', first-suppressed-msg='I/O error occurred while writing; fd=\'16\', error=\'No space left on device (28)\'' Oct 30 07:17:19 s-metcld-0002 syslog-ng[8467]: I/O error occurred while writing; fd='16', error='No space left on device (28)' Workaround: We have the mcollective client installed on each server so we were able to kill the syslog-ng process by mcollective. when syslog-ng is killed, the system stops freezing immedialty, the ssh connection is back. I'm not sure to understand how syslog-ng freezes the system but it is worth reporting I guess. Regards, Carl Chenet -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 4

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26