January 2014 - syslog-ng - lists.balabit.hu

[Bug 211] New: dbparser pattern order breaks validation
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=211 Summary: dbparser pattern order breaks validation Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 If two log lines share a common leading portion such as; xlog: backup pg_xlog/000000010000014700000076 xlog: backup pg_xlog/000000010000014700000076 failed then the order of the patterns in the pattern database xml file is important. For example, if the two patterns used are; xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed and the shorter pattern is first, it will match BOTH of the above lines because it matches the leading portion of both lines. If the patterns are entered with the longer pattern first, then the longer pattern will fail on the shorter message, allowing the match to fall through to the shorter pattern matcher, but will match the longer message. I expect the behavior of the dbparser to match with the pattern containing the MOST total static content that also matches all of the parsing macros. Some method where the order is not important need to be used. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 261] New: DOCBUG: --rekey
by bugzilla＠bugzilla.balabit.com 15 Apr '14

15 Apr '14

https://bugzilla.balabit.com/show_bug.cgi?id=261 Summary: DOCBUG: --rekey Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 In Example 2.3. (Using the rekey() option) from §2.10.1. (Specifying data types in value-pairs) the documentation claims: "The rekey() option can be used with the format-json template-function as well, using the following syntax: $(format-json --key .cee.* --rekey --shift 4 --add-prefix events.)" However, when using this syntax, the config parser fails with the following error: "Error parsing config, Error compiling template (Error parsing value-pairs: expected an equal sign in key=value pair)" The correct syntax (which works, as suggested by algeron on IRC) would be the following: $(format-json --rekey .cee.* --add-prefix events. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

Syslog-ng & Distributed File Systems??
by Jiann-Ming Su 08 Mar '14

08 Mar '14

Is there a version of syslog-ng that can properly use distributed filesystems like GlusterFS? For example, I have two nodes each running syslog-ng but also sharing a GlusterFS filesystem. Can those independent syslog-ng processes running on different servers write to the same file on the GlusterFS?

7 20

[Bug 260] New: exit with code 134
by bugzilla＠bugzilla.balabit.com 06 Mar '14

06 Mar '14

https://bugzilla.balabit.com/show_bug.cgi?id=260 Summary: exit with code 134 Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 I just did a reload on version 3.4.4 (with latest patches) and triggered 2013-11-05T21:24:33-08:00 host daemon.crit supervise/syslog-ng[25895]: Daemon exited due to a deadlock/signal/failure, restarting; exitcode='134' Is it possible that the return code of syslog-ng is somehow linked to the return code of a child program destination? core file sent to Balazs in private e-mail -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 11

patterndb
by Fabien Wernli 03 Mar '14

03 Mar '14

Hi, I'm currently organizing my patterndb files into something easily maintainable, and I ran into the following problem, best explained using an example: sshd and pam_unix. Some of sshd specific patterns reside in a specific ruleset, e.g. <ruleset name='sshd' id='A'> <patterns> <pattern>sshd</pattern </patterns> <rules> <rule> ... </rule> </rules> </ruleset> Some of sshd's messages come from pam_unix, e.g. modules/pam_unix/support.c, and are shared among other daemons: <ruleset name='pam_unix' id='B'> <patterns> <pattern>crond</pattern> <pattern>imap</pattern> <pattern>login</pattern> <pattern>pam</pattern> <pattern>pure-ftpd</pattern> <pattern>proftpd</pattern> <pattern>sshd</pattern> <pattern>su</pattern> <pattern>sudo</pattern> ... <patterns> <rules> <rule> ... </rule> </rules> </ruleset> The problem is, when a sshd message is parsed, even if it doesn't match any rules in ruleset 'A', ruleset 'B' isn't even considered for matching. So how do I properly organize my pdb tree, without duplicating rules in multiple source files? Thanks in advance

3 4

[RFC] procfile destination
by Balazs Scheidler 11 Feb '14

11 Feb '14

Hi, Based on a request I have now created a new destination driver, called "procfile". A working PoC is available on this branch: https://github.com/balabit/syslog-ng/tree/3.6/f/procfile It can be used to change kernel settings dynamically, based on a trigger within syslog-ng. The difference between procfile() and file(): - no template in the name of the file - processing is performed immediately as read by the source, no queueing is performed (e.g. no loss is possible, but takes CPU time from the source) - opens and closes the destination file after each and every write, writes to offset zero e.g. it doesn't append further data. I'd be grateful if you could provide feedback around these questions or the feature in general: - name of the driver, right now it is called procfile as it was the original use-case, however it works with stuff in /dev too, and might work with simple named pipes. - right now the file must exist and is not created (e.g. it really requires an existing file); do we need that? - do we need append mode in some cases - whether we need append mode (if we think of general files, this would probably be needed) Thanks in advance, Bazsi

5 6

[Bug 269] New: some code doesn't follow glib recomendations
by bugzilla＠bugzilla.balabit.com 04 Feb '14

04 Feb '14

https://bugzilla.balabit.com/show_bug.cgi?id=269 Summary: some code doesn't follow glib recomendations Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: iippolitov(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 I'm using rhel 5.9 and syslog-ng 3.5.3. tests/unit/test_nvtable fails on line 69. It is a know bug: https://bugzilla.redhat.com/show_bug.cgi?id=714409#c4 . As per https://bugzilla.redhat.com/show_bug.cgi?id=716447 : >From the glib2 documentation: ... Note that neither keys nor values are copied when inserted into the GHashTable, so they must exist for the lifetime of the GHashTable. This means that the use of static strings is OK, but temporary strings (i.e. those created in buffers and those returned by GTK+ widgets) should be copied with g_strdup() before being inserted. If keys or values are dynamically allocated, you must be careful to ensure that they are freed when they are removed from the GHashTable, and also when they are overwritten by new insertions into the GHashTable. It is also not advisable to mix static strings and dynamically-allocated strings in a GHashTable, because it then becomes difficult to determine whether the string should be freed. ... The code in lib/nv_table.c uses a pointer to a string that is changed during a program execution to insert values into GHashTable. This causes some unit tests to fail at least in rhel5. nv_registry_alloc_handle was obviously fixed to g_strdup() string from function parameters, while nv_registry_add_alias still uses a pointer the function gets as a parameter: Here is a workaround: void nv_registry_add_alias(NVRegistry *self, NVHandle handle, const gchar *alias) { g_static_mutex_lock(&nv_registry_lock); - g_hash_table_insert(self->name_map, (gchar*)alias, GUINT_TO_POINTER((glong) handle)); + gchar * st_alias = g_strdup(alias); + g_hash_table_insert(self->name_map, st_alias, GUINT_TO_POINTER((glong) handle)); g_static_mutex_unlock(&nv_registry_lock); } AFAIK, g_hash_table_insert is used widely around syslog-ng code. Could someone please look it through and ensure that any value added into gHashTable is not changed during hashtable lifetime? Feel free to contact me via email or gtalk to find out any details I might miss. Thanks in advance. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

Help with long log messages
by Edward Gleeck 03 Feb '14

03 Feb '14

Syslog-ng seems to only allow up to 8192 bytes of log_msg_size and after which it splits up the log message into multiple log messages. Setting this up in the global options or on the source option to use more than 8192 does not seem to work. I was wondering if there are other options that I need to put in order that very long logs aren't split up. I realize this might be a very rare case on the need for long log messages, but the application logging was designed poorly and we need this functionality currently while the logging is being fixed. In looking at the source code, it seems that the log_msg_size is stored as a gint type, which in my recollection allows me to store up to +32787 right? If the max I could put is 8192, then I guess I'll have to come up with something else to process the split logs, otherwise any help would be appreciated.

2 1

[Bug 267] New: $YEAR on event in RFC 3164 format
by bugzilla＠bugzilla.balabit.com 31 Jan '14

31 Jan '14

https://bugzilla.balabit.com/show_bug.cgi?id=267 Summary: $YEAR on event in RFC 3164 format Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: dtgarry(a)hotmail.com Type of the Report: --- Estimated Hours: 0.0 I am using syslog-ng 3.3.4 Revision: /3.3.4.dfsg-2ubuntu1 Revision: /3.3.4.dfsg-2ubuntu1 I have a problem on the macro $YEAR when the event is in RFC 3164 format sometime the $YEAR extract the correct value of year, but not always. I test this problem on a computer with the current date, 9 january 2014, sending some event with these dates: - Dec 31 23:59:59, syslog-ng return the 2013 year; - Jan 1 00:00:01, syslog-ng return the 2014 year; - Feb 1 00:00:01, syslog-ng return the 2014 year; - Mar 1 00:00:01, syslog-ng return the 2013 year; how syslog-ng extract the year from an event when there is no year? Is it possible to configure correctly the syslog-ng? for testing I have configured syslog-ng to call my program "mytest" like this: destination d_tst { program("/etc/syslog-ng/sbin/mytest" template("<$PRI>$DATE $HOST $PROGRAM $MSGONLY $HOST $FACILITY $PRIORITY $LEVEL $TAG $YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC $PROGRAM\n") template-escape(yes) ); }; on mytest program I log what syslog-ng has parsed; today, that is the 9th january 2014, I have sent to syslog-ng this string: <131>Dec 31 23:59:59 hostTest processTest some test on mytest I will receive: <131>Dec 31 23:59:59 hostTest processTest some test hostTest local0 err err 83 2013-12-31 23:59:59 processTest note the year is 2013 If I send this: <131>Jan 1 00:00:01 hostTest processTest some test I will receive: <131>Jan 1 00:00:01 hostTest processTest some test hostTest local0 err err 83 2014-01-01 00:00:01 processTest now the year is 2014 But I send this event that is in the future: <131>Feb 1 00:00:01 hostTest processTest some test I will receive: <131>Feb 1 00:00:01 hostTest processTest some test hostTest local0 err err 83 2014-02-01 00:00:01 processTest the year is 2014 If I send an event of march: <131>Mar 1 00:00:01 hostTest processTest some test I will receive: <131>Mar 1 00:00:01 hostTest processTest some test hostTest local0 err err 83 2013-03-01 00:00:01 processTest where the year is 2013 thank you. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

Re: [syslog-ng] Issue with pdbtool patterndb and percent symbols
by Mark Shetka 29 Jan '14

29 Jan '14

My Zsh shell was somehow interfering with pdbtool and the % symbols. I reran with Bash and the patterns match just fine, so I'm all good. Thanks! -- Mark Shetka Information Technology Systems & Services University of Minnesota - Duluth (218) 726-7682 On Wed, Jan 29, 2014 at 8:40 AM, Mark Shetka <mshetka(a)d.umn.edu> wrote: > I am setting up some patterns to parse Cisco syslog messages. I noticed > that pdbtool will not complete if I have a "%F" anywhere in the string. > > Example log message: > %FWSM-1-109006: Authentication failed for user 'test' from > 131.212.1.1/43250 to 10.1.1.1/22 on interface management > > This does not complete: > pdbtool match -p cisco.xml -M "%FWSM-1-109006: Authentication failed for > user 'test' from 131.212.1.1/43250 to 10.1.1.1/22 on interface management" > > Nor does simply %F: > pdbtool match -p cisco.xml -M "%F" > > It is fine without the %: > pdbtool match -p cisco.xml -M "FWSM-1-109006: Authentication failed for > user 'test' from 131.212.1.1/43250 to 10.1.1.1/22 on interface management" > > MESSAGE=FWSM-1-109006: Authentication failed for user 'test' from > 131.212.1.1/43250 to 10.1.1.1/22 on interface management > .classifier.class=login > .classifier.rule_id=5cfbcb23-cfe4-4120-85c1-918df65c0edc > usracct.username=test > usracct.device=131.212.1.1 > usracct.service=22 > usracct.type=login > usracct.sessionid= > usracct.application= > secevt.verdict=REJECT > TAGS=.classifier.login,usracct,secevt > > It also seems to have issues with "%S", although not quite in the same > way. Any ideas what could be causing this? > > Mark > > > -- > Mark Shetka > Information Technology Systems & Services > University of Minnesota - Duluth > (218) 726-7682 >

1 0