November 2011 - syslog-ng - lists.balabit.hu

An interesting assert in 3.3.1
by Gergely Nagy 01 Dec '11

01 Dec '11

A while ago I noticed that my syslog-ng 3.3 segfaults every time I boot up my desktop machine at home (but then it gets restarted by the supervisor), and once its booted up, I can't reproduce the problem anymore. I assume it has something to do with the system being in a reasonably early state, or the load being particularly high. While I didn't have time to debug it further, I'd like to record the current state of investigation here: Core was generated by `/usr/sbin/syslog-ng -p /var/run/syslog-ng.pid --enable-core'. Program terminated with signal 6, Aborted. #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007fd927cc9680 in *__GI_abort () at abort.c:92 #2 0x00007fd928ae6e31 in g_assertion_message (domain=<optimized out>, file=<optimized out>, line=<optimized out>, func=0x7fd929609dc0 "log_writer_work_perform", message=0x7fd920007a60 "assertion failed: ((self->super.flags & PIF_INITIALIZED) != 0)") at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1358 #3 0x00007fd928ae73d0 in g_assertion_message_expr (domain=0x0, file=0x7fd929609acb "../../../lib/logwriter.c", line=128, func=0x7fd929609dc0 "log_writer_work_perform", expr=<optimized out>) at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1369 #4 0x00007fd9295d3570 in log_writer_work_perform (s=0x1d68080) at ../../../lib/logwriter.c:128 #5 0x00007fd9295d3897 in log_writer_io_flush_output (s=0x1d68080) at ../../../lib/logwriter.c:198 #6 0x00007fd9295ff986 in iv_run_active_list (active=0x7fff3da27700) at ../../../../../lib/ivykis/lib/iv_main.c:226 #7 0x00007fd9295ffb0d in iv_main () at ../../../../../lib/ivykis/lib/iv_main.c:269 #8 0x00007fd9295d8082 in main_loop_run () at ../../../lib/mainloop.c:722 #9 0x0000000000401a22 in main (argc=1, argv=0x7fff3da27878) at ../../../syslog-ng/main.c:260 (gdb) The particular writer is a program destination (sshguard), and at that point, it's probably not running yet. (For the record, I don't have threaded(yes) in my config at the moment). I'll look into this further, as time permits. -- |8]

1 1

Bug in rewrite logic, 3.3.3
by Dave Rawks 30 Nov '11

30 Nov '11

I've got two different log paths defined that have the same rewrite included in them. The second log path always causes the rewrite to overwrite the incorrect macro value. > ######################## > # Destinations > ######################## > > destination dn_json { > tcp( "logstash" > port(10514) > so_keepalive(yes) > template("$(format_json --pair \@timestamp=\"$ISODATE\" message=\"$MESSAGE\" priority=\"$PRIORITY\" severity=\"$SEVERITY\" facility=\"$FACILITY\" program=\"$PROGRAM\" pid=\"$PID\" date=\"$R_DATE\" host=\"$HOST\" )\n") > ); > }; > > ######################## > #Rewrites > ######################## > > rewrite r_no_pid{ > set("0", value("PID") condition("$PID" =="")); > }; > > ######################## > # Log paths > # order matters if you use "flags(final);" to mark the end of processing in a > # "log" statement. ORDER MATTERS! > ####################### > > > Send all log events on to logstash > log { > source(s_remote); > rewrite(r_no_pid); > destination(dn_json); > }; > > log { > source(s_netloghost); > rewrite(r_no_pid); > destination(df_netloghost); > destination(dn_opennms); > destination(dn_json); > }; If the above configuration is used the events that pass through the second log path have their MESSAGE macro overwritten as "0" instead of their PID macro as is defined in the r_no_pid rewrite rule. The events that pass through the first logpath behave as expected. If I comment out the first logpath then the second behaves as expected. -Dave

2 1

distro URLs for syslog-ng
by Peter Czanik 29 Nov '11

29 Nov '11

Hello, Please take a look at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… It has now information for many Linux distributions, where up-to-date syslog-ng package information is available, unofficial repositories with latest and/or backported syslog-ng, bugreporting, etc. Let me know, if you have info or links to additional distributions or repositories! Bye, -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 147] New: missing share/xsd/patterndb-4.xsd schema definition
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=147 Summary: missing share/xsd/patterndb-4.xsd schema definition Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Is OSE 3.3.3 I downloaded the source and installed, but only go schema definitions for database formats 1,2 and 3. The manual for Syslog-ng 3.3.3 has a secion on "What's new in the syslog-ng pattern database format V4" so I can only assume that the version 4 schema should be present. Can you provide one? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 148] New: .classifier. unknown not set when patterndb unable to classify message
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=148 Summary: .classifier.unknown not set when patterndb unable to classify message Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 filter f_tag_unknown { tags(".classifier.unknown"); }; Using this filter results in no messages going to a destination, but using filter f_tag_unknown { not tags(".classifier.system"); }; logs all of the unclassified messages. The problem is that I am going to have a lot more than just .classifier.system in my pattern database, so this is not a viable workaround. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 145] New: Comilation on Solaris 10 with gcc 3.4.3 fails
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=145 Summary: Comilation on Solaris 10 with gcc 3.4.3 fails Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Solaris Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: andyh(a)nominet.org.uk Type of the Report: bug Estimated Hours: 0.0 Hi, Trying to compile 3.3.3 from source on Solaris 10 with gcc 3.4.4 and get this: libtool: compile: /usr/sfw/bin/gcc -DHAVE_CONFIG_H -I. -I.. -D_GNU_SOURCE -I../lib/include -I../lib/include -I../lib -I../modules/include -Wall -m64 -mcpu=v9 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event.lo -MD -MP -MF .deps/iv_event.Tpo -c iv_event.c -fPIC -DPIC -o iv_event.o iv_event.c: In function `iv_event_run_pending_events': iv_event.c:74: error: unrecognizable insn: (insn 80 79 81 2 (set (reg:DI 139) (lo_sum:DI (reg:DI 139) (unspec:DI [ (symbol_ref:DI ("__tls") [flags 0xa] <var_decl fefbb480 __tls>) ] 0))) -1 (nil) (nil)) iv_event.c:74: internal compiler error: in extract_insn, at recog.c:2083 Please submit a full bug report, with preprocessed source if appropriate. See <URL:http://gcc.gnu.org/bugs.html> for instructions. It appears (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21613) that there may be a workaround, but I'm not a developer and can't understand hwo to apply the workaround -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Problems with custom patterndb
by Clements, Frank 29 Nov '11

29 Nov '11

Hello Everyone! I've been trying to get some custom patterns put together to do some log correlation and I'm having one hell of a time getting a working pattern. I think I need a second, third, forth set of eyes on this ... Any help is appreciated! Message: Nov 25 12:02:27 GENERATED NAT-UDP-C: 192.168.107.132:12260 -> 207.44.101.104:12260 to 48.70.67.223:940 Pattern: @IPvANY:.dict.insideAddr@:@NUMBER:.dict.insidePort@ -> @IPvANY:.dict.outsideAddr@:@NUMBER:.dict.outsidePort@ to @IPvANY:.dict.destAddr@:@NUMBER:.dict.destPort@ I've looked at a few examples from the community patterns, but nothing in this stands out as being "wrong". Thanks - Frank W Clements

2 4

Multi-threaded in open source?
by Avleen Vig 25 Nov '11

25 Nov '11

Hi everyone, I'm running an infrastructure with several hundred hosts logging many hundreds of Gb of logs to a central server. Syslog-ng on the central server is using 100% CPU on a single core, and I see here: http://bazsi.blogs.balabit.com/2011/07/syslog-ng-multithreaded-performance/ that there is some multi-threading option available! Is this only in the PE version or is there an OSE edition with this feature?

5 6

[PATCH (3.4)] value-pairs: Fix the error handling in the command-line parser.
by Gergely Nagy 25 Nov '11

25 Nov '11

When an error occurs during command-line parsing, do not attempt to free argv: it's not owned by us anymore. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/value-pairs.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/lib/value-pairs.c b/lib/value-pairs.c index fcaa401..c58d294 100644 --- a/lib/value-pairs.c +++ b/lib/value-pairs.c @@ -465,7 +465,6 @@ value_pairs_new_from_cmdline (GlobalConfig *cfg, { value_pairs_free (vp); g_option_context_free (ctx); - g_free (argv); return NULL; } g_option_context_free (ctx); -- 1.7.7.3

2 1

[PATCH] logproto: Fix compilation on systems without IOV_MAX.
by Gergely Nagy 25 Nov '11

25 Nov '11

Since IOV_MAX is not defined on Hurd its usage is made conditional. Errors are handled by g_malloc0() which aborts on failure, so no return value has to be checked. Reported-By: Svante Signell <svante.signell(a)telia.com> Patch-From: Svante Signell <svante.signell(a)telia.com> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/logproto.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/lib/logproto.c b/lib/logproto.c index 282064c..16ca0d7 100644 --- a/lib/logproto.c +++ b/lib/logproto.c @@ -420,9 +420,11 @@ log_proto_file_writer_new(LogTransport *transport, gint flush_lines) if (flush_lines == 0) /* the flush-lines option has not been specified, use a default value */ flush_lines = 1; +#ifdef IOV_MAX if (flush_lines > IOV_MAX) /* limit the flush_lines according to the current platform */ flush_lines = IOV_MAX; +#endif /* allocate the structure with the proper number of items at the end */ LogProtoFileWriter *self = (LogProtoFileWriter *)g_malloc0(sizeof(LogProtoFileWriter) + sizeof(struct iovec)*flush_lines); -- 1.7.7.3

2 1