- syslog-ng - lists.balabit.hu

Trouble with microseconds
by Clayton Dukes 06 Dec '13

06 Dec '13

Hi, I have a system configured to microseconds logging, but for most of entries it's always zero - actually the only program which logs it correctly is the syslog-ng itself. @1386239689.000000 13 blah @1386239859.947724 45 syslog-ng shutting down; version='3.5.2' @1386239860.304194 45 syslog-ng starting up; version='3.5.2' @1386239860.337325 45 EOF on control channel, closing connection; @1386239865.000000 13 blah Anyone have an idea what I'm doing wrong? Here's the config: destination d_mydest{ program("/somepath" template("@${UNIXTIME:--}\t${PRI:--}\t${MSGONLY}\n") flags(no_multi_line) flush_timeout(100) frac_digits(6) ); }; ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 2

syslog - sql error
by vijay 03 Dec '13

03 Dec '13

HI All, Iam using syslog-ng3.2.5 rpm in centos x86_64 environment.Past days it is working well.Suddenly iam seeing a problem in writing to mysql Error running SQL query; type='mysql', host='192.168.9.68', port='3306', user='cygnet', database='CygNetDB', error='1072: Key column \'date\' doesn\'t exist in table', query='CREATE INDEX SYSLOGS_20131130_date_idx ON SYSLOGS_20131130 (date)' Error adding missing index; table='SYSLOGS_20131130', column='date' Error running SQL query; type='mysql', host='192.168.9.68', port='3306', user='cygnet', database='CygNetDB', error='1170: BLOB/TEXT column \'facility\' used in key specification without a key length', query='CREATE INDEX SYSLOGS_20131130_facility_idx ON SYSLOGS_20131130 (facility)' Error adding missing index; table='SYSLOGS_20131130', column='facility' vijay

2 1

[PATCH] rewrite_rule_clone: unit tests for clone of rewrite rules are implemented
by Juhász Viktor 02 Dec '13

02 Dec '13

These tests evince some problems about clone implementation. The clone implementations didn't increased the reference of the condition, and it leaks the template which is used by the constructors (set, subst). These errors are corrected in this patch. Signed-off-by: Juhász Viktor <jviktor(a)balabit.hu> --- lib/rewrite/Makefile.am | 2 + lib/rewrite/rewrite-set-tag.c | 2 +- lib/rewrite/rewrite-set.c | 4 +- lib/rewrite/rewrite-subst.c | 4 +- lib/rewrite/tests/Makefile.am | 9 ++++ lib/rewrite/tests/test_clone.c | 96 ++++++++++++++++++++++++++++++++++++++++++ libtest/test_case.h | 37 ++++++++++++++++ 7 files changed, 149 insertions(+), 5 deletions(-) create mode 100644 lib/rewrite/tests/Makefile.am create mode 100644 lib/rewrite/tests/test_clone.c create mode 100644 libtest/test_case.h diff --git a/lib/rewrite/Makefile.am b/lib/rewrite/Makefile.am index b111935..bc6547f 100644 --- a/lib/rewrite/Makefile.am +++ b/lib/rewrite/Makefile.am @@ -21,3 +21,5 @@ BUILT_SOURCES += \ lib/rewrite/rewrite-expr-grammar.h EXTRA_DIST += lib/rewrite/rewrite-expr-grammar.ym + +include lib/rewrite/tests/Makefile.am diff --git a/lib/rewrite/rewrite-set-tag.c b/lib/rewrite/rewrite-set-tag.c index 45d3cbf..154973d 100644 --- a/lib/rewrite/rewrite-set-tag.c +++ b/lib/rewrite/rewrite-set-tag.c @@ -56,7 +56,7 @@ log_rewrite_set_tag_clone(LogPipe *s) log_rewrite_init(&cloned->super); cloned->super.super.clone = log_rewrite_set_tag_clone; cloned->super.process = log_rewrite_set_tag_process; - cloned->super.condition = self->super.condition; + cloned->super.condition = self->super.condition ? filter_expr_ref(self->super.condition) : NULL; cloned->tag_id = self->tag_id; cloned->value = self->value; diff --git a/lib/rewrite/rewrite-set.c b/lib/rewrite/rewrite-set.c index 5b117ff..de42c8d 100644 --- a/lib/rewrite/rewrite-set.c +++ b/lib/rewrite/rewrite-set.c @@ -56,9 +56,9 @@ log_rewrite_set_clone(LogPipe *s) LogRewriteSet *self = (LogRewriteSet *) s; LogRewriteSet *cloned; - cloned = (LogRewriteSet *) log_rewrite_set_new(log_template_ref(self->value_template)); + cloned = (LogRewriteSet *) log_rewrite_set_new(self->value_template); cloned->super.value_handle = self->super.value_handle; - cloned->super.condition = self->super.condition; + cloned->super.condition = self->super.condition ? filter_expr_ref(self->super.condition) : NULL; return &cloned->super.super; } diff --git a/lib/rewrite/rewrite-subst.c b/lib/rewrite/rewrite-subst.c index f16afae..385fb82 100644 --- a/lib/rewrite/rewrite-subst.c +++ b/lib/rewrite/rewrite-subst.c @@ -101,10 +101,10 @@ log_rewrite_subst_clone(LogPipe *s) LogRewriteSubst *self = (LogRewriteSubst *) s; LogRewriteSubst *cloned; - cloned = (LogRewriteSubst *) log_rewrite_subst_new(log_template_ref(self->replacement)); + cloned = (LogRewriteSubst *) log_rewrite_subst_new(self->replacement); cloned->matcher = log_matcher_ref(self->matcher); cloned->super.value_handle = self->super.value_handle; - cloned->super.condition = self->super.condition; + cloned->super.condition = self->super.condition ? filter_expr_ref(self->super.condition) : NULL; return &cloned->super.super; } diff --git a/lib/rewrite/tests/Makefile.am b/lib/rewrite/tests/Makefile.am new file mode 100644 index 0000000..c5f740d --- /dev/null +++ b/lib/rewrite/tests/Makefile.am @@ -0,0 +1,9 @@ +lib_rewrite_tests_TESTS = lib/rewrite/tests/test_clone + +check_PROGRAMS += ${lib_rewrite_tests_TESTS} + +lib_rewrite_tests_test_clone_CFLAGS = $(TEST_CFLAGS) \ + -I${top_srcdir}/lib/rewrite/tests + +lib_rewrite_tests_test_clone_LDADD = $(TEST_LDADD) \ + $(PREOPEN_SYSLOGFORMAT) diff --git a/lib/rewrite/tests/test_clone.c b/lib/rewrite/tests/test_clone.c new file mode 100644 index 0000000..8bb6d90 --- /dev/null +++ b/lib/rewrite/tests/test_clone.c @@ -0,0 +1,96 @@ +#include "apphook.h" +#include "plugin.h" +#include "testutils.h" +#include "lib/rewrite/rewrite-expr.h" +#include "lib/rewrite/rewrite-set.h" +#include "lib/rewrite/rewrite-subst.h" +#include "lib/rewrite/rewrite-set-tag.h" +#include "lib/filter/filter-cmp.h" +#include "test_case.h" + +typedef struct _RewriteTestCase { + TestCase super; + LogRewrite *test_set_rewrite; + LogRewrite *test_subst_rewrite; + LogRewrite *test_tag_rewrite; +} RewriteTestCase; + +void +rewrite_testcase_setup(TestCase *s) +{ + RewriteTestCase *self = (RewriteTestCase *)s; + configuration = cfg_new(0x305); + + FilterExprNode *set_condition = fop_cmp_new(log_template_new(configuration, "OP1"), log_template_new(configuration, "OP2"), 10520); + FilterExprNode *subst_condition = fop_cmp_new(log_template_new(configuration, "OP1"), log_template_new(configuration, "OP2"), 10520); + FilterExprNode *set_tag_condition = fop_cmp_new(log_template_new(configuration, "OP1"), log_template_new(configuration, "OP2"), 10520); + + LogTemplate *test_template = log_template_new(configuration, "TEST"); + + self->test_set_rewrite = log_rewrite_set_new(test_template); + self->test_subst_rewrite = log_rewrite_subst_new(test_template); + + self->test_tag_rewrite = log_rewrite_set_tag_new("test_tag", FALSE); + + log_rewrite_set_condition(self->test_set_rewrite, set_condition); + log_rewrite_set_condition(self->test_subst_rewrite, subst_condition); + log_rewrite_set_condition(self->test_tag_rewrite, set_tag_condition); + + log_rewrite_subst_set_matcher(self->test_subst_rewrite, log_matcher_posix_re_new()); + + log_template_unref(test_template); +} + +void +rewrite_testcase_teardown(TestCase *s) +{ + RewriteTestCase *self = (RewriteTestCase *)s; + log_pipe_unref((LogPipe *)self->test_subst_rewrite); + log_pipe_unref((LogPipe *)self->test_set_rewrite); + log_pipe_unref((LogPipe *)self->test_tag_rewrite); + cfg_free(configuration); +} + +void +test_clone(LogRewrite *rule) +{ + LogPipe *cloned_rule = log_pipe_clone((LogPipe *)rule); + assert_guint32(rule->condition->ref_cnt, 2, ASSERTION_ERROR("Bad reference number of condition")); + log_pipe_unref(cloned_rule); + assert_guint32(rule->condition->ref_cnt, 1, ASSERTION_ERROR("Bad reference number of condition")); +} + +void +test_reference_on_set_condition_cloned(TestCase *s) +{ + RewriteTestCase *self = (RewriteTestCase *) s; + test_clone(self->test_set_rewrite); +} + +void +test_reference_on_subst_condition_cloned(TestCase *s) +{ + RewriteTestCase *self = (RewriteTestCase *) s; + test_clone(self->test_subst_rewrite); +} + +void +test_reference_on_tag_condition_cloned(TestCase *s) +{ + RewriteTestCase *self = (RewriteTestCase *) s; + test_clone(self->test_tag_rewrite); +} + +int +main(int argc, char **argv) +{ + app_startup(); + putenv("TZ=MET-1METDST"); + tzset(); + + start_grabbing_messages(); + RUN_TESTCASE(RewriteTestCase, rewrite_testcase, test_reference_on_set_condition_cloned); + RUN_TESTCASE(RewriteTestCase, rewrite_testcase, test_reference_on_subst_condition_cloned); + RUN_TESTCASE(RewriteTestCase, rewrite_testcase, test_reference_on_tag_condition_cloned); + stop_grabbing_messages(); +} diff --git a/libtest/test_case.h b/libtest/test_case.h new file mode 100644 index 0000000..4a49c9a --- /dev/null +++ b/libtest/test_case.h @@ -0,0 +1,37 @@ +/* + * test_case.h + * + * Created on: Nov 27, 2013 + * Author: jviktor + */ + +#ifndef TEST_CASE_H_ +#define TEST_CASE_H_ + +typedef struct _TestCase TestCase; + +struct _TestCase { + void (*setup)(TestCase *self); + void (*run)(TestCase *self); + void (*teardown)(TestCase *self); +}; + + +static void +run_test_case(TestCase *tc) +{ + if (tc->setup) tc->setup(tc); + tc->run(tc); + if (tc->teardown) tc->teardown(tc); +} + +#define RUN_TESTCASE(class_name, test_case_prefix, test_function) { \ + class_name *tc = g_new0(class_name, 1); \ + tc->super.setup = test_case_prefix ## _setup; \ + tc->super.teardown = test_case_prefix ## _teardown; \ + tc->super.run = test_function;\ + run_test_case(&tc->super); \ + g_free(tc); \ +} + +#endif /* TEST_CASE_H_ */ -- 1.8.3.2

2 1

[Bug 264] New: docbug: all-hard-macros does not exist in value-pairs
by bugzilla＠bugzilla.balabit.com 02 Dec '13

02 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=264 Summary: docbug: all-hard-macros does not exist in value-pairs Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 using this yields "Error parsing config, Error compiling template (Error parsing value-pairs: unknown scope all-hard-macros)" Indeed, the key "all-hard-macros" is not being referenced in 'lib/value-pairs.c' -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

Syslog-ng OSE Question.
by Malkin, Vlad 02 Dec '13

02 Dec '13

Hi, Is it possible to have syslog-ng client to invoke an external (to it) software to acknowledge when each individual log statement is sent to the server? Background: logs should not be lost. The system needs to store logs internally until they are transmitted to the server (use of TCP transport provides a level of reliable delivery). After each log statement is transmitted by syslog-ng client, it needs to be removed from internal store to preserve allocated space. Thank you for your help! --Vlad

5 6

syslog-ng 3.5.2 has been released
by devel＠balabit.hu 29 Nov '13

29 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.5.2 SUMMARY : new stable release DATE : Nov 29, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.5.2) has been released. For latest fixes in the 3.5.x feature branch you are recommended to upgrade to this version. CHANGES: 3.5.2 Fri, 29 Nov 2013 17:09:40 +0100 This is the first bug-fix release for the 3.5.x series, with a number of significant fixes over the original 3.5.1 release. Upgrading to this release is highly recommended. Bugfixes ======== * The system() source has been fixed on Solaris and HP-UX: it correctly recognises the system on Solaris, and uses /dev/log (and not /dev/pipe) on HP-UX. * The system() source has much stricter requirements to select /dev/kmsg on Linux, so it won't choose it over /proc/kmsg unless the kernel really does support it. * The syslog-format() module has been fixed to only set the PROGRAM to "kernel" if all of the previous conditions apply, and the message is a local message. * The linux-kmsg-format parser (and system() on Linux 3.5+ as a consequence) will correctly calculate the timestamp of kernel messages on 32-bit systems now. * The sun-streams() source has been fixed, it compiles and works again. * A very rare and short race condition has been fixed in the pipe() source and destination modules, which could leave the pipe available with elevated permissions for a short time. * The SQL, MongoDB and SMTP modules were fixed to not cause a segmentation fault when loaded before the socket module. * A crash was fixed in the no-tls build of the socket destination driver. * It is now possible to specify the host() and port() parameters for the redis() destination. * Various stability fixes, such as: - A use-after-free condition was corrected in the queue backlog acknowledgement code. - Always use setsid() when detaching, even if we're not detaching from a TTY. - Memory leaks were fixed in the persist state handling code. - Changing stats-level() and reloading syslog-ng produced a crash, which has been fixed. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Lecuyer Balazs Scheidler <bazsi(a)balabit.hu> Daniel Gados <gdani(a)balabit.hu> Gergely Nagy <algernon(a)balabit.hu> Howard Tsai <htsai(a)skytap.com> Imre Lazar <imre(a)balabit.hu> Jakub Wilk <jwilk(a)debian.org> Laszlo Budai <lbudai(a)balabit.hu> Martin Bagge / brother <brother(a)bsnet.se> Matyas Koszik <koszik(a)atw.hu> Oscar Muñoz Otto Berger <otto(a)bergerdata.de> Viktor Juhasz <jviktor(a)balabit.hu> Viktor Tusa <tusa(a)balabit.hu> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

syslog-ng 3.4.6 has been released
by devel＠balabit.hu 27 Nov '13

27 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.6 SUMMARY : new stable release DATE : Nov 27, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.6) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.6 Wed, 27 Nov 2013 16:21:47 +0100 This is the fifth bug-fix release for the 3.4.x series, with a number of significant fixes over the 3.4.5 release. Upgrading to this release is highly recommended. Bugfixes ======== * The system() source has been fixed on Solaris and HP-UX: it correctly recognises the system on Solaris, and uses /dev/log (and not /dev/pipe) on HP-UX. * The sun-streams() source has been fixed, it compiles and works again. * A very rare and short race condition has been fixed in the pipe() source and destination modules, which could leave the pipe available with elevated permissions for a short time. * The SQL, MongoDB and SMTP modules were fixed to not cause a segmentation fault when loaded before the socket module. * The syslog-format() module has been fixed to only set the PROGRAM to "kernel" if all of the previous conditions apply, and the message is a local message. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Lecuyer Balazs Scheidler <bazsi(a)balabit.hu> Gergely Nagy <algernon(a)balabit.hu> Imre Lazar <imre(a)balabit.hu> Laszlo Budai <lbudai(a)balabit.hu> Matyas Koszik <koszik(a)atw.hu> Oscar Muñoz DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Mongodb
by Nilo Cabeza Diaz 27 Nov '13

27 Nov '13

how send log to mongodb??? this is what I do in syslog-ng.conf and does nothing for me destination d_dw_v3_mongo { mongodb( host("localhost") port(27017) database("syslog") collection("messages") value-pairs( key("ISODATE") key("PRIORITY") pair("svc_id" "${.SDATA.syslog(a)0.svcid}") pair("tid" "${.SDATA.syslog(a)0.tid}") pair("act_id" "${.SDATA.syslog(a)0.id}") pair("act_eid" "${.SDATA.syslog(a)0.eid}") key("MESSAGE") key("SDATA") key("HOST") ); };

2 1

Error of parser
by Nilo Cabeza Diaz 26 Nov '13

26 Nov '13

hello: look this error Failed to start Syslog-NG server : * Starting system logging syslog-ng syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted' Error parsing config, syntax error, unexpected KW_DESTINATION, expecting ';' in /etc/syslog-ng/syslog-ng.conf at line 94, column 1: destination d_mongodb { ^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng ...fail!

2 1

syslog-ng 3.6 rebase
by Balazs Scheidler 25 Nov '13

25 Nov '13

Hi, After integrating everything I could, I would like to rebase the current master to merge fixups into the base patches. This is the last such rebase in 3.6. You can see the rebased series in the "next" branch, I intend to reset the master branch to that history tomorrow evening. the HEAD of both branches are identical, only the history differs. In the meantime, you can submit patches either the current "master" or "next" branches, I'll integrate stuff to "next" myself. Thanks. -- Bazsi

1 1