- syslog-ng - lists.balabit.hu

Re: [syslog-ng] libdbi too old?
by Len Conrad 02 Jun '11

02 Jun '11

OK. one of your msgs had the right IP and that's what I used at work just now. I'll see if it works from home. ---------- Original Message ---------------------------------- From: John Kristoff <jtk(a)cymru.com> Reply-To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Date: Thu, 2 Jun 2011 08:38:27 -0500 >On Sun, 22 May 2011 13:32:31 +0200 >Balazs Scheidler <bazsi(a)balabit.hu> wrote: > >> It's not difficult to compile from sources however, and the exact >> libdbi version that we use in PE binary packages can be found on >> git.balabit.hu/bazsi/libdbi.git and libdbi-drivers.git. > >>From my notes, here are the steps used to build a recent version of >syslog-ng on a CentOS box with libdbi and specifically pgsql support. > > # ld.so.conf might need to contain: > /usr/local/lib > /usr/local/lib/dbd > /usr/local/pgsql/lib > > sudo yum install gtk2-devel > sudo yum install libdbi-devel > sudo yum install libdbi-dbd-pgsql > > cd ~/workdir > wget http://www.balabit.com/downloads/files/eventlog/0.2/eventlog_0.2.12.tar.gz > tar zxvf eventlog_0.2.12.tar.gz > cd eventlog-0.2.12 > ./configure > make > sudo make install > > cd ~/workdir > # syslog-ng 3.2.x needs newer libdbi than what CentOS has, build from source > # get http://sourceforge.net/projects/libdbi/files/libdbi/libdbi-0.8.3/ > cd libdb-0.8.3 > ./configure --disable-docs && make && sudo make install > make > sudo make install > > cd ~/workdir > # syslog-ng 3.2.x needs newer libdbi-drivers than what CentOS has, build from source > # get http://sourceforge.net/projects/libdbi-drivers/files/libdbi-drivers/libdbi-… > tar zxvf libdb-drivers-0.8.3-1.tar.gz > cd libdbi-drivers-0.8.3-1 > ./configure --with-dbi-incdir=/usr/local/include/dbi --with-dbi-libdir=/usr/local/lib --with-pgsql-dir=/usr/local/pgsql --with-pgsql --disable-docs > make > sudo make install > > cd ~/workdir > PKG_CONFIG_PATH=/usr/lib64/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/ssl/lib/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig > export PKG_CONFIG_PATH > wget http://www.balabit.com/downloads/files/syslog-ng/sources/3.2.4/source/syslo… > tar zxvf syslog-ng_3.2.4.tar.gz > cd syslog-ng-3.2.4 > LIBDBI_LIBS="-L/usr/local/lib -ldbi" LIBDBI_CFLAGS="-I/usr/local/include -I/usr/local/include/dbi" ./configure --enable-ssl --enable-sql > # NOTE: there is a bug in syslog-ng 3.2.4 and below and it should be patched. For details, see this bug_id: https://bugzilla.balabit.com/show_bug.cgi?id=120 > make > sudo make install > >John >______________________________________________________________________________ >Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >FAQ: http://www.balabit.com/wiki/syslog-ng-faq > >

1 0

libdbi too old?
by John Kristoff 02 Jun '11

02 Jun '11

I ran into a problem upgrading from an OSE syslog-ng 3.1.1 to OSE syslog 3.2.4 on a CentOS 64-bit box running under a VM. When starting 3.2.4, here is what I got: Starting syslog-ng... /usr/lib64/dbd/libdbdpgsql.so: undefined symbol: _dbd_parse_datetime libdbi: Failed to load driver: /usr/lib64/dbd/libdbdpgsql.so The database access library (DBI) reports no usable SQL drivers, perhaps DBI drivers are not installed properly; Error initializing dest driver; dest='d_foo', id='d_foo#0' Error initializing message pipeline; The following relevant packages are installed: libdbi-dbd-pgsql.x86_64 0.8.1a-1.2.2 libdbi-devel.i386 0.8.1-2.1 libdbi-devel.x86_64 0.8.1-2.1 It looks like I have an libdbi that is probably just too old? Not an unheard of issue when using CentOS and RedHat, but if there is an easy work around, I'd be interested in hearing about any ideas. Thanks, John

2 2

OSX 10.6.7 not logging and has errors
by Frank Scalzo 01 Jun '11

01 Jun '11

thanks scott, i saw the errors that was my fault for being tired and a little lazy that night made adjustments to file now i get no errors when doing the syslog-ng --syntax-only but I am not getting any logging of the normal items like login attempt and failures and such the only log i got was below looks like its just the stats and this was in messages and debug not the main log Jun 1 11:48:05 mcm-001 syslog-ng[120]: syslog-ng starting up; version='3.0.8' Jun 1 11:58:06 mcm-001 syslog-ng[120]: Log statistics; processed='destination(user)=0', processed='destination(mailwarn)=0', processed='destination(uucp)=0', processed='destination(lpr)=0', processed='destination(daemon)=0', processed='destination(debug)=1', processed='center(received)=0', processed='destination(mailerr)=0', processed='destination(d_syslog)=1', processed='destination(kern)=0', processed='source(src)=1', processed='destination(mailinfo)=0', processed='destination(newsnotice)=0', processed='destination(newserr)=0', processed='destination(messages)=1', processed='destination(authlog)=0', processed='destination(ppp)=0', processed='center(queued)=0', processed='destination(mail)=0', processed='destination(newscrit)=0', processed='destination(cron)=0', processed='src.internal(src#1)=1', stamp='src.internal(src#1)=1306943285', processed='destination(console)=0', processed='destination(console_all)=1' here is the revised latest config @version: 3.0 # # Syslog-ng example configuration for for Debian GNU/Linux # # Copyright (c) 1999 anonymous # Copyright (c) 1999 Balazs Scheidler # $Id: syslog-ng.conf.sample,v 1.3 2003/05/20 08:57:27 asd Exp $ # # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # # replaced with Line below fxs options { long_hostnames(off); sync(0); }; options { long_hostnames(off); flush_lines(0); }; source src { unix-stream("/var/run/syslog"); internal(); }; source net { udp(); }; destination authlog { file("/var/log/auth.log"); }; # replaced with Line below fxs destination syslog { file("/var/log/syslog"); }; destination d_syslog { file("/var/log/syslog.log"); }; #destination d_syslog { file("/var/log/system.log"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination kern { file("/var/log/kern.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/user.log"); }; destination uucp { file("/var/log/uucp.log"); }; destination ppp { file("/var/log/ppp.log"); }; destination mail { file("/var/log/mail.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; #destination console_all { file("/dev/tty12"); }; destination console_all { file("/dev/console"); }; #destination loghost { udp("loghost" port(999)); }; destination xconsole { pipe("/dev/xconsole"); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; #Changed it to make it filter the messages from the localO fxs filter f_syslog { not facility(authpriv, mail); }; filter f_syslog { facility(local0); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(cron); }; filter f_ppp { facility(local2); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; log { source(src); filter(f_authpriv); destination(authlog); }; # replaced with Line below fxs log { source(src); filter(f_syslog); destination(syslog); }; log { source(src); filter(f_syslog); destination(d_syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; log { source(src); filter(f_news); filter(f_err); destination(newserr); }; log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; log { source(src); filter(f_ppp); destination(ppp); }; log { source(src); destination(console_all); }; On Wed, Jun 1, 2011 at 6:00 AM, <syslog-ng-request(a)lists.balabit.hu> wrote: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: syslog-ng Digest, Vol 73, Issue 37 (Frank Scalzo) > 2. [Bug 122] [3.2.4]: mixed linking broken for > libsyslog-ng.so.* (bugzilla(a)bugzilla.balabit.com) > 3. [Bug 122] [3.2.4]: mixed linking broken for > libsyslog-ng.so.* (bugzilla(a)bugzilla.balabit.com) > 4. Re: syslog-ng Digest, Vol 73, Issue 37 (Scott Rochford) > 5. [Bug 122] [3.2.4]: mixed linking broken for > libsyslog-ng.so.* (bugzilla(a)bugzilla.balabit.com) > 6. [Bug 122] [3.2.4]: mixed linking broken for > libsyslog-ng.so.* (bugzilla(a)bugzilla.balabit.com) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 31 May 2011 08:51:17 -0400 > From: Frank Scalzo <fscalzo(a)gmail.com> > Subject: Re: [syslog-ng] syslog-ng Digest, Vol 73, Issue 37 > To: syslog-ng(a)lists.balabit.hu > Message-ID: <BANLkTikpWETLmnTSzSDL_4Cx4p0=gPT+6w(a)mail.gmail.com> > Content-Type: text/plain; charset="windows-1252" > > Gergely, > > sorry after fighting with this for hours i forgot to upload. > > @version: 3.0 > # > # Syslog-ng example configuration for for Debian GNU/Linux > # > # Copyright (c) 1999 anonymous > # Copyright (c) 1999 Balazs Scheidler > # $Id: syslog-ng.conf.sample,v 1.3 2003/05/20 08:57:27 asd Exp $ > # > # Syslog-ng configuration file, compatible with default Debian syslogd > # installation. > # > > > # replaced with Line below fxs options { long_hostnames(off); sync(0); }; > options { long_hostnames(off); flush_lines(0); }; > > source src { unix-stream("/var/run/syslog"); internal(); }; > source net { udp(); }; > > destination authlog { file("/var/log/auth.log"); }; > # replaced with Line below fxs destination syslog { > file("/var/log/syslog"); > }; > destination d_syslog { file(?/var/log/system.log?); }; > destination cron { file("/var/log/cron.log"); }; > destination daemon { file("/var/log/daemon.log"); }; > destination kern { file("/var/log/kern.log"); }; > destination lpr { file("/var/log/lpr.log"); }; > destination user { file("/var/log/user.log"); }; > destination uucp { file("/var/log/uucp.log"); }; > destination ppp { file("/var/log/ppp.log"); }; > destination mail { file("/var/log/mail.log"); }; > > destination mailinfo { file("/var/log/mail.info"); }; > destination mailwarn { file("/var/log/mail.warn"); }; > destination mailerr { file("/var/log/mail.err"); }; > > destination newscrit { file("/var/log/news/news.crit"); }; > destination newserr { file("/var/log/news/news.err"); }; > destination newsnotice { file("/var/log/news/news.notice"); }; > > destination debug { file("/var/log/debug"); }; > destination messages { file("/var/log/messages"); }; > destination console { usertty("root"); }; > #destination console_all { file("/dev/tty12"); }; > destination console_all { file(?/dev/console?); }; > #destination loghost { udp("loghost" port(999)); }; > > > destination xconsole { pipe("/dev/xconsole"); }; > > filter f_auth { facility(auth); }; > filter f_authpriv { facility(auth, authpriv); }; > #Changed it to make it filter the messages from the localO fxs filter > f_syslog { not facility(authpriv, mail); }; > filter f_syslog { facility(local0); }; > filter f_cron { facility(cron); }; > filter f_daemon { facility(daemon); }; > filter f_kern { facility(kern); }; > filter f_lpr { facility(lpr); }; > filter f_mail { facility(mail); }; > filter f_user { facility(user); }; > filter f_uucp { facility(cron); }; > filter f_ppp { facility(local2); }; > filter f_news { facility(news); }; > filter f_debug { not facility(auth, authpriv, news, mail); }; > filter f_messages { level(info..warn) > and not facility(auth, authpriv, mail, news); }; > filter f_emergency { level(emerg); }; > > filter f_info { level(info); }; > filter f_notice { level(notice); }; > filter f_warn { level(warn); }; > filter f_crit { level(crit); }; > filter f_err { level(err); }; > > log { source(src); filter(f_authpriv); destination(authlog); }; > # replaced with Line below fxs log { source(src); filter(f_syslog); > destination(syslog); }; > log { source(src); filter(f_syslog); destination(d_syslog); }; > log { source(src); filter(f_cron); destination(cron); }; > log { source(src); filter(f_daemon); destination(daemon); }; > log { source(src); filter(f_kern); destination(kern); }; > log { source(src); filter(f_lpr); destination(lpr); }; > log { source(src); filter(f_mail); destination(mail); }; > log { source(src); filter(f_user); destination(user); }; > log { source(src); filter(f_uucp); destination(uucp); }; > log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); > }; > log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); > }; > log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; > log { source(src); filter(f_news); filter(f_crit); destination(newscrit); > }; > log { source(src); filter(f_news); filter(f_err); destination(newserr); }; > log { source(src); filter(f_news); filter(f_notice); > destination(newsnotice); }; > log { source(src); filter(f_debug); destination(debug); }; > log { source(src); filter(f_messages); destination(messages); }; > log { source(src); filter(f_emergency); destination(console); }; > log { source(src); filter(f_ppp); destination(ppp); }; > log { source(src); destination(console_all); }; >

1 0

syslong-ng configuration not working on osx 10.6.7
by Frank Scalzo 31 May '11

31 May '11

I am not getting any logs though its running i did find i was getting 2 errors in the debug message log though Error opening file for writing; filename='\xff\xff\xff/dev/console\xff\xff\xff', error='No such file or directory (2)' syslog-ng[126]: syslog-ng internal() messages are looping back, preventing loop by suppressing further messages; recurse_count='2' anyone that could help would be great Frank

2 1

keep_hostname (yes); and source file
by Steven Blatchford 28 May '11

28 May '11

Hello, I have a file, /srv/http/foo/var/log/system.log that I would like sent to our central log. This config was not working: @version: 3.0 # # /etc/syslog-ng.conf # options { stats_freq (0); flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames(off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); perm(0640); group("log"); }; source s_foo { file("/srv/http/foo/var/log/system.log"); }; log { source(s_foo); destination(d_net); }; destination d_net { udp("host.org"); }; It does work if I comment out 'keep_hostname (yes);'. Can someone explain to me why this is? -steve

1 0

Re: [syslog-ng] Running script for checking syslog-ng status
by Henry Xu 27 May '11

27 May '11

Hi All, Yes, the "supervising syslog-ng" process starts by default. I did tests and killed the syslog-ng process when the supervising process is running. But seems it didn't work.The new syslog-ng process didn't start, and the supervising process stopped as well. What's wrong? Best, Henry ------------------------------ Message: 3 Date: Thu, 26 May 2011 23:47:51 +0200 From: Gergely Nagy <algernon(a)balabit.hu> Subject: Re: [syslog-ng] Running script for checking syslog-ng status To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <874o4hdth4.fsf(a)luthien.mhp> Content-Type: text/plain Matthew Hall <mhall(a)mhcomputing.net> writes: > On Thu, May 26, 2011 at 10:49:08AM +0200, Gergely Nagy wrote: >> It was also brought to my attention, that syslog-ng has a built-in >> supervisor, that will restart it in case it crashes. This can be >> enabled by starting syslog-ng with the --process-mode=safe-background >> command-line argument. >> >> (What do you know? I never knew this was there! One learns something >> new every day!) > > Gergely, > > I thought the supervisor was on by default. Am I mistaken to believe > this? You are correct.

2 1

Multiple Interface interaction with syslogng
by Pramod Pillai 27 May '11

27 May '11

Hi I am using Solaris box which has multiple IP interface cards. When I run syslogng client ( instance ) it picks up one of the available IP for use. I want the syslogng to pick up the IP which I want. By this way on the syslong-ng server I will have the correct IP address. EX: The box has following IP address. 10.232.1.1 10.232.1.2 10.232.1.3 10.232.1.4 I want the syslogn-ng to pick up "10.232.1.3". There by the syslog-ng server instance will always print the source IP as "10.232.1.3" Regards Pramod

2 1

Running script for checking syslog-ng status
by Henry Xu 26 May '11

26 May '11

Hi All, Does syslog-ng have a running script that check the syslog-ng running status? If it crashed or stopped running,this script can make sure to start a new syslog-ng process. This morning,the syslog-ng on my server crashed,and somehow resulted in the server halt. Thanks. Best, Henry

4 5

[PATCH] tfjson: Support either json-c or json-glib.
by Gergely Nagy 25 May '11

25 May '11

Add the ability to build the module with either json-c, or json-glib (version 0.12 or later). If both are present, we'll prefer json-c. Requested-by: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 19 ++++++++++-- modules/tfjson/tfjson.c | 73 ++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 75 insertions(+), 17 deletions(-) diff --git a/configure.in b/configure.in index 9c03b65..87de97f 100644 --- a/configure.in +++ b/configure.in @@ -25,7 +25,8 @@ EVTLOG_MIN_VERSION="0.2.12" OPENSSL_MIN_VERSION="0.9.8" LIBDBI_MIN_VERSION="0.8.0" IVYKIS_MIN_VERSION="0.18" -JSON_MIN_VERSION="0.7" +JSON_C_MIN_VERSION="0.7" +JSON_GLIB_MIN_VERSION="0.12" PCRE_MIN_VERSION="6.1" dnl *************************************************************************** @@ -563,7 +564,10 @@ fi dnl *************************************************************************** dnl json headers/libraries dnl *************************************************************************** -PKG_CHECK_MODULES(JSON, json >= $JSON_MIN_VERSION,, JSON_LIBS="") +PKG_CHECK_MODULES(JSON, json >= $JSON_C_MIN_VERSION, JSON_TYPE="C", JSON_LIBS="") +if test "x$JSON_TYPE" = "x"; then + PKG_CHECK_MODULES(JSON, json-glib-1.0 >= $JSON_GLIB_MIN_VERSION, JSON_TYPE="glib", JSON_LIBS="") +fi dnl *************************************************************************** dnl pcre headers/libraries @@ -813,7 +817,7 @@ if test "x$enable_json" = "xauto"; then enable_json="yes" fi elif test "x$enable_json" = "xyes" -a -z "$JSON_LIBS"; then - AC_MSG_ERROR(Cannot find json-c version >= $JSON_MIN_VERSION: is pkg-config in path?) + AC_MSG_ERROR(Cannot find json-c version >= $JSON_C_MIN_VERSION or json-glib-1.0 >= $JSON_GLIB_MIN_VERSION: is pkg-config in path?) fi if test "x$enable_systemd" = "xauto"; then @@ -964,6 +968,15 @@ AC_DEFINE_UNQUOTED(ENABLE_PCRE, `enable_value $enable_pcre`, [Enable PCRE suppor AC_DEFINE_UNQUOTED(ENABLE_ENV_WRAPPER, `enable_value $enable_env_wrapper`, [Enable environment wrapper support]) AC_DEFINE_UNQUOTED(ENABLE_SYSTEMD, `enable_value $enable_systemd`, [Enable systemd support]) +case "$JSON_TYPE" in + "C") + AC_DEFINE_UNQUOTED(HAVE_JSON_C, 1, [Have json-c]) + ;; + "glib") + AC_DEFINE_UNQUOTED(HAVE_JSON_GLIB, 1, [Have json-glib]) + ;; +esac + AM_CONDITIONAL(ENABLE_ENV_WRAPPER, [test "$enable_env_wrapper" = "yes"]) AM_CONDITIONAL(ENABLE_SYSTEMD, [test "$enable_systemd" = "yes"]) AM_CONDITIONAL(ENABLE_SSL, [test "$enable_ssl" = "yes"]) diff --git a/modules/tfjson/tfjson.c b/modules/tfjson/tfjson.c index 07fc76e..7155f15 100644 --- a/modules/tfjson/tfjson.c +++ b/modules/tfjson/tfjson.c @@ -25,7 +25,15 @@ #include "cfg.h" #include "value-pairs.h" +#include "config.h" + +#ifdef HAVE_JSON_C #include <json.h> +#endif + +#ifdef HAVE_JSON_GLIB +#include <json-glib/json-glib.h> +#endif static gboolean tf_json_prepare(LogTemplateFunction *self, LogTemplate *parent, @@ -35,6 +43,8 @@ tf_json_prepare(LogTemplateFunction *self, LogTemplate *parent, { ValuePairs *vp; + g_type_init (); + vp = value_pairs_new_from_cmdline (parent->cfg, argc, argv, error); if (!vp) return FALSE; @@ -45,6 +55,7 @@ tf_json_prepare(LogTemplateFunction *self, LogTemplate *parent, return TRUE; } +#if HAVE_JSON_C static gboolean tf_json_foreach (const gchar *name, const gchar *value, gpointer user_data) { @@ -57,16 +68,57 @@ tf_json_foreach (const gchar *name, const gchar *value, gpointer user_data) return FALSE; } -static struct json_object * -tf_json_format_message(ValuePairs *vp, LogMessage *msg) +static void +tf_json_append(GString *result, ValuePairs *vp, LogMessage *msg) +{ + struct json_object *json; + + json = json_object_new_object(); + + value_pairs_foreach(vp, tf_json_foreach, msg, 0, json); + + g_string_append(result, json_object_to_json_string (json)); + json_object_put(json); +} +#endif + +#if HAVE_JSON_GLIB +static gboolean +tf_json_foreach (const gchar *name, const gchar *value, gpointer user_data) { - struct json_object *root; + JsonBuilder *builder = (JsonBuilder *)user_data; + + json_builder_set_member_name(builder, name); + json_builder_add_string_value(builder, value); + + return FALSE; +} + +static void +tf_json_append(GString *result, ValuePairs *vp, LogMessage *msg) +{ + JsonBuilder *builder; + JsonGenerator *gen; + gchar *str; + + builder = json_builder_new(); + json_builder_begin_object(builder); + + value_pairs_foreach(vp, tf_json_foreach, msg, 0, builder); + + json_builder_end_object(builder); + + gen = json_generator_new(); + json_generator_set_root(gen, json_builder_get_root(builder)); + str = json_generator_to_data(gen, NULL); - root = json_object_new_object(); - value_pairs_foreach (vp, tf_json_foreach, msg, 0, root); + g_object_unref(gen); + g_object_unref(builder); - return root; + g_string_append(result, str); + g_free(str); } +#endif static void tf_json_call(LogTemplateFunction *self, gpointer state, GPtrArray *arg_bufs, @@ -77,14 +129,7 @@ tf_json_call(LogTemplateFunction *self, gpointer state, GPtrArray *arg_bufs, ValuePairs *vp = (ValuePairs *)state; for (i = 0; i < num_messages; i++) - { - LogMessage *msg = messages[i]; - struct json_object *json; - - json = tf_json_format_message(vp, msg); - g_string_append(result, json_object_to_json_string (json)); - json_object_put(json); - } + tf_json_append(result, vp, messages[i]); } static void -- 1.7.2.5

4 4

dropped udp packets and help with config
by Zeek Anow 24 May '11

24 May '11

Something really wrong with syslog-ng or my config. I'm dropping way too many packets. I will admit that my configuration is probably really a large part of the problem and would appreciate it if someone could take a look at it and offer some suggestions. There is another thread going about a similar problem on a similar platform. We recently upgraded to Solaris 10 from Solaris 9 and I don't recall us dropping that many packets before. And we also upgraded from a very older Sylog-ng version to 3.1.2. I am basing the dropped packets on the udp stats, not syslog-ng stats. Syslog-ng stats has NO dropped packets. UDP udpInDatagrams -4599313 udpInErrors - 0 udpOutDatagrams - 3421 udpOutErrors - 0 tcpInErrs - 0 udpNoPorts -2587612 udpInCksumErrs - 0 udpInOverflows -95806254 The above is a 3 hour sample and it is from our syslog server that does not get that much traffic. ____________________________ Here is the current version info: Solaris 10, syslog-ng 3.1.2 Installer-Version: 3.1.2 Revision: ssh+git://bazsi@git.balabit //var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8bf13c304b6ab5fc1a372b49d55c78370efe14ca Compile-Date: Oct 25 2010 23:56:18 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: on Enable-Sun-Door: on Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on _____________________________ Below is a very small sampling of our syslog-ng.conf. We are filtering on about 1400 devices most of which are Firewalls and routers. The IPs in the following sample have been made up. One of my questions is "Does the number of devices we are filtering on make a difference? (1400)" We have several sites and use just one version of the syslogng.conf file. It is a lot easier to maintain one copy:)) Also notice the format: ("^10\.123\.10\.133$") for the filters. All 1400 are in that format. I was hoping this would help a little but don't really know for sure:)) The source statementbelow "...external_Future_tcp" has not yet been implemented. Since we are dropping so many packets, I was going to try configuring the devices to log TCP instead of UDP. @version: 3.0 # Created: 01 March 2011 #----------------[ GLOBAL OPTIONS ]------------------------- options { create_dirs(yes); use_dns(no); time_reopen(10); time_reap(360); keep_timestamp(yes); }; #---------------------[ SOURCES ]--------------------------- source s_local { sun-stream("/dev/log" door("/etc/.syslog_door")); internal(); }; source s_external { udp(); }; source s_external_tcp { tcp(max-connections(50) port(514)); }; source s_external_Future_tcp { tcp(max-connections(1400) port(1470)); }; #---------------------[ DESTINATION ]--------------------------- destination d_local { file("/var/adm/messages" perm(0655) dir_perm(0655)); }; destination d_network_file { file("/logs/$YEAR/$MONTH/$DAY/network.log" perm(0655) dir_perm(0655)); }; destination d_bacsit { udp("10.11.13.114" port(2514) spoof-source(yes)); }; destination d_network_syslogd { udp("10.11.13.116" port(1514) spoof-source(yes)); }; destination d_firewall_file { file("/logs/$YEAR/$MONTH/$DAY/firewall/$HOST.log" perm(0655) dir_perm(0655)); }; destination d_mrv_file { file("/logs/$YEAR/$MONTH/$DAY/mrv.log" perm(0655) dir_perm(0655)); }; destination d_mail_file { file("/logs/$YEAR/$MONTH/$DAY/mail/$HOST.log" perm(0655) dir_perm(0655)); }; destination d_f567_file { file("/logs/$YEAR/$MONTH/$DAY/f5s/$HOST.log" perm(0655) dir_perm(0655)); }; #---------------------[ FILTERS ]--------------------------- filter f_f567 { host("^10\.123\.10\.133$") or # Host B host("^10\.100\.10\.200$") or # Host A host("^10\.115\.10\.246$") or # Host C host("^10\.121\.10\.102$") or # Host D host("^10\.117\.10\.99$"); # Host F }; filter f_mrv { host("^10\.68\.69\.100$") or # host("^10\.100\166\.10$") or # }; . . . and so on #---------------------[ LOGS ]--------------------------- log { source(s_local); destination(d_local); }; log { source(s_external); filter(f_f567); destination(d_f5_file); }; log { source(s_external); source(s_external_tcp); filter(f_firewall); destination(d_bacsit); }; log { source(s_external); filter(f_network); destination(d_bacsit); }; log { source(s_external); source(s_external_tcp); filter(f_firewall); destination(d_combo_file); }; log { source(s_external); filter(f_mail); destination(d_mail_file); }; ....and so on I'm grateful for all help and suggestions. Thanks!!

5 5