- syslog-ng - lists.balabit.hu

Parsing Cisco FWSM with device-id
by Martin Holste 30 Oct '11

30 Oct '11

So I have a customer who has enabled the device-id configuration directive on his FWSM, and that means that instead of this from a normal FWSM: <174>%FWSM-6-302013: Built inbound TCP... it sends this: <174>FWSMHostName %FWSM-6-30203: Built inbound TCP... This means that the program name does not get properly parsed as syslog-ng pushes it into the msg field. If I can't convince the customer to remove the device-id setting, what's the least CPU-intensive way of coping with this so that program and msg are set correctly?

3 4

syslog-ng does not start when a destination hostname is unresolvable
by Heiko Gerstung 29 Oct '11

29 Oct '11

Hi! I am using syslog-ng for my network appliance and came across a problem with unresolved destination hosts. But first of all I have to say that from all the syslog daemons I looked at, syslog-ng really stands out as the most versatile and flexible solution. So, thank you for this great piece of software! In my configuration it is possible to specify a number of remote syslog servers, to which my device forwards all log entries via TCP or UDP/514 in addition to logging everything to local files (/var/log/messages etc.). Now, when a customer enters an unresolvable hostname, syslog-ng does not start at all and therefore the whole system is without a syslog daemon. My current workaround is to try and resolve the configured hostnames myself and only write them to the generated syslog-ng.conf file when this succeeds, but I wonder if there is no configuration option or commandline switch that prevents syslog-ng from completely stopping to work when it cannot resolve a hostname. And, is this really the intended default behavior? On my system (and on almost every other system I assume), syslog-ng starts long before network connectivity is available in order to provide logging services during the boot sequence. I would have expected that, if one of the destinations cannot be resolved, syslog-ng starts without it and periodically tries to resolve it later. I did not find anything about this in the documentation (manpages of syslog-ng and syslog-ng.conf) or in the FAQ, the only thing I found was a bug in bugzilla (https://bugzilla.balabit.com/show_bug.cgi?id=63) which has been filed in 11/2009 and reports that --syntax-only does not catch unresolved _sources_ although this is considered a critical error and prevents syslog-ng from starting. I would assume that this is the same for unresolved destinations. Any comments on this? I know that ntpd tries to resolve configured servers in a separate thread which retries resolving them every once in a while and starts using such a host as soon as it could be resolved. Best Regards, Heiko

2 1

[PATCH] logproto: Fix log_proto_file_writer_flush()'s partial construction.
by Gergely Nagy 29 Oct '11

29 Oct '11

When log_proto_file_writer_flush() encountered a partial write, and tried to construct a self->partial buffer, that contains all the data we didn't write yet, the starting offset computation was... interesting. What we want to copy into self->partial, is whatever is left in the last, partially processed buffer (i0), then append the full contents of the rest. Calculating the starting offset of this first buffer was done in a quite convoluted way, involving the sum of the length of the buffers we already touched, the lenght of the last fully processed buffer, and the total number of bytes already written. This makes my head spin, especially since it can be simplified to substracting the number of bytes to copy from the first buffer from the total length of it. The attached patch simplifies the offset calculation, and it also fixes a subtle bug! Lets assume 5 buffers, the first one 100, the second 200, the third 300, the fourth 400 and the fifth 500 bytes, totalling to 1500 bytes. Lets assume we managed to write out the first three buffers, and half of the fourht: 800 bytes. In this case, rc will be 800, sum will be 1000, i0 will be 3, and ofs will be 200. So far, all is well. But when we try to compute the starting offset within the first partial buffer, we end up with: rc - (sum - buf[i0 - 1].length), which, when values are substituted becomes: 800 - (1000 - 300) = 100! This is obviously not correct: we've already written out half of the fourth buffer, yet, we start copying from the 100th byte instead of the 200th! After the fix, when the starting position is buffer[i0].length - ofs, we get the correct value of 200, and all is well. Reported-by: SZALAY Attila <sasa(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/logproto.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/lib/logproto.c b/lib/logproto.c index bdf9695..5047c98 100644 --- a/lib/logproto.c +++ b/lib/logproto.c @@ -299,7 +299,7 @@ log_proto_file_writer_flush(LogProto *s) /* allocate and copy the remaning data */ self->partial = (guchar *)g_malloc(self->partial_len); ofs = sum - rc; /* the length of the remaning (not processed) chunk in the first message */ - memcpy(self->partial, self->buffer[i0].iov_base + rc - (i0 > 0 ? (sum - self->buffer[i0 - 1].iov_len) : 0), ofs); + memcpy(self->partial, self->buffer[i0].iov_len - ofs, ofs); i = i0 + 1; while (i < self->buf_count) { -- 1.7.7.1

3 5

[PATCH] tfjson: Make $(format-json) have less unneccessary whitespace.
by Gergely Nagy 29 Oct '11

29 Oct '11

When compiling with jsonc, use a custom formatter, so that the generated JSON does not have superflous whitespace between elements, and the only whitespace in the JSON is in the content parts. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/tfjson/tfjson.c | 32 ++++++++++++++++++++++++++++++++ 1 files changed, 32 insertions(+), 0 deletions(-) diff --git a/modules/tfjson/tfjson.c b/modules/tfjson/tfjson.c index 65daadd..eba2147 100644 --- a/modules/tfjson/tfjson.c +++ b/modules/tfjson/tfjson.c @@ -28,7 +28,9 @@ #include "config.h" #ifdef HAVE_JSON_C +#include <printbuf.h> #include <json.h> +#include <json_object_private.h> #endif #ifdef HAVE_JSON_GLIB @@ -54,6 +56,35 @@ tf_json_prepare(LogTemplateFunction *self, LogTemplate *parent, } #if HAVE_JSON_C +static int +tf_json_object_to_string (struct json_object *jso, + struct printbuf *pb) +{ + int i = 0; + struct json_object_iter iter; + sprintbuf (pb, "{"); + + json_object_object_foreachC (jso, iter) + { + gchar *esc; + + if (i) + sprintbuf (pb, ","); + sprintbuf (pb, "\""); + esc = g_strescape (iter.key, NULL); + sprintbuf (pb, esc); + g_free (esc); + sprintbuf (pb, "\":"); + if (iter.val == NULL) + sprintbuf (pb, "null"); + else + iter.val->_to_json_string (iter.val, pb); + i++; + } + + return sprintbuf(pb, "}"); +} + static gboolean tf_json_foreach (const gchar *name, const gchar *value, gpointer user_data) { @@ -72,6 +103,7 @@ tf_json_append(GString *result, ValuePairs *vp, LogMessage *msg) struct json_object *json; json = json_object_new_object(); + json->_to_json_string = tf_json_object_to_string; value_pairs_foreach(vp, tf_json_foreach, msg, 0, json); -- 1.7.7.1

2 3

Re: [syslog-ng] syslog-ng 100% cpu utilization issue
by Jain, Vaibhav (GE Healthcare) 29 Oct '11

29 Oct '11

Hi, I am using RHEL 5.5 and the syslog-ng version present in below link is for EL 6. Can you please share syslog-ng 64 bit for RHEL 5.5 http://download.fedora.redhat.com/pub/epel/6/x86_64/repoview/syslog-ng.h tml Thanks in advance for your support. Regards, Vaibhav -----Original Message----- From: Pillai Pramod-a23020 [mailto:pramod.p_pillai@nsn.com] Sent: Friday, October 28, 2011 6:38 PM To: Jain, Vaibhav (GE Healthcare) Subject: RE: [syslog-ng] syslog-ng 100% cpu utilization issue The client and server uses the same software. The client and server is distinguished by the config file. Pramod -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Jain, Vaibhav (GE Healthcare) Sent: Friday, October 28, 2011 6:32 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng 100% cpu utilization issue Thanks for sharing syslog-ng. Can you please share syslog-ng client and server both ? In the below link only one RPM is there. I don't know whether this is client or server. http://download.fedora.redhat.com/pub/epel/6/x86_64/repoview/syslog-ng.h tml -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Thursday, October 27, 2011 8:20 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng 100% cpu utilization issue On Thu, 2011-10-27 at 16:12 +0200, Gergely Nagy wrote: > "Jain, Vaibhav (GE Healthcare)" <Vaibhav.Jain(a)ge.com> writes: > > > Can you please share the syslog-ng client and server RPMs of > > syslog-ng > > 3.2.4 or above 64 bit for RHEL OS? > > http://download.fedora.redhat.com/pub/epel/6/x86_64/repoview/syslog-ng > .html > > If that's for a too new OS version, then I do not know any ready made > RPMs for older RHEL versions. > syslog-ng binaries are supplied as a part of EPEL (community maintained packages for RHEL) for free and as far as I remember syslog-ng is part of that. ======================================================================== ======= shameless plug follows (don't read if you are not interested in paid offerings) ======================================================================== ======= [Please Scroll down] The Premium Edition of syslog-ng does support a wide range of RHEL versions (starting with RHEL2 to RHEL6). With syslog-ng PE 4.0 it support 46 different OS version / architecture combinations, which are quality checked with thourough automatic testing on each. So if you have a wide variety of platforms, you can check that out. Not to mention that in case you have a problem, you can ask support for help. -- Bazsi ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

2 3

syslog-ng 100% cpu utilization issue
by Jain, Vaibhav (GE Healthcare) 28 Oct '11

28 Oct '11

Hi, I am getting 100% CPU utilization problem on all the syslog-ng client machines. All the clients are pointing to one syslog-ng server. I don't know why this is happening can you please help me on this. PS:I am using syslog-ng 3.0.1 (open source, 64bit). Thanks, Vaibhav

4 6

[Bug 141] New: Crash while reloading config
by bugzilla＠bugzilla.balabit.com 27 Oct '11

27 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=141 Summary: Crash while reloading config Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: eric(a)sevenscale.com Type of the Report: --- Estimated Hours: 0.0 We have experienced multiple instances of crashes on reloading a config with 3.3.1. Here is the backtrace: (gdb) where #0 afsocket_sd_remove_and_kill_connection (self=0x0, connection=0x1d64010) at afsocket.c:343 #1 0x00007f3b91790560 in afsocket_sd_kill_connection_list (list=<value optimized out>) at afsocket.c:367 #2 0x00007f3b9225a4f2 in persist_config_entry_free (self=0x1eb9180) at cfg.c:66 #3 0x0000003b62a2dd3e in ?? () from /lib64/libglib-2.0.so.0 #4 0x0000003b62a2e1ad in g_hash_table_remove_all () from /lib64/libglib-2.0.so.0 #5 0x0000003b62a2e245 in g_hash_table_destroy () from /lib64/libglib-2.0.so.0 #6 0x00007f3b9225a78c in persist_config_free (self=0x35de9e0) at cfg.c:83 #7 0x00007f3b922730d4 in main_loop_reload_config_apply () at mainloop.c:491 #8 0x00007f3b9228f81b in iv_signal_event (_this=0x7f3b924bbee0) at iv_signal.c:80 #9 0x00007f3b9228f3a9 in iv_event_raw_got_event (_this=0x7f3b924bbf08) at iv_event_raw.c:82 #10 0x00007f3b9228cdf6 in iv_run_active_list () at iv_main.c:219 #11 iv_main () at iv_main.c:269 #12 0x00007f3b92272c3e in main_loop_run () at mainloop.c:722 #13 0x00000000004015b1 in main () (gdb) p *connection $5 = {super = {ref_cnt = {counter = 2}, flags = 0, cfg = 0x0, pipe_next = 0x1a2e5b0, queue_data = 0x0, queue = 0, init = 0x7f3b917901b0 <afsocket_sc_init>, deinit = 0x7f3b91790470 <afsocket_sc_deinit>, free_fn = 0x7f3b9178fc60 <afsocket_sc_free>, notify = 0x7f3b91790570 <afsocket_sc_notify>}, owner = 0x0, reader = 0x1d64430, sock = 386, peer_addr = 0x0} (gdb) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

Latest version of syslog-ng open source 64 bit
by Jain, Vaibhav (GE Healthcare) 27 Oct '11

27 Oct '11

Hi All, I want to download the latest version of syslog-ng (client and server) open source 64 bit. Can you please share with me? Thanks Vaibhav

2 1

Request: GitHub mirror of git bazsi/syslog-ng-3.2.git and bazsi/syslog-ng-3.3.git
by Eric Lindvall 26 Oct '11

26 Oct '11

Hello, GitHub has the ability to setup an automated mirror of a remote git repository to allow for users of GitHub to interact with the code more easily. I would really appreciate having such a mirror setup for syslog-ng to make development easier to track in how I work day-to-day. To get the mirror setup, a free Organization needs to be created for "syslog-ng", and then someone needs to send an email to support(a)github.comasking to setup the mirrors. If this is something that interests anyone, I would be more than willing to help out in getting the process completed. Thanks, Eric

2 2

Cannot install syslog-ng 3.2.4 under cygwin with Oracle support
by Hery Fanomezantsoa 26 Oct '11

26 Oct '11

Hi all, Prior to installing syslog-ng on cygwin, i installed libdbi and libdbi-drivers with oracle instant client on windows. After that, i installed eventlog Finally i tried to install syslog-ng and it failed. Here is my configure: ./configure EVTLOG_CFLAGS="-I/usr/local/eventlog/include/eventlog" \ > EVTLOG_LIBS="-L/usr/local/eventlog/lib -levtlog" \ > GLIB_CFLAGS="-I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include" \ > GLIB_LIBS="-L/usr/lib -lglib-2.0 -lintl" \ > LIBDBI_CFLAGS="-I/usr/local/libdbi/include/dbi-0.8" \ > LIBDBI_LIBS="-L/usr/local/libdbi/lib" \ > --enable-sql I've got this warning: *** Warning: This system can not link to static lib archive /usr/local/eventlog/lib/libevtlog.la. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have. *** Warning: linker path does not have real file for library -lrt. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have *** because I did check the linker path looking for a file starting *** with librt and none of the candidates passed a file format test *** using a file magic. Last file checked: /lib/librt.a *** Warning: linker path does not have real file for library -lresolv. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have *** because I did check the linker path looking for a file starting *** with libresolv and none of the candidates passed a file format test *** using a file magic. Last file checked: /lib/libresolv.a *** Warning: linker path does not have real file for library -ldl. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have *** because I did check the linker path looking for a file starting *** with libdl and none of the candidates passed a file format test *** using a file magic. Last file checked: /lib/libdl.a *** The inter-library dependencies that have been dropped here will be *** automatically added whenever a program is linked with this library *** or is declared to -dlopen it. *** Since this library must not contain undefined symbols, *** because either the platform does not support them or *** it was explicitly requested with -no-undefined, *** libtool will only create a static version of it. And this error ../../lib/.libs/libsyslog-ng.a(plugin.o): In function `plugin_load_module': /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:154: undefined reference to `_g_module_symbol' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:172: undefined reference to `_g_module_build_path' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:206: undefined reference to `_g_module_open' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:217: undefined reference to `_g_module_make_resident' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:219: undefined reference to `_g_module_symbol' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:221: undefined reference to `_g_module_error' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:210: undefined reference to `_g_module_error' /cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/lib/plugin.c:146: undefined reference to `_g_module_open' collect2: ld returned 1 exit status make[5]: *** [pdbtool.exe] Error 1 make[5]: Leaving directory `/cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/modules/dbparser' make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory `/cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/modules/dbparser' make[3]: *** [all] Error 2 make[3]: Leaving directory `/cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/modules/dbparser' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4/modules' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/cygdrive/q/STAGE2011/Soft/syslog-ng-3.2.4' make: *** [all] Error 2 What do I miss? Thanks in advance.

2 3