July 2012 - syslog-ng - lists.balabit.hu

[Bug 161] New: Syslog-ng stops processing after reload
by bugzilla＠bugzilla.balabit.com 10 Sep '12

10 Sep '12

https://bugzilla.balabit.com/show_bug.cgi?id=161 Summary: Syslog-ng stops processing after reload Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 syslog-ng version: 3.3.4 OS: RHEL 6.2 Whenever I issue a reload to our syslog-ng instance syslog-ng stops processing messages and sits there doing nothing. The issue only seems to occur when the server is processing a high log volume. This server is part of a 2-node active/passive HA group. So we have 2 servers with the exact same configuration. Whenever I issue the reload on the server that is not currently active, it works fine. I can issue a failover and the server that was able to reload fine now hangs. The active server in this HA group processes around 35,000 messages per second. I havent posted the config because it is fairly large, however I can if desired. The most notable thing in it is that threading is enabled. Below is the backtrace of syslog-ng stuck in this state. I took a gcore, so I can gather any additional information that is needed. Also, if I try to do a full backtrace ('where full') in gdb on any thread other than thread 5, gdb segfaults. Not sure if that has any relevance or not. (gdb) info threads 5 Thread 0x7fb9c654ab00 (LWP 11040) 0x000000353cedd967 in writev () from /lib64/libc.so.6 4 Thread 0x7fb9c25fb700 (LWP 11049) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 3 Thread 0x7fb9c2ffc700 (LWP 11048) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 2 Thread 0x7fb9c39fd700 (LWP 11046) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 * 1 Thread 0x7fb9c43fe700 (LWP 11045) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c43fb9a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x14fc610) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc6e0) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 2 [Switching to thread 2 (Thread 0x7fb9c39fd700 (LWP 11046))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c39fa9a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x14fc8e0) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc9b0) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 3 [Switching to thread 3 (Thread 0x7fb9c2ffc700 (LWP 11048))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c2ff99a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x15635c0) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc380) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 4 [Switching to thread 4 (Thread 0x7fb9c25fb700 (LWP 11049))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c25f89a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x1566d20) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x1566a70) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 5 [Switching to thread 5 (Thread 0x7fb9c654ab00 (LWP 11040))]#0 0x000000353cedd967 in writev () from /lib64/libc.so.6 (gdb) where #0 0x000000353cedd967 in writev () from /lib64/libc.so.6 #1 0x00007fb9c65820f8 in log_proto_file_writer_flush (s=0x171b5a0) at logproto.c:270 #2 0x00007fb9c658c403 in log_proto_flush (s=0x171b5a0) at logproto.h:86 #3 0x00007fb9c658ec3e in log_writer_flush (self=0x171adb0, flush_mode=LW_FLUSH_QUEUE) at logwriter.c:1032 #4 0x00007fb9c658f006 in log_writer_deinit (s=0x171adb0) at logwriter.c:1106 #5 0x00007fb9c63399d6 in log_pipe_deinit (s=0x171adb0) at ../../lib/logpipe.h:254 #6 0x00007fb9c633b3cd in affile_dw_deinit (s=0x171ac20) at affile.c:627 #7 0x00007fb9c63399d6 in log_pipe_deinit (s=0x171ac20) at ../../lib/logpipe.h:254 #8 0x00007fb9c633c211 in affile_dd_deinit_writer (key=0x171ad60, value=0x171ac20, user_data=0x0) at affile.c:974 #9 0x000000353e22a7a3 in g_hash_table_foreach () from /lib64/libglib-2.0.so.0 #10 0x00007fb9c633c349 in affile_dd_deinit (s=0x1539990) at affile.c:996 #11 0x00007fb9c6575d86 in log_pipe_deinit (s=0x1539990) at logpipe.h:254 #12 0x00007fb9c65761b4 in log_dest_group_deinit (s=0x1539ef0) at dgroup.c:76 #13 0x00007fb9c656f54c in log_pipe_deinit (s=0x1539ef0) at logpipe.h:254 #14 0x00007fb9c657078c in log_center_deinit (self=0x14f35c0) at center.c:576 #15 0x00007fb9c6571296 in cfg_deinit (cfg=0x14f1ca0) at cfg.c:261 #16 0x00007fb9c6590cac in main_loop_reload_config_apply () at mainloop.c:494 #17 0x00007fb9c6590aab in main_loop_io_worker_job_complete (self=0x156d160) at mainloop.c:417 #18 0x00007fb9c65d3ccc in iv_work_event (_pool=0x14f1bf0) at iv_work.c:221 #19 0x00007fb9c65d2862 in iv_event_run_pending_events (_dummy=0x0) at iv_event.c:67 #20 0x00007fb9c65d2cb5 in iv_event_raw_got_event (_this=0x7fb9c654aa18) at iv_event_raw.c:82 #21 0x00007fb9c65cefec in iv_run_active_list (active=0x7fff33ed17b0) at iv_main.c:219 #22 0x00007fb9c65cf1cc in iv_main () at iv_main.c:269 #23 0x00007fb9c65913e9 in main_loop_run () at mainloop.c:731 #24 0x00000000004016d0 in main () -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

[Bug 144] New: incremental statistics and on reload and stop
by bugzilla＠bugzilla.balabit.com 07 Sep '12

07 Sep '12

https://bugzilla.balabit.com/show_bug.cgi?id=144 Summary: incremental statistics and on reload and stop Product: syslog-ng Version: unspecified Platform: All OS/Version: All Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mdione.ext(a)orange.com Type of the Report: --- Estimated Hours: 0.0 syslog-ng log statistics in /var/log/messages (or some other defined destination), but it does it accumulatively. this makes it harder to make graphs, because the data collector must detect reloads and stop/starts. this most probably should not be done if the statistics are asked via syslog-ng-ctl. also, it does not print statistics on reload or stop, so the last values before the reload or stop are lost. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[PATCH (3.4)] confgen: Implement a built-in system() generator
by Gergely Nagy 19 Aug '12

19 Aug '12

The system SCL plugin relies on a script (generate-system-source.sh) to generate the block system() expands into. While this is flexible and easy to modifiy, it has the disadvantage of running an external script at startup, which is undesirable in many situations, including when using a strict AppArmor profile. However, a generated system() is useful in a lot of situations, ranging from creating portable, system-agnostic configuration files to supporting sysvinit and systemd with the same config. For these reasons, this patch implements a built-in system() generator, that does not rely on the generate-system-source.sh script at all, but does everything itself, from C. It should be used similarly to the existing generator: @module confgen context(source) name(system) system(yes) This will result in the very same system() expansion as the script's as of this patch. The downside is that if one of the generator gets extended, the other must be updated too, to keep them in sync. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/confgen/confgen-plugin.c | 174 +++++++++++++++++++++++++++++++++++++- scl/system/plugin.conf | 5 ++ 2 files changed, 175 insertions(+), 4 deletions(-) diff --git a/modules/confgen/confgen-plugin.c b/modules/confgen/confgen-plugin.c index ffd24b2..90428b9 100644 --- a/modules/confgen/confgen-plugin.c +++ b/modules/confgen/confgen-plugin.c @@ -28,8 +28,13 @@ #include "messages.h" #include "plugin.h" +#include <sys/utsname.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <stdlib.h> #include <string.h> #include <errno.h> +#include <unistd.h> gboolean confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, gpointer user_data) @@ -87,10 +92,157 @@ confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, g return TRUE; } +static void +confgen_sysblock_add_unix_dgram(GString *sysblock, const gchar *path, + const gchar *perms) +{ + g_string_append_printf(sysblock, "unix-dgram(\"%s\"", path); + if (perms) + g_string_append_printf(sysblock, " perms(%s)", perms); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_file(GString *sysblock, const gchar *path, + gint follow_freq, const gchar *prg_override, + const gchar *flags) +{ + g_string_append_printf(sysblock, "file(\"%s\"", path); + if (follow_freq >= 0) + g_string_append_printf(sysblock, " follow-freq(%d)", follow_freq); + if (prg_override) + g_string_append_printf(sysblock, " program-override(\"%s\")", prg_override); + if (flags) + g_string_append_printf(sysblock, " flags(%s)", flags); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_module(GString *sysblock, const gchar *mod) +{ + g_string_append_printf(sysblock, "@module %s\n", mod); +} + +static void +confgen_sysblock_add_sun_streams(GString *sysblock, const gchar *path, + const gchar *door) +{ + g_string_append_printf(sysblock, "sun-streams(\"%s\"", path); + if (door) + g_string_append_printf(sysblock, " door(\"%s\")", door); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_pipe(GString *sysblock, const gchar *path, gint pad_size) +{ + g_string_append_printf(sysblock, "pipe(\"%s\"", path); + if (pad_size >= 0) + g_string_append_printf(sysblock, " pad_size(%d)", pad_size); + g_string_append(sysblock, ");\n"); +} + +gboolean +confgen_generate_system(CfgLexer *lexer, gint type, const gchar *name, + CfgArgs *args, gpointer user_data) +{ + gchar buf[256]; + GString *sysblock; + struct utsname u; + + g_snprintf(buf, sizeof(buf), "%s confgen %s", cfg_lexer_lookup_context_name_by_type(type), name); + if (!cfg_args_validate(args, NULL, buf)) + { + msg_error("confgen: confgen invocations do not process arguments, but your argument list is not empty", + evt_tag_str("context", cfg_lexer_lookup_context_name_by_type(type)), + evt_tag_str("block", name), + NULL); + return FALSE; + } + + sysblock = g_string_sized_new(1024); + + if (uname(&u) != 0) + { + msg_error("confgen: Cannot get information about the running kernel", + evt_tag_errno("error", errno), + NULL); + return FALSE; + } + + if (strcmp(u.sysname, "Linux") == 0) + { + char *log = "/dev/log"; + + if (getenv("LISTEN_FDS") != NULL) + { + struct stat sbuf; + + if (stat("/run/systemd/journal/syslog", &sbuf) == 0) + { + if (S_ISSOCK(sbuf.st_mode)) + log = "/run/systemd/journal/syslog"; + } + } + + confgen_sysblock_add_unix_dgram(sysblock, log, NULL); + confgen_sysblock_add_file(sysblock, "/proc/kmsg", -1, "kernel", "kernel"); + } + else if (strcmp(u.sysname, "SunOS") == 0) + { + confgen_sysblock_add_module(sysblock, "afstreams"); + + if (strcmp(u.release, "5.8") == 0) + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", NULL); + else if (strcmp(u.release, "5.9") == 0) + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", "/etc/.syslog_door"); + else + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", "/var/run/syslog_door"); + } + else if (strcmp(u.sysname, "FreeBSD") == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/log", NULL); + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/logpriv", "0600"); + confgen_sysblock_add_file(sysblock, "/dev/klog", 0, "kernel", "no-parse"); + } + else if (strcmp(u.sysname, "GNU/kFreeBSD") == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/log", NULL); + confgen_sysblock_add_file(sysblock, "/dev/klog", 0, "kernel", NULL); + } + else if (strcmp(u.sysname, "HP-UX") == 0) + { + confgen_sysblock_add_pipe(sysblock, "/dev/pipe", 2048); + } + else if (strcmp(u.sysname, "AIX") == 0 || + strcmp(u.sysname, "OSF1") == 0 || + strncmp(u.sysname, "CYGWIN", 6) == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/dev/log", NULL); + } + else + { + msg_error("system(): Error detecting platform, unable to define the system() source. " + "Please send your system information to the developers!", + evt_tag_str("sysname", u.sysname), + evt_tag_str("release", u.release), + NULL); + return FALSE; + } + + if (!cfg_lexer_include_buffer(lexer, buf, sysblock->str, sysblock->len)) + { + g_string_free(sysblock, TRUE); + return FALSE; + } + + return TRUE; +} + gboolean confgen_module_init(GlobalConfig *cfg, CfgArgs *args) { - const gchar *name, *context, *exec; + const gchar *name, *context, *exec, *sys; name = cfg_args_get(args, "name"); if (!name) @@ -107,13 +259,27 @@ confgen_module_init(GlobalConfig *cfg, CfgArgs *args) return FALSE; } exec = cfg_args_get(args, "exec"); - if (!exec) + sys = cfg_args_get(args, "system"); + + if (!exec && !sys) + { + msg_error("confgen: exec or system argument expected", + NULL); + return FALSE; + } + + if (exec && sys) { - msg_error("confgen: exec argument expected", + msg_error("confgen: exec and system arguments are mutually excluse", NULL); return FALSE; } - cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), name, confgen_generate, g_strdup(exec), g_free); + + if (exec) + cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), name, confgen_generate, g_strdup(exec), g_free); + if (sys) + cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), + name, confgen_generate_system, NULL, NULL); return TRUE; } diff --git a/scl/system/plugin.conf b/scl/system/plugin.conf index 5ba652e..d150fee 100644 --- a/scl/system/plugin.conf +++ b/scl/system/plugin.conf @@ -21,3 +21,8 @@ ############################################################################# @module confgen context(source) name(system) exec("`scl-root`/system/generate-system-source.sh") + +# To use a built-in discovery system, that does not rely on a script, +# comment out the line above, and uncomment the next one: + +#@module confgen context(source) name(system) system(yes) -- 1.7.10

2 2

[PATCH (3.4)] jsonparser: Clone the marker() setting when cloning the parser
by Gergely Nagy 18 Aug '12

18 Aug '12

Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/jsonparser/jsonparser.c | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/jsonparser/jsonparser.c b/modules/jsonparser/jsonparser.c index 7331ad9..891b9f0 100644 --- a/modules/jsonparser/jsonparser.c +++ b/modules/jsonparser/jsonparser.c @@ -201,6 +201,7 @@ log_json_parser_clone (LogPipe *s) cloned = (LogJSONParser *) log_json_parser_new (); log_json_parser_set_prefix ((LogParser *)cloned, self->prefix); + log_json_parser_set_marker ((LogParser *)cloned, self->marker); return &cloned->super.super; } -- 1.7.10

2 2

[Bug 178] New: Spoofed source address bug introduced in 3.3.5
by bugzilla＠bugzilla.balabit.com 10 Aug '12

10 Aug '12

https://bugzilla.balabit.com/show_bug.cgi?id=178 Summary: Spoofed source address bug introduced in 3.3.5 Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Solaris Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: marvin.nipper(a)stream.com Type of the Report: regression Estimated Hours: 0.0 Hi. Sorry if this is not the right way to do this. There was an email thread regarding this on 4/30, but I figured that the right way to do this way to simply open an official bug report. As this is my 1st one for syslog-ng, apologies if I screw it up! This all pertains to 3.3.5, on Solaris 10 x86. Per my original email, changes made between 3.3.4 and 3.3.5 have caused a situation wherein forwarded UDP packets, with spoofed source addressing, all revert to null addresses, the instant that a HUP is issued against syslog-ng (e.g. during log rotation). Also, as per that email, with Gergely's excellent (and educated) guess, I backed out the patch noted here: http://git.madhouse-project.org/debian/syslog-ng/patch/?id=a898014482f733e9…, and the result was that the null addressing issue was resolved. Obviously, as there were (no doubt) other reasons for that patch, I'm just wanting to be sure that this problem doesn't get lost in the shuffle, and that the patch gets reworked (to achieve its originally desired results, while managing to also resolve the null address problem that it caused with spoofing). Let me know if you need any further information regarding this problem. And, as always, I'm MORE than happy to be a test guinea pig, if need be. THANKS. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 21

Syslog-ng OSE 3.3.5 - High CPU usage
by satish pattar 31 Jul '12

31 Jul '12

Hi All, I am using syslog-ng OSE & seeing a high CPU usage ? Could you please point out if this could be a config/rules issue or syslog-ng internal issue. CPU usage in sorted order : 0.0 11.9 16.6 22.2 25.0 25.0 25.0 25.0 33.3 33.3 50.0 ------------------------------------------ # syslog-ng --version syslog-ng 3.3.5 Installer-Version: 3.3.5 Revision: Compile-Date: Apr 20 2012 04:47:44 Default-Modules: afsocket,affile,afprog,afuser,basicfuncs,syslogformat,dbparser Available-Modules: syslogformat,dbparser,afuser,basicfuncs,afmongodb,convertfuncs,afprog,dummy,afsocket,confgen,affile,csvparser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off ----------- Here is the code snippet to send syslog messages to syslogger for(i=0; i<500;i++) { // this is the message format we are following syslog(LOG_INFO, "%s, %s, %d, %s, %s, %s %d, %s", "00:00:18","XYZ",getpid(),"Info","xyz.c", "Func1", 100, "WelCome to Syslog" ); } --------------------- # cat /proc/cpuinfo Processor : ARMv7 Processor rev 10 (v7l) processor : 0 BogoMIPS : 1988.29 processor : 1 BogoMIPS : 1988.29 Features : swp half thumb fastmult vfp edsp neon vfpv3 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 ------------------------------ syslog-ng-confg: @version: 3.3 ###### # options options { # disable the chained hostname format in logs # (default is enabled) chain_hostnames(0); # the time to wait before a died connection is re-established # (default is 60) time_reopen(10); # the time to wait before an idle destination file is closed # (default is 60) time_reap(360); # the number of lines buffered before written to file # you might want to increase this if your disk isn't catching with # all the log messages you get or if you want less disk activity flush_lines(5); flush_timeout(500); # the number of lines fitting in the output queue log_fifo_size(10000); # enable or disable directory creation for destination files create_dirs(yes); # default owner, group, and permissions for log files # (defaults are 0, 0, 0600) #owner(root); group(root); perm(0640); frac_digits(4); # Override the global timestamp format #(set in the global ts_format() parameter) for the specific destination. ts_format(iso); # default owner, group, and permissions for created directories # (defaults are 0, 0, 0700) #dir_owner(root); dir_perm(0755); # enable or disable DNS usage # syslog-ng blocks on DNS queries, so enabling DNS may lead to # a Denial of Service attack # (default is yes) use_dns(no); # maximum length of message in bytes # this is only limited by the program listening on the /dev/log Unix # socket, glibc can handle arbitrary length log messages, but -- for # example -- syslogd accepts only 1024 bytes # (default is 2048) #log_msg_size(2048); # print statistic information only once every hour (it is not really needed) stats_freq(3600); }; ###### # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-dgram("/dev/log" program_override("mylogger: ")); # messages from the kernel file("/proc/kmsg" program_override("kernel: ")); # use the following line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) # udp(); }; ######## #templates #template t_default { template("${ISODATE} ${HOST} ${MSGHDR}${MSG}\n"); template_escape(no); }; #template t_logtty { template("${MSG}\n"); template_escape(no); }; template t_restmsg_notempty { template("${PROGRAM} ${date} ${cname}[${processId}] ${type} (${source}:${line}) ${Restmsg}\n"); }; template t_restmsg_empty { template("${PROGRAM} ${FACILITY} ${LEVEL} ${MSG}\n"); }; ###### # destinations destination d_log_notempty { file("/var/my_log" template(t_restmsg_notempty)); }; #destination d_log_notempty { file("/dev/console" template(t_restmsg_notempty)); }; destination d_log_empty { file("/var/my_log" template(t_restmsg_empty)); }; destination authlog { file("/var/log/auth.log"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination _syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/user.log"); }; destination uucp { file("/var/log/uucp.log"); }; destination mail { file("/var/log/mail.log"); }; destination kern { file("/var/log/kern.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; ###### # filters # all messages from the auth and authpriv facilities filter f_restmsg_notempty {match(".+" value("Restmsg"));}; filter f_restmsg_empty {not match(".+" value("Restmsg"));}; # all messages except from the auth and authpriv facilities filter f_auth { facility(auth); }; filter f_authpriv { not facility(auth, authpriv); }; #filter f_debug { level(notice) and not facility(auth, authpriv, news, mail); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; #filter f_syslog { not facility(authpriv, mail ); }; filter f_syslog { program(syslog-ng); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_mail { facility(mail); }; filter f_user { facility(user); and level(error) }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news); }; filter f_all_info { not match("Info", value("type"), type("string")); }; filter f_all_generic{ not match("Generic", value("cname"), type("string")); }; parser p_db {db_parser( file("/etc/syslog-ng/rules/rules.xml") );}; log {source(s_all);parser(p_db);filter(f_restmsg_notempty);destination(d_log_notempty); }; #log { source(s_all); filter(f_authpriv); destination(authlog); }; log { source(s_all); filter(f_syslog); destination(_syslog); }; log { source(s_all); filter(f_cron); destination(cron); }; log { source(s_all); filter(f_daemon); destination(daemon); }; log { source(s_all); filter(f_kern); destination(kern); }; log { source(s_all); filter(f_mail); destination(mail); }; log { source(s_all); filter(f_user); destination(user); }; #log { source(s_all); filter(f_debug); destination(debug); }; ------------------------------------------------------------------------------------- rules.xml: <patterndb version='4' pub_date='2012-02-17'> <ruleset name='root' id='123456678'> <pattern>mylogger</pattern> <rules> <rule provider='test' id='182437592347598' class='system'> <patterns> <pattern>@ESTRING:date:,@ @ESTRING:cname:,@ @ESTRING:processId:,@ @ESTRING:type:,@ @ESTRING:source:,@ @ESTRING:line:,@ @ANYSTRING:Restmsg: @</pattern> </patterns> </rule> </rules> </ruleset> </patterndb> --------------------------------------------------------------------------------------------- Thanks, pattarsatish

2 1

Installation Error
by kibirango moses 28 Jul '12

28 Jul '12

Hullo users, I am installing syslog-ng ver.3.3.5 on slackware linux 13.1 and i am getting the errors below, when i do a make usr/lib64/gcc/x86_64-slackware-linux/4.4.4/../../../../x86_64-slackware-linux/bin/ld: /usr/local/lib64/libssl.a(s23_srvr.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC /usr/local/lib64/libssl.a: could not read symbols: Bad value Could somebody helpme with a fix to the problem thanx in advnace

2 1

Logging To MongoDB and specifying data types
by Michael Dillon 27 Jul '12

27 Jul '12

I'm using the syslog-ng windows client to forward IIS logs to syslog-ng. I'm using the CSV parser to split it into pairs and pick and choose my pairs. I'm not able to figure out how to insert a field from these pairs (or any others) into MongoDB as a date time field. They are being stored as strings. Also I'd like to store quite a few of the other columns/elements as integers. Obviously I could run a job to do conversions after they are inserted into the database I'd just hate to setup a rube golberg machine if I don't have to. -- Michael Dillon

2 1

Able to connect but not logging
by kibirango moses 27 Jul '12

27 Jul '12

Hullo Users; I am using syslog-ng 3.3.4 and my client is able to connect but not logging to the central logging server. my syslog-ng.conf on server side is @version: 3.3 # Default syslog-ng configuration for Slackware # For info about the format of this file, see "man syslog-ng.conf" # Converted from sysklogd (syslog.conf) by mario(a)slackverse.org options { flush_lines (0); time_reopen (10); time_reap(360); # log_fifo_size (10240); # log_msg_size (8192); chain_hostnames (no); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); owner("root"); group("root"); dir_owner(root); # dir_group(logs); dir_perm(0750); perm(0640); stats_freq (3600); check_hostname (yes); dns_cache (yes); threaded(yes); mark_freq(600); }; source system { internal(); unix-stream("/dev/log"); udp(); tcp(max_connections(100)); }; filter f_messages { level(info,notice) and not facility(authpriv,cron,mail,news); }; filter f_syslog { level(warn..emerg) and not facility(authpriv,cron,mail,news); }; filter f_debug { level(debug); }; filter f_authpriv { facility(authpriv); }; filter f_cron { facility(cron); }; filter f_mail { facility(mail); }; filter f_emerg { level(emerg); }; filter f_uucp { facility(uucp); }; destination d_messages { file("/var/log/messages"); }; destination d_syslog { file("/var/log/syslog"); }; destination d_debug { file("/var/log/debug"); }; destination d_secure { file("/var/log/secure"); }; destination d_cron { file("/var/log/cron"); }; destination d_maillog { file("/var/log/maillog"); }; destination d_usertty { usertty("*"); }; destination d_spooler { file("/var/log/spooler"); }; # Log anything 'info' or higher, but lower than 'warn'. # Exclude authpriv, cron, mail, and news. These are logged elsewhere. log { source(system); filter(f_messages); destination(d_messages); }; # Log anything 'warn' or higher. # Exclude authpriv, cron, mail, and news. These are logged elsewhere. log { source(system); filter(f_syslog); destination(d_syslog); }; # Debugging information is logged here. log { source(system); filter(f_debug); destination(d_debug); }; # Private authentication message logging: log { source(system); filter(f_authpriv); destination(d_secure); }; # Cron related logs: log { source(system); filter(f_cron); destination(d_cron); }; # Mail related logs: log { source(system); filter(f_mail); destination(d_maillog); }; # Emergency level messages go to all users: log { source(system); filter(f_emerg); destination(d_usertty); }; # This log is for news and uucp errors: log { source(system); filter(f_uucp); destination(d_spooler); }; # Uncomment this to see kernel messages on the console. #filter f_kern { facility(kern); }; #destination d_console { file("/dev/console"); }; #log { source(system); filter(f_kern); destination(d_console); }; # Uncomment these if you'd like INN to keep logs on everything. # You won't need this if you don't run INN (the InterNetNews daemon). #filter f_news_crit { facility(news) and level(crit); }; #filter f_news_err { facility(news) and level(err); }; #filter f_news_notice { facility(news) and level(notice); }; #destination d_news_crit { file("/var/log/news/news.crit"); }; #destination d_news_err { file("/var/log/news/news.err"); }; #destination d_news_notice { file("/var/log/news/news.notice"); }; #log { source(system); filter(f_news_crit); destination(d_news_crit); }; #log { source(system); filter(f_news_err); destination(d_news_err); }; #log { source(system); filter(f_news_notice); destination(f_news_notice); }; # Remote logging source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; destination logpile { file("/var/log/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) create_dirs(yes) dir_perm(0700)); }; log { source(system); destination(logpile); }; Cleint syslog-ng.conf I just added the lines belo to the server file above # Remote logging destination logpile { file("/var/log/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) create_dirs(yes) dir_perm(0700)); }; destination remote { tcp("196.43.133.98" port(514)); }; log { source(system); destination(logpile); }; log { source(system); destination(remote); }; testing for connection root@client:/# lsof -i | grep syslog-ng syslog-ng 18339 root 9u IPv4 37485820 0t0 UDP *:syslog syslog-ng 18339 root 10u IPv4 37485822 0t0 TCP *:shell (LISTEN) syslog-ng 18339 root 11u IPv4 37485824 0t0 TCP client:48305-> syslog server:shell (ESTABLISHED) I am not seeing any logging being registered . How can i solve this and where have i gone wrong. Thanx i advance

2 1

[Bug 187] New: Syslog-ng truncates SDATA PARAM VALUE to 256 chars
by bugzilla＠bugzilla.balabit.com 27 Jul '12

27 Jul '12

https://bugzilla.balabit.com/show_bug.cgi?id=187 Summary: Syslog-ng truncates SDATA PARAM VALUE to 256 chars Product: syslog-ng Version: 3.2.x Platform: All OS/Version: All Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: cloun-rulez(a)yandex.ru QAContact: cloun-rulez(a)yandex.ru Type of the Report: enhancement Estimated Hours: 0.0 As you can see there are no limitations of SDATA PARAM VALUE length in rfc5424 http://tools.ietf.org/html/rfc5424 As I have saw in tests/unit/test_msgparse.c this is default behavior of syslog-ng. // too long sdata value gets truncated testcase("<132>1 2006-10-29T01:59:59.156+01:00 mymachine evntslog - - [a i=\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"] An application event log entry...", LP_SYSLOG_PROTOCOL, NULL, 132, // pri 1162083599, 156000, 3600, // timestamp (sec/usec/zone) "mymachine", // host "evntslog", //app "An application event log entry...", // msg "[a i=\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"]", //sd_str "",//processid "",//msgid expected_sd_pairs_test_5b ); I'm trying to increase limit of truncation but still haven't success. Need help. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5