- syslog-ng - lists.balabit.hu

[PATCH] [cfg] make dns_cache_hosts default to /etc/hosts
by balint.kovacs＠balabit.com 25 Aug '12

25 Aug '12

From: Balint Kovacs <balint.kovacs(a)balabit.com> The default for dns_cache_hosts was empty until now, which resulted in use_dns(persist_only) not to work out-of-the box until the user set the dns_cache_hosts to a sane value. This should make the configuration more intuitive in most use-cases. Signed-off-by: Balint Kovacs <blint(a)balabit.hu> --- lib/cfg.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/lib/cfg.c b/lib/cfg.c index ed4cedf..8ba22df 100644 --- a/lib/cfg.c +++ b/lib/cfg.c @@ -315,6 +315,7 @@ cfg_new(gint version) self->dns_cache_size = 1007; self->dns_cache_expire = 3600; self->dns_cache_expire_failed = 60; + self->dns_cache_hosts = "/etc/hosts"; self->threaded = FALSE; log_template_options_defaults(&self->template_options); -- 1.7.0.4

2 1

[RFC]: $(format-json) improvements, and other things
by Gergely Nagy 23 Aug '12

23 Aug '12

Hi! An old goal we had (and which was a GSoC project for a while)) is to support nested structures in $(format-json), so that you can have output that looks like this: {"msg":{"text":"Hi, I can has structure!"}} Sadly, $(format-json) was not able to do that, as all keys within syslog-ng are stored in a flat structure, that has strong resemblance to JSON's dot notation, and format-json was not able to translate from that notation back to nested structures. With a set of patches I just pushed to my repository at github[1] (feature/3.4/value-pairs/walk branch), this limitation is no more. If we have a template like the following: $(format-json msg.text='Hi, I can has structure!') Then the output will be nested, like the first JSON showed in this mail. The implementation is fairly inefficient at the moment, and has a few bugs in it still, but the API looks and feels resonable, so I figured I'll give you all a heads up on what to expect in the not too distant future. The branch also merges the tfjson and jsonparser modules, and drops json-glib support: json-c is required for both format-json and the json-parser, and they come in the same module now (json-plugin). The work done on this branch also paves way for major changes within the MongoDB destination driver: moving away from upserts to inserts, and then on to batch inserts, from which I expect a significant performance boost. Code reviews, comments and whatnot are most appreciated. Do keep in mind however, that as I work, I will likely rebase the branch a few more times, without notice. [1]: https://github.com/algernon/syslog-ng/commits/feature/3.4/value-pairs/walk -- |8]

1 1

syslog-ng 3.4.0alpha2 failed to start using ipv6 address.
by huican ping 22 Aug '12

22 Aug '12

I tried to make syslog-ng listening on ipv6 network interface, but it failed to start. Can anyone help me? my syslog-ng version is: syslog-ng 3.4.0alpha2 /localdisk/syslog-ng-3.4.0alpha2>syslog-ng WARNING: Configuration file format is too old, please update it to use the 3.4 format as some constructs might operate inefficiently; Error resolving hostname; host='fdfb:8edf:1466::191' Error initializing message pipeline; My simple config file is: I already put the line below at /etc/hosts file: fdfb:8edf:1466::191 iclab098v6 iclab098v6.ic.intel.com ================== @version: 3.3 @include "scl.conf" options { use_dns(persist_only); dns_cache_hosts(/etc/hosts); }; source s_local { system(); internal(); }; source s_network { udp(ip(10.230.34.191) port(514)); tcp(ip('fdfb:8edf:1466::191') port(514)); }; destination d_local { file("/var/log/syslogmessages"); }; log { source(s_local); source(s_network); destination(d_local); };

1 1

include all config files under etc/include/
by sagar naravane 22 Aug '12

22 Aug '12

Hi, My syslog-ng config location is is /apps/syslog-ng/etc/ I want to include all *.conf under /apps/syslog-ng/etc/include/ How do i do that ? I've tried following : 1.* @include "include/"* < This works but I want to restrict it to .conf only 2.* @include "/apps/syslog-ng/etc/include/*.conf"* < DOESN'T WORK 3. *include "/apps/syslog-ng/etc/include/*.conf"* < DOESN'T WORK Regards, Sagar Naravane

2 1

Auto routing to specific relay based on hostname
by sagar naravane 22 Aug '12

22 Aug '12

Guys, I am a newbie to syslog-ng solution. I am trying to create a common config file to be distributed across client machines in San Jose, Dallas and Dublin colos. Every colo has 1 dedicated relay which forwards log data to a central server (SJ|DU|DA)_ client --> (SJ|DU|DA)_ relay --> SYSLOG-NG server the clients located in these colos have sj,du & da in their hostnames eg da1xxx058, du1xxx059, sj1xxx060. I want to setup the config in a way that based on hostname the config decides what relay to connect to. Is this possible ? -- Regards, Sagar Naravane

2 6

syslog-ng with MySQL
by James McDonald 21 Aug '12

21 Aug '12

Trying to incorporate a MySQL db (on a separate server) with my two syslog-ng servers, so that the logs can be searchable. Does anyone have any experience with this? Any tips or documentation would be helpful. I didn't find a whole lot on the Balabit website when I searched for it.

4 10

special characters in filenames
by Daniel Neubacher 21 Aug '12

21 Aug '12

Hello there, I've got a pretty annoying problem with syslog-ng 3.3.5 and log filenames. I think my syslog-ng.cfg is a little unconventional because I've configured for each log file a file statement and a uniq program name to keep the filenames from the client on the server. The cfg I've attached bellow is only a small part of it because I can't show you everything. In the original syslog-ng.cfg I've configured about 120 individual log files. Now to the problem. Each day I'm getting log file names like this on my log server: /log/applogs/2012/08/17/perl-1.hk.fra1.xing.com/t.file(d_applogs#0,/log/applogs/2012/08/17/mx2-2.xing.com/syslog-ng)=0', /log/applogs/2012/08/15/syslog-2.log.fra2.xing.com/ropped='dst.file(d_syslog#0,/log/syslog /log/syslog/2012/08/17/ fai-1.ops.fra1 fai? This is pretty annoying because my script which is compressing the logs is failing on these files. Does anybody know this problem? Server cfg: @version: 3.3 options { threaded(yes); owner("root"); group("root"); perm(0660); dir_owner("root"); dir_group("root"); dir_perm(0770); create_dirs(yes); stats_freq(600); stats_level(2); chain_hostnames(no); check_hostname(yes); keep_hostname(no); dns_cache(yes); dns_cache_size(16384); dns_cache_expire(3600); dns_cache_expire_failed(60); use_fqdn(yes); log_msg_size(128000); log_fifo_size(1000000); }; template t_plain { template("$MSG\n"); template_escape(no); }; filter f_syslog { program(^auth.log) or program(^cron.log) or program(^daemon.log) or program(^kern.log) or program(^lpr.log) or program(^mail.log) or program(^syslog.log) or program(^user.log) or program(^uucp.log) or program(^mail.info) or program(^mail.warn) or program(^mail.err) or program(^news.crit) or program(^news.err) or program(^news.notice) or program(^debug.log) or program(^error.log) or program(^messages.log) or program(^ppp.log); }; filter f_applogs { not filter(f_syslog) and program(".*.log"); }; source s_src { unix-dgram("/dev/log" max-connections(500)); internal(); file("/proc/kmsg" program_override("kernel")); }; source s_net { udp( log_fetch_limit(400) so_rcvbuf(51200000) keep_timestamp(yes) port(514) ); tcp( max-connections(1000) so_rcvbuf(51200000) so_keepalive(yes) keep_timestamp(yes) port(514) ); syslog(); }; destination d_messages { file("/var/log/messages"); }; destination d_syslog { file("/log/syslog/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST/$PROGRAM" template(t_plain)); }; destination d_syslog_onefile { file("/log/syslog_onefile/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST"); }; destination d_applogs { file("/log/applogs/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST/$PROGRAM" template(t_plain)); }; destination d_perlhk {tcp("perl-1.hk.fra1.xing.com" port(514));}; log { source(s_src); destination(d_messages); }; log { source(s_net); filter(f_syslog); destination(d_syslog); }; log { source(s_net); filter(f_syslog); destination(d_syslog_onefile); }; log { source(s_net); filter(f_applogs); destination(d_applogs); }; log { source(s_net); filter(f_applogs); destination(d_perlhk); }; Client: @version: 3.3 options { threaded(yes); use_dns(yes); use_fqdn(yes); dns_cache(yes); dns_cache_size(16384); dns_cache_expire(3600); dns_cache_expire_failed(10); log_msg_size(128000); log_fifo_size(100000); normalize_hostnames(yes); check_hostname(yes); bad_hostname("^gconfd$"); create_dirs(yes); owner("root"); group("root"); perm(0640); stats_freq(3600); time_reopen(30); }; # Applogs source s_perl_applogs { file(/www/applogs/admin.log follow_freq(1) flags(no-parse) program_override(admin.log)); file(/www/applogs/fcgid.log follow_freq(1) flags(no-parse) program_override(fcgid.log)); }; ##Ruby source s_ruby_applogs { file("/virtual/cra/shared/log/production.log" follow_freq(1) flags(no-parse) program_override(production.log)); }; source s_syslog { file("/var/log/auth.log" follow_freq(1) flags(no-parse) program_override("auth.log")); file("/var/log/cron.log" follow_freq(1) flags(no-parse) program_override("cron.log")); file("/var/log/daemon.log" follow_freq(1) flags(no-parse) program_override("daemon.log")); file("/var/log/kern.log" follow_freq(1) flags(no-parse) program_override("kern.log")); file("/var/log/lpr.log" follow_freq(1) flags(no-parse) program_override("lpr.log")); file("/var/log/mail.log" follow_freq(1) flags(no-parse) program_override("mail.log")); file("/var/log/syslog" follow_freq(1) flags(no-parse) program_override("syslog.log")); file("/var/log/user.log" follow_freq(1) flags(no-parse) program_override("user.log")); file("/var/log/uucp.log" follow_freq(1) flags(no-parse) program_override("uucp.log")); file("/var/log/mail/mail.info" follow_freq(1) flags(no-parse) program_override("mail.info")); file("/var/log/mail/mail.warn" follow_freq(1) flags(no-parse) program_override("mail.warn")); file("/var/log/mail/mail.err" follow_freq(1) flags(no-parse) program_override("mail.err")); file("/var/log/news/news.crit" follow_freq(1) flags(no-parse) program_override("news.crit")); file("/var/log/news/news.err" follow_freq(1) flags(no-parse) program_override("news.err")); file("/var/log/news/news.notice" follow_freq(1) flags(no-parse) program_override("news.notice")); file("/var/log/debug" follow_freq(1) flags(no-parse) program_override("debug.log")); file("/var/log/error" follow_freq(1) flags(no-parse) program_override("error.log")); file("/var/log/messages" follow_freq(1) flags(no-parse) program_override("messages.log")); file("/var/log/ppp.log" follow_freq(1) flags(no-parse) program_override("ppp.log")); }; destination syslog-1.log.fra1 { udp("syslog-1.log.fra1.xing.com" port(514)); }; destination syslog-2.log.fra1 { tcp("syslog-2.log.fra1.xing.com" port(514)); }; destination syslog-1.log.fra2 { tcp("syslog-1.log.fra2.xing.com" port(514)); }; destination syslog-2.log.fra2 { tcp("syslog-2.log.fra2.xing.com" port(514)); }; log { source(s_all); destination(syslog-1.log.fra1); }; log { source(s_syslog); source(s_perl_applogs); source(s_ruby_applogs); destination(syslog-2.log.fra1); destination(syslog-1.log.fra2); destination(syslog-2.log.fra2); }; -- Daniel Neubacher, Network Administrator daniel.neubacher(a)xing.com<mailto:daniel.neubacher@xing.com> XING AG Gaensemarkt 43, 20354 Hamburg, Germany Tel. +49 40 419131-28, Fax +49 40 419131-11 Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807 Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.

3 3

[PATCH (3.4)] confgen: Implement a built-in system() generator
by Gergely Nagy 19 Aug '12

19 Aug '12

The system SCL plugin relies on a script (generate-system-source.sh) to generate the block system() expands into. While this is flexible and easy to modifiy, it has the disadvantage of running an external script at startup, which is undesirable in many situations, including when using a strict AppArmor profile. However, a generated system() is useful in a lot of situations, ranging from creating portable, system-agnostic configuration files to supporting sysvinit and systemd with the same config. For these reasons, this patch implements a built-in system() generator, that does not rely on the generate-system-source.sh script at all, but does everything itself, from C. It should be used similarly to the existing generator: @module confgen context(source) name(system) system(yes) This will result in the very same system() expansion as the script's as of this patch. The downside is that if one of the generator gets extended, the other must be updated too, to keep them in sync. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/confgen/confgen-plugin.c | 174 +++++++++++++++++++++++++++++++++++++- scl/system/plugin.conf | 5 ++ 2 files changed, 175 insertions(+), 4 deletions(-) diff --git a/modules/confgen/confgen-plugin.c b/modules/confgen/confgen-plugin.c index ffd24b2..90428b9 100644 --- a/modules/confgen/confgen-plugin.c +++ b/modules/confgen/confgen-plugin.c @@ -28,8 +28,13 @@ #include "messages.h" #include "plugin.h" +#include <sys/utsname.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <stdlib.h> #include <string.h> #include <errno.h> +#include <unistd.h> gboolean confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, gpointer user_data) @@ -87,10 +92,157 @@ confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, g return TRUE; } +static void +confgen_sysblock_add_unix_dgram(GString *sysblock, const gchar *path, + const gchar *perms) +{ + g_string_append_printf(sysblock, "unix-dgram(\"%s\"", path); + if (perms) + g_string_append_printf(sysblock, " perms(%s)", perms); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_file(GString *sysblock, const gchar *path, + gint follow_freq, const gchar *prg_override, + const gchar *flags) +{ + g_string_append_printf(sysblock, "file(\"%s\"", path); + if (follow_freq >= 0) + g_string_append_printf(sysblock, " follow-freq(%d)", follow_freq); + if (prg_override) + g_string_append_printf(sysblock, " program-override(\"%s\")", prg_override); + if (flags) + g_string_append_printf(sysblock, " flags(%s)", flags); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_module(GString *sysblock, const gchar *mod) +{ + g_string_append_printf(sysblock, "@module %s\n", mod); +} + +static void +confgen_sysblock_add_sun_streams(GString *sysblock, const gchar *path, + const gchar *door) +{ + g_string_append_printf(sysblock, "sun-streams(\"%s\"", path); + if (door) + g_string_append_printf(sysblock, " door(\"%s\")", door); + g_string_append(sysblock, ");\n"); +} + +static void +confgen_sysblock_add_pipe(GString *sysblock, const gchar *path, gint pad_size) +{ + g_string_append_printf(sysblock, "pipe(\"%s\"", path); + if (pad_size >= 0) + g_string_append_printf(sysblock, " pad_size(%d)", pad_size); + g_string_append(sysblock, ");\n"); +} + +gboolean +confgen_generate_system(CfgLexer *lexer, gint type, const gchar *name, + CfgArgs *args, gpointer user_data) +{ + gchar buf[256]; + GString *sysblock; + struct utsname u; + + g_snprintf(buf, sizeof(buf), "%s confgen %s", cfg_lexer_lookup_context_name_by_type(type), name); + if (!cfg_args_validate(args, NULL, buf)) + { + msg_error("confgen: confgen invocations do not process arguments, but your argument list is not empty", + evt_tag_str("context", cfg_lexer_lookup_context_name_by_type(type)), + evt_tag_str("block", name), + NULL); + return FALSE; + } + + sysblock = g_string_sized_new(1024); + + if (uname(&u) != 0) + { + msg_error("confgen: Cannot get information about the running kernel", + evt_tag_errno("error", errno), + NULL); + return FALSE; + } + + if (strcmp(u.sysname, "Linux") == 0) + { + char *log = "/dev/log"; + + if (getenv("LISTEN_FDS") != NULL) + { + struct stat sbuf; + + if (stat("/run/systemd/journal/syslog", &sbuf) == 0) + { + if (S_ISSOCK(sbuf.st_mode)) + log = "/run/systemd/journal/syslog"; + } + } + + confgen_sysblock_add_unix_dgram(sysblock, log, NULL); + confgen_sysblock_add_file(sysblock, "/proc/kmsg", -1, "kernel", "kernel"); + } + else if (strcmp(u.sysname, "SunOS") == 0) + { + confgen_sysblock_add_module(sysblock, "afstreams"); + + if (strcmp(u.release, "5.8") == 0) + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", NULL); + else if (strcmp(u.release, "5.9") == 0) + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", "/etc/.syslog_door"); + else + confgen_sysblock_add_sun_streams(sysblock, "/dev/log", "/var/run/syslog_door"); + } + else if (strcmp(u.sysname, "FreeBSD") == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/log", NULL); + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/logpriv", "0600"); + confgen_sysblock_add_file(sysblock, "/dev/klog", 0, "kernel", "no-parse"); + } + else if (strcmp(u.sysname, "GNU/kFreeBSD") == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/var/run/log", NULL); + confgen_sysblock_add_file(sysblock, "/dev/klog", 0, "kernel", NULL); + } + else if (strcmp(u.sysname, "HP-UX") == 0) + { + confgen_sysblock_add_pipe(sysblock, "/dev/pipe", 2048); + } + else if (strcmp(u.sysname, "AIX") == 0 || + strcmp(u.sysname, "OSF1") == 0 || + strncmp(u.sysname, "CYGWIN", 6) == 0) + { + confgen_sysblock_add_unix_dgram(sysblock, "/dev/log", NULL); + } + else + { + msg_error("system(): Error detecting platform, unable to define the system() source. " + "Please send your system information to the developers!", + evt_tag_str("sysname", u.sysname), + evt_tag_str("release", u.release), + NULL); + return FALSE; + } + + if (!cfg_lexer_include_buffer(lexer, buf, sysblock->str, sysblock->len)) + { + g_string_free(sysblock, TRUE); + return FALSE; + } + + return TRUE; +} + gboolean confgen_module_init(GlobalConfig *cfg, CfgArgs *args) { - const gchar *name, *context, *exec; + const gchar *name, *context, *exec, *sys; name = cfg_args_get(args, "name"); if (!name) @@ -107,13 +259,27 @@ confgen_module_init(GlobalConfig *cfg, CfgArgs *args) return FALSE; } exec = cfg_args_get(args, "exec"); - if (!exec) + sys = cfg_args_get(args, "system"); + + if (!exec && !sys) + { + msg_error("confgen: exec or system argument expected", + NULL); + return FALSE; + } + + if (exec && sys) { - msg_error("confgen: exec argument expected", + msg_error("confgen: exec and system arguments are mutually excluse", NULL); return FALSE; } - cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), name, confgen_generate, g_strdup(exec), g_free); + + if (exec) + cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), name, confgen_generate, g_strdup(exec), g_free); + if (sys) + cfg_lexer_register_block_generator(cfg->lexer, cfg_lexer_lookup_context_type_by_name(context), + name, confgen_generate_system, NULL, NULL); return TRUE; } diff --git a/scl/system/plugin.conf b/scl/system/plugin.conf index 5ba652e..d150fee 100644 --- a/scl/system/plugin.conf +++ b/scl/system/plugin.conf @@ -21,3 +21,8 @@ ############################################################################# @module confgen context(source) name(system) exec("`scl-root`/system/generate-system-source.sh") + +# To use a built-in discovery system, that does not rely on a script, +# comment out the line above, and uncomment the next one: + +#@module confgen context(source) name(system) system(yes) -- 1.7.10

2 2

[PATCH (3.4)] afmongodb: Support connecting to unix sockets
by Gergely Nagy 18 Aug '12

18 Aug '12

The next version of libmongo-client (0.1.6) will support connecting via unix sockets, and the mongodb destination driver should follow. To achieve this, the internal copy was updated to a version that already supports this, and the minimum required version was also bumped to 0.1.6. With that in place, the path() option is introduced, which, when set, will tell the driver to connect via the specified path to a unix socket. It does this by setting port to MONGO_CONN_LOCAL, that's how a path is recognised by libmongo-client. However, setting path() conflicts with host(), port() and servers(): the first cannot be used when any of the other three was previously set, nor the other way around. The configuration grammar was updated to notice this case, and bail out with an error if such a scenario is detected. Based on the work of Conrad Hoffmann <ch(a)bitfehler.net>. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 2 +- modules/afmongodb/afmongodb-grammar.ym | 26 ++++++- modules/afmongodb/afmongodb-parser.c | 1 + modules/afmongodb/afmongodb.c | 118 +++++++++++++++++++++++--------- modules/afmongodb/afmongodb.h | 3 + modules/afmongodb/libmongo-client | 2 +- 6 files changed, 115 insertions(+), 37 deletions(-) diff --git a/configure.in b/configure.in index a0410ed..294066a 100644 --- a/configure.in +++ b/configure.in @@ -28,7 +28,7 @@ IVYKIS_MIN_VERSION="0.30.1" JSON_C_MIN_VERSION="0.9" JSON_GLIB_MIN_VERSION="0.12" PCRE_MIN_VERSION="6.1" -LMC_MIN_VERSION="0.1.0" +LMC_MIN_VERSION="0.1.6" dnl *************************************************************************** dnl Initial setup diff --git a/modules/afmongodb/afmongodb-grammar.ym b/modules/afmongodb/afmongodb-grammar.ym index 71dd46d..f108f26 100644 --- a/modules/afmongodb/afmongodb-grammar.ym +++ b/modules/afmongodb/afmongodb-grammar.ym @@ -52,6 +52,7 @@ extern ValuePairsTransformSet *last_vp_transset; %token KW_COLLECTION %token KW_SERVERS %token KW_SAFE_MODE +%token KW_PATH %% @@ -69,14 +70,33 @@ afmongodb_options ; afmongodb_option - : KW_SERVERS '(' string_list ')' { afmongodb_dd_set_servers(last_driver, $3); } + : KW_SERVERS '(' string_list ')' + { + CHECK_ERROR(afmongodb_dd_check_address(last_driver, FALSE), @3, "Can't mix path() & servers()"); + afmongodb_dd_set_servers(last_driver, $3); + } + | KW_HOST '(' string ')' + { + CHECK_ERROR(afmongodb_dd_check_address(last_driver, FALSE), @3, "Can't mix path() & host()"); + afmongodb_dd_set_host(last_driver, $3); + free($3); + } + | KW_PORT '(' LL_NUMBER ')' + { + CHECK_ERROR(afmongodb_dd_check_address(last_driver, FALSE), @3, "Can't mix path() & port()"); + afmongodb_dd_set_port(last_driver, $3); + } + | KW_PATH '(' string ')' + { + CHECK_ERROR(afmongodb_dd_check_address(last_driver, TRUE), @3, "Can't mix path() with host() or server()"); + afmongodb_dd_set_path(last_driver, $3); + free($3); + } | KW_DATABASE '(' string ')' { afmongodb_dd_set_database(last_driver, $3); free($3); } | KW_COLLECTION '(' string ')' { afmongodb_dd_set_collection(last_driver, $3); free($3); } | KW_USERNAME '(' string ')' { afmongodb_dd_set_user(last_driver, $3); free($3); } | KW_PASSWORD '(' string ')' { afmongodb_dd_set_password(last_driver, $3); free($3); } | KW_SAFE_MODE '(' yesno ')' { afmongodb_dd_set_safe_mode(last_driver, $3); } - | KW_HOST '(' string ')' { afmongodb_dd_set_host(last_driver, $3); free($3); } - | KW_PORT '(' LL_NUMBER ')' { afmongodb_dd_set_port(last_driver, $3); } | value_pair_option { afmongodb_dd_set_value_pairs(last_driver, $1); } | dest_driver_option ; diff --git a/modules/afmongodb/afmongodb-parser.c b/modules/afmongodb/afmongodb-parser.c index 9d72878..8aa55c1 100644 --- a/modules/afmongodb/afmongodb-parser.c +++ b/modules/afmongodb/afmongodb-parser.c @@ -38,6 +38,7 @@ static CfgLexerKeyword afmongodb_keywords[] = { { "safe_mode", KW_SAFE_MODE }, { "host", KW_HOST }, { "port", KW_PORT }, + { "path", KW_PATH }, { NULL } }; diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 5af34c9..aeed3d3 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -50,7 +50,7 @@ typedef struct gchar *coll; GList *servers; - gchar *host; + gchar *address; gint port; gboolean safe_mode; @@ -116,8 +116,8 @@ afmongodb_dd_set_host(LogDriver *d, const gchar *host) { MongoDBDestDriver *self = (MongoDBDestDriver *)d; - g_free(self->host); - self->host = g_strdup(host); + g_free(self->address); + self->address = g_strdup(host); } void @@ -138,6 +138,39 @@ afmongodb_dd_set_servers(LogDriver *d, GList *servers) } void +afmongodb_dd_set_path(LogDriver *d, const gchar *path) +{ + MongoDBDestDriver *self = (MongoDBDestDriver *)d; + + g_free(self->address); + self->address = g_strdup(path); + self->port = MONGO_CONN_LOCAL; +} + +gboolean +afmongodb_dd_check_address(LogDriver *d, gboolean local) +{ + MongoDBDestDriver *self = (MongoDBDestDriver *)d; + + if (local) + { + if ((self->port != 0 || + self->port != MONGO_CONN_LOCAL) && + self->address != NULL) + return FALSE; + if (self->servers) + return FALSE; + } + else + { + if (self->port == MONGO_CONN_LOCAL && + self->address != NULL) + return FALSE; + } + return TRUE; +} + +void afmongodb_dd_set_database(LogDriver *d, const gchar *database) { MongoDBDestDriver *self = (MongoDBDestDriver *)d; @@ -182,8 +215,12 @@ afmongodb_dd_format_stats_instance(MongoDBDestDriver *self) { static gchar persist_name[1024]; - g_snprintf(persist_name, sizeof(persist_name), - "mongodb,%s,%u,%s,%s", self->host, self->port, self->db, self->coll); + if (self->port == MONGO_CONN_LOCAL) + g_snprintf(persist_name, sizeof(persist_name), + "mongodb,%s,%s,%s", self->address, self->db, self->coll); + else + g_snprintf(persist_name, sizeof(persist_name), + "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); return persist_name; } @@ -192,8 +229,12 @@ afmongodb_dd_format_persist_name(MongoDBDestDriver *self) { static gchar persist_name[1024]; - g_snprintf(persist_name, sizeof(persist_name), - "afmongodb(%s,%u,%s,%s)", self->host, self->port, self->db, self->coll); + if (self->port == MONGO_CONN_LOCAL) + g_snprintf(persist_name, sizeof(persist_name), + "afmongodb(%s,%s,%s)", self->address, self->db, self->coll); + else + g_snprintf(persist_name, sizeof(persist_name), + "afmongodb(%s,%u,%s,%s)", self->address, self->port, self->db, self->coll); return persist_name; } @@ -221,7 +262,7 @@ afmongodb_dd_connect(MongoDBDestDriver *self, gboolean reconnect) if (reconnect && self->conn) return TRUE; - self->conn = mongo_sync_connect(self->host, self->port, FALSE); + self->conn = mongo_sync_connect(self->address, self->port, FALSE); if (!self->conn) { msg_error ("Error connecting to MongoDB", NULL); @@ -467,31 +508,45 @@ afmongodb_dd_init(LogPipe *s) value_pairs_add_scope(self->vp, "nv-pairs"); } - if (self->host) - { - gchar *srv = g_strdup_printf ("%s:%d", self->host, - (self->port) ? self->port : 27017); - self->servers = g_list_prepend (self->servers, srv); - g_free (self->host); - } - - self->host = NULL; - self->port = 27017; - if (!mongo_util_parse_addr(g_list_nth_data(self->servers, 0), &self->host, - &self->port)) + if (self->port != MONGO_CONN_LOCAL) { - msg_error("Cannot parse the primary host", - evt_tag_str("primary", g_list_nth_data(self->servers, 0)), - NULL); - return FALSE; + if (self->address) + { + gchar *srv = g_strdup_printf ("%s:%d", self->address, + (self->port) ? self->port : 27017); + self->servers = g_list_prepend (self->servers, srv); + g_free (self->address); + } + + if (!self->servers) + afmongodb_dd_set_servers((LogDriver *)self, g_list_append (NULL, g_strdup ("127.0.0.1:27017"))); + + self->address = NULL; + self->port = 27017; + if (!mongo_util_parse_addr(g_list_nth_data(self->servers, 0), + &self->address, + &self->port)) + { + msg_error("Cannot parse the primary host", + evt_tag_str("primary", g_list_nth_data(self->servers, 0)), + NULL); + return FALSE; + } } - msg_verbose("Initializing MongoDB destination", - evt_tag_str("host", self->host), - evt_tag_int("port", self->port), - evt_tag_str("database", self->db), - evt_tag_str("collection", self->coll), - NULL); + if (self->port == MONGO_CONN_LOCAL) + msg_verbose("Initializing MongoDB destination", + evt_tag_str("address", self->address), + evt_tag_str("database", self->db), + evt_tag_str("collection", self->coll), + NULL); + else + msg_verbose("Initializing MongoDB destination", + evt_tag_str("address", self->address), + evt_tag_int("port", self->port), + evt_tag_str("database", self->db), + evt_tag_str("collection", self->coll), + NULL); self->queue = log_dest_driver_acquire_queue(&self->super, afmongodb_dd_format_persist_name(self)); @@ -548,7 +603,7 @@ afmongodb_dd_free(LogPipe *d) g_free(self->coll); g_free(self->user); g_free(self->password); - g_free(self->host); + g_free(self->address); string_list_free(self->servers); if (self->vp) value_pairs_free(self->vp); @@ -609,7 +664,6 @@ afmongodb_dd_new(void) self->super.super.super.queue = afmongodb_dd_queue; self->super.super.super.free_fn = afmongodb_dd_free; - afmongodb_dd_set_servers((LogDriver *)self, g_list_append (NULL, g_strdup ("127.0.0.1:27017"))); afmongodb_dd_set_database((LogDriver *)self, "syslog"); afmongodb_dd_set_collection((LogDriver *)self, "messages"); afmongodb_dd_set_safe_mode((LogDriver *)self, FALSE); diff --git a/modules/afmongodb/afmongodb.h b/modules/afmongodb/afmongodb.h index e1f4b57..6a61616 100644 --- a/modules/afmongodb/afmongodb.h +++ b/modules/afmongodb/afmongodb.h @@ -38,5 +38,8 @@ void afmongodb_dd_set_user(LogDriver *d, const gchar *user); void afmongodb_dd_set_password(LogDriver *d, const gchar *password); void afmongodb_dd_set_value_pairs(LogDriver *d, ValuePairs *vp); void afmongodb_dd_set_safe_mode(LogDriver *d, gboolean state); +void afmongodb_dd_set_path(LogDriver *d, const gchar *path); + +gboolean afmongodb_dd_check_address(LogDriver *d, gboolean local); #endif diff --git a/modules/afmongodb/libmongo-client b/modules/afmongodb/libmongo-client index 58f3814..7896ce7 160000 --- a/modules/afmongodb/libmongo-client +++ b/modules/afmongodb/libmongo-client @@ -1 +1 @@ -Subproject commit 58f3814cad94bcd78216c7ac971c8435d17a9242 +Subproject commit 7896ce7dbc239e4069787fee336875a9b9a6f6f5 -- 1.7.10.4

2 1

[PATCH (3.4)] afsocket: Implement tcp_keep_alive_time()
by Gergely Nagy 18 Aug '12

18 Aug '12

With the new tcp_keep_alive_time() option one can control on a per-source basis the time after which a TCP connection will start sending TCP keepalive probes (assuming so-keepalive(yes)). While it is accepted for all stream sources, it only makes sense for TCP-based streams. By default, it's value is 0, which means using the kernel default (2 hours). Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsocket/afsocket-grammar.ym | 2 ++ modules/afsocket/afsocket-parser.c | 1 + modules/afsocket/afsocket.c | 11 +++++++++++ modules/afsocket/afsocket.h | 2 ++ 4 files changed, 16 insertions(+), 0 deletions(-) diff --git a/modules/afsocket/afsocket-grammar.ym b/modules/afsocket/afsocket-grammar.ym index 7970ca9..48b0526 100644 --- a/modules/afsocket/afsocket-grammar.ym +++ b/modules/afsocket/afsocket-grammar.ym @@ -83,6 +83,7 @@ TLSContext *last_tls_context; %token KW_SO_SNDBUF %token KW_SO_RCVBUF %token KW_SO_KEEPALIVE +%token KW_TCP_KEEP_ALIVE_TIME %token KW_SPOOF_SOURCE %token KW_KEEP_ALIVE @@ -252,6 +253,7 @@ source_afinet_tcp_option source_afsocket_stream_params : KW_KEEP_ALIVE '(' yesno ')' { afsocket_sd_set_keep_alive(last_driver, $3); } + | KW_TCP_KEEP_ALIVE_TIME '(' LL_NUMBER ')' { afsocket_sd_set_keep_alive_time(last_driver, $3); } | KW_MAX_CONNECTIONS '(' LL_NUMBER ')' { afsocket_sd_set_max_connections(last_driver, $3); } ; diff --git a/modules/afsocket/afsocket-parser.c b/modules/afsocket/afsocket-parser.c index 44671ca..bd8f5d6 100644 --- a/modules/afsocket/afsocket-parser.c +++ b/modules/afsocket/afsocket-parser.c @@ -64,6 +64,7 @@ static CfgLexerKeyword afsocket_keywords[] = { { "so_sndbuf", KW_SO_SNDBUF }, { "so_keepalive", KW_SO_KEEPALIVE }, { "tcp_keep_alive", KW_SO_KEEPALIVE, 0, KWS_OBSOLETE, "so_keepalive" }, + { "tcp_keep_alive_time",KW_TCP_KEEP_ALIVE_TIME }, { "spoof_source", KW_SPOOF_SOURCE }, { "transport", KW_TRANSPORT }, { "max_connections", KW_MAX_CONNECTIONS }, diff --git a/modules/afsocket/afsocket.c b/modules/afsocket/afsocket.c index a7cc7bc..4a71b11 100644 --- a/modules/afsocket/afsocket.c +++ b/modules/afsocket/afsocket.c @@ -43,6 +43,7 @@ #include <arpa/inet.h> #include <sys/stat.h> #include <unistd.h> +#include <netinet/tcp.h> #if ENABLE_TCP_WRAPPER #include <tcpd.h> @@ -79,6 +80,8 @@ afsocket_setup_socket(gint fd, SocketOptions *sock_options, AFSocketDirection di setsockopt(fd, SOL_SOCKET, SO_BROADCAST, &sock_options->broadcast, sizeof(sock_options->broadcast)); } setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &sock_options->keepalive, sizeof(sock_options->keepalive)); + if (sock_options->keepalive_time > 0) + setsockopt(fd, SOL_TCP, TCP_KEEPIDLE, &sock_options->keepalive_time, sizeof(sock_options->keepalive_time)); return TRUE; } @@ -386,6 +389,14 @@ afsocket_sd_set_keep_alive(LogDriver *s, gint enable) } void +afsocket_sd_set_keep_alive_time(LogDriver *s, gint timeout) +{ + AFSocketSourceDriver *self = (AFSocketSourceDriver *) s; + + self->keepalive_timeout = timeout; +} + +void afsocket_sd_set_max_connections(LogDriver *s, gint max_connections) { AFSocketSourceDriver *self = (AFSocketSourceDriver *) s; diff --git a/modules/afsocket/afsocket.h b/modules/afsocket/afsocket.h index 392e249..fd11d28 100644 --- a/modules/afsocket/afsocket.h +++ b/modules/afsocket/afsocket.h @@ -58,6 +58,7 @@ typedef struct _SocketOptions gint rcvbuf; gint broadcast; gint keepalive; + gint keepalive_time; } SocketOptions; gboolean afsocket_setup_socket(gint fd, SocketOptions *sock_options, AFSocketDirection dir); @@ -78,6 +79,7 @@ struct _AFSocketSourceDriver gint max_connections; gint num_connections; gint listen_backlog; + gint keepalive_timeout; GList *connections; SocketOptions *sock_options_ptr; -- 1.7.9

2 1

[PATCH] db-parser(): implement a MAC address parser
by Balint Kovacs 18 Aug '12

18 Aug '12

While talking about netfilter logs recently, we found, that a MAC address parser may come in handy when parsing these messages, so here it comes. I hope others might find it useful as well. Balint

2 2

[PATCH (3.4)] jsonparser: Clone the marker() setting when cloning the parser
by Gergely Nagy 18 Aug '12

18 Aug '12

Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/jsonparser/jsonparser.c | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/jsonparser/jsonparser.c b/modules/jsonparser/jsonparser.c index 7331ad9..891b9f0 100644 --- a/modules/jsonparser/jsonparser.c +++ b/modules/jsonparser/jsonparser.c @@ -201,6 +201,7 @@ log_json_parser_clone (LogPipe *s) cloned = (LogJSONParser *) log_json_parser_new (); log_json_parser_set_prefix ((LogParser *)cloned, self->prefix); + log_json_parser_set_marker ((LogParser *)cloned, self->marker); return &cloned->super.super; } -- 1.7.10

2 2

[PATCH (3.4)] afsmtp: Fix the module linkage
by Gergely Nagy 18 Aug '12

18 Aug '12

Use $(MODULE_DEPS_LIBS) and $(MODULE_LDFLAGS) instead of trying to grab all the libs. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsmtp/Makefile.am | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/afsmtp/Makefile.am b/modules/afsmtp/Makefile.am index bb5b94d..34c0e98 100644 --- a/modules/afsmtp/Makefile.am +++ b/modules/afsmtp/Makefile.am @@ -6,10 +6,10 @@ export top_srcdir if ENABLE_SMTP module_LTLIBRARIES = libafsmtp.la -libafsmtp_la_CFLAGS = $(LIBESMTP_CFLAGS) $(IVYKIS_CFLAGS) $(EVENTLOG_CFLAGS) $(SYSLOG_NG_CFLAGS) +libafsmtp_la_CFLAGS = $(LIBESMTP_CFLAGS) libafsmtp_la_SOURCES = afsmtp-grammar.y afsmtp.c afsmtp.h afsmtp-parser.c afsmtp-parser.h -libafsmtp_la_LIBADD = $(LIBESMTP_LIBS) $(IVYKIS_LIBS) $(EVENTLOG_LIBS) $(SYSLOG_NG_LIBS) -libafsmtp_la_LDFLAGS = -avoid-version +libafsmtp_la_LIBADD = $(LIBESMTP_LIBS) $(MODULE_DEPS_LIBS) +libafsmtp_la_LDFLAGS = $(MODULE_LDFLAGS) endif -- 1.7.10.4

2 1

[Bug 62] PROGRAM macro problem
by bugzilla＠bugzilla.balabit.com 18 Aug '12

18 Aug '12

https://bugzilla.balabit.com/show_bug.cgi?id=62 Gergely Nagy <algernon(a)balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |algernon(a)balabit.hu Resolution| |FIXED Status|NEW |RESOLVED --- Comment #4 from Gergely Nagy <algernon(a)balabit.hu> 2012-08-18 13:26:40 --- syslog-ng 3.4 has a $(sanitize) template function, which will replace "/" with "_", so this issue can be considered resolved, I believe. The same function can be configured to replace parens and pretty much anything else too, if so need be. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[PATCH 0/1] [RESEND] libmongo-client: Add support for Unix sockets
by Conrad Hoffmann 16 Aug '12

16 Aug '12

Hey once more. Here is a first revision of my earlier patch. Changes to last version: - Better error handling/input checking (thanks to Balazs for the feedback) - Free everything that was allocated before bailing out after errors - Some style fixes, hope it's ok now Further feedback welcome! Conrad Hoffmann (1): Add support for Unix sockets. examples/mongo-dump.c | 22 +++++++++++++++++----- src/mongo-client.c | 39 +++++++++++++++++++++++++++++++++++++++ src/mongo-client.h | 12 ++++++++++++ src/mongo-sync.c | 23 +++++++++++++++++++++++ src/mongo-sync.h | 15 +++++++++++++++ 5 files changed, 106 insertions(+), 5 deletions(-) -- 1.7.11.4

2 7

Compiling 3.4
by Conrad Hoffmann 14 Aug '12

14 Aug '12

Greetings! I am having some difficulties compiling current master (at the time of this writing) of syslog-ng 3.4. The first error is this: filter-expr-parser.c:30:5: error: conflicting types for 'filter_expr_parse' In file included from filter-expr-parser.c:27:0: filter-expr-grammar.h:404:5: note: previous declaration of 'filter_expr_parse' was here make[4]: *** [filter-expr-parser.lo] Error Now, at first I thought, since filter-expr-grammar.h is generated, it might have something to do with my bison version or the likes. But then I realized that the offending declaration in filter-expr-parser.c:30: int filter_expr_parse(CfgLexer *lexer, FilterExprNode **node); directly contradicts its use int the same file below (line 74): .parse = (gint (*)(CfgLexer *, gpointer *, gpointer)) filter_expr_parse, filter-expr-parser.c is not generated, so I don't quite get how this compiles at all for anyone? Am I missing something? The actual usage (line 74) matches the generated declaration, so I can just comment out the offending one in line 30. But I suspect this it not how it's supposed to work... Same thing happens for parser-expr-parser and rewrite-expr-parser... Thanks for any hints, Conrad

3 5

phpsyslogng-2.8.
by Jose Aguero M. 14 Aug '12

14 Aug '12

Hi all: I have installed ubuntu server with syslog-ng 2.8, but I need to know how to install a client on linux centos. I need help thanks

2 1

[PATCH] [0/1] libmongo-client: add support for Unix sockets
by Conrad Hoffmann 14 Aug '12

14 Aug '12

Greetings! This is patch agains the master branch of syslog-ng-3.4. It's more of a proof-of-concept thing, but in a quick session I put support for Unix sockets into libmongo-client. It may not be stunningly beautiful, but I actually used it (along with a second patch, which I will send shortly) to have syslog-ng write to mongodb via a Unix socket. I realize libmongo-client is also on GitHub, but the submodule pointed to a different repo, so I decided to just dump this here. :) If you have any interest at all, I could also clean this up some more. Maybe split across two commits, too. Let me know. Conrad Hoffmann (1): Add support for Unix sockets. examples/mongo-dump.c | 22 +++++++++++++++++----- src/mongo-client.c | 40 ++++++++++++++++++++++++++++++++++++++++ src/mongo-client.h | 12 ++++++++++++ src/mongo-sync.c | 23 +++++++++++++++++++++++ src/mongo-sync.h | 15 +++++++++++++++ 5 files changed, 107 insertions(+), 5 deletions(-) -- 1.7.11.4

3 3

3.4 alpha: Deadlock in MongoDB plugin?
by Conrad Hoffmann 14 Aug '12

14 Aug '12

Hi, it's me bugging again :) (btw, I'm on the list, no need to CC me) In my quest to test my patches with the actual 3.4 version syslog, I noticed today that the mongodb plugin doesn't seem to work very at all on todays master branch (vanilla, without any patches). No log messages are ever written to the DB, and after a few messages syslog locks up. The mongodb threads seems to idle in its normal event loop while the main thread seems to be stuck in some afmongodb-related lock. Once again, I need to go to bed before I can take a closer look, but I just thought I'd let you know. Find here a backtrace of the main thread in the state it is stuck in: http://pastebin.com/QZSam3CH At the bottom is also the backtrace for the mongodb thread, as far as I can tell it looks as always. Good night for now, Conrad

2 4

[PATCH] [0/1] Allow mongodb destination to use Unix sockets
by Conrad Hoffmann 12 Aug '12

12 Aug '12

As promised this is actually the second part of the previous patch, but since the other one was for a different repo I decided to keep them seperate. Yet, this one depends on the former. This patch is also for syslog-ng-3.4, master branch. Like I mentioned, I already used Unix sockets successfully, though I must add that this was with a modified version of the path for syslog-ng-3.3, because I had trouble getting 3.4 to work at all. I will take another look at this tomorrow, but maybe you can give some feedback already? Conrad Hoffmann (1): Allow Unix sockets for MongoDB connection. modules/afmongodb/afmongodb-grammar.ym | 2 ++ modules/afmongodb/afmongodb-parser.c | 1 + modules/afmongodb/afmongodb.c | 25 +++++++++++++++++++++---- modules/afmongodb/afmongodb.h | 1 + 4 files changed, 25 insertions(+), 4 deletions(-) -- 1.7.11.4

3 4

[PATCH] tfhash: New template function to calculate the SHA1 of its input
by Peter Gyongyosi 12 Aug '12

12 Aug '12

tfhash: New template function to calculate the SHA1 of its input This patch adds a new template function called $(hash) which can be used to calculate the SHA1 hash of its inputs. As it uses the SHA1() function from OpenSSL, it is only available if syslog-ng is compiled using --enable-ssl. One use case for it can be when you'd like to anonymize sensitive parts of the log message (eg. the user name) you parsed out using patterndb before storing or forwarding it for further analysis but would like to retain the ability to correlate messages along this value. Beware though that such simple anonymization can be broken if there's enough data available, see http://arxiv.org/pdf/cs/0610105v2.pdf for references and examples. Thanks for the original idea to Martin Grauel. Signed-off-by: Peter Gyongyosi <gyp(a)balabit.hu> --- [see the patch itself at https://github.com/gyp/syslog-ng-3.4/commit/5c6e97d94694eef2aa76fa61a1c8928… ] <https://github.com/gyp/syslog-ng-3.4/commit/5c6e97d94694eef2aa76fa61a1c8928…>

5 12

lib/ivykis submodule pointing to stale commit?
by Conrad Hoffmann 12 Aug '12

12 Aug '12

Hi folks, apparently the commit that lib/ivykis submodule is set to (syslog-ng-3.4, master branch) does not exist in the tree anymore? GitHub actually still shows it, but it's not checked out: (See lines beginning with *) dionysos:~/tmp $>git clone https://github.com/balabit/syslog-ng-3.4.git Cloning into 'syslog-ng-3.4'... remote: Counting objects: 13157, done. remote: Compressing objects: 100% (2834/2834), done. remote: Total 13157 (delta 10256), reused 13100 (delta 10213) Receiving objects: 100% (13157/13157), 3.18 MiB | 1.12 MiB/s, done. Resolving deltas: 100% (10256/10256), done. dionysos:~/tmp $>cd syslog-ng-3.4/ dionysos:~/tmp/syslog-ng-3.4 $>git submodule init Submodule 'lib/ivykis' (git://github.com/buytenh/ivykis.git) registered for path 'lib/ivykis' Submodule 'modules/afmongodb/libmongo-client' (git://git.balabit.hu/bazsi/libmongo-client.git) registered for path 'modules/afmongodb/libmongo-client' dionysos:~/tmp/syslog-ng-3.4 $>git submodule update Cloning into 'lib/ivykis'... remote: Counting objects: 3678, done. remote: Compressing objects: 100% (917/917), done. remote: Total 3678 (delta 2764), reused 3635 (delta 2732) Receiving objects: 100% (3678/3678), 1.06 MiB | 682 KiB/s, done. Resolving deltas: 100% (2764/2764), done. *fatal: reference is not a tree: 73c0c27a19755f700b268081a5a59acdd04b421e Cloning into 'modules/afmongodb/libmongo-client'... remote: Generating pack... remote: Done counting 4658 objects. remote: Deltifying 4658 objects... remote: 100% (4658/4658) done remote: Total 4658 (delta 3123), reused 4585 (delta 3053) Receiving objects: 100% (4658/4658), 702.36 KiB | 1.17 MiB/s, done. Resolving deltas: 100% (3123/3123), done. Submodule path 'modules/afmongodb/libmongo-client': checked out '58f3814cad94bcd78216c7ac971c8435d17a9242' *Unable to checkout '73c0c27a19755f700b268081a5a59acdd04b421e' in submodule path 'lib/ivykis' I can't find a similar in the tree neither. Any ideas? Regards, Conrad

3 2

Re: [syslog-ng] Issue about using perl script in Receiving messages from external applications
by Evan Rempel 10 Aug '12

10 Aug '12

You are probably suffering from perl's buffering. I forget the exact syntax right now but a quick google of disabling Perl auto flush should give you what you need. Jian Zhu <zhujian0805(a)gmail.com> wrote: Hello, Could anyone please help me on this below issue? my syslog-ng.conf file look like below: The test.sh script work fine, I can see it's output in the test.sh.log file. But the test.pl<http://test.pl> doesn't work. the test.sh and test.pl<http://test.pl> have the same function of printing something to STDOUT. ------------------------------ ------------------------------------------------------------------------------------------------------------------- @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator’s guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { # Number of syslog lines stored in memory before being written to files flush_lines (0); log_fifo_size (2048); create_dirs (yes); perm (0640); dir_perm (0750); }; source ppp { program("/root/perl/test.pl<http://test.pl>"); }; destination ppp { file ("/var/log/syslog-ng/$HOST/perl.pl.log" create_dirs(yes)); }; log { source(ppp); destination(ppp); }; source sss { program("/root/shell/test.sh"); }; destination sss { file ("/var/log/syslog-ng/$HOST/test.sh.log" create_dirs(yes)); }; log { source(sss); destination(sss); }; ------------------------------------------------------------------------------------------------------------------------------------------------- -- Thanks & Best Regards James Zhu Mobile: +86 186 8482 2818 Email: zhujian0805(a)gmail.com<mailto:zhujian0805@gmail.com>

3 3

[Bug 178] New: Spoofed source address bug introduced in 3.3.5
by bugzilla＠bugzilla.balabit.com 10 Aug '12

10 Aug '12

https://bugzilla.balabit.com/show_bug.cgi?id=178 Summary: Spoofed source address bug introduced in 3.3.5 Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Solaris Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: marvin.nipper(a)stream.com Type of the Report: regression Estimated Hours: 0.0 Hi. Sorry if this is not the right way to do this. There was an email thread regarding this on 4/30, but I figured that the right way to do this way to simply open an official bug report. As this is my 1st one for syslog-ng, apologies if I screw it up! This all pertains to 3.3.5, on Solaris 10 x86. Per my original email, changes made between 3.3.4 and 3.3.5 have caused a situation wherein forwarded UDP packets, with spoofed source addressing, all revert to null addresses, the instant that a HUP is issued against syslog-ng (e.g. during log rotation). Also, as per that email, with Gergely's excellent (and educated) guess, I backed out the patch noted here: http://git.madhouse-project.org/debian/syslog-ng/patch/?id=a898014482f733e9…, and the result was that the null addressing issue was resolved. Obviously, as there were (no doubt) other reasons for that patch, I'm just wanting to be sure that this problem doesn't get lost in the shuffle, and that the patch gets reworked (to achieve its originally desired results, while managing to also resolve the null address problem that it caused with spoofing). Let me know if you need any further information regarding this problem. And, as always, I'm MORE than happy to be a test guinea pig, if need be. THANKS. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 21

syslog-ng upgrade
by James McDonald 10 Aug '12

10 Aug '12

Does anyone know what needs to be done to successfully upgrade from 4.1 to 4.2 (F2)? I tried simply stopping the service and running 'rpm -ivh syslog-ng..rpm -force' but it did not complete successfully. I added the -force command because it was complaining of file conflicts. Balabit has requested I upgrade to this version, as they try to help me figure out my other problem I'm having with the syslog-ng service, but I was unable to find any documentation or get a response from Balabit yet. I thought I would see if anyone here had performed the upgrade and knew what needed to be done. Here is the output I got when I tried to run it: rpm -ivh syslog-ng-premium-edition-4.2.3-1.rhel5.i386.r pm --force warning: syslog-ng-premium-edition-4.2.3-1.rhel5.i386.rpm: Header V4 DSA signature: NOKEY, key ID 2aa28252 Preparing... ########################################### [100%] *IMPORTANT NOTICE - NEW LICENSE REQUIRED FOR MULTI-CPU OPERATION* Please note that the 4.2 version of syslog-ng Premium Edition includes multi-threaded processing with a 2 CPU core limit. If you are upgrading your syslog-ng server from version 4.1 to 4.2 and your syslog-ng deployment requires more than 2 CPU cores, you will need to obtain a new license file, free of charge, from the BalaBit sales department. Please contact the sales department at sales(a)balabit.com. Shutting down kernel logger: [FAILED] Shutting down system logger: [FAILED] Stopping syslog-ng: 1:syslog-ng-premium-editiwarning: /opt/syslog-ng/etc/syslog-ng.conf created as /opt/syslog-ng/etc/syslog-ng.conf.rpmnew ########################################### [100%] Starting syslog-ng: WARNING: window sizing for network sources has changed in syslog-ng 4.1. The window size is the value of the log_iw_size() option divided by the value of max_connections(). If log_iw_size()/max_connections() is lower than 100, it is automatically increased to 100. Adjust the log_fifo_size() option if needed to avoid message loss.; orig_window_size='1', new_window_size='100', min_log_fifo_size='100000' [ OK ] Thanks, James McDonald System Administrator RHCSA, Security+ jmcdonald(a)lce.com

3 2