- syslog-ng - lists.balabit.hu

UDP Dropping packets
by Aldrich, Jamie S 30 Aug '11

30 Aug '11

I am trying to send the output from an application log to a LogLogic device, and it appears we are getting UDP drops repeatedly. Any ideas on what I can do to fix them? Aug 25 15:24:55 lxfwossecp3 syslog-ng[2297]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=6376', processed='center(received)=6376', processed='destination(d_messages)=6374', processed='destination(d_bo ot)=0', processed='destination(d_auth)=0', processed='destination(d_cron)=1', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=1', processed='destination(d_cons)=0', processed='des tination(d_spol)=0', processed='destination(d_mail)=0', processed='source(s_sys)=2', processed='source(s_file)=6374', suppressed='udp(10.13.33.11:514)=0' Aug 25 15:34:55 lxfwossecp3 syslog-ng[2297]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=6378', processed='center(received)=6378', processed='destination(d_messages)=6374', processed='destination(d_bo ot)=0', processed='destination(d_auth)=0', processed='destination(d_cron)=2', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=2', processed='destination(d_cons)=0', processed='des tination(d_spol)=0', processed='destination(d_mail)=0', processed='source(s_sys)=4', processed='source(s_file)=6374', suppressed='udp(10.13.33.11:514)=0' Jamie

4 6

3.3 git
by Peter Czanik 30 Aug '11

30 Aug '11

Hello, Just tested the current 3.3 git on openSUSE and FreeBSD. I made a 'make dist' and then used the resulting tgz for builds. I also did some quick functionality tests. While file names mention beta2 everywhere, it's actually a git checkout from this morning. Work in progress openSUSE packages for 11.4 and Factory are available at http://download.opensuse.org/repositories/home:/czanik:/syslog-ng33/ (it fails at post build checks on 11.3 and it does not compile on SLES). openSUSE package sources are available at https://build.opensuse.org/package/files?package=syslog-ng&project=home%3Ac… It's almost the same as the beta2 version included in factory, but I could remove all the patches hand picked from git. If you want to build syslog-ng 3.3 git on FreeBSD, you can download the source from the above URL, copy it to /usr/ports/distfiles/syslog-ng_3.3.0beta2.tar.gz and "rm distinfo" from /usr/ports/sysutils/syslog-ng3-devel, as it contains info about the original sources. Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

malformed syslog packets?
by Matt Zagrabelny 30 Aug '11

30 Aug '11

Hi! I've got a central log server running the OSE 3.1.3 version of syslog-ng: dpkg -l syslog-ng Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-========================================-========================================-================================================================================================ ii syslog-ng 3.1.3-3 Next generation logging daemon I have dns lookup turned on via: options { use_dns(yes); dns_cache(2000); dns_cache_expire(87600); }; And this seems to work just fine...except for a certain type of device on our network. We have a number of UPSes that log to our central log server and it seems that the dns look ups do not work for those (types of devices). % cd /var/log/syslog-ng/remote_clients % ls -d 10.* 10.25.32.4 10.25.5.15 10.25.5.19 10.25.5.26 10.25.5.35 10.25.5.4 10.25.5.44 10.25.5.51 10.25.5.6 10.25.5.65 10.25.5.69 10.25.5.76 10.25.5.1 10.25.5.16 10.25.5.2 10.25.5.27 10.25.5.36 10.25.5.40 10.25.5.49 10.25.5.52 10.25.5.60 10.25.5.66 10.25.5.7 10.25.5.79 10.25.5.10 10.25.5.17 10.25.5.20 10.25.5.28 10.25.5.37 10.25.5.41 10.25.5.5 10.25.5.55 10.25.5.61 10.25.5.67 10.25.5.72 10.25.5.81 10.25.5.14 10.25.5.18 10.25.5.23 10.25.5.3 10.25.5.38 10.25.5.43 10.25.5.50 10.25.5.58 10.25.5.62 10.25.5.68 10.25.5.75 10.25.5.9 When I look up those IP addresses, they are *all* APC batteries (UPSes). For instance: % dig -x 10.25.5.43 +short kplz246Abat1.d.umn.edu. Is it possible that they are sending some sort of munged data to the log server and syslog-ng is not able to perform the (reverse) name lookup? Any advice? Thanks, much! -Matt Zagrabelny

3 5

syslog-ng.persist file error
by Scott McLane Gardner 29 Aug '11

29 Aug '11

Hello, When I came in this morning, syslog-ng was not running, and the following error is produced when trying to start it with init script: >Error loading persistent configuration file; name='//var/syslog-ng.persist' I'm guessing that additional forward slash is not supposed to be there, but I have no idea how this is getting set or where to change it. Can anyone help me? Thanks, Scott

3 3

[PATCH] templates: Add a C_* family of timestamp macros.
by Gergely Nagy 29 Aug '11

29 Aug '11

The C_* family of timestamp macros work very similarly to R_ (received) and S_ (stamp), except C_ uses the current system time as of the macro expansion. The patch below uses cached_g_current_time() for performance reasons, but in case the minor caching is undesired, it's easy to convert to something else. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/templates.c | 39 +++++++++++++++++++++++++++++++++++++-- 1 files changed, 37 insertions(+), 2 deletions(-) diff --git a/lib/templates.c b/lib/templates.c index cb0c1b5..63000a3 100644 --- a/lib/templates.c +++ b/lib/templates.c @@ -86,6 +86,7 @@ enum M_RECVD_OFS = M_TIME_MACROS_MAX, M_STAMP_OFS = 2 * M_TIME_MACROS_MAX, + M_CSTAMP_OFS = 3 * M_TIME_MACROS_MAX, }; #define M_TIME_MACROS 15 @@ -171,6 +172,29 @@ LogMacroDef macros[] = { "S_TZ", M_STAMP_OFS + M_TZ }, { "S_UNIXTIME", M_STAMP_OFS + M_UNIXTIME }, + { "C_DATE", M_CSTAMP_OFS + M_DATE }, + { "C_FULLDATE", M_CSTAMP_OFS + M_FULLDATE }, + { "C_ISODATE", M_CSTAMP_OFS + M_ISODATE }, + { "C_STAMP", M_CSTAMP_OFS + M_STAMP }, + { "C_YEAR", M_CSTAMP_OFS + M_YEAR }, + { "C_YEAR_DAY", M_CSTAMP_OFS + M_YEAR_DAY }, + { "C_MONTH", M_CSTAMP_OFS + M_MONTH }, + { "C_MONTH_WEEK", M_CSTAMP_OFS + M_MONTH_WEEK }, + { "C_MONTH_ABBREV", M_CSTAMP_OFS + M_MONTH_ABBREV }, + { "C_MONTH_NAME", M_CSTAMP_OFS + M_MONTH_NAME }, + { "C_DAY", M_CSTAMP_OFS + M_DAY }, + { "C_HOUR", M_CSTAMP_OFS + M_HOUR }, + { "C_MIN", M_CSTAMP_OFS + M_MIN }, + { "C_SEC", M_CSTAMP_OFS + M_SEC }, + { "C_WEEKDAY", M_CSTAMP_OFS + M_WEEK_DAY_ABBREV }, /* deprecated */ + { "C_WEEK_DAY", M_CSTAMP_OFS + M_WEEK_DAY }, + { "C_WEEK_DAY_ABBREV",M_CSTAMP_OFS + M_WEEK_DAY_ABBREV }, + { "C_WEEK_DAY_NAME", M_CSTAMP_OFS + M_WEEK_DAY_NAME }, + { "C_WEEK", M_CSTAMP_OFS + M_WEEK }, + { "C_TZOFFSET", M_CSTAMP_OFS + M_TZOFFSET }, + { "C_TZ", M_CSTAMP_OFS + M_TZ }, + { "C_UNIXTIME", M_CSTAMP_OFS + M_UNIXTIME }, + { "SDATA", M_SDATA }, { "MSGHDR", M_MSGHDR }, { "SOURCEIP", M_SOURCE_IP }, @@ -421,7 +445,7 @@ log_macro_expand(GString *result, gint id, gboolean escape, LogTemplateOptions * gchar buf[64]; gint length; time_t t; - LogStamp *stamp; + LogStamp *stamp, sstamp; glong zone_ofs; if (id >= M_TIME_FIRST && id <= M_TIME_LAST) @@ -438,7 +462,18 @@ log_macro_expand(GString *result, gint id, gboolean escape, LogTemplateOptions * id -= M_STAMP_OFS; stamp = &msg->timestamps[LM_TS_STAMP]; } - else + else if (id >= M_TIME_FIRST + M_CSTAMP_OFS && id <= M_TIME_LAST + M_CSTAMP_OFS) + { + GTimeVal tv; + + id -= M_CSTAMP_OFS; + cached_g_current_time(&tv); + sstamp.tv_sec = tv.tv_sec; + sstamp.tv_usec = tv.tv_usec; + sstamp.zone_offset = -1; + stamp = &sstamp; + } + else { g_assert_not_reached(); break; -- 1.7.0.4

2 1

Need help with filtering messages - new user with syslog-ng
by Cosmin Neagu 29 Aug '11

29 Aug '11

Hello, First of all, i started to use syslog-ng on Ubuntu a few days ago and it seams a great syslog server. But today i stumble on a problem. I configured snmptrapd with TRAPDOPTS='-Lsd ' and this means that snmptrapd will send the trap received to syslog-ng. Now, syslog-ng puts those traps by default in /var/log/syslog because of this default configurations: source s_src { unix-dgram("/dev/log"); internal(); file("/proc/kmsg" program_override("kernel")); destination d_syslog { file("/var/log/syslog"); }; filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); }; log { source(s_src); filter(f_syslog3); destination(d_syslog); }; What i want to acomplish is to have traps from diferent host put in diferent files, not all together in the same file like it happens now. At first i tried to filter based on the host's ip address that was sending the trap, but i realized that the snmptrapd process is the one that sends the trap to syslog-ng process, not the device directly: Aug 29 11:42:48 Dell *snmptrapd*[3801]: 2011-08-29 11:42:43 10.90.0.252 [UDP: [10.90.0.252]:49364->[192.168.53.151]]: iso.3.6.1.2.1.1.3.0 = Timeticks: (1563318974) 180 days, 22:33:09.74 iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.9.9.41.2.0.1 iso.3.6.1.4.1.9.9.41.1.2.3.1.2.31 = STRING: "LINK" iso.3.6.1.4.1.9.9.41.1.2.3.1.3.31 = INTEGER: 4 iso.3.6.1.4.1.9.9.41.1.2.3.1.4.31 = STRING: "UPDOWN" iso.3.6.1.4.1.9.9.41.1.2.3.1.5.31 = STRING: "Interface Serial0/0/0, changed state to down" iso.3.6.1.4.1.9.9.41.1.2.3.1.6.31 = Timeticks: (1563318974) 180 days, 22:33:09.74 So maibe you have done this - how can i filter based on the program that it sending the message (like snmptrapd). And also, can filters based on the text itself can be used? Like: - if the mesage contains "10.90.0.252 [UDP: [10.90.0.252]:XXXXX->[192.168.53.151]" put the mesage in "this" file - if the mesage contains "10.90.1.22 [UDP: [10.90.1.22]:XXXXX->[192.168.53.151]" put the mesage in "that" file Thanks --

2 1

[PATCH] tfjson: Typo (& build) fix.
by Gergely Nagy 28 Aug '11

28 Aug '11

My previous patch contained a typo (an extra . at the end of a line) which resulted in breaking compilation. This patch fixes that typo. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/tfjson/tfjson.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/modules/tfjson/tfjson.c b/modules/tfjson/tfjson.c index 3d54201..65daadd 100644 --- a/modules/tfjson/tfjson.c +++ b/modules/tfjson/tfjson.c @@ -160,7 +160,7 @@ const ModuleInfo module_info = { .canonical_name = "tfjson", .version = VERSION, - .description = "The tfjson module provides a JSON formatting template function for syslog-ng.",. + .description = "The tfjson module provides a JSON formatting template function for syslog-ng.", .core_revision = SOURCE_REVISION, .plugins = builtin_tmpl_func_plugins, .plugins_len = G_N_ELEMENTS(builtin_tmpl_func_plugins), -- 1.7.2.5

1 0

[PATCH] scl: Add support for GNU/kFreeBSD
by Gergely Nagy 28 Aug '11

28 Aug '11

GNU/kFreeBSD is GNU userland (including libc) on top of a FreeBSD kernel. It's very similar to FreeBSD, but not completely. On GNU/kFreeBSD, the system log source is /var/run/log, while the kernel messages go to /dev/klog, in correctly formatted legacy BSD format. Tested on Debian GNU/kFreeBSD (squeeze). Signed-off-by: Gergely Nagy <algernon(a)madhouse-project.org> --- scl/system/generate-system-source.sh | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/scl/system/generate-system-source.sh b/scl/system/generate-system-source.sh index fab7135..1088b63 100755 --- a/scl/system/generate-system-source.sh +++ b/scl/system/generate-system-source.sh @@ -75,6 +75,12 @@ unix-dgram("/var/run/logpriv" perm(0600)); file("/dev/klog" follow-freq(0) program-override("kernel") flags(no-parse)); EOF ;; + GNU/kFreeBSD) + cat <<EOF +unix-dgram("/var/run/log"); +file("/dev/klog" follow-freq(0) program-override("kernel")); +EOF + ;; HP-UX) cat <<EOF pipe("/dev/log" pad_size(2048)); -- 1.7.2.5

2 1

[PATCH] tfjson: Add module_info.
by Gergely Nagy 28 Aug '11

28 Aug '11

In order for syslog-ng --module-registry to be able to display meaningful information about a module, module_info must be set in said modules. Add that to tfjson, as it was lacking one. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/tfjson/tfjson.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/modules/tfjson/tfjson.c b/modules/tfjson/tfjson.c index 39a6cdb..3d54201 100644 --- a/modules/tfjson/tfjson.c +++ b/modules/tfjson/tfjson.c @@ -155,3 +155,13 @@ tfjson_module_init(GlobalConfig *cfg, CfgArgs *args) plugin_register(cfg, builtin_tmpl_func_plugins, G_N_ELEMENTS(builtin_tmpl_func_plugins)); return TRUE; } + +const ModuleInfo module_info = +{ + .canonical_name = "tfjson", + .version = VERSION, + .description = "The tfjson module provides a JSON formatting template function for syslog-ng.",. + .core_revision = SOURCE_REVISION, + .plugins = builtin_tmpl_func_plugins, + .plugins_len = G_N_ELEMENTS(builtin_tmpl_func_plugins), +}; -- 1.7.2.5

2 1

[PATCH] lib/Makefile.am: Include cfg-grammar.h in the package headers too.
by Gergely Nagy 28 Aug '11

28 Aug '11

cfg-grammar.h is included by some of the public headers we install, therefore cfg-grammar.h should be installed alongside aswell. Signed-off-by: Gergely Nagy <algernon(a)madhouse-project.org> --- lib/Makefile.am | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/lib/Makefile.am b/lib/Makefile.am index d941a8d..6527bc8 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -24,6 +24,7 @@ pkginclude_HEADERS = \ block-ref-parser.h \ center.h \ cfg.h \ + cfg-grammar.h \ cfg-lexer.h \ cfg-parser.h \ children.h \ -- 1.7.2.5

2 1