- syslog-ng - lists.balabit.hu

Solaris 10 support(?) in latest versions (e.g. 3.3.2)
by Marvin Nipper 22 Dec '11

22 Dec '11

Apologies upfront for the length, but this addresses two different problems. Environment: Solaris 10 x64 U10, fully patched (i.e. pretty much "new"). Currently operating (successfully) with 3.0.5, compiled on that platform, and distributed to a number of other Solaris 10 x64 targets. Problem: Trying to "make the leap" to 3.3.2, and get current, but that version seems to have numerous Solaris compatibility issues. Just trying to understand the status of Solaris support. Problem #1 Latest packaging no longer works with Solaris-based "make". After a clean configure (output here): syslog-ng Open Source Edition 3.3.2 configured Compiler options: compiler : gcc -std=gnu99 compiler options : -g -O2 -Wall -pthread -D_REENTRANT -D_PTHREADS -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -I$(top_srcdir)/lib/ivykis/lib/include -I$(top_builddir)/lib/ivykis/lib/include -I$(top_srcdir)/lib/ivykis/modules/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : -R/usr/local/lib -lpthread prefix : /usr/local linking mode : dynamic __thread keyword : yes Submodules: ivykis : internal libmongo-client : internal Features: Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : no Linux capability support : no PCRE support : no Env wrapper support : no systemd support : no (unit dir: none) Modules: Module search path : /usr/local/lib/syslog-ng Default module list : affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afstreams Sun STREAMS support (module): yes SSL support (module) : yes SQL support (module) : no PACCT module (EXPERIMENTAL) : no MongoDB destination (module): yes JSON support (module) : no (using no) ..... a simple "make" attempt now immediately results in this: # make make all-recursive Making all in lib make: Fatal error in reader: Makefile, line 991: Unexpected end of line seen Current working directory /var/opt/packages/syslog-ng-3.3.2/lib *** Error code 1 The following command caused the error: fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='lib modules syslog-ng scripts tests doc contrib scl debian tgz2build build'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /var/opt/packages/syslog-ng-3.3.2 *** Error code 1 make: Fatal error: Command failed for target `all' Not being a developer, per se, I spent time in Google-land, the general result that I find is: "These are caused because Solaris "make" doesn't accept the 'GNU multi-line Makefile format'. Use gmake instead." But there are obviously folks who also say that this can be fixed by "correcting" the problems in the Makefiles (so that would seem to be a preferable alternative). Problem #2 Can no longer perform successful compilations, due to this error: ld: fatal: relocation error: R_386_GOTOFF Same configure output as previously shown, followed by gmake (to get around the first problem). Numerous modules compile successfully, but then this happens: gcc -DHAVE_CONFIG_H -I. -I../.. -D_GNU_SOURCE -I../../lib/include -I../../lib/include -I../../modules/include -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event_test.o -MD -MP -MF .deps/iv_event_test.Tpo -c -o iv_event_test.o iv_event_test.c mv -f .deps/iv_event_test.Tpo .deps/iv_event_test.Po /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -R/usr/local/lib -o iv_event_test iv_event_test.o ../../lib/libivykis.la ../libivykis-modules.la -lrt -lpthread -lsocket libtool: link: gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -o iv_event_test iv_event_test.o ../../lib/.libs/libivykis.a ../.libs/libivykis-modules.a /var/opt/packages/syslog-ng-3.3.2/lib/ivykis/lib/.libs/libivykis.a -lrt -lpthread -lsocket -R/usr/local/lib ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a(iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a(iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a(iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a(iv_event.o): symbol __tls: relocation illegal for TLS symbol collect2: ld returned 1 exit status gmake[7]: *** [iv_event_test] Error 1 gmake[7]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/modules/test' gmake[6]: *** [all-recursive] Error 1 gmake[6]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/modules' gmake[5]: *** [all-recursive] Error 1 gmake[5]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis' gmake[4]: *** [all] Error 2 gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis' gmake[3]: *** [all-recursive] Error 1 gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2' gmake: *** [all] Error 2 Again, adopting the "Google is my friend" approach, this appears to be an issue with "Solaris not supporting Visibility". (http://www.ilkda.com/compile/Build_Problems.htm) Most of the sites where I see this being discussed, with other (various) source components, I also see the developers subsequently modifying their environment to "adjust for" the Visibility issue in Solaris. So, again, not being a C developer myself, I guess I'm wondering whether I'm just doing something wrong with 3.3.2 (i.e. something that I need to do different from my 3.0.5 compilation), or whether this is truly just a "Solaris thing" that has not been accounted for in the current code base? Sorry for not being better at sorting this out myself. Any and all help/guidance would certainly be appreciated. GLAD to do some sort of testing here, if that will help. Also, just for reference, here is the full "configure" output from the 3.0.5 build, that compiles clean, and works just fine: checking pkg-config is at least version 0.9.0... yes checking for a BSD-compatible install... ./install-sh -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... ./install-sh -c -d checking for gawk... no checking for mawk... no checking for nawk... nawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for gcc... (cached) gcc checking whether we are using the GNU C compiler... (cached) yes checking whether gcc accepts -g... (cached) yes checking for gcc option to accept ISO C89... (cached) none needed checking dependency style of gcc... (cached) gcc3 checking for bison... bison -y checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking whether make sets $(MAKE)... (cached) yes checking for ranlib... ranlib checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 checking how to enable static linking for certain libraries... GNU or Solaris checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /usr/sfw/bin/ggrep checking for egrep... /usr/sfw/bin/ggrep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking dmalloc.h usability... no checking dmalloc.h presence... no checking for dmalloc.h... no checking for strings.h... (cached) yes checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking stropts.h usability... yes checking stropts.h presence... yes checking for stropts.h... yes checking sys/strlog.h usability... yes checking sys/strlog.h presence... yes checking for sys/strlog.h... yes checking door.h usability... yes checking door.h presence... yes checking for door.h... yes checking sys/capability.h usability... no checking sys/capability.h presence... no checking for sys/capability.h... no checking sys/prctl.h usability... no checking sys/prctl.h presence... no checking for sys/prctl.h... no checking tcpd.h usability... yes checking tcpd.h presence... yes checking for tcpd.h... yes checking whether struct tm is in sys/time.h or time.h... time.h checking for struct tm.tm_gmtoff... no checking for I_CONSLOG... yes checking for O_LARGEFILE... yes checking for struct sockaddr_storage... yes checking for struct sockaddr_in6... yes checking for PR_SET_KEEPCAPS... no checking for door_create in -ldoor... yes checking for socket in -lsocket... yes checking for nanosleep in -lrt... yes checking for gethostbyname in -lnsl... yes checking for regexec in -lregex... no checking for res_init in -lresolv... yes checking for cap_set_proc in -lcap... no checking for PCRE... no configure: WARNING: Cannot find pcre version >= 7.3. pcre support disabled. checking for strdup... yes checking for strtol... yes checking for strtoll... yes checking for strtoimax... yes checking for inet_aton... no checking for inet_ntoa... no checking for getopt_long... yes checking for getaddrinfo... no checking for getutent... yes checking for pread... yes checking for pwrite... yes checking for strcasestr... no checking for TCP wrapper library... -lwrap checking for OPENSSL... yes checking for dlsym in -ldl... yes checking for inflate in -lz... yes checking for dlsym in -ldl... (cached) yes checking for LIBDBI... no checking for dbi_initialize in -ldbi... no checking pthread.h usability... yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for LIBNET... stty: : No such device or address stty: : No such device or address yes checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable IPv6 support... yes checking whether to enable SQL support... no checking whether to enable Linux capability support... no checking whether to enable PCRE support... no checking pkg-config is at least version 0.14... yes checking for GLIB - version >= 2.10.1... yes (version 2.13.5) checking for EVTLOG... yes checking for LIBNET... stty: : No such device or address stty: : No such device or address yes checking for static GLib libraries... yes checking sanity checking Glib headers... yes configure: creating ./config.status config.status: creating dist.conf config.status: creating Makefile config.status: creating syslog-ng.spec config.status: creating src/Makefile config.status: creating doc/Makefile config.status: creating doc/docvars.xml config.status: creating contrib/Makefile config.status: creating tests/Makefile config.status: creating debian/Makefile config.status: creating tgz2build/Makefile config.status: creating tests/unit/Makefile config.status: creating tests/functional/Makefile config.status: creating tests/loggen/Makefile config.status: creating config.h config.status: executing depfiles commands syslog-ng Open Source Edition 3.0.5 configured Compiler options: compiler : gcc compiler options : -g -O2 -Wall -I/usr/local/include/glib-2.0 -I/u sr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl /include -DLIBNET_LIL_ENDIAN -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET _BITS=64 linker flags : -R/usr/local/lib -lpthread prefix : /usr/local linking mode : dynamic Features: Sun STREAMS support : yes Sun Door support : yes Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : no SSL support : yes SQL support : no Linux capability support : no PCRE support : no Env wrapper support : no This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

4 15

Pattern matching.
by Anup Shetty 22 Dec '11

22 Dec '11

I am trying to match the pattern for DC logs and here is my XML format Here's the patterndb.xml file at /opt/syslog-ng/var/patterndb.xml" --------------------------------------- <?xml version='1.0' encoding='UTF-8'?> <patterndb version='3' pub_date='2011-12-21'> <ruleset id='90c9b341f4e3d63c5ed8b29950491bf8' name='Domain Ctrls'> <rules> <rule provider='localtest' id='012c230f236d6a3f761ba956e7dff26a' class='system'> <patterns> <pattern> @ESTRING:user::@ Security Microsoft Windows security auditing.: [Success Audit] A computer account was changed. Subject: Security ID: S-1-5-7 Account Name: ANONYMOUS LOGON Account Domain: NT AUTHORITY Logon ID: 0x3e6 Computer Account That Was Changed: Security ID: @ESTRING:: @Account Name: @ESTRING:*ACC_NAME*: @ Account Domain: testdomain Changed Attributes: SAM Account Name: - Display Name: - User Principal Name: - Home Directory: - Home Drive: - Script Path: - Profile Path: - User Workstations: - Password Last Set: @ESTRING:: @@ESTRING:: @ Account Expires: - Primary Group ID: - AllowedToDelegateTo: - Old UAC Value: - New UAC Value: - User Account Control: - User Parameters: - SID History: - Logon Hours: - DNS Host Name: - Service Principal Names: - Additional Information: Privileges: - (EventID 4742) </pattern> </patterns> </rule> </rules> </ruleset> </patterndb> --------------------------------------- Here's the syslog-ng conf extract: --------------------------------------- parser pattern_db { db_parser( file("/opt/syslog-ng/var/patterndb.xml") ); }; destination patt_d{ file("/data/test/${R_YEAR}/${R_MONTH}/${R_DAY}/Domain_Ctrl__${SOURCEIP}_${R_YEAR}_${R_MONTH}_${R_DAY}.log" owner("test") group("test") perm(0660) dir-owner("test") dir-group("test") dir-perm(0770) template("$*ACC_NAME*\n $MSG\n") ); }; --------------------------------- but the *ACC_NAME* returns blank, although the log contains that field. -- Thanks Anup

4 4

Quick filter question again
by Lay, James 22 Dec '11

22 Dec '11

Hey all! Hope this is quick/easy. So I'm matching IP's that I don't want to see...filter below: message("66\.220\.") My issues is, this will match not only say 66.220.1.1, but also 166.220.1.1. I've tried changing to: message(" 66\.220\.") but that'd didn't fly. Any suggestions on how to more exactly match IP blocks? Thanks all. James

2 2

[PATCH (3.4) 0/1]: value-pairs: Add value_pairs_foreach_sorted().
by Gergely Nagy 22 Dec '11

22 Dec '11

The patch that will follow shortly implements a value_pairs_foreach_sorted() function, that can be used for multiple purposes in the future: - Template functions that want to format their output with keys always in a predictable order. - Various other parts of the code, that work better when operating on a sorted collection. I'm submitting it separately, so that it'll be easier for me to build upon it in the not too distant future. (And because I almost forgot to send it, even though it was origianlly made in early november...)

2 3

Program stop/start bug in 3.3.x
by Martin Holste 22 Dec '11

22 Dec '11

I'm getting a weird bug with the program destination in which on shutdown it sends TERM and then sends a start: <snip> Sending destination program a TERM signal; cmdline='perl /usr/local/elsa/node/elsa.pl -c /etc/elsa_node.conf', child_pid='21421' Starting destination program; cmdline='perl /usr/local/elsa/node/elsa.pl -c /etc/elsa_node.conf' </snip> 3.2.4 is fine.

2 1

[PATCH (3.4)] afsql: Add support for specifying custom SQL backend options.
by Gergely Nagy 22 Dec '11

22 Dec '11

When one is using an SQL backend that supports a set of options that syslog-ng does not set itself, yet, one does want these options set, there was previously no way to accomplish that without changing the source. This patch adds a new option: dbd-option(OPTION_NAME VALUE) (where OPTION_NAME is always a string, and value can be either a string, or a number). The options specified this way will always be set with libdbi whenever we (re)connect to the database. A simple use-case: destination d_slow_sql_test { sql(type("null") dbd-option("null.sleep.connect" 1) dbd-option("null.sleep.query" 5) columns("dummy") values("1") ); }; Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsql/afsql-grammar.ym | 3 +++ modules/afsql/afsql-parser.c | 2 ++ modules/afsql/afsql.c | 40 ++++++++++++++++++++++++++++++++++++++++ modules/afsql/afsql.h | 4 ++++ 4 files changed, 49 insertions(+), 0 deletions(-) diff --git a/modules/afsql/afsql-grammar.ym b/modules/afsql/afsql-grammar.ym index 580fb80..d21a3bf 100644 --- a/modules/afsql/afsql-grammar.ym +++ b/modules/afsql/afsql-grammar.ym @@ -54,6 +54,7 @@ extern LogDriver *last_driver; %token KW_DEFAULT %token KW_RETRIES +%token KW_DBD_OPTION %type <ptr> dest_afsql %type <ptr> dest_afsql_params @@ -100,6 +101,8 @@ dest_afsql_option free($3); } | KW_USERNAME '(' string ')' { afsql_dd_set_user(last_driver, $3); free($3); } + | KW_DBD_OPTION '(' string LL_NUMBER ')' { afsql_dd_add_dbd_option_numeric(last_driver, $3, $4); free($3); } + | KW_DBD_OPTION '(' string string ')' { afsql_dd_add_dbd_option(last_driver, $3, $4); free($3); free($4); } | KW_PASSWORD '(' string ')' { afsql_dd_set_password(last_driver, $3); free($3); } | KW_DATABASE '(' string ')' { afsql_dd_set_database(last_driver, $3); free($3); } | KW_TABLE '(' string ')' { afsql_dd_set_table(last_driver, $3); free($3); } diff --git a/modules/afsql/afsql-parser.c b/modules/afsql/afsql-parser.c index 877389a..291e495 100644 --- a/modules/afsql/afsql-parser.c +++ b/modules/afsql/afsql-parser.c @@ -55,6 +55,8 @@ static CfgLexerKeyword afsql_keywords[] = { { "flush_lines", KW_FLUSH_LINES }, { "flush_timeout", KW_FLUSH_TIMEOUT }, { "flags", KW_FLAGS }, + + { "dbd_option", KW_DBD_OPTION }, { NULL } }; diff --git a/modules/afsql/afsql.c b/modules/afsql/afsql.c index b3fec86..a327cdd 100644 --- a/modules/afsql/afsql.c +++ b/modules/afsql/afsql.c @@ -103,6 +103,9 @@ typedef struct _AFSqlDestDriver StatsCounterItem *dropped_messages; StatsCounterItem *stored_messages; + GHashTable *dbd_options; + GHashTable *dbd_options_numeric; + /* shared by the main/db thread */ GThread *db_thread; GMutex *db_thread_mutex; @@ -127,6 +130,22 @@ static const char *s_freetds = "freetds"; #define MAX_FAILED_ATTEMPTS 3 void +afsql_dd_add_dbd_option(LogDriver *s, const gchar *name, const gchar *value) +{ + AFSqlDestDriver *self = (AFSqlDestDriver *) s; + + g_hash_table_insert(self->dbd_options, g_strdup(name), g_strdup(value)); +} + +void +afsql_dd_add_dbd_option_numeric(LogDriver *s, const gchar *name, gint value) +{ + AFSqlDestDriver *self = (AFSqlDestDriver *) s; + + g_hash_table_insert(self->dbd_options_numeric, g_strdup(name), GINT_TO_POINTER(value)); +} + +void afsql_dd_set_type(LogDriver *s, const gchar *type) { AFSqlDestDriver *self = (AFSqlDestDriver *) s; @@ -608,6 +627,19 @@ afsql_dd_disconnect(AFSqlDestDriver *self) g_hash_table_remove_all(self->validated_tables); } +static void +afsql_dd_set_dbd_opt(gpointer key, gpointer value, gpointer user_data) +{ + dbi_conn_set_option((dbi_conn)user_data, (gchar *)key, (gchar *)value); +} + +static void +afsql_dd_set_dbd_opt_numeric(gpointer key, gpointer value, gpointer user_data) +{ + dbi_conn_set_option_numeric((dbi_conn)user_data, (gchar *)key, + GPOINTER_TO_INT(value)); +} + /** * afsql_dd_insert_db: * @@ -645,6 +677,10 @@ afsql_dd_insert_db(AFSqlDestDriver *self) dbi_conn_set_option(self->dbi_ctx, "sqlite_dbdir", ""); dbi_conn_set_option(self->dbi_ctx, "sqlite3_dbdir", ""); + /* Set user-specified options */ + g_hash_table_foreach(self->dbd_options, afsql_dd_set_dbd_opt, self->dbi_ctx); + g_hash_table_foreach(self->dbd_options_numeric, afsql_dd_set_dbd_opt_numeric, self->dbi_ctx); + if (dbi_conn_connect(self->dbi_ctx) < 0) { const gchar *dbi_error; @@ -1200,6 +1236,8 @@ afsql_dd_free(LogPipe *s) string_list_free(self->values); log_template_unref(self->table); g_hash_table_destroy(self->validated_tables); + g_hash_table_destroy(self->dbd_options); + g_hash_table_destroy(self->dbd_options_numeric); if(self->session_statements) string_list_free(self->session_statements); log_dest_driver_free(s); @@ -1235,6 +1273,8 @@ afsql_dd_new(void) self->num_retries = MAX_FAILED_ATTEMPTS; self->validated_tables = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); + self->dbd_options = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, g_free); + self->dbd_options_numeric = g_hash_table_new_full(g_str_hash, g_int_equal, g_free, NULL); log_template_options_defaults(&self->template_options); init_sequence_number(&self->seq_num); diff --git a/modules/afsql/afsql.h b/modules/afsql/afsql.h index ee2ae68..ce2e18a 100644 --- a/modules/afsql/afsql.h +++ b/modules/afsql/afsql.h @@ -56,6 +56,8 @@ void afsql_dd_set_flags(LogDriver *s, gint flags); LogDriver *afsql_dd_new(); gint afsql_dd_lookup_flag(const gchar *flag); void afsql_dd_set_retries(LogDriver *s, gint num_retries); +void afsql_dd_add_dbd_option(LogDriver *s, const gchar *name, const gchar *value); +void afsql_dd_add_dbd_option_numeric(LogDriver *s, const gchar *name, gint value); #else @@ -69,6 +71,8 @@ void afsql_dd_set_retries(LogDriver *s, gint num_retries); #define afsql_dd_set_columns(s, c) #define afsql_dd_set_values(s, v) #define afsql_dd_set_null_value(s, v) +#define afsql_dd_add_dbd_option(s, n, v) +#define afsql_dd_add_dbd_option_numeric(s, n, v) #define afsql_dd_new() 0 -- 1.7.7.3

2 4

Minimum Syslog Message Length
by vincent＠ragosta.net 22 Dec '11

22 Dec '11

Is it possible to specify a restriction on the minimum length of a syslog message? We often receive small, malformed messages that we do not want to transmit. Thanks, Vincent

3 3

trap divide error
by Dmitry Gilev 22 Dec '11

22 Dec '11

Hi, I've started to get the following messages since upgrade to 3.3.3 (from 3.0.7: >From dmesg logs: syslog-ng[30744] trap divide error ip:7fa0f8ed5a56 sp:7fa0f64423a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[30798] trap divide error ip:7fa0f8ed5a56 sp:7fa0df5fb3a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[31043] trap divide error ip:7fa0f8ed5a56 sp:7fa0d2bfa3a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[31059] trap divide error ip:7fa0f8ed5a56 sp:7fa0e6bfa3a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[31087] trap divide error ip:7fa0f8ed5a56 sp:7fa0de1f93a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[31145] trap divide error ip:7fa0f8ed5a56 sp:7fa0f64423a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[32733] trap divide error ip:7fa0f8ed5a56 sp:7fa0f64423a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[1404] trap divide error ip:7fa0f8ed5a56 sp:7fa0f6e433a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[1531] trap divide error ip:7fa0f8ed5a56 sp:7fa0f64423a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[1688] trap divide error ip:7fa0f8ed5a56 sp:7fa0e7ffc3a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[2013] trap divide error ip:7fa0f8ed5a56 sp:7fa0f4c2c3a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] syslog-ng[2019] trap divide error ip:7fa0f8ed5a56 sp:7fa0f6e433a0 error:0 in libsyslog-ng-3.3.3.so[7fa0f8ea3000+7b000] Does anybody know what is wrong here? Is it known bug? # syslog-ng -V syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#master#d199a1980be6b23fe24189e86a882812288e292c Compile-Date: Dec 20 2011 15:17:13 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat Available-Modules: basicfuncs,confgen,affile,csvparser,dbparser,afsocket,dummy,afsocket-notls,syslogformat,afprog,convertfuncs,afuser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off # ldd libsyslog-ng-3.3.3.so linux-vdso.so.1 => (0x00007fffc41fc000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f42f3d64000) libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f42f3b61000) libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007f42f395c000) librt.so.1 => /lib64/librt.so.1 (0x00007f42f3754000) libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f42f346e000) libevtlog.so.0 => /lib64/libevtlog.so.0 (0x00007f42f3269000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f42f3065000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f42f2e49000) libc.so.6 => /lib64/libc.so.6 (0x00007f42f2aa7000) /lib64/ld-linux-x86-64.so.2 (0x00000035d0200000) # uname -a Linux xxx.xxx.com 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux (CentOS 6.2) Thanks! Dmitry

3 4

patterndb & not describing the entire log message
by Christophe Brocas 22 Dec '11

22 Dec '11

Hello As I said before in other message, I currently work with patterndb to write a pattern file describing Windows log messages related to file system. These messages are sent to Syslog-NG through Snare software. Example : Aug 12 15:08:48 10.0.0.1 MSWinEventLog 1 Security 541 ven. août 12 15:08:46 201 4656 Microsoft-Windows-Security-Auditing D00000001\accout1 N/A Success Audit hostname.d00000001.subdom1.dom1.fr Système de fichiers Un handle vers un objet a été demandé. Sujet : ID de sécurité : S-1-5-21-514083418-1667312537-733111063-1179 Nom du compte : account1 Domaine du compte : D00000001 ID d’ouverture de session : 0x8e90858 Objet : Serveur de l’objet : Security Type d’objet : File Nom de l’objet : E:\xxx\yyyy\filename.doc.doc ID du handle : 0x2b0 Informations sur le processus : ID du processus : 0xeb0 Nom du processus : C:\Windows\System32\inetsrv\w3wp.exe Informations sur la demande d’accès : ID de la transaction : {00000000-0000-0000-0000-000000000000} Accès : DELETE SYNCHRONIZE ReadAttributes Masque d’accès : 0x110080 Privilèges utilisés pour les vérifications d’accès : - Nombre de SID restreints : 0 540 Question : That kind of messages are extremely long and some areas seem empty in my example but may be not in the future. Is there a way with pattern to not describe the entire message but only the relevant part for us (the equivalent of wildcard in regexp) ? Thanks Christophe ***************************************************** "Le contenu de ce courriel et ses éventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire. Attention : L'organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'organisme sauf s'il en est disposé autrement dans le présent courriel." ******************************************************

4 7

Message parameter substitution
by Anup Shetty 21 Dec '11

21 Dec '11

I am new to syslog-ng and would like some help on the pattern matching and the substitution option. Currently the requirement is to substitute a parameter in the message with a random value in order to anonymize it. *For example:* Dec 31 23:13:25 servername sshd[25218]: Failed keyboard-interactive/pam for *user1* from 10.x.x.x port 47325 ssh2 If I create a pattern database for this message and pick out the username using the string and substitute it user1 to say anon1, will I be able to store the original-substituted value pair for this user and use it repeatedly? Would I be able to do it for all the subsequent logs? To be more clear, an example substitution process that must happen as the logs arrive and the patterns are matched. log with user1 arrives and is substituted by anon1 log with user2 arrives and is substituted by anon2 again log with user1 arrives and is again substituted by anon1 log with user3 arrives and is substituted by anon3 again log with user2 arrives and is again substituted by anon2 . . . . This is required so that once the usernames are substituted for attaining anonymity, there must be a way to reverse them for audit purposes. -- Thanks and regards, AS

3 5