- syslog-ng - lists.balabit.hu

setting facility/level of source log messages ?
by bana＠docisland.org 29 Apr '09

29 Apr '09

Hi, I often meet the case where I need to centralize some purely applicative log messages (ie they don't even have a PRI header). So syslog-ng sets a default facility and severity for these messages, like kern facility and emergency severity for a source file with a "no-parse" option. Is there a clean way to change these default values by modifying the configuration file ? (and not using a source program() nor a tail | logger trick). I've already written a quick and dirty patch for my needs, but it would be great to have a native way of doing the same thing (I may have missed something in the documentation). Best Regards. -- Xavier

2 9

Parser-DB Memory Leak
by Martin Holste 27 Apr '09

27 Apr '09

I'm running tests with the db-parser() parser module and finding that the memory grows exponentially until the program finally dies with: ***MEMORY-ERROR***: syslog-ng[2769]: GSlice: failed to allocate 248 bytes (alignment: 256): Cannot allocate memory Here is my configuration: @version: 3.0 options { }; source s_tcp { tcp(ip(0.0.0.0) port(445)); }; parser p_db { db-parser(); }; filter f_security_class { match("security" value(".classifier.class")); }; filter f_class_unknown { match("unknown" value(".classifier.class")); }; filter f_not_class_unknown { not match("unknown" value(".classifier.class")); }; filter f_FIREWALL_ACCESS_DENY { match("2" value(".classifier.rule_id")); }; template t_FIREWALL_ACCESS_DENY { template("$R_UNIXTIME\t$SOURCEIP\t$FACILITY\t$PROGRAM\t${.classifier.class}\t${.classifier.rule_id}\t${FIREWALL.proto}\t${FIREWALL.o_int}\t${FIREWALL.srcip}\t${FIREWALL.srcport}\t${FIREWALL.i_int}\t${FIREWALL.dstip}\t${FIREWALL.dstport}\t${FIREWALL.access_group}\t${FWSM.PRIORITY}\t${FWSM.MSG_CODE}\t${MSGONLY}\n"); }; destination d_FIREWALL_ACCESS_DENY { fifo("/tmp/db_parsed_fifo", template(t_FIREWALL_ACCESS_DENY)); }; filter f_FIREWALL_CONNECTION_END { match("3" value(".classifier.rule_id")); }; template t_FIREWALL_CONNECTION_END { template("$R_UNIXTIME\t$SOURCEIP\t$FACILITY\t$PROGRAM\t${.classifier.class}\t${.classifier.rule_id}\t${FIREWALL.proto}\t${FIREWALL.o_int}\t${FIREWALL.srcip}\t${FIREWALL.srcport}\t${FIREWALL.i_int}\t${FIREWALL.dstip}\t${FIREWALL.dstport}\t${FIREWALL.conn_duration}\t${FIREWALL.conn_bytes}\t$MSGONLY\n"); }; destination d_FIREWALL_CONNECTION_END { fifo("/tmp/db_parsed_fifo", template(t_FIREWALL_CONNECTION_END)); }; template t_DEFAULT { template("$R_UNIXTIME\t$SOURCEIP\t\$FACILITY\t$PROGRAM\t1\t1\t$MSGONLY\n"); }; destination d_default_fifo { fifo("/tmp/db_parsed_fifo", template(t_DEFAULT)); }; log { source(s_tcp); log { parser(p_db); filter(f_not_class_unknown); log { filter(f_FIREWALL_ACCESS_DENY); destination(d_FIREWALL_ACCESS_DENY); }; log { filter(f_FIREWALL_CONNECTION_END); destination(d_FIREWALL_CONNECTION_END); }; flags(final); }; log { destination(d_default_fifo); }; }; var/patterndb.xml: <patterndb version='1' pub_date='2009-04-16'> <program name='FWSM'> <pattern>%FWSM</pattern> <rule id='2' class='2'> <pattern>Deny@QSTRING:FIREWALL.proto: @src@QSTRING:FIREWALL.o_int: :@@IPv4:FIREWALL.srcip:@/@NUMBER:FIREWALL.srcport:@ dst@QSTRING:FIREWALL.i_int: :@@IPv4:FIREWALL.dstip:@/@NUMBER:FIREWALL.dstport:@ by access-group @QSTRING:FIREWALL.access_group:"@</pattern> </rule> <rule id='3' class='3'> <pattern>Teardown@QSTRING:FIREWALL.proto: @connection @NUMBER::@ for@QSTRING:FIREWALL.o_int: :@@IPv4:FIREWALL.srcip:@ /@NUMBER:FIREWALL.srcport@ to@QSTRING:FIREWALL.i_int: :@@IPv4:FIREWALL.dstip:@/@NUMBER:FIREWALL.dstport@ duration@QSTRING:FIREWALL.conn_duration: @bytes @NUMBER:FIREWALL.conn_bytes:@</pattern> </rule> </program> </patterndb> syslog-ng -V syslog-ng 3.0.1+binpkg4 Revision: ssh+git://bazsi@git.balabit //var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#555574a984eaef9410a2869db0af1be0d52b269b Compile-Date: Apr 16 2009 14:42:21 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: off Enable-Pcre: on Should I recompile with memtrace on? Thanks, Martin

4 10

syslog-ng 3.0.1 dies on reload with program() destination
by Jan Rekorajski 24 Apr '09

24 Apr '09

Sorry for possible double post, I sent the first from wrong email. Hi, Syslog-ng 3.0.1 dies on reload (kill -HUP pid) when config file contains log { destination { program() } } directive set. 100% repeatable. Here are the relevant config lines: destination d_sshguard { program("/usr/sbin/sshguard -a 6"); }; # A desperate experiment #destination d_sshguard { program("read"); }; log { source(s_sys); filter(f_authpriv); destination(d_sshguard); }; >From what I managed to find with strace/ltrace, syslog-ng kills the program, does not clear some data structures and then falls face down with sigsegv/term/ill trying to restart/reattach(?) to the program. The exact same configuration works perfectly with version 2.0.10. -- Jan Rekorajski | ALL SUSPECTS ARE GUILTY. PERIOD! baggins<at>mimuw.edu.pl | OTHERWISE THEY WOULDN'T BE SUSPECTS, WOULD THEY? BOFH, MANIAC | -- TROOPS by Kevin Rubio

5 8

sunfreeware syslog-ng v 3.0.1
by Ngim, Hyrum CTR ADCF/Arrowpoint 23 Apr '09

23 Apr '09

I get this error when I introduce the owner when defining a destination: destination firewall { file("/data/firewall/fwlog" owner(sysadmin) group(sysadmin) perm(0644)); }; When I run a syntax check I get this: # syslog-ng -s Error resolving user; user='sysadmin' # pkginfo -l SMCsyslng PKGINST: SMCsyslng NAME: syslogng CATEGORY: application ARCH: sparc VERSION: 3.0.1 BASEDIR: /usr/local VENDOR: BalaBit IT Ltd PSTAMP: Steve Christensen INSTDATE: Apr 15 2009 11:56 EMAIL: steve(a)smc.vnet.net STATUS: completely installed FILES: 91 installed pathnames 4 shared pathnames 25 directories 7 executables 1610 blocks used (approx) I also checked the misc.c file in the sunfreeware source and it didn't seem like it had the same errors that were identified in posts like this one: https://lists.balabit.hu/pipermail/syslog-ng/2009-April/012761.html

2 2

sunfreeware syslog-ng v 3.0.1
by Ngim, Hyrum CTR ADCF/Arrowpoint 23 Apr '09

23 Apr '09

I have installed and am configuring the sunfreeware package for syslog-ng. I ran into this issue that I saw on some other emails: Error resolving user; user='sysadmin' Is there a way around this to get it working? TIA

2 1

IPv6 addresses logged as 127.0.0.1
by Michael Granzow 22 Apr '09

22 Apr '09

When using the template parameter $SOURCEIP, connections originating from IPv6 addresses are logged as "127.0.0.1". The following patch fixes that issue. Listening on :: on dual-stack hosts will lead to IPv4 connections being logged in IPv6-mapped format, e.g. ::ffff:10.11.12.13. This is not what people usually expect, but it's not wrong. If desired, the place to fix that would be g_sockaddr_inet6_format().

2 2

Error compiling syslog-ng-3.0.1 on HP-UX 11.23 (Itanium)
by Monz Roswitha 22 Apr '09

22 Apr '09

Hi, I am trying to build syslog-ng-3.0.1 on HP-UX Itanium (64-bit) box. I am running ./configure CFLAGS="-U_XOPEN_SOURCE -U_XOPEN_SOURCE_EXTENDED -D_HPUX_SOURCE" and it is giving following error messages while 'make': if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/hpux32/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -U_XOPEN_SOURCE -U_XOPEN_SOURCE_EXTENDED -D_HPUX_SOURCE -Wall -U_XOPEN_SOURCE -U_XOPEN_SOURCE_EXTENDED -D_HPUX_SOURCE -MT compat.o -MD -MP -MF ".deps/compat.Tpo" -c -o compat.o compat.c; \ then mv -f ".deps/compat.Tpo" ".deps/compat.Po"; else rm -f ".deps/compat.Tpo"; exit 1; fi compat.c:35: error: redefinition of '__pread' /usr/local/lib/gcc/ia64-hp-hpux11.23/3.4.3/include/sys/unistd.h:584: error: previous definition of '__pread' was here compat.c:57: error: redefinition of '__pwrite' /usr/local/lib/gcc/ia64-hp-hpux11.23/3.4.3/include/sys/unistd.h:586: error: previous definition of '__pwrite' was here *** Error exit code 1 Stop. *** Error exit code 1 Stop. *** Error exit code 1 Stop. How can I solve this problem? Is a binary for HP-UX 11.23 Itanium (IA64) planed? Thanks for help. Kind regards, Roswitha Mit freundlichen Grüßen Roswitha Monz, OCP -- Systemadministratorin Unix & Oracle Server-Netzwerk-Betrieb _____________________________________________________________ Gemeinnützige Salzburger Landeskliniken Betriebsgesellschaft mbH Servicebereich Informatik und Medizintechnik Müllner Hauptstraße 48, A-5020 Salzburg Tel.: +43-(0)662-4482-4938 Mob.: +43(0)664-604141-4938 Fax +43(0)662 4482-4908 mailto:r.monz@salk.at Internet: http://www.salk.at

2 1

Failure to source one file prevents entire syslog-ng logger from running
by Hari Sekhon 22 Apr '09

22 Apr '09

Hi, While I was away on holiday, another sysadmin changed the path of a 3rd party application logfile that I was I sourcing in syslog-ng (it cannot log directly to syslog). This caused the whole syslog-ng logger to basically fail to work/(re)start, giving an the error: Starting syslog-ng: Persistent configuration file is in invalid format, ignoring; Error opening file for reading; filename='/path/to/file.log', error='No such file or directory (2)' Error initializing source driver; source='s_file_blah' And auto-restarting the logger of course didn't work either since every time a restart is initiated the thing simply bailed out with this error. Shouldn't this error simply throw up a warning and continue logging the rest of what's happening on the system, rather than leaving a hole in the audit trail because it cannot lock in one source??? (Even better: start regardless, do what it can and then periodically check the source again, and every time it cannot get the source open, log a warning in syslog so that it can be caught by the monitoring infrastructure) Any suggestions for a workaround/option to prevent anything like this ever happening again in future would be welcome. -h -- Hari Sekhon Always open to interesting opportunities http://www.linkedin.com/in/harisekhon

2 3

db-parser QSTRING delmiter limitations
by Martin Holste 20 Apr '09

20 Apr '09

Hi, I'm new to the list and syslog-ng in general. I'm building a centralized log collector and am very interested in the power of the db-parser() parsing module. It really has amazing potential, and I'm eager to implement it. I've been playing with it quite a bit with a proof-of-concept to parse firewall logs from Cisco FWSM blades. The $MSGONLY part looks like this for a firewall deny: Deny udp src OUTSIDE:10.0.0.0/1234 dst INSIDE:192.168.0.0/5678 by access-group "OUTSIDE" [0xb74026ad, 0x0] My working parser entry is thus: <patterndb version='1' pub_date='2009-04-17'> <program name='FWSM'> <pattern>%FWSM</pattern> <rule id='1' class='security'> <pattern>Deny@QSTRING:FIREWALL.DENY_PROTO: @src</pattern> </rule> </program> </patterndb> This works great and returns udp and tcp in the ${FIREWALL.DENY_PROTO} macro for logging, along with the ${.classifier.class} and ${.classifier.rule_id} macros. However, when I try to parse out the interface, IP, and port numbers from "OUTSIDE:10.0.0.0/1234" part, the delimiters fail to capture correctly and the whole pattern misses. Here's what I'm trying to do: <patterndb version='1' pub_date='2009-04-17'> <program name='FWSM'> <pattern>%FWSM</pattern> <rule id='1' class='security'> <pattern>Deny@QSTRING:FIREWALL.DENY_PROTO: @src@QSTRING:FIREWALL.DENY_O_INT: @:@IPv4$:FIREWALL.DENY_SRCIP:@/@NUMBER:FIREWALL.DENY_SRCPORT: @dst</pattern> </rule> </program> </patterndb> After much debugging, it appears that there is a problem using QSTRING to match non-space-delimited parsing boundaries. That is, you cannot parse arbitrarily, you have to match on space boundaries. Is this true, or am I doing something wrong? I even tried to parse the 'n' out of the word 'Deny' with a pattern like <pattern>De@QSTRING:test: @y</pattern> and that fails. >From the debug, it appears that unless there is a space present, the radix key is off by one: Looking up node in the radix tree; i='0', nodelen='0', keylen='138', root_key='', key='Deny udp src<snip></snip>' Looking up node in the radix tree; i='2', nodelen='2', keylen='138', root_key='De', key='Deny udp src<snip></snip>' It looks like the key for the second entry should be key='ny udp src<snip></snip>' since the original 'De' match already hit. I put a lot of printf debugging statements in the code to see if I could figure out what was going wrong, but I havent' been able to conclude what the problem is yet, assuming arbitrary pattern delimiting was the intended goal. Is anyone able to successfully get db-parser() to parse on arbitrary characters? Also, the source code refers to STRING and ESTRING, how are those different from QSTRING? It looked like ESTRING was probably just an offset-based version of QSTRING. Thanks, Martin

2 3

message source hostname not displayed relay's hostname displaied instedd
by gatfi sami 17 Apr '09

17 Apr '09

hi i have syslog-ng 2.O.9-27 on sles11 32bite (SUSE Entreprise server 11) on 3 virtual machine i use the following syslog organisation a *syslog clien*t (read error log and sends them to a relay) the *relay* frorwords the message to a syslog-server The *syslog server* writes the messages in a pipe binde to a mysql database untill now everithing is working perfectly but in the syslog server i olways founf the relai host name ip adress i tried HOST_FROM in the template writing to the pipe it is always the same the relay's hostname destination d_mysql { pipe("/opt/topnet/mnt/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST_FROM', '$FACILITY', '$PRIORITY', '$LEVEL','$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; imagine if i have aot of cilents with each one a diiferent error log i will see nothing but the relai's hostname need hepl here thks

2 1