- syslog-ng - lists.balabit.hu

Invitation to connect on LinkedIn
by Roc Zhou via LinkedIn 28 Feb '11

28 Feb '11

LinkedIn ------------Roc Zhou requested to add you as a connection on LinkedIn: ------------------------------------------ Cosmin, I'd like to add you to my professional network on LinkedIn. - Roc Accept invitation from Roc Zhou http://www.linkedin.com/e/-uz7o7o-gkp2i0wr-3r/2QmX6bZJs-PJn0df2BKk2xSyRmW36… View invitation from Roc Zhou http://www.linkedin.com/e/-uz7o7o-gkp2i0wr-3r/2QmX6bZJs-PJn0df2BKk2xSyRmW36… ------------------------------------------ DID YOU KNOW you can conduct a more credible and powerful reference check using LinkedIn? Enter the company name and years of employment or the prospective employee to find their colleagues that are also in your network. This provides you with a more balanced set of feedback to evaluate that new hire. http://www.linkedin.com/e/-uz7o7o-gkp2i0wr-3r/rsr/inv-27/ -- (c) 2011, LinkedIn Corporation

1 0

Why the 'a' versions came out first ?
by Mr. James W. Laferriere 25 Feb '11

25 Feb '11

Hello Bazsi (& anyone else in the know) , While I am not a PE customer and more of a listener on these maillists I noticed this release with a kind of "Now why did they release the 'a' versions before the non-updated ?" . This just did not sound right So I thought I'd ask . Feb 24 devel(a)balabit.hu (4867) [syslog-ng-announce] syslog-ng Premium Edition 3.0.7a has been released Feb 24 devel(a)balabit.hu (4917) [syslog-ng-announce] syslog-ng Premium Edition 4.0.1a has been released Feb 25 devel(a)balabit.hu (6657) [syslog-ng-announce] syslog-ng Premium Edition 3.0.7 has been released Feb 25 devel(a)balabit.hu (8134) [syslog-ng-announce] syslog-ng Premium Edition 4.0.1 has been released Feb 25 devel(a)balabit.hu (4919) [syslog-ng-announce] syslog-ng Premium Edition 3.2.1b has been released Tia , JimL -- +------------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network&System Engineer | 3237 Holden Road | Give me Linux | | babydr(a)baby-dragons.com | Fairbanks, AK. 99709 | only on AXP | +------------------------------------------------------------------+

2 1

Firewalling with syslog-ng, a working prototype
by Valentijn Sessink 24 Feb '11

24 Feb '11

Hi list, For a week or so, I'm gathering the building blocks for a sort of low-tech intrusion detection/prevention system. My "itch": having a system that acts "real time" on the log messages that various daemons produce; having it low profile; easy to get it to act (i.e. no scripts that call scripts that call other scripts). For example, if sshd says "invalid user", I'd like the firewall to act on this, with as little steps in between as possible. Luckily, syslog-ng is able to find patterns all by itself, so I'm able to "skip the middle man", i.e. I can use syslog-ng directly on the firewalling rules. And what is better: I'm not even using the program() call! I'm currently running such a system in pre-production and I'm delighted. It's really easy to build. It works like a charm. Here's how: 0) Install syslog-ng. 1) To get started, get the sshd pattern from the Balabit git web frontend: http://git.balabit.hu/?p=bazsi/syslog-ng-patterndb.git;a=blob;f=access/sshd… and put it in /etc/syslog-ng/patterndb.d/, run pdbtool merge -D . -p /var/lib/syslog-ng/patterndb.xml 2) Setup your syslog-ng.conf file. Add, in the appropriate sections: # a destination; we have a dual destination here, a file to see the # blocked hosts, and an iptables-destination in /proc to block them. destination d_syslogblock { file("/proc/net/xt_recent/syslogblock" template("+${usracct.device}\n")); file("/var/log/syslogblock"); }; # a parser for the pattern-DB we made in step 1 parser pattern_db { db_parser( file("/var/lib/syslog-ng/patterndb.xml")); }; # a filter to filter the parser results filter f_syslogblock { tags("secevt") and match("REJECT" value("secevt.verdict")); }; # and finally, the log itself: log { source(s_src); parser(pattern_db); filter(f_syslogblock); destination(d_syslogblock); }; 3) Now set up your firewalling. A minimalistic firewall would look like: ip6tables -N block ip6tables -A INPUT -i lo -j ACCEPT ip6tables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT ip6tables -A INPUT -m state --state INVALID -j DROP ip6tables -A INPUT -m recent --rcheck --name block \ --seconds 3600 -j DROP # see note (1) below ip6tables -A INPUT -m recent --rcheck --name syslogblock \ --seconds 900 --hitcount 15 -j block ip6tables -A block -m recent --rcheck --name block --set 4) Repeat step 3, but with "iptables" instead of "ip6tables". (I used ip6tables on purpose, because it's time you get ready for IPv6 if you aren't). Ready. Not too hard, isn't it? It works as follows. syslog-ng will put the IP-address of any wrong password or username into the /proc/net/xt_recent/syslogblock file, with a "+" in front of the address. This is the iptables way of telling the "recent" module it should add a certain address. Now the IPtables rules tell, that if someone is seen in this "syslogblock" list more that 15 times within 15 minutes, then (s)he will be added to the "block" list. And if someone is in the "block" list, (s)he will be blocked for an hour. Note (1): if you replace the "--rcheck" here with an "--update" statement, the block will last even longer: "--rcheck" says: were you blocked less than an hour ago? Then we can't let you in. While "--update" says: were you blocked less than an hour ago, but you're here *again*? We'll block you for *another* hour, starting now. So the latter means that you actually need to be quiet for 60 minutes to be able to log in again. There is only one caveat in this whole setup: if you (or syslog-ng) inadvertedly send garbage to the /proc/net/xt_recent/syslogblock file, then this file will be closed. Syslog-ng will reopen it after the "reopen" timeout, so your message will probably come through; but should something change in the working of syslog-ng, like should syslog-ng try to *re-send* such a broken message, then this will, effectively, block the rest of the blocking messages. (I don't think this will happen, because "fprintf" is not a reliable transport, so to say, but you'll never know). I'll be implementing this solution for a couple of other servers in the next weeks or so, and I'll report if there is more to say about it. Oh, BTW: I think that these two lines: ip6tables -A INPUT -m recent --rcheck --name syslogblock --seconds 900 --hitcount 15 -j block ip6tables -A block -m recent --rcheck --name block --set can be concatenated so that you don't need a separate "block" chain anymore, as follows: ip6tables -A INPUT -m recent --rcheck --name syslogblock --seconds 900 --hitcount 15 -m recent --rcheck --name block --set But I'm not sure, I'll have to check. Best regards, Valentijn -- http://www.openoffice.nl/ Open Office - Linux Office Solutions Valentijn Sessink v.sessink#openoffice.nl +31(0)20-4214059

5 14

Syntax check / reload / duplicate messages issues in 3.2.2 (and 3.1.1)
by Jakub Jankowski 23 Feb '11

23 Feb '11

Hi, I've came across a weird bug in syslog-ng OSE 3.1.1, which is also present in 3.2.2: when I add a non-existant destination to syslog-ng config: [root@test-syslog-ng ~]# echo 'log { source(s_sys); filter(f_default); destination(d_nonexistent); };' >> /etc/syslog-ng/syslog-ng.conf it passes syntax checking ($SYSLOGNG_OPTIONS is basically --no-caps -p /pid/file/path - I need no-caps because this is virtual environment): [root@test-syslog-ng ~]# . /etc/sysconfig/syslog-ng && syslog-ng -s $SYSLOGNG_OPTIONS [root@test-syslog-ng ~]# echo $? 0 but after /etc/init.d/syslog-ng reload (HUP) this happens: Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Error in configuration, unresolved destination reference; destination='d_nonexistent' Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Error in configuration, unresolved destination reference; destination='d_nonexistent' Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Error initializing new configuration, reverting to old config; Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Error initializing new configuration, reverting to old config; Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Configuration reload request received, reloading configuration; Feb 18 14:17:40 test-syslog-ng syslog-ng[30198]: Configuration reload request received, reloading configuration; Afterwards, everything is logged twice(!) [root@test-syslog-ng ~]# R=$RANDOM; echo $R; logger -t test $R 578 [root@test-syslog-ng ~]# grep "test: $R" /var/log/messages Feb 18 14:38:28 test-syslog-ng test: 578 Feb 18 14:38:28 test-syslog-ng test: 578 [root@test-syslog-ng ~]# Another reload, and afterwards every message is logged 3 times: [root@test-syslog-ng ~]# service syslog-ng reload Reloading syslog-ng: [ OK ] [root@test-syslog-ng ~]# R=$RANDOM; echo $R; logger -t test $R 13143 [root@test-syslog-ng ~]# grep "test: $R" /var/log/messages Feb 18 14:39:57 test-syslog-ng test: 13143 Feb 18 14:39:57 test-syslog-ng test: 13143 Feb 18 14:39:57 test-syslog-ng test: 13143 [root@test-syslog-ng ~]# Can anyone confirm this behaviour? It's all fine if I stop syslog-ng and start it again (well, try to start): [root@test-syslog-ng ~]# service syslog-ng start Starting syslog-ng: Error in configuration, unresolved destination reference; destination='d_nonexistent' [FAILED] I see two bugs here: 1) syntax checking (-s) is, well, just syntax checking - but doesn't guarantee that config is valid. 2) after reloading configuration with unresolved destination N times, every message is logged 1+N times. My environment: CentOS5, syslog-ng (without any patches) built from my own .spec, which I made by adapting EPEL srpm (2.x) to 3.x. [root@test-syslog-ng ~]# syslog-ng -V syslog-ng 3.2.2 Installer-Version: 3.2.2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.2#master#1d3f396485eb47b1ff6aa18ac4f1c4cd51c0ea4c Compile-Date: Jan 26 2011 11:44:50 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: on Enable-Pcre: off Enable-Pacct: off [root@test-syslog-ng ~]# Config file is pretty straightforward: [root@test-syslog-ng ~]# cat /etc/syslog-ng/syslog-ng.conf @version: 3.2 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel") flags(no-multi-line)); unix-stream ("/dev/log" flags(no-multi-line)); internal(); # udp(ip(0.0.0.0) port(514) flags(no-multi-line)); }; destination d_cons { file("/dev/console" group("root")); }; destination d_mesg { file("/var/log/messages" group("root")); }; destination d_auth { file("/var/log/secure" group("root")); }; destination d_mail { file("/var/log/maillog" group("root") flush_lines(10)); }; destination d_spol { file("/var/log/spooler" group("root")); }; destination d_boot { file("/var/log/boot.log" group("root")); }; destination d_cron { file("/var/log/cron" group("root")); }; destination d_kern { file("/var/log/kern" group("root")); }; destination d_mlal { usertty("*"); }; # Facilities/levels filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; [root@test-syslog-ng ~]# Regards, -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

2 1

PatternDB: macros extraction from URLs
by Ioan Indreias 23 Feb '11

23 Feb '11

Hello all, We are using OSE 3.2.1 version and till now we have managed to configure most of the patterns we need. However we have reached to a point where we need some hints from the users of this list. The problem is the following: how to extract macros when the order of them are not known (like in a URL). For example we would like to extract 'user' and 'action' from an URL like the one mentioned below: APP[9988]: WEB[0011]: http://abc.example.com/query.php?user=test1&action=login&host=prod1&device=… HTTP 1.1 Unfortunately the "user" and "action" could be placed anywhere in the URL (as the URL is not created by aour application) thus we have to create something like this: <pattern>http://abc.example.com/query.php@ESTRING::u@ser=@ESTRING:user:&@action=…</pattern> <pattern>http://abc.example.com/query.php@ESTRING::u@ser=@ESTRING:user:&@ESTRING…</pattern> <pattern>http://abc.example.com/query.php@ESTRING::a@ction=@ESTRING:action:&@use…</pattern> <pattern>http://abc.example.com/query.php@ESTRING::a@ction=@ESTRING:action:&@EST…</pattern> and so on.... Not to mention if we need to extract the 'device' macro as well - the number of patterns grow significantly. Have somebody some hints on how to optimize the extraction of macros when them are not in an known order? Thank you in advance, Ioan

6 7

Newer libtool and autoconf versions?
by Corinna Vinschen 23 Feb '11

23 Feb '11

Hi, Right now, the current 3.2.1 sources can't be build on Cygwin. Apart from two minor problems which are fixed in git (bugzilla items 101 and 102) and a refresh of the files in contrib/cygwin-packaging (bugzilla entry 103), there's a somewhat more troubeling problem. The source tarballs contain libtool/autoconf-generated files which have been created with older versions of these tools, namely libtool 2.2.6b and autoconf 2.65. In order to get a working libtool *.la dependency mechanism on Cygwin, these files have to be created with pretty new versions of libtool/autoconf, right now at least libtool 2.4 and autoconf 2.67 (automake is not affected). Otherwise, with the old versions, building the shared libs fails due to a libtool bug. While I can rectify this situation locally by running autoreconf, it would be nice if building syslog-ng from a tarball would work out of the box on Cygwin, just as on other systems. So, here's my question: Is it possible to generate subsequent source tarballs using these newer versions of libtool and autoconf? Thanks in advance, Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat

5 32

Help required for Syslog-ng compilation on AIX 5.2.
by Rakesh Rajasekharan 23 Feb '11

23 Feb '11

Hi, I need some urgent help or advice to address a make error that I am getting during my compilation of syslog-ng on AIX machine using the --enable-static-linking option I am using gcc3.3.2 versoin. Below was my configure options that I used ./configure --enable-mixed-linking --prefix=/opt/soe/local/syslog-ng-3.0.7 configure goes fine but the make breaks with the below error. I first doubted flex for this but I do have that installed on my machine. mv -f .deps/main.Tpo .deps/main.Po gcc -std=gnu99 -g -O2 -Wall -D_LARGE_FILES=1 -L/usr/local/lib -o syslog-ng main.o libsyslog-ng.a -static -lrt -lnsl -lfl -L/usr/local/lib -lglib-2.0 -lintl -liconv -L/opt/soe/local/syslog-ng-3.0.7/eventlog/lib -levtlog -ldl globals.o ld: 0711-340 SEVERE ERROR Symbol yyout in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yyin in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yyout in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yyin in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yyout in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yyout in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-340 SEVERE ERROR Symbol yylineno in object libsyslog-ng.a[cfg-lex.o] A csect with multiple labels cannot be moved into the TOC. ld: 0711-224 WARNING: Duplicate symbol: macros ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. Please provide your inputs. Thanks & Reagrds, Rakesh

1 0

[Bug 113] New: hang on disk access seem to cause network logging to stop
by bugzilla＠bugzilla.balabit.com 23 Feb '11

23 Feb '11

https://bugzilla.balabit.com/show_bug.cgi?id=113 Summary: hang on disk access seem to cause network logging to stop Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: bug Estimated Hours: 0.0 syslog-ng deadlocks in some disk operation (faulty hardware; controler somehow locks up etc) then network logging to another host is also stopping. This is very unfortunate because network logging could give provide me a hints on why disks deadlocks etc. Also if we have few hard disks mounted in /mnt/test1, /mnt/test2, we log things into /mnt/test1/logfile, /mnt/test2/logfile and test test1 disk deadlocks then syslog-ng will also stop logging to test2/logfile. Basically one deadlocked device locks everything else. Major pain here. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

syslog_ng 3.0.7 make fails on RHEL.
by Rakesh Rajasekharan 23 Feb '11

23 Feb '11

Hi, I am finding no success trying to compile syslog-ng 3.0.7 on the RHEL machine using the --enable-static-linking mode. I get stuck in the make stage. I am using GNU gcc 3.4.5 version .Below is the error that appears in make. gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/opt/soe/local/syslog-ng-3.0.7/eventlog/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c mv -f .deps/main.Tpo .deps/main.Po gcc -std=gnu99 -g -O2 -Wall -o syslog-ng main.o libsyslog-ng.a -static -lrt -lnsl -lfl -L/usr/local/lib -lglib-2.0 -liconv -L/opt/soe/local/syslog-ng-3.0.7/eventlog/lib -levtlog -ldl -lwrap -lcap globals.o /usr/local/lib/gcc/i686-pc-linux-gnu/3.4.5/../../../../i686-pc-linux-gnu/bin/ld: cannot find -lcap collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/compile/syslog-ng-3.0.7/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/compile/syslog-ng-3.0.7' make: *** [all] Error 2 [root@CSCSVSLGG189v syslog-ng-3.0.7]# which ld /usr/local/bin/ld [root@CSCSVSLGG189v syslog-ng-3.0.7]# ld -V GNU ld version 2.17 Supported emulations: elf_i386 i386linux Can you please advice on this. Thanks & Regards, Rakesh

3 4

using correlation to filter out some messages ?
by Guillaume Rousse 22 Feb '11

22 Feb '11

Hello. I've had a look at message correlation in the 3.2 manual, but it's still a bit obscure for me to understand if it could allow me to achieve filtering of pairs of log messages, as described here: https://lists.balabit.hu/pipermail/syslog-ng/2010-January/013857.html -- BOFH excuse #360: Your parity check is overdrawn and you're out of cache.

5 19