April 2011 - syslog-ng - lists.balabit.hu

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 42] New: capabilities, chown, chmod
by bugzilla＠bugzilla.balabit.com 20 Dec '12

20 Dec '12

https://bugzilla.balabit.com/show_bug.cgi?id=42 Summary: capabilities, chown, chmod Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: bug Estimated Hours: 0.0 Let's take a look at syslog-ng-3.0.1/src/affile.c lines 60-83 1. CAP_SYS_ADMIN is needed only for /proc/kmsg, it is added w/o check 2. CAP_DAC_READ_SEARCH should be added only if open fail with errno 13 2a. CAP_DAC_OVERRIDE should be added only if open fail with errno 13 and with CAP_DAC_READ_SEARCH set 3. fchown needs CAP_CHOWN unconditionaly 4. fchmod needs CAP_FOWNER if file owner != euid (root here) 5. all caps should be restored summary: - CAP_SYS_ADMIN and CAP_DAC_OVERRIDE are set always even if unnecessary, and permanently - owner, group and perm doesn't work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 10

[Bug 111] New: RPM build errors: Installed (but unpackaged) file(s) found:
by bugzilla＠bugzilla.balabit.com 14 Nov '11

14 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=111 Summary: RPM build errors: Installed (but unpackaged) file(s) found: Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: owen.mann(a)interactivedata.com Type of the Report: bug Estimated Hours: 0.0 when attempting `rpmbuild -tb syslog-ng_3.2beta1.tar.gz`, the compilation completes, but: [...] Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/syslog-ng-root error: Installed (but unpackaged) file(s) found: /bin/pdbtool /bin/update-patterndb /etc/syslog-ng/modules.conf /etc/syslog-ng/scl.conf /lib/libsyslog-ng.la /lib/libsyslog-ng.so /lib/libsyslog-ng.so.0 /lib/libsyslog-ng.so.0.0.0 /lib/syslog-ng/libaffile.la /lib/syslog-ng/libaffile.so /lib/syslog-ng/libafprog.la /lib/syslog-ng/libafprog.so /lib/syslog-ng/libafsocket-notls.la /lib/syslog-ng/libafsocket-notls.so /lib/syslog-ng/libafsocket-tls.la /lib/syslog-ng/libafsocket-tls.so /lib/syslog-ng/libafsocket.so /lib/syslog-ng/libafuser.la /lib/syslog-ng/libafuser.so /lib/syslog-ng/libbasicfuncs.la /lib/syslog-ng/libbasicfuncs.so /lib/syslog-ng/libconfgen.la /lib/syslog-ng/libconfgen.so /lib/syslog-ng/libconvertfuncs.la /lib/syslog-ng/libconvertfuncs.so /lib/syslog-ng/libcsvparser.la /lib/syslog-ng/libcsvparser.so /lib/syslog-ng/libdbparser.la /lib/syslog-ng/libdbparser.so /lib/syslog-ng/libdummy.la /lib/syslog-ng/libdummy.so /lib/syslog-ng/libsyslogformat.la /lib/syslog-ng/libsyslogformat.so /sbin/syslog-ng-ctl /share/include/scl/pacct/plugin.conf /share/include/scl/syslogconf/README /share/include/scl/syslogconf/convert-syslogconf.awk /share/include/scl/syslogconf/plugin.conf /share/include/scl/system/generate-system-source.sh /share/include/scl/system/plugin.conf /share/xsd/patterndb-1.xsd /share/xsd/patterndb-2.xsd /share/xsd/patterndb-3.xsd [...] If these files are added to the %files section in syslog-ng.spec, the build completes successfully. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

Loganalyzer
by Peter Czanik 11 Nov '11

11 Nov '11

Hello, Recently I found many e-mail and forum posts looking for a web GUI for syslog-ng. One of them, Loganalyzer is getting more popular, as a completely free web application. There is just one catch, it was not designed for syslog-ng originally. So I did some research and tests, and using my instructions quite easy to get started with it. Of course Loganalyzer has some rough edges, but it works for me on a number of different systems. For a step by step tutorial to get started, read my blog at http://czanik.blogs.balabit.com/2011/02/loganalyzer/ Have a nice weekend! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 3

[Bug 96] New: pdbtool misleading error message
by bugzilla＠bugzilla.balabit.com 01 Nov '11

01 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=96 Summary: pdbtool misleading error message Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: czanik(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 When trying to test a non-existent file with pdbtool it gives misleading error messages and even calls an external application before failing. czanik@linux-6y8u:~/patterndb> pdbtool test --validate uw-imapd.ceee uw-imapd.ceee: Unable to detect patterndb version, please write the <patterndb> tag on a single line warning: failed to load external entity "/usr/share/syslog-ng/xsd/patterndb-0.xsd" Schemas parser error : Failed to locate the main schema resource at '/usr/share/syslog-ng/xsd/patterndb-0.xsd'. WXS schema /usr/share/syslog-ng/xsd/patterndb-0.xsd failed to compile warning: failed to load external entity "uw-imapd.ceee" uw-imapd.ceee: xmllint returned an error, the executed command was: xmllint --noout --nonet --schema /usr/share/syslog-ng/xsd/patterndb-0.xsd uw-imapd.ceeeError opening classifier configuration file; filename='uw-imapd.ceee', error='No such file or directory (2)' I think, that the proper behavior would be, that pdbtool detect that the file is nonexistent, prints a message about it (without any other misleading messages) and quits without calling external apps. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 115] New: libsyslog-ng.so is not linked with glib/cap but uses glib/ cap symbols
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=115 Summary: libsyslog-ng.so is not linked with glib/cap but uses glib/cap symbols Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: --- Estimated Hours: 0.0 [arekm@carme-pld ~/rpm/BUILD/syslog-ng-3.2.2/lib]$ ldd .libs/libsyslog-ng.so linux-vdso.so.1 => (0x00007fff46f80000) libc.so.6 => /lib64/libc.so.6 (0x00007fb64c4b8000) /lib64/ld-linux-x86-64.so.2 (0x0000003fba800000) but libsyslog-ng.so uses many glib and cap symbols: libtool: link: x86_64-pld-linux-gcc -std=gnu99 -I../../../lib -I../../../lib -I../../../modules/dbparser -I.. -Wno-pointer-sign -O2 -fno-strict-aliasing -fwrapv -march=x86-64 -gdwarf-3 -g2 -Wall .libs/test_patterndbS.o -Wl,--as-needed -Wl,--no-copy-dt-needed-entries -Wl,-z -Wl,relro -Wl,-z -Wl,combreloc -o .libs/test_patterndb test_patterndb.o ../.libs/patternize.o -Wl,-Bstatic -Wl,--whole-archive -Wl,--export-dynamic -pthread -Wl,--no-whole-archive -Wl,-Bdynamic ../../syslogformat/.libs/libsyslogformat.so ../../../lib/.libs/libsyslog-ng.so ../.libs/libsyslog-ng-patterndb.so /home/users/arekm/rpm/BUILD/syslog-ng-3.2.2/lib/.libs/libsyslog-ng.so -lssl -lcrypto -lresolv /usr/lib64/libgmodule-2.0.so /usr/lib64/libgthread-2.0.so -lpthread /usr/lib64/libglib-2.0.so /usr/lib64/libevtlog.so /usr/lib64/libpcre.so -lcap -ldl -lrt -pthread -Wl,-rpath -Wl,/lib64/syslog-ng ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_set_proc' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_from_text' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_build_path' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_free' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_make_resident' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_thread_init' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_symbol' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_open' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_error' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_set_flag' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_to_text' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_get_proc' That's in mixed linking, CORE_DEPS_LIBS is then empty. CORE_DEPS_LIBS should contain static linking to glib and dynamic to cap then. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 5

[Bug 114] New: configure strips -ldl -lrt for static linking but doesn' t add back with dynamic linking
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=114 Summary: configure strips -ldl -lrt for static linking but doesn't add back with dynamic linking Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: --- Estimated Hours: 0.0 configure.in when detecting glib strips -ldl and -lrt to prevent static linking with these libraries. # strip out -ldl & -lrt as it cannot be linked statically GLIB_LIBS=`echo $GLIB_LIBS | tr ' ' '\n' | egrep -v "^(-ldld?)|(-lrt)$" | tr '\n' ' '` This is fine. Unfortunately it doesn't add -ldl and -lrt back with dynamic linking and that causes newer glib detection to fail: configure:14223: checking for g_hash_table_new configure:14223: x86_64-pld-linux-gcc -std=gnu99 -o conftest -O2 -fno-strict-aliasing -fwrapv -march=x86-64 -gdwarf-3 -g2 -Wall -D_FORTIFY_SOURCE=2 -Wl,--as-needed -Wl,--no-copy-dt-needed-entries -Wl,-z,relro -Wl,-z,combreloc conftest.c -Wl,-Bstatic -Wl,--export-dynamic -pthread -lgmodule-2.0 -lgthread-2.0 -lglib-2.0 -Wl,-Bdynamic >&5 /usr/lib64/gcc/x86_64-pld-linux/4.5.2/../../../../lib64/libglib-2.0.a(gmain.o): In function `g_get_monotonic_time': (.text+0x2e12): undefined reference to `clock_gettime' collect2: ld returned 1 exit status -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 108] New: 2.6.38+ will require CAP_SYSLOG (CAP_SYS_ADMIN not enough)
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=108 Summary: 2.6.38+ will require CAP_SYSLOG (CAP_SYS_ADMIN not enough) Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: enhancement Estimated Hours: 0.0 2.6.38+ kernels (linus git) reports: [ 9.890305] WARNING: at kernel/printk.c:430 do_syslog+0x427/0x452() [ 9.890318] Hardware name: 2764CTO [ 9.890320] Attempt to access syslog with CAP_SYS_ADMIN but no CAP_SYSLOG (deprecated and denied). when syslog tries to do syslogging. Additionally I'm geting: [root@t400 ~]# /usr/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf Error opening file for reading; filename='/proc/kmsg', error='Operation not permitted (1)' Error initializing source driver; source='s_sys', id='s_sys#0' Error initializing message pipeline; which likely is due to the same problem. Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h… -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 44

[Bug 107] New: when trying to log to postgres database if port is not specified it gets defaulted to 0
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=107 Summary: when trying to log to postgres database if port is not specified it gets defaulted to 0 Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: jhnsmth64(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 syslog-ng.conf ... destination d_pgsql { sql(type(pgsql) host("localhost") username("user") password("pass") ... }; ... If destination stays defined like above, without port specified explicitly, syslog sends log messages to port 0 instead defaulting port to the Postgres port (5432). Destination settings below works: ... destination d_pgsql { sql(type(pgsql) host("localhost") port(5432) username("user") password("pass") ... }; ... -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 4

[Bug 106] New: Syslog-ng memory strdup and g_malloc leaks
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=106 Summary: Syslog-ng memory strdup and g_malloc leaks Product: syslog-ng Version: 3.2.x Platform: Other OS/Version: Solaris Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: daniel_novak(a)kb.cz Type of the Report: refactorization Estimated Hours: 0.0 We tried to compile a few last git version of syslog-3.2.1(syslog-ng-3.2.git-5f25ce47ab57774f8fe8df17bc96a006e535cb53) as well as 3.2.1 stable release. Everything is running well, but there are several minor memory leaks.We run on Solaris 10/08 on sparc platform.With libumem.so in memory debug mode MDB output is following : > ::findleaks CACHE LEAKED BUFCTL CALLER 00052288 2 00110d98 libc.so.1`strdup+0xc 00052788 1 0007de00 libglib-2.0.so.0.2513.0`g_malloc0+0x64 ---------------------------------------------------------------------- Total 3 buffers, 64 bytes =========================================================================================== > ::findleaks -dvf findleaks: maximum buffers => 4121 findleaks: actual buffers => 3967 mdb: [fef80000, fef84000): couldn't read 16384 bytes at fef80000: no mapping for address findleaks: findleaks: potential pointers => 341574 findleaks: dismissals => 196071 (57.4%) findleaks: misses => 130325 (38.1%) findleaks: dups => 11214 ( 3.2%) findleaks: follows => 3964 ( 1.1%) findleaks: findleaks: elapsed wall time => 0 seconds findleaks: CACHE LEAKED BUFCTL CALLER 00052288 2 00110d98 libc.so.1`strdup+0xc 00052788 1 0007de00 libglib-2.0.so.0.2513.0`g_malloc0+0x64 ---------------------------------------------------------------------- Total 3 buffers, 64 bytes umem_alloc_16 leak: 2 buffers, 16 bytes each, 32 bytes total ADDR BUFADDR TIMESTAMP THREAD CACHE LASTLOG CONTENTS 110d98 1097e0 1b08cf3fb0c96c 1 52288 43af4 0 libumem.so.1`umem_cache_alloc+0x210 libumem.so.1`umem_alloc+0x60 libumem.so.1`malloc+0x28 libc.so.1`strdup+0xc libsyslog-ng.so.0.0.0`cfg_lexer_lookup_keyword+0xf0 libsyslog-ng.so.0.0.0`_cfg_lexer_lex+0x18a4 libsyslog-ng.so.0.0.0`cfg_lexer_lex+0x284 libsyslog-ng.so.0.0.0`main_lex+0x1c libsyslog-ng.so.0.0.0`main_parse+0x7a4 libsyslog-ng.so.0.0.0`cfg_parser_parse+0xcc libsyslog-ng.so.0.0.0`cfg_run_parser+0x1ac libsyslog-ng.so.0.0.0`cfg_read_config+0x98 initial_init+0x5c main+0x244 _start+0x5c umem_alloc_32 leak: 1 buffer, 32 bytes ADDR BUFADDR TIMESTAMP THREAD CACHE LASTLOG CONTENTS 7de00 77b98 1b08cf406e21b0 1 52788 373ec 0 libumem.so.1`umem_cache_alloc+0x210 libumem.so.1`umem_alloc+0x60 libumem.so.1`malloc+0x28 libumem.so.1`calloc+0x58 libglib-2.0.so.0.2513.0`g_malloc0+0x64 libsyslog-ng.so.0.0.0`serialize_file_archive_new+0x1c libsyslog-ng.so.0.0.0`persist_state_load+0x54 libsyslog-ng.so.0.0.0`persist_state_start+0x34 libsyslog-ng.so.0.0.0`cfg_initial_init+0x28 initial_init+0xbc main+0x244 _start+0x5c ========================================================================= findleaks: using cached results (-f will force a full run) CACHE LEAKED BUFCTL CALLER 00052288 2 00110d98 libc.so.1`strdup+0xc 00052788 1 0007de00 libglib-2.0.so.0.2513.0`g_malloc0+0x64 ---------------------------------------------------------------------- Total 3 buffers, 64 bytes > 00110d98::umem_verify mdb: cache 110d98 () does not have redzone checking enabled > 00052288::umem_verify Summary for cache 'umem_alloc_16' clean > 00052788::umem_verify Summary for cache 'umem_alloc_32' clean ============================================================================= -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4