August 2011 - syslog-ng - lists.balabit.hu

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 121] New: After service syslog-ng reload, the first log don' t transfer to remote servicer via TLS
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=121 Summary: After service syslog-ng reload, the first log don't transfer to remote servicer via TLS Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: yong.zheng(a)nsn.com Type of the Report: bug Estimated Hours: 0.0 The version:syslog-ng 3.0.5 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq (3600); }; I will upload the logs to remote syslog-ng server via TLS, I encounter a issue, On client: after reload the configure via below command: #service syslog-ng reload. I write a log: #logger -p local1.info "this is test" but I don't see the log to transfer to server immediately, I wait many minutes, still don't see. I must write the second log to triger that client send the 2 logs to server #logger -p local1.info "this is test 2" the 2 logs have transfered to server immediately successfully, I can see the 2 logs on server. It is 100% reproduced. Is this a known issue, or not ? Thanks. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 42] New: capabilities, chown, chmod
by bugzilla＠bugzilla.balabit.com 20 Dec '12

20 Dec '12

https://bugzilla.balabit.com/show_bug.cgi?id=42 Summary: capabilities, chown, chmod Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: bug Estimated Hours: 0.0 Let's take a look at syslog-ng-3.0.1/src/affile.c lines 60-83 1. CAP_SYS_ADMIN is needed only for /proc/kmsg, it is added w/o check 2. CAP_DAC_READ_SEARCH should be added only if open fail with errno 13 2a. CAP_DAC_OVERRIDE should be added only if open fail with errno 13 and with CAP_DAC_READ_SEARCH set 3. fchown needs CAP_CHOWN unconditionaly 4. fchmod needs CAP_FOWNER if file owner != euid (root here) 5. all caps should be restored summary: - CAP_SYS_ADMIN and CAP_DAC_OVERRIDE are set always even if unnecessary, and permanently - owner, group and perm doesn't work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 10

[Bug 111] New: RPM build errors: Installed (but unpackaged) file(s) found:
by bugzilla＠bugzilla.balabit.com 14 Nov '11

14 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=111 Summary: RPM build errors: Installed (but unpackaged) file(s) found: Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: owen.mann(a)interactivedata.com Type of the Report: bug Estimated Hours: 0.0 when attempting `rpmbuild -tb syslog-ng_3.2beta1.tar.gz`, the compilation completes, but: [...] Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/syslog-ng-root error: Installed (but unpackaged) file(s) found: /bin/pdbtool /bin/update-patterndb /etc/syslog-ng/modules.conf /etc/syslog-ng/scl.conf /lib/libsyslog-ng.la /lib/libsyslog-ng.so /lib/libsyslog-ng.so.0 /lib/libsyslog-ng.so.0.0.0 /lib/syslog-ng/libaffile.la /lib/syslog-ng/libaffile.so /lib/syslog-ng/libafprog.la /lib/syslog-ng/libafprog.so /lib/syslog-ng/libafsocket-notls.la /lib/syslog-ng/libafsocket-notls.so /lib/syslog-ng/libafsocket-tls.la /lib/syslog-ng/libafsocket-tls.so /lib/syslog-ng/libafsocket.so /lib/syslog-ng/libafuser.la /lib/syslog-ng/libafuser.so /lib/syslog-ng/libbasicfuncs.la /lib/syslog-ng/libbasicfuncs.so /lib/syslog-ng/libconfgen.la /lib/syslog-ng/libconfgen.so /lib/syslog-ng/libconvertfuncs.la /lib/syslog-ng/libconvertfuncs.so /lib/syslog-ng/libcsvparser.la /lib/syslog-ng/libcsvparser.so /lib/syslog-ng/libdbparser.la /lib/syslog-ng/libdbparser.so /lib/syslog-ng/libdummy.la /lib/syslog-ng/libdummy.so /lib/syslog-ng/libsyslogformat.la /lib/syslog-ng/libsyslogformat.so /sbin/syslog-ng-ctl /share/include/scl/pacct/plugin.conf /share/include/scl/syslogconf/README /share/include/scl/syslogconf/convert-syslogconf.awk /share/include/scl/syslogconf/plugin.conf /share/include/scl/system/generate-system-source.sh /share/include/scl/system/plugin.conf /share/xsd/patterndb-1.xsd /share/xsd/patterndb-2.xsd /share/xsd/patterndb-3.xsd [...] If these files are added to the %files section in syslog-ng.spec, the build completes successfully. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

Loganalyzer
by Peter Czanik 11 Nov '11

11 Nov '11

Hello, Recently I found many e-mail and forum posts looking for a web GUI for syslog-ng. One of them, Loganalyzer is getting more popular, as a completely free web application. There is just one catch, it was not designed for syslog-ng originally. So I did some research and tests, and using my instructions quite easy to get started with it. Of course Loganalyzer has some rough edges, but it works for me on a number of different systems. For a step by step tutorial to get started, read my blog at http://czanik.blogs.balabit.com/2011/02/loganalyzer/ Have a nice weekend! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 3

[Bug 96] New: pdbtool misleading error message
by bugzilla＠bugzilla.balabit.com 01 Nov '11

01 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=96 Summary: pdbtool misleading error message Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: czanik(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 When trying to test a non-existent file with pdbtool it gives misleading error messages and even calls an external application before failing. czanik@linux-6y8u:~/patterndb> pdbtool test --validate uw-imapd.ceee uw-imapd.ceee: Unable to detect patterndb version, please write the <patterndb> tag on a single line warning: failed to load external entity "/usr/share/syslog-ng/xsd/patterndb-0.xsd" Schemas parser error : Failed to locate the main schema resource at '/usr/share/syslog-ng/xsd/patterndb-0.xsd'. WXS schema /usr/share/syslog-ng/xsd/patterndb-0.xsd failed to compile warning: failed to load external entity "uw-imapd.ceee" uw-imapd.ceee: xmllint returned an error, the executed command was: xmllint --noout --nonet --schema /usr/share/syslog-ng/xsd/patterndb-0.xsd uw-imapd.ceeeError opening classifier configuration file; filename='uw-imapd.ceee', error='No such file or directory (2)' I think, that the proper behavior would be, that pdbtool detect that the file is nonexistent, prints a message about it (without any other misleading messages) and quits without calling external apps. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 129] New: Add ability to synchronously flush logs on demand
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=129 Summary: Add ability to synchronously flush logs on demand Product: syslog-ng Version: 1.6.x Platform: All OS/Version: FreeBSD Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: fermulator(a)sympatico.ca Type of the Report: enhancement Estimated Hours: 0.0 As per the below e-mail thread with Bazsi, I was trying to figure out why syslog-ng doesn't write logs immediately. As I discovered syslog-ng by default has the concept of “flush_timeout” and “flush_lines”, where flush_timeout is how frequently we'll write logs (default 10s), and flush_lines is the maximum number of lines we'll hold in the buffer before writing (default 10,000). Currently, there is no way to synchronously force syslog-ng to "write logs now". This ticket is an enhancement to syslog-ng to add the ability, in a synchronous way, ask syslog-ng to flush the buffer now, writing all logs in the queue/buffer immediately/ASAP, and block until the operation is completed. Version Information: * syslog-ng 1.6.8 * FreeBSD 6.1 ----------- From: bazsi(a)balabit.hu To: fermulator(a)sympatico.ca Subject: Re: Flush (write logs now) for syslog-ng? Date: Fri, 22 Jul 2011 06:31:36 +0200 hi, we do have a bugzilla at bugzilla.balabit.com, however it's primarily used for bug reports. you can file enhancement requests though. ----- Original message ----- > Thanks for the reply. > > I tried SIGHUP, and as you mentioned, it's not synchronous. syslog-ng > does write the logs, but only after a few seconds (presumably this is > less than the timeout though at least). > > Do you have any "ticket system", that has the "enhancement" to do this > synchronously that I could reference? > > ~Matt > > > -------- Original Message -------- > From: Balazs Scheidler <bazsi(a)balabit.hu> > Sent: Thu 21 Jul 2011 02:31:24 PM EST > To: Fermulator Fermulator <fermulator(a)sympatico.ca> > Cc: > Subject: Re: Flush (write logs now) for syslog-ng? > > > hi, > > > > sending a SIGHUP should flush all buffers. this is not synchronous > > though, so you have to sleep some time to let syslog-ng do the thing. > > > > we have plans to add both on-demand synchronous reloads and config > > file flushes. > > > > ----- Original message ----- > > > > > > > > > > > > Hi Bazsi, > > > > > > > > > > > > I hope you don’t mind, I was trying to search for this in > > > the syslog-ng mail archives, but couldn’t find my answer, and found > > > your e-mail in the process. > > > > > > > > > > > > > > > As I understand it, syslog-ng has “flush_timeout” and “flush_lines” > > > parameters which determine how often we write out the buffer/queue of > > > log messages, and what the maximum number of lines is that we’ll > > > buffer, respectively. > > > > > > > > > > > > > > > > > > I’m trying to figure out: How can I tell syslog-ng to “write > > > logs now” on demand? (i.e. I > > > have a script that upon failure, scrapes some log files … but > > > instead of waiting for the flush_timeout, I’d rather “poke” > > > syslog-ng to “write all logs now” (i.e. temporarily set the > > > flush_timeout to 0, or temporarily set the flush_lines to 0) > > > > > > > > > > > > Is it possible? > > > > > > > > > > > > Thanks so much > > > > > > > > > ~Fermulator > > > (Matt Callaghan) > > > > > -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 115] New: libsyslog-ng.so is not linked with glib/cap but uses glib/ cap symbols
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=115 Summary: libsyslog-ng.so is not linked with glib/cap but uses glib/cap symbols Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: --- Estimated Hours: 0.0 [arekm@carme-pld ~/rpm/BUILD/syslog-ng-3.2.2/lib]$ ldd .libs/libsyslog-ng.so linux-vdso.so.1 => (0x00007fff46f80000) libc.so.6 => /lib64/libc.so.6 (0x00007fb64c4b8000) /lib64/ld-linux-x86-64.so.2 (0x0000003fba800000) but libsyslog-ng.so uses many glib and cap symbols: libtool: link: x86_64-pld-linux-gcc -std=gnu99 -I../../../lib -I../../../lib -I../../../modules/dbparser -I.. -Wno-pointer-sign -O2 -fno-strict-aliasing -fwrapv -march=x86-64 -gdwarf-3 -g2 -Wall .libs/test_patterndbS.o -Wl,--as-needed -Wl,--no-copy-dt-needed-entries -Wl,-z -Wl,relro -Wl,-z -Wl,combreloc -o .libs/test_patterndb test_patterndb.o ../.libs/patternize.o -Wl,-Bstatic -Wl,--whole-archive -Wl,--export-dynamic -pthread -Wl,--no-whole-archive -Wl,-Bdynamic ../../syslogformat/.libs/libsyslogformat.so ../../../lib/.libs/libsyslog-ng.so ../.libs/libsyslog-ng-patterndb.so /home/users/arekm/rpm/BUILD/syslog-ng-3.2.2/lib/.libs/libsyslog-ng.so -lssl -lcrypto -lresolv /usr/lib64/libgmodule-2.0.so /usr/lib64/libgthread-2.0.so -lpthread /usr/lib64/libglib-2.0.so /usr/lib64/libevtlog.so /usr/lib64/libpcre.so -lcap -ldl -lrt -pthread -Wl,-rpath -Wl,/lib64/syslog-ng ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_set_proc' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_from_text' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_build_path' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_free' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_make_resident' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_thread_init' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_symbol' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_open' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `g_module_error' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_set_flag' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_to_text' ../../../lib/.libs/libsyslog-ng.so: undefined reference to `cap_get_proc' That's in mixed linking, CORE_DEPS_LIBS is then empty. CORE_DEPS_LIBS should contain static linking to glib and dynamic to cap then. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 5

[Bug 114] New: configure strips -ldl -lrt for static linking but doesn' t add back with dynamic linking
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=114 Summary: configure strips -ldl -lrt for static linking but doesn't add back with dynamic linking Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: --- Estimated Hours: 0.0 configure.in when detecting glib strips -ldl and -lrt to prevent static linking with these libraries. # strip out -ldl & -lrt as it cannot be linked statically GLIB_LIBS=`echo $GLIB_LIBS | tr ' ' '\n' | egrep -v "^(-ldld?)|(-lrt)$" | tr '\n' ' '` This is fine. Unfortunately it doesn't add -ldl and -lrt back with dynamic linking and that causes newer glib detection to fail: configure:14223: checking for g_hash_table_new configure:14223: x86_64-pld-linux-gcc -std=gnu99 -o conftest -O2 -fno-strict-aliasing -fwrapv -march=x86-64 -gdwarf-3 -g2 -Wall -D_FORTIFY_SOURCE=2 -Wl,--as-needed -Wl,--no-copy-dt-needed-entries -Wl,-z,relro -Wl,-z,combreloc conftest.c -Wl,-Bstatic -Wl,--export-dynamic -pthread -lgmodule-2.0 -lgthread-2.0 -lglib-2.0 -Wl,-Bdynamic >&5 /usr/lib64/gcc/x86_64-pld-linux/4.5.2/../../../../lib64/libglib-2.0.a(gmain.o): In function `g_get_monotonic_time': (.text+0x2e12): undefined reference to `clock_gettime' collect2: ld returned 1 exit status -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 108] New: 2.6.38+ will require CAP_SYSLOG (CAP_SYS_ADMIN not enough)
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=108 Summary: 2.6.38+ will require CAP_SYSLOG (CAP_SYS_ADMIN not enough) Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: enhancement Estimated Hours: 0.0 2.6.38+ kernels (linus git) reports: [ 9.890305] WARNING: at kernel/printk.c:430 do_syslog+0x427/0x452() [ 9.890318] Hardware name: 2764CTO [ 9.890320] Attempt to access syslog with CAP_SYS_ADMIN but no CAP_SYSLOG (deprecated and denied). when syslog tries to do syslogging. Additionally I'm geting: [root@t400 ~]# /usr/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf Error opening file for reading; filename='/proc/kmsg', error='Operation not permitted (1)' Error initializing source driver; source='s_sys', id='s_sys#0' Error initializing message pipeline; which likely is due to the same problem. Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h… -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 44