February 2012 - syslog-ng - lists.balabit.hu

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 155] New: Compilation failure in 3.3.3: the Darwin linker does not accept "--whole-archive"
by bugzilla＠bugzilla.balabit.com 26 Aug '13

26 Aug '13

https://bugzilla.balabit.com/show_bug.cgi?id=155 Summary: Compilation failure in 3.3.3: the Darwin linker does not accept "--whole-archive" Product: syslog-ng Version: 3.3.x Platform: Macintosh OS/Version: Mac OS Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: jacknagel(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Trying to compile 3.3.3 on Mac OS X 10.6.8/XCode 4.2, I get this: libtool: link: /usr/bin/llvm-gcc -std=gnu99 -dynamiclib -o .libs/libsyslog-ng-3.3.3.dylib [...] ld: unknown option: --whole-archive collect2: ld returned 1 exit status some cursory googling reveals that the Darwin linker does not accept "--whole-archive". I'd be happy to help test a patch. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 11

[Bug 121] New: After service syslog-ng reload, the first log don' t transfer to remote servicer via TLS
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=121 Summary: After service syslog-ng reload, the first log don't transfer to remote servicer via TLS Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: yong.zheng(a)nsn.com Type of the Report: bug Estimated Hours: 0.0 The version:syslog-ng 3.0.5 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq (3600); }; I will upload the logs to remote syslog-ng server via TLS, I encounter a issue, On client: after reload the configure via below command: #service syslog-ng reload. I write a log: #logger -p local1.info "this is test" but I don't see the log to transfer to server immediately, I wait many minutes, still don't see. I must write the second log to triger that client send the 2 logs to server #logger -p local1.info "this is test 2" the 2 logs have transfered to server immediately successfully, I can see the 2 logs on server. It is 100% reproduced. Is this a known issue, or not ? Thanks. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 42] New: capabilities, chown, chmod
by bugzilla＠bugzilla.balabit.com 20 Dec '12

20 Dec '12

https://bugzilla.balabit.com/show_bug.cgi?id=42 Summary: capabilities, chown, chmod Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: bug Estimated Hours: 0.0 Let's take a look at syslog-ng-3.0.1/src/affile.c lines 60-83 1. CAP_SYS_ADMIN is needed only for /proc/kmsg, it is added w/o check 2. CAP_DAC_READ_SEARCH should be added only if open fail with errno 13 2a. CAP_DAC_OVERRIDE should be added only if open fail with errno 13 and with CAP_DAC_READ_SEARCH set 3. fchown needs CAP_CHOWN unconditionaly 4. fchmod needs CAP_FOWNER if file owner != euid (root here) 5. all caps should be restored summary: - CAP_SYS_ADMIN and CAP_DAC_OVERRIDE are set always even if unnecessary, and permanently - owner, group and perm doesn't work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 10

[Bug 151] New: syslog-ng opens /dev/ log device on every HUP without closing it first
by bugzilla＠bugzilla.balabit.com 08 Oct '12

08 Oct '12

https://bugzilla.balabit.com/show_bug.cgi?id=151 Summary: syslog-ng opens /dev/log device on every HUP without closing it first Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Solaris Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: michael.hocke(a)nyu.edu Type of the Report: bug Estimated Hours: 0.0 On Solaris 10 the /dev/log device can be held open concurrently for up to only 16 instances. syslog-ng opens a new instance of /dev/log after HUPing the process, i.e. after the 16th HUP syslog-ng quits because the new configuration fails and so does the previous configuration. This has been working fine in 3.0. I cannot test if this started to happen in 3.1, 3.2, or if this is new for 3.3. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 161] New: Syslog-ng stops processing after reload
by bugzilla＠bugzilla.balabit.com 10 Sep '12

10 Sep '12

https://bugzilla.balabit.com/show_bug.cgi?id=161 Summary: Syslog-ng stops processing after reload Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 syslog-ng version: 3.3.4 OS: RHEL 6.2 Whenever I issue a reload to our syslog-ng instance syslog-ng stops processing messages and sits there doing nothing. The issue only seems to occur when the server is processing a high log volume. This server is part of a 2-node active/passive HA group. So we have 2 servers with the exact same configuration. Whenever I issue the reload on the server that is not currently active, it works fine. I can issue a failover and the server that was able to reload fine now hangs. The active server in this HA group processes around 35,000 messages per second. I havent posted the config because it is fairly large, however I can if desired. The most notable thing in it is that threading is enabled. Below is the backtrace of syslog-ng stuck in this state. I took a gcore, so I can gather any additional information that is needed. Also, if I try to do a full backtrace ('where full') in gdb on any thread other than thread 5, gdb segfaults. Not sure if that has any relevance or not. (gdb) info threads 5 Thread 0x7fb9c654ab00 (LWP 11040) 0x000000353cedd967 in writev () from /lib64/libc.so.6 4 Thread 0x7fb9c25fb700 (LWP 11049) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 3 Thread 0x7fb9c2ffc700 (LWP 11048) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 2 Thread 0x7fb9c39fd700 (LWP 11046) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 * 1 Thread 0x7fb9c43fe700 (LWP 11045) 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c43fb9a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x14fc610) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc6e0) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 2 [Switching to thread 2 (Thread 0x7fb9c39fd700 (LWP 11046))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c39fa9a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x14fc8e0) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc9b0) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 3 [Switching to thread 3 (Thread 0x7fb9c2ffc700 (LWP 11048))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c2ff99a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x15635c0) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x14fc380) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 4 [Switching to thread 4 (Thread 0x7fb9c25fb700 (LWP 11049))]#0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 (gdb) where #0 0x000000353cee5d03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fb9c65cfa3e in iv_epoll_poll (numfds=1, active=0x7fb9c25f89a0, msec=10000) at iv_method_epoll.c:73 #2 0x00007fb9c65cf1bb in iv_main () at iv_main.c:265 #3 0x00007fb9c65d3c36 in iv_work_thread (_thr=0x1566d20) at iv_work.c:196 #4 0x00007fb9c65d42a8 in iv_thread_handler (_thr=0x1566a70) at iv_thread.c:100 #5 0x000000353d2077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x000000353cee570d in clone () from /lib64/libc.so.6 (gdb) thread 5 [Switching to thread 5 (Thread 0x7fb9c654ab00 (LWP 11040))]#0 0x000000353cedd967 in writev () from /lib64/libc.so.6 (gdb) where #0 0x000000353cedd967 in writev () from /lib64/libc.so.6 #1 0x00007fb9c65820f8 in log_proto_file_writer_flush (s=0x171b5a0) at logproto.c:270 #2 0x00007fb9c658c403 in log_proto_flush (s=0x171b5a0) at logproto.h:86 #3 0x00007fb9c658ec3e in log_writer_flush (self=0x171adb0, flush_mode=LW_FLUSH_QUEUE) at logwriter.c:1032 #4 0x00007fb9c658f006 in log_writer_deinit (s=0x171adb0) at logwriter.c:1106 #5 0x00007fb9c63399d6 in log_pipe_deinit (s=0x171adb0) at ../../lib/logpipe.h:254 #6 0x00007fb9c633b3cd in affile_dw_deinit (s=0x171ac20) at affile.c:627 #7 0x00007fb9c63399d6 in log_pipe_deinit (s=0x171ac20) at ../../lib/logpipe.h:254 #8 0x00007fb9c633c211 in affile_dd_deinit_writer (key=0x171ad60, value=0x171ac20, user_data=0x0) at affile.c:974 #9 0x000000353e22a7a3 in g_hash_table_foreach () from /lib64/libglib-2.0.so.0 #10 0x00007fb9c633c349 in affile_dd_deinit (s=0x1539990) at affile.c:996 #11 0x00007fb9c6575d86 in log_pipe_deinit (s=0x1539990) at logpipe.h:254 #12 0x00007fb9c65761b4 in log_dest_group_deinit (s=0x1539ef0) at dgroup.c:76 #13 0x00007fb9c656f54c in log_pipe_deinit (s=0x1539ef0) at logpipe.h:254 #14 0x00007fb9c657078c in log_center_deinit (self=0x14f35c0) at center.c:576 #15 0x00007fb9c6571296 in cfg_deinit (cfg=0x14f1ca0) at cfg.c:261 #16 0x00007fb9c6590cac in main_loop_reload_config_apply () at mainloop.c:494 #17 0x00007fb9c6590aab in main_loop_io_worker_job_complete (self=0x156d160) at mainloop.c:417 #18 0x00007fb9c65d3ccc in iv_work_event (_pool=0x14f1bf0) at iv_work.c:221 #19 0x00007fb9c65d2862 in iv_event_run_pending_events (_dummy=0x0) at iv_event.c:67 #20 0x00007fb9c65d2cb5 in iv_event_raw_got_event (_this=0x7fb9c654aa18) at iv_event_raw.c:82 #21 0x00007fb9c65cefec in iv_run_active_list (active=0x7fff33ed17b0) at iv_main.c:219 #22 0x00007fb9c65cf1cc in iv_main () at iv_main.c:269 #23 0x00007fb9c65913e9 in main_loop_run () at mainloop.c:731 #24 0x00000000004016d0 in main () -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

[Bug 144] New: incremental statistics and on reload and stop
by bugzilla＠bugzilla.balabit.com 07 Sep '12

07 Sep '12

https://bugzilla.balabit.com/show_bug.cgi?id=144 Summary: incremental statistics and on reload and stop Product: syslog-ng Version: unspecified Platform: All OS/Version: All Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mdione.ext(a)orange.com Type of the Report: --- Estimated Hours: 0.0 syslog-ng log statistics in /var/log/messages (or some other defined destination), but it does it accumulatively. this makes it harder to make graphs, because the data collector must detect reloads and stop/starts. this most probably should not be done if the statistics are asked via syslog-ng-ctl. also, it does not print statistics on reload or stop, so the last values before the reload or stop are lost. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 159] New: on signal HUP program destinations are not closed
by bugzilla＠bugzilla.balabit.com 26 Jun '12

26 Jun '12

https://bugzilla.balabit.com/show_bug.cgi?id=159 Summary: on signal HUP program destinations are not closed Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 In previous versions of syslog-ng, when a HUP signal was sent to syslog-ng it would close all of its destinations and reopen them. In version 3.3.4 (perhaps earlier ones) the destination to programs is not closed, but a new one is opened. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 12

[Bug 157] New: syslog-ng 3.2.x/3.3.x: test_sql.py and 64-bit systems
by bugzilla＠bugzilla.balabit.com 01 Jun '12

01 Jun '12

https://bugzilla.balabit.com/show_bug.cgi?id=157 Summary: syslog-ng 3.2.x/3.3.x: test_sql.py and 64-bit systems Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: jpo(a)di.uminho.pt Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=45) --> (https://bugzilla.balabit.com/attachment.cgi?id=45) Add '/usr/lib64/dbd/' to the search paths Hi, In 64-bits systems we need to add lib64 directories to the libdbi/libdbdsqlite3 module search path in order to successfully run the test_sql functional test. Location of libdbdsqlite3.so in 64-bit Fedora/RHEL: ---------- $ rpm -ql libdbi-dbd-sqlite /usr/lib64/dbd/libdbdsqlite3.so ---------- Affected syslog-ng versions: At least the 3.2 and 3.3 branches (I haven't check the 3.4 development branch). Fix: The attached patch adds the '/usr/lib64/dbd/' directory to the search path. Adding '/usr/local/lib64/dbd' should also make sense (but it is not done by the attached patch). /jpo -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 143] New: syslog-ng supervisor dies on SIGHUP
by bugzilla＠bugzilla.balabit.com 31 May '12

31 May '12

https://bugzilla.balabit.com/show_bug.cgi?id=143 Summary: syslog-ng supervisor dies on SIGHUP Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 The process which supervises syslog-ng and restarts it in the event of a crash dies when it receives a SIGHUP. This is a problem as when sending a HUP to syslog-ng to have it reload its config, the HUP will get sent to both the supervisor process and the worker process if doing something like `pkill -HUP syslog-ng`. I attached a gdb to the supervisor and its not a segfault or anything, it just quits. I'm guessing the supervisor doesnt have a signal handler for HUP, so it uses the default action which is to quit. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5