December 2011 - syslog-ng - lists.balabit.hu

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 121] New: After service syslog-ng reload, the first log don' t transfer to remote servicer via TLS
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=121 Summary: After service syslog-ng reload, the first log don't transfer to remote servicer via TLS Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: yong.zheng(a)nsn.com Type of the Report: bug Estimated Hours: 0.0 The version:syslog-ng 3.0.5 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq (3600); }; I will upload the logs to remote syslog-ng server via TLS, I encounter a issue, On client: after reload the configure via below command: #service syslog-ng reload. I write a log: #logger -p local1.info "this is test" but I don't see the log to transfer to server immediately, I wait many minutes, still don't see. I must write the second log to triger that client send the 2 logs to server #logger -p local1.info "this is test 2" the 2 logs have transfered to server immediately successfully, I can see the 2 logs on server. It is 100% reproduced. Is this a known issue, or not ? Thanks. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 42] New: capabilities, chown, chmod
by bugzilla＠bugzilla.balabit.com 20 Dec '12

20 Dec '12

https://bugzilla.balabit.com/show_bug.cgi?id=42 Summary: capabilities, chown, chmod Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: bug Estimated Hours: 0.0 Let's take a look at syslog-ng-3.0.1/src/affile.c lines 60-83 1. CAP_SYS_ADMIN is needed only for /proc/kmsg, it is added w/o check 2. CAP_DAC_READ_SEARCH should be added only if open fail with errno 13 2a. CAP_DAC_OVERRIDE should be added only if open fail with errno 13 and with CAP_DAC_READ_SEARCH set 3. fchown needs CAP_CHOWN unconditionaly 4. fchmod needs CAP_FOWNER if file owner != euid (root here) 5. all caps should be restored summary: - CAP_SYS_ADMIN and CAP_DAC_OVERRIDE are set always even if unnecessary, and permanently - owner, group and perm doesn't work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 10

[Bug 151] New: syslog-ng opens /dev/ log device on every HUP without closing it first
by bugzilla＠bugzilla.balabit.com 08 Oct '12

08 Oct '12

https://bugzilla.balabit.com/show_bug.cgi?id=151 Summary: syslog-ng opens /dev/log device on every HUP without closing it first Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Solaris Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: michael.hocke(a)nyu.edu Type of the Report: bug Estimated Hours: 0.0 On Solaris 10 the /dev/log device can be held open concurrently for up to only 16 instances. syslog-ng opens a new instance of /dev/log after HUPing the process, i.e. after the 16th HUP syslog-ng quits because the new configuration fails and so does the previous configuration. This has been working fine in 3.0. I cannot test if this started to happen in 3.1, 3.2, or if this is new for 3.3. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 144] New: incremental statistics and on reload and stop
by bugzilla＠bugzilla.balabit.com 07 Sep '12

07 Sep '12

https://bugzilla.balabit.com/show_bug.cgi?id=144 Summary: incremental statistics and on reload and stop Product: syslog-ng Version: unspecified Platform: All OS/Version: All Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mdione.ext(a)orange.com Type of the Report: --- Estimated Hours: 0.0 syslog-ng log statistics in /var/log/messages (or some other defined destination), but it does it accumulatively. this makes it harder to make graphs, because the data collector must detect reloads and stop/starts. this most probably should not be done if the statistics are asked via syslog-ng-ctl. also, it does not print statistics on reload or stop, so the last values before the reload or stop are lost. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 143] New: syslog-ng supervisor dies on SIGHUP
by bugzilla＠bugzilla.balabit.com 31 May '12

31 May '12

https://bugzilla.balabit.com/show_bug.cgi?id=143 Summary: syslog-ng supervisor dies on SIGHUP Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 The process which supervises syslog-ng and restarts it in the event of a crash dies when it receives a SIGHUP. This is a problem as when sending a HUP to syslog-ng to have it reload its config, the HUP will get sent to both the supervisor process and the worker process if doing something like `pkill -HUP syslog-ng`. I attached a gdb to the supervisor and its not a segfault or anything, it just quits. I'm guessing the supervisor doesnt have a signal handler for HUP, so it uses the default action which is to quit. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 154] New: Processes forked via program() do not properly terminate
by bugzilla＠bugzilla.balabit.com 23 Mar '12

23 Mar '12

https://bugzilla.balabit.com/show_bug.cgi?id=154 Summary: Processes forked via program() do not properly terminate Product: syslog-ng Version: 3.2.x Platform: Other OS/Version: Solaris Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: hendrik.volker(a)de.verizonbusiness.com Type of the Report: --- Estimated Hours: 0.0 I am using a SyslogNG 3.2.4 build for our company internal version of Solaris 10 on some SUN V245 SPARC servers. In a project I am using the following config: ---8<--- source s_whsvoipgwv2_ngrep_bge1 { program ( "/prod/tools/ngrep -d bge1 -t -q -W single SIP port 5060" flags (no-parse) tags ("IF1") ); }; parser p_whsvoipgwv2_ngrep { csv-parser ( columns ("NGREP.PROTO" "NGREP.DATE", "NGREP.TIME", "NGREP.SRCADDR", "NGREP.DUMMY", "NGREP.DSTADDR", "NGREP.MSG" ) flags (drop-invalid, escape-none, greedy, strip-whitespace) delimiters (" ") ); }; template t_whsvoipgwv2_ngrep { template ( "$MONTH_ABBREV $DAY ${NGREP.TIME} $HOST ${NGREP.PROTO} ${NGREP.SRCADDR} ${NGREP.DSTADDR} ${ NGREP.MSG}\n" ); }; destination d_whsvoipgwv2_sip_voipslp { file ("/var/home/voipslp/sip/sip.$YEAR$MONTH$DAY$HOUR.log" owner("voipslp") group("voipsl") perm(0644) template(t_whsvoipgwv2_ngrep) ); }; log { source(s_whsvoipgwv2_ngrep_bge1); parser(p_whsvoipgwv2_ngrep); destination(d_whsvoipgwv2_sip_voipslp); flags (final); }; ---8<--- With the log file rotation every hour it seems that SyslogNG is starting new instances of ngrep, while not terminating the old instances. This results in lots! of running but unused ngreps. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 11

[Bug 150] New: Possible memory leak for TLS connections
by bugzilla＠bugzilla.balabit.com 23 Mar '12

23 Mar '12

https://bugzilla.balabit.com/show_bug.cgi?id=150 Summary: Possible memory leak for TLS connections Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: eric(a)sevenscale.com Type of the Report: --- Estimated Hours: 0.0 I've noticed that memory usage seems to grow as clients using TLS connect and disconnect. I ran syslog-ng under valgrind while I caused a number of clients to connect and disconnect a few times and saved the leak summary from valgrind: https://gist.github.com/893a9687bf98dc53f0e7 -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 152] New: Vers 3.3.3 pipe source does not parse host from input line
by bugzilla＠bugzilla.balabit.com 14 Feb '12

14 Feb '12

https://bugzilla.balabit.com/show_bug.cgi?id=152 Summary: Vers 3.3.3 pipe source does not parse host from input line Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Try as I might, I can not get a pipe source to maintain the hostname. Syslog-ng includes the hostname as the $MSGHDR (or the $PROGRAM). my config # ----------------------------------- source patterndb { pipe("/var/log/syslog.pipes/patterndb" log_fetch_limit(500) log_iw_size(100000) ); }; source int { internal(); }; template t_standardfile { template("$S_ISODATE $FULLHOST $FACILITY.$LEVEL $MSGHDR][$PROGRAM][$MESSAGE\n"); template_escape(no); }; destination d_var_patterndb { file("/var/log/patterndb.$R_YEAR$R_MONTH$R_DAY.000000" owner("root") group("syslogs") perm(0640) template(t_standardfile)); }; log { source(patterndb); destination(d_var_patterndb); }; # ----------------------------------- data I write tothe pipe # ----------------------------------- <12>2011-11-25T00:00:30-08:00 somehost.uvic.ca mmfs: Fri Nov 25 00:00:29.618 2011: Accepted and connected to 172.20.102.38 hermes0080 <c0n350> <12>2011-11-25T00:00:30-08:00 somehost.uvic.ca mmfs: Fri Nov 25 00:00:29.620 2011: Connecting to 172.20.107.23 nestor0167 <c0n200> <12>2011-11-25T00:00:30-08:00 somehost.uvic.ca mmfs: Fri Nov 25 00:00:29.621 2011: Connected to 172.20.107.23 nestor0167 <c0n200> # ----------------------------------- The output file # ----------------------------------- 2011-11-25T00:00:30-08:00 patterndb(a)catamount.comp.uvic.ca user.warning somehost.uvic.ca ][somehost.uvic.ca][mmfs: Fri Nov 25 00:00:29.618 2011: Accepted and connected to 172.20.102.38 hermes0080 <c0n350> 2011-11-25T00:00:30-08:00 patterndb(a)catamount.comp.uvic.ca user.warning somehost.uvic.ca ][somehost.uvic.ca][mmfs: Fri Nov 25 00:00:29.620 2011: Connecting to 172.20.107.23 nestor0167 <c0n200> 2011-11-25T00:00:30-08:00 patterndb(a)catamount.comp.uvic.ca user.warning somehost.uvic.ca ][somehost.uvic.ca][mmfs: Fri Nov 25 00:00:29.621 2011: Connected to 172.20.107.23 nestor0167 <c0n200> # ----------------------------------- I would expect the output file to read # ----------------------------------- 2011-11-25T00:00:30-08:00 somehost.uvic.ca user.warning mmfs: ][mmfs][Fri Nov 25 00:00:29.618 2011: Accepted and connected to 172.20.102.38 hermes0080 <c0n350> 2011-11-25T00:00:30-08:00 somehost.uvic.ca user.warning mmfs: ][mmfs][Fri Nov 25 00:00:29.620 2011: Connecting to 172.20.107.23 nestor0167 <c0n200> 2011-11-25T00:00:30-08:00 somehost.uvic.ca user.warning mmfs: ][mmfs][Fri Nov 25 00:00:29.621 2011: Connected to 172.20.107.23 nestor0167 <c0n200> # ----------------------------------- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

kernel logging feature requests
by Patrick H. 01 Feb '12

01 Feb '12

While setting up a new server at home I've come across 2 feature requests that would be fairly nice to have (and not that hard to implement I would think). 1) When the 'kernel' flag is set on a file() source (like for reading /proc/kmsg), look for the printk time (eg "[ 1234.567890]") and calculate when the message would have occurred instead of just using when the line was read off the file. Basically check to see the number of seconds the system has been up, subtract the printk time, and then subtract that from current time. 2) I grab all kernel messages with priority of crit or higher and send it to the usertty() destination, but this destination doesnt support templates. It'd be nice to be able to define the template. I mostly just want to change the time format and remove the hostname (since these will only come from localhost on my setup).

4 8