July 2010 - syslog-ng - lists.balabit.hu

cannot find -lglib-2.0 while cross compiling
by Hasan Ümit Ezerçe 10 Jul '10

10 Jul '10

Hi, I need to run syslog-ng on an itanium RHEL 4.7 server but cannot find any RPM's for ia64 arch. So decided to cross compile it. I am trying to build it on a centos 4.7 VM and get the following error: ia64-unknown-linux-gnu-gcc -Os -Wall -o syslog-ng main.o libsyslog-ng.a -lnsl -lrt -Wl,-Bstatic -lglib-2.0 -levtlog -lnet -Wl,-Bdynamic -ldl /opt/crosstool/gcc-3.4.5-glibc-2.3.6/ia64-unknown-linux-gnu/lib/gcc/ia64-unknown-linux-gnu/3.4.5/../../../../ia64-unknown-linux-gnu/bin/ld: cannot find -lglib-2.0 collect2: ld returned 1 exit status However it seems that I already have glib-2.0 libs installed: $ pkg-config glib-2.0 --cflags --libs -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -lglib-2.0 Can you tell what am I missing here? Best regards.

3 2

syslog-ng 3.0.8 has been released
by devel＠balabit.hu 09 Jul '10

09 Jul '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.8 SUMMARY : new stable release DATE : Jul 9, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.8) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.8 Fri, 09 Jul 2010 09:27:37 +0200 Bugfixes: * The flush_timeout() fix in 3.0.7 was not working properly for file() destinations. This has been fixed. * Fixed the use of the greedy and drop-invalid flags for csv-parser. * Fixed a possible but rare memory leak in the handling of message suppression. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Syslog-NG to parse HTTP query strings?
by Steve Barnes 09 Jul '10

09 Jul '10

Hello Is Syslog-NG capable of parsing HTTP query string key=value pairs? I've read through both the CSV and pattern DB parser documentation and it seems they both presume a consistent ordering in the $MSG field. Neither seem appropriate when if the order of query string pairs changes, such as: http://somehost.com/somefile.cgi?keyA=valueA&keyB=valueB or http://somehost.com/somefile.cgi?keyB=valueB&keyA=valueA Basically, I want to grab keyA's value irrespective of which order it appears and use it as a value in an SQL insert statement. Is this possible with Syslog-NG's native parsers or am I going to have to handle this in a Perl script/program? Cheers Steve

2 3

[Bug 90] New: csv-parser's flags(drop-invalid) doesn' t work with escape-{backslash, double-char}
by bugzilla＠bugzilla.balabit.com 09 Jul '10

09 Jul '10

https://bugzilla.balabit.com/show_bug.cgi?id=90 Summary: csv-parser's flags(drop-invalid) doesn't work with escape-{backslash,double-char} Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: andrewn(a)locus.net Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=19) --> (https://bugzilla.balabit.com/attachment.cgi?id=19) Possible fix The example parser p_apache in syslog-ng-v3.0-guide-admin-en.pdf on numbered page 63 does not work when drop-invalid is added. The 2 main loops of log_csv_parser_process seem to treat cur_column and src differently upon termination. The patch attached attempts to fix that. It also checks to see that the whole string was parsed. I don't know if this is the correct fix, but so far it seems to Work For Me (tm). -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 86] New: flush_timeout has no effect
by bugzilla＠bugzilla.balabit.com 09 Jul '10

09 Jul '10

https://bugzilla.balabit.com/show_bug.cgi?id=86 Summary: flush_timeout has no effect Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 After upgrading to version 3.0.7 OSE the flush_timeout option has no effect for file destinations. The messages are buffered until flush_lines is reached and then they are flushed. TCP destinations are no affected by this issue. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

Files still opened after time_reap
by Erwan Miran 09 Jul '10

09 Jul '10

Hi, I'm using syslog-ng 2.0.0 (yes I know...) on a debian loghost. Files generated via df_monitor (see configuration extract below) are deleted by the monitoring process about 4 min after having been written by syslog-ng. My problem is that these files remain opened (lsof | grep deleted) by syslog-ng after deleted whereas time_reap option is set to 30s. I googled the issue and just found https://lists.balabit.hu/pipermail/syslog-ng/2003-January/004416.html which is not my exact case (macros are actually used in filenames). I didn't find any bugfix/feature on time_reap in the changelog of subsequent versions ... Thanks in advance for your help Regards Erwan options { chain_hostnames(0); time_reopen(10); time_reap(30); log_fifo_size(8192); create_dirs(yes); group(adm); perm(0660); dir_group(adm); dir_perm(0755); use_dns(no); log_msg_size(4096); use_fqdn(yes); keep_hostname(yes); use_time_recvd(no); }; source s_machines { tcp(ip("890.123.456.789") port(514) max_connections(100)); }; template tpl_monitor { template("$S_YEAR-$S_MONTH-$S_DAY $S_HOUR:$S_MIN:$S_SEC $FULLHOST ==$LEVEL== [$PROGRAM] $MESSAGE\n"); template_escape(no); }; destination df_monitor { file("/var/log/monitor/$R_YEAR-$R_MONTH-$R_DAY-$R_HOUR-$R_MIN" template(tpl_monitor) group(monitor) dir_perm(0775) dir_group(monitor) owner(loghost-maintainer)); }; destination dh_backup-loghost { tcp("123.456.789.890" port(514)); }; log{ source(s_machines); destination(df_monitor); destination(dh_backup-loghost); };

2 3

file() source driver on regular file
by Risto Vaarandi 07 Jul '10

07 Jul '10

hi all, is file() driver intended for reading kernel messages only on Linux, or does it work for regular files also? I've tried to get the following simple configuration working, but without success: source textfile { file("/home/risto/test.log" flags(no-parse) follow-freq(1)); }; destination testlog { file("/var/log/test.log"); }; log { source(textfile); destination(testlog); }; The source definition is identical to the example from the syslog-ng administrator guide (I've played with the options, but the result is the same). When syslog-ng starts up, it produces the following error messages: Follow-mode file source not found, deferring open; filename='/home/risto/test.log' The current log file has a mismatching size/inode information, restarting from the beginning; filename='/home/risto/test.log' If /home/risto/test.log does not exist, only the first message is produced. Is reading from regular files only supported by PE? (I am using the 3.0.7 open source version of syslog-ng.) kind regards, risto

2 1

The latest version of eventlog?
by Sergei Zhirikov 05 Jul '10

05 Jul '10

Hi, I'm wondering what is the latest version of the eventlog library? If I look here: http://www.balabit.com/downloads/files/eventlog/0.2/ then the latest seems to be 0.2.9. Although on my other computer I have 0.2.10 installed (built from source), but I can't remember where I got it from. Is there something missing on the download page? -- Thanks and regards, Sergei.

2 2

POLLERR occurred while idle
by Tom Ritter 05 Jul '10

05 Jul '10

I'm getting a large volume of messages that look like this: 23:09:53 hn syslog-ng[30]: POLLERR occurred while idle; fd='13' 23:09:54 hn syslog-ng[30]: POLLERR occurred while idle; fd='14' 23:09:55 hn syslog-ng[30]: POLLERR occurred while idle; fd='13' 23:09:56 hn syslog-ng[30]: POLLERR occurred while idle; fd='14' 23:09:57 hn syslog-ng[30]: POLLERR occurred while idle; fd='13' 23:09:58 hn syslog-ng[30]: POLLERR occurred while idle; fd='14' 23:09:59 hn syslog-ng[30]: POLLERR occurred while idle; fd='13' I have no idea what they are but they won't stop =( I looked at the file descriptors: # ls /proc/30/fd total 0 lr-x------ 1 root root 64 Jul 4 23:21 0 -> /dev/null l-wx------ 1 root root 64 Jul 4 23:21 1 -> /dev/null l-wx------ 1 root root 64 Jul 4 23:21 10 -> /var/log/messages/messages l-wx------ 1 root root 64 Jul 4 23:21 11 -> /var/log/hn/20100704.log l-wx------ 1 root root 64 Jul 4 23:21 13 -> pipe:[28238824] l-wx------ 1 root root 64 Jul 4 23:21 2 -> /dev/null lrwx------ 1 root root 64 Jul 4 23:21 3 -> socket:[28238645] l-wx------ 1 root root 64 Jul 4 23:21 4 -> pipe:[28238641] lrwx------ 1 root root 64 Jul 4 23:21 5 -> socket:[28238650] lr-x------ 1 root root 64 Jul 4 23:21 6 -> /proc/kmsg l-wx------ 1 root root 64 Jul 4 23:21 7 -> /var/log/syslog/syslog.log l-wx------ 1 root root 64 Jul 4 23:21 8 -> /var/log/fw/important.log l-wx------ 1 root root 64 Jul 4 23:21 9 -> /var/log/debug/debug.log And I ran strace looking for fd's 13 and 14. Here are some grep results: strace.log.30964:open("/dev/null", O_WRONLY) = 14 ---- strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 29014) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 27100) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 25102) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 23291) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 21398) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 19680) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = ? ERESTART_RESTARTBLOCK (To be restarted) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 18183) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 16697) = 1 ([{fd=13, revents=POLLERR}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 0) = 0 (Timeout) strace.log.30962:poll([{fd=13, events=POLLOUT}], 1, 0) = 1 ([{fd=13, revents=POLLOUT}]) strace.log.30962:poll([{fd=13, events=0}, {fd=3, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}], 4, 15221) = 1 ([{fd=13, revents=POLLERR}]) ----- strace.log.30962:fcntl(13, F_GETFL) = 0x1 (flags O_WRONLY) strace.log.30962:fcntl(13, F_SETFL, O_WRONLY|O_NONBLOCK) = 0 strace.log.30962:write(13, "POLLERR occurred while idle; fd="..., 168) = 168 strace.log.30962:close(13) = 0 I mussed some pid's around to try and format the email as best I could. Can anyone suggest more debugging steps, or what might be causing this? I haven't seen this error elsewhere, the search results are barren, except the for commit logs. I'm on gentoo using 3.1.1: # syslog-ng --version syslog-ng 3.1.1 Installer-Version: 3.1.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8747d74491eac3fdf5052194f47a68e659633ca9 Compile-Date: Jun 17 2010 13:37:53 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on -tom

3 2

Syslog-ng writing to files once per minute
by Eric Cables 01 Jul '10

01 Jul '10

I'm hoping this is a really stupid question, but after implementing what I thought to be the fix to this, I can't seem to get it working. I have a number of hosts logging to facility local5, and the destination log file is only updated once per minute (HH:MM.30s to be precise). I've implemented flush_lines(0); and flush_interval(0);, to the global options, but it doesn't seem to have an effect. syslog-ng 3.0.3 My goal is to have logs written to the destination file with minimal latency after syslog-ng receives the message. -- Eric Cables

2 6