- syslog-ng - lists.balabit.hu

[Bug 46] New: Syslog-ng can go into and endless loop, consume all memory, and crash
by bugzilla＠bugzilla.balabit.com 07 May '09

07 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=46 Summary: Syslog-ng can go into and endless loop, consume all memory, and crash Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: support(a)sikkerhed.org Type of the Report: bug Estimated Hours: 0.0 I've discovered a scenario where syslog-ng will eat all memory, and then either crash or get OOM-killed. I'm using the following config: {{{ options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); owner(root); group(adm); perm(0640); dir_owner(root); dir_group(adm); dir_perm(0755); use_dns(yes); use_fqdn(yes); stats_freq(3600); }; source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); }; destination du_all { usertty("*"); }; destination df_logs { file("/var/log/$FACILITY/log-$YEAR-$MONTH-$DAY"); file("/var/log/$FACILITY/current"); }; destination df_everything { file("/var/log/everything/log-$YEAR-$MONTH-$DAY"); file("/var/log/everything/current"); }; filter f_messages { level(info,notice,warn) and not facility(auth,authpriv); }; filter f_emerg { level(emerg); }; log { source(s_all); destination(df_logs); }; log { source(s_all); filter(f_messages); destination(df_everything); }; log { source(s_all); filter(f_emerg); destination(du_all); }; }}} The problem is the "/var/log/$FACILITY/current" output. I had some old output left over from the earlier logger, which meant that /var/log/syslog was a file. When syslog-ng expects this node to be a directory, not a file, it tries to mkdir() it, and fails. This fail generates a syslog message, which needs to get logged to /var/log/syslog/current... which doesn't exist, is mkdir()'ed, fails, is syslogged, etc. In other words, it never starts up, and doesn't log anything useful about what is going on. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Having syslog-ng send to a central server via local6
by Patrick Topping 06 May '09

06 May '09

I am setting up syslog-ng to tail a database query log and have it sent to a central server for analysis / archiving. I have it partially working. I am able to send the UDP packets to the central server but I cannot get them to be sent to central server @ local6. Below is my source, destination, & log lines: *source s_general_db_log { file ("/var/lib/mysql/mysql/general_log.CSV" flags(no-parse) follow_freq(1) program_override("vchat3dev-db002_mysql_db_log:") ); }; destination d_central { udp( "10.0.4.162" port(514) ); }; log { source(s_general_db_log); destination(d_central); }; *I have tried adding in a facility option to the destination but it does not work. Any help with what I am missing would be greatly appreciated. Thanks in advance. -Patrick

4 4

[Bug 43] New: syslog-ng 2.1.4 consumes all CPU when processing "source file "
by bugzilla＠bugzilla.balabit.com 05 May '09

05 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=43 Summary: syslog-ng 2.1.4 consumes all CPU when processing "source file" Product: syslog-ng Version: 2.1.x Platform: Macintosh OS/Version: Mac OS Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: denis.laplante(a)ubc.ca Type of the Report: bug Estimated Hours: 0.0 Summary: syslog-ng version 2.1.4 consumes all CPU when processing "source file" This happens with MacOsX-10.5 (via MacPorts compilation environment), and SunOS-5.8 (manual compilation). Version 2.0.9 on another MacOsX-10.5 compiled earlier behaves normally: lines are copied to destination file, and CPU usage remains low. Test Sequence: rm testlog.txt syslog-ng -f ./syslog-ng.conf -d -F [start without daemonizing] [ 2 lines appended to syslogng.log : ] > Apr 15 09:31:16 s_syslogng@greygum syslog-ng[28695]: Follow-mode file source not found, deferring open; filename='.../testlog.txt' > Apr 15 09:31:16 s_syslogng@greygum syslog-ng[28695]: syslog-ng starting up; version='2.1.4' touch testlog.txt [file gets opened, CPU is quiet] echo "hello" > testlog.txt [CPU immediately jumps to max, but no lines copied to destination] [ While CPU at max, process unkillable except "kill -9" ] [ While CPU at max, use GUI "Activity Monitor" option "Sample", results below: ] > Sampling process 28695 for 3 seconds with 1 millisecond of run time between samples > Sampling completed, processing symbols... > Analysis of sampling syslog-ng (pid 28695) every 1 millisecond > Call graph: > 1373 Thread_2503 > 1373 start > 1373 main > 1373 main_loop_run > 1373 g_main_context_iteration > 1373 g_main_context_iterate > 1373 g_main_context_dispatch > 1373 log_reader_fd_dispatch > 1301 fd_do_read > 1287 read$UNIX2003 > 1287 read$UNIX2003 > 13 fd_do_read > 1 dyld_stub_read$UNIX2003 > 1 dyld_stub_read$UNIX2003 > 58 log_reader_iterate_buf > 48 log_reader_iterate_buf > 8 __memcpy > 8 __memcpy > 1 log_reader_find_eom > 1 log_reader_find_eom > 1 memmove > 1 memmove > 13 log_reader_fd_dispatch > 1 g_sockaddr_unref > 1 g_sockaddr_unref [ number probably counts when program counter was found in routine ] On SunOs, truss showed very fast cycling between alarm() and read(). [ syslog-ng.conf ] source s_testfile{ file(".../testlog.txt" follow_freq(10) flags(no-parse)); }; destination d_test_local{ file(".../testlog-copy.txt" perm(0644) ); }; log{ source(s_testfile); destination(d_test_local); }; source s_syslogng { internal(); }; # Messages internal to syslog-ng destination d_syslogng { file(".../syslogng.log" perm(0644) ); }; log{ source(s_syslogng); destination(d_syslogng); }; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 04 May '09

04 May '09

OK, it's now the top line and it works, the program now recognizes that it is a 3.0 version config file. Also I tried to change the .0 to a .2 and that also caused the same error. So it must be tied to the version of the program and not the maj.min of the configuration file. It would be nice to have that explained a little more clearly in the instructions. Thanks all for helping. Message: 1 Date: Fri, 1 May 2009 16:57:19 -0500 From: "Charles Jennings" <jennings.charles.e.security(a)gmail.com> Subject: Re: [syslog-ng] Error when starting syslog-ng 3.0.1 To: "'Syslog-ng users' and developers' mailing list'" <syslog-ng(a)lists.balabit.hu> Message-ID: <DF2F4E8F406F42EF9DC28946F540AAE5(a)its.local> Content-Type: text/plain; charset="us-ascii" It needs to be the very first line in the file. You have it as the second line. -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Bristol, Gary L. Sent: Friday, May 01, 2009 4:51 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Error when starting syslog-ng 3.0.1 Did as suggested. The errors are now a lot less but now syslog-ng won't start. Here's the error I get [root@soc-nsm-002 init.d]# ./syslog-ng restart Shutting down system logger: [ OK ] Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; syntax error in /etc/syslog-ng/syslog-ng.conf at line 2. # syslog-ng configuration file. @version: 3.0 # # Thi

1 0

syslog-ng 3.0.1 dies on reload with program() destination
by Bartosz Lis 04 May '09

04 May '09

Hello, I had the same problem as Jan Rekorajski with syslog-ng 3.0.1 and 2.0.10: > Hi, > Syslog-ng 3.0.1 dies on reload (kill -HUP pid) when config file contains > log { destination { program() } } directive set. 100% repeatable. > > Here are the relevant config lines: > > destination d_sshguard { program("/usr/sbin/sshguard -a 6"); }; > # A desperate experiment > #destination d_sshguard { program("read"); }; > > log { source(s_sys); filter(f_authpriv); destination(d_sshguard); }; pstree revealed that the command /usr/sbin/sshguard was run by a shell (in my case pdksh). To make the shell go out of my way I've put exec before /usr/sbin/sshguard . See my syslog-ng.conf : destination d_sshguard { program("exec /usr/sbin/sshguard" template("$DATE $FULLHOST $MESSAGE\n")); } Since I've changed this and restarted syslog-ng, it survives killing it with HUP, now. Furthermore, pstree revealed that killing syslog-ng with HUP makes it restart sshguard. Bazsi suggested that some pocess sends either SIGINT or SIGTERM to syslog-ng's process group. In my opinion it is the shell to do that. When syslog-ng reloads its configuration it destroys FIFO to sshguard. This action awakes the shell and triggers it to send some destructive signal to its process group. Regards, -- Bartosz Lis @ Institute of Comp. Science, Technical University of Lodz, Poland bartoszl @ ics.p.lodz.pl

3 3

syslog-ng PID not getting set correctly
by Ngim, Hyrum CTR ADCF/Arrowpoint 04 May '09

04 May '09

We are running the sunfreeware 3.0.1 version of syslog-ng. I'm noticing that the PID is not being set correctly. I saw a bug report but it's not the same as what I'm seeing: https://bugzilla.balabit.com/show_bug.cgi?id=23 Here is what I am experiencing: # ps -ef | grep syslog root 1662 1 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng root 1663 1662 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng # cat /var/run/syslog-ng.pid 1663 Shouldn't the PID be 1662? I've also seen inconsistencies when I use the svcadm commands to disable/enable/refresh the syslog-ng service. Many times it won't stop the service properly and the /var/run/syslog-ng.pid file remains in /var/run. When I try to restart syslog-ng, it won't restart until I run an svcadm disable syslog-ng, remove the /var/run/syslog-ng.pid file, then run svcadm enable syslog-ng. Thanks In Advance for any help.

3 3

[Bug 45] New: syslog-ng fails to build with openssl-1.0.0-beta2
by bugzilla＠bugzilla.balabit.com 04 May '09

04 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=45 Summary: syslog-ng fails to build with openssl-1.0.0-beta2 Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 This bug occurs with 3.0.1 as well as with HEAD of git. Configured with --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-ssl --enable-dynamic-linking --enable-ipv6 --disable-sql --with-pidfile-dir=/var/run. gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT cfg-grammar.o -MD -MP -MF .deps/cfg-grammar.Tpo -c -o cfg-grammar.o cfg-grammar.c In file included from /usr/include/openssl/crypto.h:129, from /usr/include/openssl/comp.h:5, from /usr/include/openssl/ssl.h:149, from tlscontext.h:31, from cfg-grammar.y:15: /usr/include/openssl/safestack.h:105: error: expected identifier or ‘(’ before numeric constant make[2]: *** [cfg-grammar.o] Error 1 make[2]: Leaving directory `/tmp/syslog-ng-3.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/syslog-ng-3.0' make: *** [all] Error 2 I know what you're thinking, and no, there's nothing wrong with the openssl headers nor libraries on my system because OpenSSH 5.2p1 and a patched PHP 5.3.0RC1 successfully builds, links and runs with openssl-1.0.0-beta2 on my system. Perhaps some macros used internally by openssl-1.0.0-beta2 have been redefined by syslog-ng? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng OSE and oracle sql() driver
by Crowder, Steven D. 04 May '09

04 May '09

I don't think the first one made it through. According to the page here: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/ It says that the syslog-ng OSE has "Direct database access", however I have run into a few problems with both the precompiled OSE 3.0HEAD package and compiling from source. The binary package simply logs to /var/log/messages that it doesn't know about an "oracl" DBI driver. I thought that maybe I had to setup the ORACLE_HOME and other env vars and it would fix this, but no dice. Then I tried compiling from source (syslog-ng_3.0.1). It complains that it cannot find dbi.h. I know that the the PE version of syslog-ng will allow direct oracle access based on the documentation, but it appears that this is supposed to work in OSE as well based on the page listed above. If this is just an error, and there's no way that the OSE can have "Direct" database access, I understand, I just want to check first. Thanks, Steven

3 3

Tag support in syslog-ng
by ILLES, Marton 03 May '09

03 May '09

Hi, In the recent discussion on db-parser() I mentioned that I would like to add tag support for messages. The whole feature is not 100%, but here is the first phase. I wrote about it in my blog: http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html and here it is: I had some time so I created a tagging framework in syslog-ng. The patches could be found in my syslog-ng git repo (http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=summary) At the current state it is possible to create tags on the fly, attach them to messages and latter match on them in a filter() expression. A tag is simply identified by it's name, obviously these names must be unique. Internally tags are stored as bitmask vector so it is very fast to set/unset or match on them. Though as of now the internal representation limits the number of tags to 8159. I think it ought to be enough for the near future at least. I dare not to say that it ought to be enough for anyone. :) As of now messages could be tagged on sources only and messages are automatically tagged on sources with a special tag representing the source ".source.". Matching on the source this way is much faster than matching on the ${SOURCE} macro as macro matching involves a string comparison or even a regular expression matching while tag matching is a simple bitwise comparison. (I am planning to add automatic tags on sources for the source driver and other hopefully useful information.) Here is an example on how to use it: source s_tcp1 { tcp(ip(192.168.1.1) port(1514) tags("tcp", "router")); }; source s_tcp2 { tcp(ip(192.168.1.2) port(1514) tags("tcp", "windows)); }; source s_udp1 { udp(ip(192.168.1.1) port(1514) tags("udp", "router")); }; #Match messages from s_tcp1 source filter f_tcp1 { tags(".source.s_tcp1"); }; filter f_router { tags("router"); }; #Match on tags "tcp" or "udp" filter f_net { tags("tcp", "udp"); }; Next step is to add tagging support for the db-parser() so messages can be tagged that way and add support for dynamic SQL table scheme for tags. This w ay one can easily match and parse messages and put the parsed information in to different SQL tables. cheers, Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

Did as suggested. The errors are now a lot less but now syslog-ng won't start. Here's the error I get [root@soc-nsm-002 init.d]# ./syslog-ng restart Shutting down system logger: [ OK ] Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; syntax error in /etc/syslog-ng/syslog-ng.conf at line 2. # syslog-ng configuration file. @version: 3.0 # # Thi Message: 2 Date: Fri, 1 May 2009 13:15:10 -0700 From: "Ngim, Hyrum CTR ADCF/Arrowpoint" <hyrum.ngim.ctr(a)awps.army.mil> Subject: Re: [syslog-ng] Error when starting syslog-ng 3.0.1 To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <8B986F9B35D04C428A6B98996432C79B05ADE7311E(a)fedmail01.fed.awps.army.mil> Content-Type: text/plain; charset="iso-8859-1" At the very top of your config file, it will need to have: @version: 3.0 You'll just need to remove the # sign.

4 3

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

I'm getting the following error when I start the syslog-ng process. I'm running version 3.0.1 I have tried several different formats for getting it to recognize that fact that I'm running or attempting to run a version 3.0 config. The other problem I'm having is that I'm not getting any local logs. The directory creation and the forwarding of the syslog to the remote system works just fine. Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; Error resolving group; group='log' WARNING: template: the default value for template-escape is changing to 'no' in version 3.0, please update your configuration file accordingly; WARNING: input: sources do not remove new-line characters from messages by default in version 3.0, please add 'no-multi-line' flag to your configuration if you want to retain this functionality; WARNING: file source: default value of follow_freq in file sources is changing in 3.0 to '1' for all files except /proc/kmsg; WARNING: filters do not store matches in macros by default in 3.0, please update your configuration by using an explicit 'store-matches' flag to achieve that; WARNING: template: the meaning of the $MSG/$MESSAGE macros is changing in version 3.0, please prepend a $MSGHDR when upgrading to 3.0 config format; Error resolving user; user='root' Here is the configuration file at /etc/syslog-ng/syslog0ng.conf # syslog-ng configuration file. #@version: 3.0 # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # 20000925 gb(a)sysfive.com # # configuration file for syslog-ng, customized for remote logging # #options { flush_lines (0); # time_reopen (10); # log_fifo_size (1000); # long_hostnames (off); # use_dns (no); # use_fqdn (no); # keep_hostname (yes); # }; options { stats_freq (0); chain_hostnames (no); flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames(off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); perm(0640); group("log"); }; source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # #source s_local { # unix-dgram("/dev/log"); # file("/proc/kmsg" log_prefix("kernel:")); #}; # source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; # network logs come from the local network source src { unix-stream("/dev/log"); internal(); }; source remote { udp(ip ("0.0.0.0")); }; source remote_tcp { tcp(ip ("xxx.xx.xx.xxx")); }; source kernsrc { file("/proc/kmsg"); }; ## Local filters filter f_filter1 { facility(kern); }; filter f_filter2 { level(info) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; # Filters for remote logs added for RSA filter f_paygate { host ("xxx.xxx.xxx.xxx");}; filter f_payweb { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1 { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2 { host ("xxx.xxx.xxx.xxx");}; filter f_roomview { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1a { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2a { host ("xxx.xxx.xxx.xxx");}; #filter f_dc { host ("xxx.xxx.xxx.xxx");}; # Local logs - order DOES matter ! #log { source(s_sys); filter(f_filter1); destination(d_cons); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_auth); }; log { source(s_sys); filter(f_filter4); destination(d_mail); }; log { source(s_sys); filter(f_filter5); destination(d_mlal); }; log { source(s_sys); filter(f_filter6); destination(d_spol); }; log { source(s_sys); filter(f_filter7); destination(d_boot); }; log { source(s_sys); filter(f_filter8); destination(d_cron); }; # Remote logs for RSA Appliance log { source(remote); filter(f_paygate); destination(d_rsa); }; log { source(remote); filter(f_payweb); destination(d_rsa); }; log { source(remote); filter(f_paymarket1); destination(d_rsa); }; log { source(remote); filter(f_paymarket2); destination(d_rsa); }; log { source(remote); filter(f_roomview); destination(d_rsa); }; log { source(remote); filter(f_paymarket1a); destination(d_rsa); }; log { source(remote); filter(f_paymarket2a); destination(d_rsa); }; #log { source(remote); filter(f_dc); destination(d_rsa); }; # log by host log { source(remote); destination(hosts); }; log { source(remote_tcp); destination(hosts); }; # Local destinations destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog"); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; # added for re-direct to RSA Appliance destination d_rsa { udp("xxx.xxx.xxx.xxx" port (514) template("<$PRI> $DATE $HOST $MESSAGE"));}; # automatic host sorting # by current year, month, day, host # files created are also by year, month, day, host, facility # set it up destination hosts { file("/var/syslog/HOSTS/$YEAR/$MONTH/$DAY/$HOST/$YEAR-$MONTH-$DAY-$HOST-$FAC ILITY.log" owner(root) group(root) perm(0655) dir_perm(0 755) create_dirs(yes)); }; Gary L. Bristol CISSP, RHCE University of Oklahoma 200 Felgar St., Suite 226 Norman, OK 73019 405-325-2236 ******************************************** ---------------------------------------------------------------------------- + Ranked 10th in PC Magazine's 2007 Top 20 Wired Campuses + Computerworld 2006 100 Best Places to Work in IT ---------------------------------------------------------------------------- ********************************************************************** This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you **********************************************************************

3 2

[Bug 44] New: name of the file and content of the file contains different timestamp
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=44 Summary: name of the file and content of the file contains different timestamp Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Friendly reminder that this bug needs to be fixed. I can help you with this if you can let me know which source file to look at. >From https://lists.balabit.hu/pipermail/syslog-ng/2009-February/012567.html 2009/2/15 Balazs Scheidler <bazsi at balabit.hu> > On Thu, 2009-02-12 at 09:52 +0800, Charles Chan wrote: > > I use time_zone instead timezone,configuration like this > > =========================================================== > > template apache_template { template("$DAY-$HOUR $S_DAY-$S_HOUR > > $R_DAY-$R_HOUR $MSGONLY\n"); template_escape(no); }; > > destination us_int_apache > > { file("/disk2/apache_logs/$YEAR-$MONTH-$DAY/$HOST.apache.log" > > timezone("PST8PDT") template(apache_template)); }; > > ============================================================ > > service can start,but the problem still exists. > > the file is stored in the /disk2/apache_logs/2009-02-12 > > generated log is: > > 11-17 11-17 11-17 222.69.201.23 - - - [11/Feb/2009:17:37:03 -0800] > > "GET /atmgateway.client.html ................... > > two date string still difference. I use syslog-ng 3.0.1 OSE version,is > > this OSE version problem? > > The PE and OSE versions are the same in this regard. Apart from the > concrete differences listed on our website, I work hard to keep the two > versions in sync. > > So let me try to understand your problem a bit deeper... Sorry for not > having done so previously. Sometimes I just reply without really > thinking through problems in the question. > > hmm.. as I see your problem seems to be that the name of the file and > the _content_ of the file contains a different timestamp. Right? > > This is intentional. > > When expanding the macros in the name of the files, syslog-ng _always_ > uses the server-side, local timezone. The timezone used in the content > of the file can be controlled using the time_zone() option. > > Currently there's no way to control the filename zone offset. The > reasoning behind that was that I already found the timezone related > configuration options too complex, and I didn't see the need so far. > I felt that agregating logs based on the _received timestamp_ is more > trustworthy (because clients are free to specify a timestamp they > feel-like), and that is in local timezone anyway. Also, noone else has > complained previously :) > > That said, I'm willing to add another option to control the timezone > used when expanding the filename, if you really think that solves your > problem. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 2

[Bug 21] New: call shutdown() before closing a socket
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=21 Summary: call shutdown() before closing a socket Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bazsi(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 syslog-ng should call shutdown() before closing a socket to ensure the contents of the socket buffer are emptied. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 23] New: MySQL default configuration causes error on 'pid' column
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=23 Summary: MySQL default configuration causes error on 'pid' column Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mark(a)la-croix.net Type of the Report: bug Estimated Hours: 0.0 When configuring a MySQL destination, the default 'pid' column type is INT. However, if a message comes through with no PID, the SQL command tries to insert an empty quoted string, which causes an error in MySQL 5.0. Example error: Error running SQL query; type='mysql', host='!MY_DB_HOST!', port='', user='syslog-ng', database='syslog-ng', error='1366: Incorrect integer value: \'\' for column \'pid\' at row 1', query='INSERT INTO !MY_LOG_CLIENT_HOST!_2008_09_23 (date, facility, level, host, program, pid, message) VALUES (\'2008-09-23 12:24:35\', \'1\', \'5\', \'!MY_LOG_CLIENT!\', \'root\', \'\', \'test\')' The message is coming from a TCP source from which a large percentage of messages have no PID associated. The workaround is to explicitly set the PID column to VARCHAR so it can handle empty strings, but it would be nice to see the empty PID case gracefully handled. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 34] New: After sighup master doe not write to logs, but clients think the server is writing to the logs
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=34 Summary: After sighup master doe not write to logs, but clients think the server is writing to the logs Product: syslog-ng Version: 2.1.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: michael(a)crowdscience.com QAContact: michael(a)crowdscience.com Type of the Report: --- Estimated Hours: 0.0 We encountered a problem when a sighup is sent to syslo-ng as part of our process to rotate the log file on the syslog-ng master. After the signal is sent and the log file has been rotated syslog-ng creates a new log file, but does not write to it. During this time the syslog-ng clients think that the master is up and continue to send their messages to the server (ie not queuing the messages locally). This resulted in a period of time where messages where not logged and required a stop and start of the syslog-ng service to get the messages logging again. The server is running Solaris Nevada snv_67 X86. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 28] New: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7. 1-RELEASE
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=28 Summary: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7.1-RELEASE Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Other Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: syslog-ng(a)ee.lbl.gov Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=6) --> (https://bugzilla.balabit.com/attachment.cgi?id=6) syslogd lines from /etc.conf I'm using the FreeBSD syslog-ng2 package (2.0.9_2) under 7.1-RELEASE (see rc.conf.txt attachment). I find that syslog-ng is cpu bound following a reboot. This problem occurs on both i386 and amd64 installs. Either stopping and starting ("/etc/rc.d/syslogd restart") or reloading ("kill -HUP `cat /var/run/syslog.pid`") results in normal operation. I used ktrace and it poll() is called followed by four calls to gettimeofday() (see ktrace.txt attachment). gcore and gdb show the loop to be inside glib (see gdb.txt attachment). lsof shows the file descriptor that /dev/console should be on has been revoked (see lsof.txt attachment). I believe something happens early during the bootup process that causes /dev/console to be reconfigured. I think that devd is running when syslog-ng starts up. Forcing the syslogd started to happen after syscons doesn't help. Note: In order to get syslog-ng to work with /etc/rc.d/syslogd the startup script must be tweaked (see syslogd.patch.txt attachment). My syslog-ng.conf is attached (syslog-ng.conf.txt) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

[Bug 24] New: .pid file not set correctly (STOP not working)
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=24 Summary: .pid file not set correctly (STOP not working) Product: syslog-ng Version: unspecified Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: man(a)inspektsecurity.com Type of the Report: --- Estimated Hours: 0.0 Hi. in installed syslog-ng 3.0.1 PE (syslog-ng-premium-edition-3.0.1-linux-i386.run). The installation is on a CentOS 5 system (RedHat spin-off). Although the program itself works fine, the RC scripts STOP command does not work. I traced the problem down to the .pid file not being set correctly. According to the RC script the .pid file shoud be in: PIDFILE=$SYSLOGNG_PREFIX/var/run/syslog-ng.pid resulting in the below when running STOP/RESTART ------------ [root@test-server:~] /etc/init.d/syslog-ng restart -n Restarting syslog-ng: -n Stopping syslog-ng: -n Starting syslog-ng: Error binding socket; addr='AF_INET(0.0.0.0:514)', error='Address already in use (98)' Error initializing source driver; source='tcp_input_syslog', id='tcp_input_syslog#0' Error initializing message pipeline; failed ------------- I found out that the pid file set is actually: /var/run/syslogd.pid So changing the RC script to: PIDFILE=/var/run/syslogd.pid seems to work. br /Marc -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

syslog-ng 3.0.1, open source edition, fails write to MySQL DB
by Arnold Wang 29 Apr '09

29 Apr '09

I have installed syslog-ng 3.0.1, open source edition, binary package in RHEL 5.3 system. I'm having trouble to have it writing to MySQL DB using native driver. The current configuration file looks like: # Write the logs to MySQL DB destination dd_mysql { sql(type(mysql) host("localhost") username("syslogadmin") password("syslogadmin") database("syslog") table("logs") columns("host", "facility", "priority", "level", "tag", "datetime", "program", "msg", "seq") values("$HOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$TAG", "$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC", "$PROGRAM", "$MSG", "$SEQ") indexes("seq")); # program("/usr/bin/mysql -usyslogadmin -psyslogadmin syslog" # template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg, seq) # VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG', '$SEQ' );\n") # template-escape(yes)); }; It works if I use the program driver, as currently commented out. The table is configured as following, using the schema defined by php-syslog-ng 2.9.8m. +----------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | host | varchar(128) | YES | MUL | NULL | | | facility | varchar(10) | YES | MUL | NULL | | | priority | varchar(10) | YES | MUL | NULL | | | level | varchar(10) | YES | | NULL | | | tag | varchar(10) | YES | | NULL | | | datetime | datetime | YES | MUL | NULL | | | program | varchar(15) | YES | MUL | NULL | | | msg | text | YES | | NULL | | | seq | bigint(20) unsigned | NO | MUL | 0 | | | counter | int(11) | NO | | 1 | | | fo | datetime | YES | | NULL | | | lo | datetime | YES | | NULL | | +----------+---------------------+------+-----+---------+---------------- What is the problem? Thanks in advance for the help.

3 4

Removing Prefixes from Syslog-ng Messages
by adam.j.brendamour＠accenture.com 29 Apr '09

29 Apr '09

Hello all, I am having issues with syslog-ng adding prefixes to my messages. I currently have a syslog-ng agent sending messages from a log source to a syslog-ng central server. I am then sending the messages from the central server to a 3rd party event management program. Is there a way to forward an unchanged syslog-ng message from the relay server? For example: Message arriving from source: Apr 16 09:31:07 psrdev47 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" Message being forwarded from server: Apr 16 09:31:07 10.56.2.77 125 <0>1 2009-04-16T09:31:02-07:00 psrdev47 - - - - 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" How can I configure the syslog-ng server to relay this unchanged message to our destination? Apr 16 09:31:07 psrdev47 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" Thanks for your help! Regards, Adam This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

3 4

setting facility/level of source log messages ?
by bana＠docisland.org 29 Apr '09

29 Apr '09

Hi, I often meet the case where I need to centralize some purely applicative log messages (ie they don't even have a PRI header). So syslog-ng sets a default facility and severity for these messages, like kern facility and emergency severity for a source file with a "no-parse" option. Is there a clean way to change these default values by modifying the configuration file ? (and not using a source program() nor a tail | logger trick). I've already written a quick and dirty patch for my needs, but it would be great to have a native way of doing the same thing (I may have missed something in the documentation). Best Regards. -- Xavier

2 9

Parser-DB Memory Leak
by Martin Holste 27 Apr '09

27 Apr '09

I'm running tests with the db-parser() parser module and finding that the memory grows exponentially until the program finally dies with: ***MEMORY-ERROR***: syslog-ng[2769]: GSlice: failed to allocate 248 bytes (alignment: 256): Cannot allocate memory Here is my configuration: @version: 3.0 options { }; source s_tcp { tcp(ip(0.0.0.0) port(445)); }; parser p_db { db-parser(); }; filter f_security_class { match("security" value(".classifier.class")); }; filter f_class_unknown { match("unknown" value(".classifier.class")); }; filter f_not_class_unknown { not match("unknown" value(".classifier.class")); }; filter f_FIREWALL_ACCESS_DENY { match("2" value(".classifier.rule_id")); }; template t_FIREWALL_ACCESS_DENY { template("$R_UNIXTIME\t$SOURCEIP\t$FACILITY\t$PROGRAM\t${.classifier.class}\t${.classifier.rule_id}\t${FIREWALL.proto}\t${FIREWALL.o_int}\t${FIREWALL.srcip}\t${FIREWALL.srcport}\t${FIREWALL.i_int}\t${FIREWALL.dstip}\t${FIREWALL.dstport}\t${FIREWALL.access_group}\t${FWSM.PRIORITY}\t${FWSM.MSG_CODE}\t${MSGONLY}\n"); }; destination d_FIREWALL_ACCESS_DENY { fifo("/tmp/db_parsed_fifo", template(t_FIREWALL_ACCESS_DENY)); }; filter f_FIREWALL_CONNECTION_END { match("3" value(".classifier.rule_id")); }; template t_FIREWALL_CONNECTION_END { template("$R_UNIXTIME\t$SOURCEIP\t$FACILITY\t$PROGRAM\t${.classifier.class}\t${.classifier.rule_id}\t${FIREWALL.proto}\t${FIREWALL.o_int}\t${FIREWALL.srcip}\t${FIREWALL.srcport}\t${FIREWALL.i_int}\t${FIREWALL.dstip}\t${FIREWALL.dstport}\t${FIREWALL.conn_duration}\t${FIREWALL.conn_bytes}\t$MSGONLY\n"); }; destination d_FIREWALL_CONNECTION_END { fifo("/tmp/db_parsed_fifo", template(t_FIREWALL_CONNECTION_END)); }; template t_DEFAULT { template("$R_UNIXTIME\t$SOURCEIP\t\$FACILITY\t$PROGRAM\t1\t1\t$MSGONLY\n"); }; destination d_default_fifo { fifo("/tmp/db_parsed_fifo", template(t_DEFAULT)); }; log { source(s_tcp); log { parser(p_db); filter(f_not_class_unknown); log { filter(f_FIREWALL_ACCESS_DENY); destination(d_FIREWALL_ACCESS_DENY); }; log { filter(f_FIREWALL_CONNECTION_END); destination(d_FIREWALL_CONNECTION_END); }; flags(final); }; log { destination(d_default_fifo); }; }; var/patterndb.xml: <patterndb version='1' pub_date='2009-04-16'> <program name='FWSM'> <pattern>%FWSM</pattern> <rule id='2' class='2'> <pattern>Deny@QSTRING:FIREWALL.proto: @src@QSTRING:FIREWALL.o_int: :@@IPv4:FIREWALL.srcip:@/@NUMBER:FIREWALL.srcport:@ dst@QSTRING:FIREWALL.i_int: :@@IPv4:FIREWALL.dstip:@/@NUMBER:FIREWALL.dstport:@ by access-group @QSTRING:FIREWALL.access_group:"@</pattern> </rule> <rule id='3' class='3'> <pattern>Teardown@QSTRING:FIREWALL.proto: @connection @NUMBER::@ for@QSTRING:FIREWALL.o_int: :@@IPv4:FIREWALL.srcip:@ /@NUMBER:FIREWALL.srcport@ to@QSTRING:FIREWALL.i_int: :@@IPv4:FIREWALL.dstip:@/@NUMBER:FIREWALL.dstport@ duration@QSTRING:FIREWALL.conn_duration: @bytes @NUMBER:FIREWALL.conn_bytes:@</pattern> </rule> </program> </patterndb> syslog-ng -V syslog-ng 3.0.1+binpkg4 Revision: ssh+git://bazsi@git.balabit //var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#555574a984eaef9410a2869db0af1be0d52b269b Compile-Date: Apr 16 2009 14:42:21 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: off Enable-Pcre: on Should I recompile with memtrace on? Thanks, Martin

4 10

syslog-ng 3.0.1 dies on reload with program() destination
by Jan Rekorajski 24 Apr '09

24 Apr '09

Sorry for possible double post, I sent the first from wrong email. Hi, Syslog-ng 3.0.1 dies on reload (kill -HUP pid) when config file contains log { destination { program() } } directive set. 100% repeatable. Here are the relevant config lines: destination d_sshguard { program("/usr/sbin/sshguard -a 6"); }; # A desperate experiment #destination d_sshguard { program("read"); }; log { source(s_sys); filter(f_authpriv); destination(d_sshguard); }; >From what I managed to find with strace/ltrace, syslog-ng kills the program, does not clear some data structures and then falls face down with sigsegv/term/ill trying to restart/reattach(?) to the program. The exact same configuration works perfectly with version 2.0.10. -- Jan Rekorajski | ALL SUSPECTS ARE GUILTY. PERIOD! baggins<at>mimuw.edu.pl | OTHERWISE THEY WOULDN'T BE SUSPECTS, WOULD THEY? BOFH, MANIAC | -- TROOPS by Kevin Rubio

5 8

sunfreeware syslog-ng v 3.0.1
by Ngim, Hyrum CTR ADCF/Arrowpoint 23 Apr '09

23 Apr '09

I get this error when I introduce the owner when defining a destination: destination firewall { file("/data/firewall/fwlog" owner(sysadmin) group(sysadmin) perm(0644)); }; When I run a syntax check I get this: # syslog-ng -s Error resolving user; user='sysadmin' # pkginfo -l SMCsyslng PKGINST: SMCsyslng NAME: syslogng CATEGORY: application ARCH: sparc VERSION: 3.0.1 BASEDIR: /usr/local VENDOR: BalaBit IT Ltd PSTAMP: Steve Christensen INSTDATE: Apr 15 2009 11:56 EMAIL: steve(a)smc.vnet.net STATUS: completely installed FILES: 91 installed pathnames 4 shared pathnames 25 directories 7 executables 1610 blocks used (approx) I also checked the misc.c file in the sunfreeware source and it didn't seem like it had the same errors that were identified in posts like this one: https://lists.balabit.hu/pipermail/syslog-ng/2009-April/012761.html

2 2

sunfreeware syslog-ng v 3.0.1
by Ngim, Hyrum CTR ADCF/Arrowpoint 23 Apr '09

23 Apr '09

I have installed and am configuring the sunfreeware package for syslog-ng. I ran into this issue that I saw on some other emails: Error resolving user; user='sysadmin' Is there a way around this to get it working? TIA

2 1

IPv6 addresses logged as 127.0.0.1
by Michael Granzow 22 Apr '09

22 Apr '09

When using the template parameter $SOURCEIP, connections originating from IPv6 addresses are logged as "127.0.0.1". The following patch fixes that issue. Listening on :: on dual-stack hosts will lead to IPv4 connections being logged in IPv6-mapped format, e.g. ::ffff:10.11.12.13. This is not what people usually expect, but it's not wrong. If desired, the place to fix that would be g_sockaddr_inet6_format().

2 2