- syslog-ng - lists.balabit.hu

suppressing pidfile and control socket for syslog-ng 3.0
by Denis Laplante 08 May '09

08 May '09

To clarify my question about suppressing PID and control socket in syslog-ng version 3.0: I don't want the error message to be suppressed by having the file successfully created in syslog-ng's favourite location, I want to suppress opening the 2 files . I might want multiple syslog-ng processes running simultaneously, for example a test instance to run without interfering with production instances. In the case of PID file, I could perhaps specify "--pidfile /dev/null" ; is there a chance that the target file might get deleted or have permissions changed? A couple of minor complaints for this option: Option --pidfile does not handle relative directories (unlike syslog-ng.conf) . Option -P responds with "Error parsing command line arguments: Unknown option -P". > Message: 3 Date: Mon, 4 May 2009 10:51:20 +0200 (CEST) > From: bugzilla(a)bugzilla.balabit.com > Subject: [syslog-ng] [Bug 43] syslog-ng 2.1.4 consumes all CPU when processing "source file" > Comment #4 from Balazs Scheidler <bazsi(a)balabit.hu> 2009-05-04 10:51:21 > [...] >>> > > can you try 3.0 if it fixes your problem? >> > Yes, 3.0.1 on Solaris-8 fixes this problem. > [...] > > How do I suppress pidfile and/or control socket? >> > > Error opening control socket, bind() failed; error='Permission denied (13)', socket='/local/var/run/syslog-ng.ctl' > This is a different issue, you should probably create /local/var/run/ directory with appropriate permissions.

3 2

[Bug 46] New: Syslog-ng can go into and endless loop, consume all memory, and crash
by bugzilla＠bugzilla.balabit.com 07 May '09

07 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=46 Summary: Syslog-ng can go into and endless loop, consume all memory, and crash Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: support(a)sikkerhed.org Type of the Report: bug Estimated Hours: 0.0 I've discovered a scenario where syslog-ng will eat all memory, and then either crash or get OOM-killed. I'm using the following config: {{{ options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); owner(root); group(adm); perm(0640); dir_owner(root); dir_group(adm); dir_perm(0755); use_dns(yes); use_fqdn(yes); stats_freq(3600); }; source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); }; destination du_all { usertty("*"); }; destination df_logs { file("/var/log/$FACILITY/log-$YEAR-$MONTH-$DAY"); file("/var/log/$FACILITY/current"); }; destination df_everything { file("/var/log/everything/log-$YEAR-$MONTH-$DAY"); file("/var/log/everything/current"); }; filter f_messages { level(info,notice,warn) and not facility(auth,authpriv); }; filter f_emerg { level(emerg); }; log { source(s_all); destination(df_logs); }; log { source(s_all); filter(f_messages); destination(df_everything); }; log { source(s_all); filter(f_emerg); destination(du_all); }; }}} The problem is the "/var/log/$FACILITY/current" output. I had some old output left over from the earlier logger, which meant that /var/log/syslog was a file. When syslog-ng expects this node to be a directory, not a file, it tries to mkdir() it, and fails. This fail generates a syslog message, which needs to get logged to /var/log/syslog/current... which doesn't exist, is mkdir()'ed, fails, is syslogged, etc. In other words, it never starts up, and doesn't log anything useful about what is going on. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Having syslog-ng send to a central server via local6
by Patrick Topping 06 May '09

06 May '09

I am setting up syslog-ng to tail a database query log and have it sent to a central server for analysis / archiving. I have it partially working. I am able to send the UDP packets to the central server but I cannot get them to be sent to central server @ local6. Below is my source, destination, & log lines: *source s_general_db_log { file ("/var/lib/mysql/mysql/general_log.CSV" flags(no-parse) follow_freq(1) program_override("vchat3dev-db002_mysql_db_log:") ); }; destination d_central { udp( "10.0.4.162" port(514) ); }; log { source(s_general_db_log); destination(d_central); }; *I have tried adding in a facility option to the destination but it does not work. Any help with what I am missing would be greatly appreciated. Thanks in advance. -Patrick

4 4

[Bug 43] New: syslog-ng 2.1.4 consumes all CPU when processing "source file "
by bugzilla＠bugzilla.balabit.com 05 May '09

05 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=43 Summary: syslog-ng 2.1.4 consumes all CPU when processing "source file" Product: syslog-ng Version: 2.1.x Platform: Macintosh OS/Version: Mac OS Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: denis.laplante(a)ubc.ca Type of the Report: bug Estimated Hours: 0.0 Summary: syslog-ng version 2.1.4 consumes all CPU when processing "source file" This happens with MacOsX-10.5 (via MacPorts compilation environment), and SunOS-5.8 (manual compilation). Version 2.0.9 on another MacOsX-10.5 compiled earlier behaves normally: lines are copied to destination file, and CPU usage remains low. Test Sequence: rm testlog.txt syslog-ng -f ./syslog-ng.conf -d -F [start without daemonizing] [ 2 lines appended to syslogng.log : ] > Apr 15 09:31:16 s_syslogng@greygum syslog-ng[28695]: Follow-mode file source not found, deferring open; filename='.../testlog.txt' > Apr 15 09:31:16 s_syslogng@greygum syslog-ng[28695]: syslog-ng starting up; version='2.1.4' touch testlog.txt [file gets opened, CPU is quiet] echo "hello" > testlog.txt [CPU immediately jumps to max, but no lines copied to destination] [ While CPU at max, process unkillable except "kill -9" ] [ While CPU at max, use GUI "Activity Monitor" option "Sample", results below: ] > Sampling process 28695 for 3 seconds with 1 millisecond of run time between samples > Sampling completed, processing symbols... > Analysis of sampling syslog-ng (pid 28695) every 1 millisecond > Call graph: > 1373 Thread_2503 > 1373 start > 1373 main > 1373 main_loop_run > 1373 g_main_context_iteration > 1373 g_main_context_iterate > 1373 g_main_context_dispatch > 1373 log_reader_fd_dispatch > 1301 fd_do_read > 1287 read$UNIX2003 > 1287 read$UNIX2003 > 13 fd_do_read > 1 dyld_stub_read$UNIX2003 > 1 dyld_stub_read$UNIX2003 > 58 log_reader_iterate_buf > 48 log_reader_iterate_buf > 8 __memcpy > 8 __memcpy > 1 log_reader_find_eom > 1 log_reader_find_eom > 1 memmove > 1 memmove > 13 log_reader_fd_dispatch > 1 g_sockaddr_unref > 1 g_sockaddr_unref [ number probably counts when program counter was found in routine ] On SunOs, truss showed very fast cycling between alarm() and read(). [ syslog-ng.conf ] source s_testfile{ file(".../testlog.txt" follow_freq(10) flags(no-parse)); }; destination d_test_local{ file(".../testlog-copy.txt" perm(0644) ); }; log{ source(s_testfile); destination(d_test_local); }; source s_syslogng { internal(); }; # Messages internal to syslog-ng destination d_syslogng { file(".../syslogng.log" perm(0644) ); }; log{ source(s_syslogng); destination(d_syslogng); }; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 04 May '09

04 May '09

OK, it's now the top line and it works, the program now recognizes that it is a 3.0 version config file. Also I tried to change the .0 to a .2 and that also caused the same error. So it must be tied to the version of the program and not the maj.min of the configuration file. It would be nice to have that explained a little more clearly in the instructions. Thanks all for helping. Message: 1 Date: Fri, 1 May 2009 16:57:19 -0500 From: "Charles Jennings" <jennings.charles.e.security(a)gmail.com> Subject: Re: [syslog-ng] Error when starting syslog-ng 3.0.1 To: "'Syslog-ng users' and developers' mailing list'" <syslog-ng(a)lists.balabit.hu> Message-ID: <DF2F4E8F406F42EF9DC28946F540AAE5(a)its.local> Content-Type: text/plain; charset="us-ascii" It needs to be the very first line in the file. You have it as the second line. -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Bristol, Gary L. Sent: Friday, May 01, 2009 4:51 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Error when starting syslog-ng 3.0.1 Did as suggested. The errors are now a lot less but now syslog-ng won't start. Here's the error I get [root@soc-nsm-002 init.d]# ./syslog-ng restart Shutting down system logger: [ OK ] Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; syntax error in /etc/syslog-ng/syslog-ng.conf at line 2. # syslog-ng configuration file. @version: 3.0 # # Thi

1 0

syslog-ng 3.0.1 dies on reload with program() destination
by Bartosz Lis 04 May '09

04 May '09

Hello, I had the same problem as Jan Rekorajski with syslog-ng 3.0.1 and 2.0.10: > Hi, > Syslog-ng 3.0.1 dies on reload (kill -HUP pid) when config file contains > log { destination { program() } } directive set. 100% repeatable. > > Here are the relevant config lines: > > destination d_sshguard { program("/usr/sbin/sshguard -a 6"); }; > # A desperate experiment > #destination d_sshguard { program("read"); }; > > log { source(s_sys); filter(f_authpriv); destination(d_sshguard); }; pstree revealed that the command /usr/sbin/sshguard was run by a shell (in my case pdksh). To make the shell go out of my way I've put exec before /usr/sbin/sshguard . See my syslog-ng.conf : destination d_sshguard { program("exec /usr/sbin/sshguard" template("$DATE $FULLHOST $MESSAGE\n")); } Since I've changed this and restarted syslog-ng, it survives killing it with HUP, now. Furthermore, pstree revealed that killing syslog-ng with HUP makes it restart sshguard. Bazsi suggested that some pocess sends either SIGINT or SIGTERM to syslog-ng's process group. In my opinion it is the shell to do that. When syslog-ng reloads its configuration it destroys FIFO to sshguard. This action awakes the shell and triggers it to send some destructive signal to its process group. Regards, -- Bartosz Lis @ Institute of Comp. Science, Technical University of Lodz, Poland bartoszl @ ics.p.lodz.pl

3 3

syslog-ng PID not getting set correctly
by Ngim, Hyrum CTR ADCF/Arrowpoint 04 May '09

04 May '09

We are running the sunfreeware 3.0.1 version of syslog-ng. I'm noticing that the PID is not being set correctly. I saw a bug report but it's not the same as what I'm seeing: https://bugzilla.balabit.com/show_bug.cgi?id=23 Here is what I am experiencing: # ps -ef | grep syslog root 1662 1 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng root 1663 1662 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng # cat /var/run/syslog-ng.pid 1663 Shouldn't the PID be 1662? I've also seen inconsistencies when I use the svcadm commands to disable/enable/refresh the syslog-ng service. Many times it won't stop the service properly and the /var/run/syslog-ng.pid file remains in /var/run. When I try to restart syslog-ng, it won't restart until I run an svcadm disable syslog-ng, remove the /var/run/syslog-ng.pid file, then run svcadm enable syslog-ng. Thanks In Advance for any help.

3 3

[Bug 45] New: syslog-ng fails to build with openssl-1.0.0-beta2
by bugzilla＠bugzilla.balabit.com 04 May '09

04 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=45 Summary: syslog-ng fails to build with openssl-1.0.0-beta2 Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 This bug occurs with 3.0.1 as well as with HEAD of git. Configured with --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-ssl --enable-dynamic-linking --enable-ipv6 --disable-sql --with-pidfile-dir=/var/run. gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT cfg-grammar.o -MD -MP -MF .deps/cfg-grammar.Tpo -c -o cfg-grammar.o cfg-grammar.c In file included from /usr/include/openssl/crypto.h:129, from /usr/include/openssl/comp.h:5, from /usr/include/openssl/ssl.h:149, from tlscontext.h:31, from cfg-grammar.y:15: /usr/include/openssl/safestack.h:105: error: expected identifier or ‘(’ before numeric constant make[2]: *** [cfg-grammar.o] Error 1 make[2]: Leaving directory `/tmp/syslog-ng-3.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/syslog-ng-3.0' make: *** [all] Error 2 I know what you're thinking, and no, there's nothing wrong with the openssl headers nor libraries on my system because OpenSSH 5.2p1 and a patched PHP 5.3.0RC1 successfully builds, links and runs with openssl-1.0.0-beta2 on my system. Perhaps some macros used internally by openssl-1.0.0-beta2 have been redefined by syslog-ng? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng OSE and oracle sql() driver
by Crowder, Steven D. 04 May '09

04 May '09

I don't think the first one made it through. According to the page here: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/ It says that the syslog-ng OSE has "Direct database access", however I have run into a few problems with both the precompiled OSE 3.0HEAD package and compiling from source. The binary package simply logs to /var/log/messages that it doesn't know about an "oracl" DBI driver. I thought that maybe I had to setup the ORACLE_HOME and other env vars and it would fix this, but no dice. Then I tried compiling from source (syslog-ng_3.0.1). It complains that it cannot find dbi.h. I know that the the PE version of syslog-ng will allow direct oracle access based on the documentation, but it appears that this is supposed to work in OSE as well based on the page listed above. If this is just an error, and there's no way that the OSE can have "Direct" database access, I understand, I just want to check first. Thanks, Steven

3 3

Tag support in syslog-ng
by ILLES, Marton 03 May '09

03 May '09

Hi, In the recent discussion on db-parser() I mentioned that I would like to add tag support for messages. The whole feature is not 100%, but here is the first phase. I wrote about it in my blog: http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html and here it is: I had some time so I created a tagging framework in syslog-ng. The patches could be found in my syslog-ng git repo (http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=summary) At the current state it is possible to create tags on the fly, attach them to messages and latter match on them in a filter() expression. A tag is simply identified by it's name, obviously these names must be unique. Internally tags are stored as bitmask vector so it is very fast to set/unset or match on them. Though as of now the internal representation limits the number of tags to 8159. I think it ought to be enough for the near future at least. I dare not to say that it ought to be enough for anyone. :) As of now messages could be tagged on sources only and messages are automatically tagged on sources with a special tag representing the source ".source.". Matching on the source this way is much faster than matching on the ${SOURCE} macro as macro matching involves a string comparison or even a regular expression matching while tag matching is a simple bitwise comparison. (I am planning to add automatic tags on sources for the source driver and other hopefully useful information.) Here is an example on how to use it: source s_tcp1 { tcp(ip(192.168.1.1) port(1514) tags("tcp", "router")); }; source s_tcp2 { tcp(ip(192.168.1.2) port(1514) tags("tcp", "windows)); }; source s_udp1 { udp(ip(192.168.1.1) port(1514) tags("udp", "router")); }; #Match messages from s_tcp1 source filter f_tcp1 { tags(".source.s_tcp1"); }; filter f_router { tags("router"); }; #Match on tags "tcp" or "udp" filter f_net { tags("tcp", "udp"); }; Next step is to add tagging support for the db-parser() so messages can be tagged that way and add support for dynamic SQL table scheme for tags. This w ay one can easily match and parse messages and put the parsed information in to different SQL tables. cheers, Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0