January 2009 - syslog-ng - lists.balabit.hu

[Bug 14] New: Maintain symlinks to current and previous log output files...
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=14 Summary: Maintain symlinks to current and previous log output files... Product: syslog-ng Version: 2.0.x Platform: PC URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218688 OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: sasa(a)debian.org Type of the Report: enhancement Estimated Hours: 0.0 >From bugs.debian.org: I'm hoping that upstream might consider adding functionality to syslog-ng to allow it to track current and previous log files and maintain symlinks to them, as a user option. This replicates one of the options that I (and others) find useful in cronolog. This has been previously discussed on the syslog-ng mailing list, though apparently nothing fully came of it. The archive of that discussion is at: https://lists.balabit.hu/pipermail/syslog-ng/2002-June/003452.html Obviously, it's probably somewhat of a low priority, but it would be nice. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng client messages delayed
by mark peters 22 Mar '10

22 Mar '10

Hi, We are running into a issue where syslog-ng clients who are configured to forward to a central syslog-ng destination server that are reasonably chatty are getting there messages delayed at the source (ie. 'tcpdump'ing the client shows it is sending messages from N minutes ago, where N is anything from 5 to 60 minutes on average sometimes more). We have also seen a small percentage of loss at the destination. On these 'delayed' clients we also see the resident memory footprint on the growing. Clients are running 1.6.12 (have tried 2.0.8 also - same symptons). The central syslog-ng server is running 2.0.8. relevant config entries are: syslog-ng client: options { sync (1); time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; destination d_central { tcp("log2.xyz.com" port(5140)); }; filter f_local5 { facility(local5); }; log { source(s_sys); filter(f_local5); destination(d_central); flags(final); }; syslog-ng central server: options { time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (yes); dns_cache (yes); dns_cache_expire (28800); use_fqdn (no); create_dirs (yes); dir_perm (0755); perm (0644); keep_hostname (no); use_time_recvd (no); stats_freq(600); flush_timeout(16000); flush_lines(256); time_sleep(80); }; destination d_server { file("/logs/server.$LEVEL.log"); }; filter f_server { facility(local5) and program("server"); }; log { source(s_sys); filter(f_server); destination(d_server); flags(final); }; to give a rough idea of message volume the server is receiving approximately 50 million log lines per day on average any help/information would be appreciated thanks -------------------------------------- Easy + Joy + Powerful = Yahoo! Bookmarks x Toolbar http://pr.mail.yahoo.co.jp/toolbar/

3 4

[Bug 21] New: call shutdown() before closing a socket
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=21 Summary: call shutdown() before closing a socket Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bazsi(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 syslog-ng should call shutdown() before closing a socket to ensure the contents of the socket buffer are emptied. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 23] New: MySQL default configuration causes error on 'pid' column
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=23 Summary: MySQL default configuration causes error on 'pid' column Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mark(a)la-croix.net Type of the Report: bug Estimated Hours: 0.0 When configuring a MySQL destination, the default 'pid' column type is INT. However, if a message comes through with no PID, the SQL command tries to insert an empty quoted string, which causes an error in MySQL 5.0. Example error: Error running SQL query; type='mysql', host='!MY_DB_HOST!', port='', user='syslog-ng', database='syslog-ng', error='1366: Incorrect integer value: \'\' for column \'pid\' at row 1', query='INSERT INTO !MY_LOG_CLIENT_HOST!_2008_09_23 (date, facility, level, host, program, pid, message) VALUES (\'2008-09-23 12:24:35\', \'1\', \'5\', \'!MY_LOG_CLIENT!\', \'root\', \'\', \'test\')' The message is coming from a TCP source from which a large percentage of messages have no PID associated. The workaround is to explicitly set the PID column to VARCHAR so it can handle empty strings, but it would be nice to see the empty PID case gracefully handled. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 28] New: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7. 1-RELEASE
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=28 Summary: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7.1-RELEASE Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Other Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: syslog-ng(a)ee.lbl.gov Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=6) --> (https://bugzilla.balabit.com/attachment.cgi?id=6) syslogd lines from /etc.conf I'm using the FreeBSD syslog-ng2 package (2.0.9_2) under 7.1-RELEASE (see rc.conf.txt attachment). I find that syslog-ng is cpu bound following a reboot. This problem occurs on both i386 and amd64 installs. Either stopping and starting ("/etc/rc.d/syslogd restart") or reloading ("kill -HUP `cat /var/run/syslog.pid`") results in normal operation. I used ktrace and it poll() is called followed by four calls to gettimeofday() (see ktrace.txt attachment). gcore and gdb show the loop to be inside glib (see gdb.txt attachment). lsof shows the file descriptor that /dev/console should be on has been revoked (see lsof.txt attachment). I believe something happens early during the bootup process that causes /dev/console to be reconfigured. I think that devd is running when syslog-ng starts up. Forcing the syslogd started to happen after syscons doesn't help. Note: In order to get syslog-ng to work with /etc/rc.d/syslogd the startup script must be tweaked (see syslogd.patch.txt attachment). My syslog-ng.conf is attached (syslog-ng.conf.txt) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

[Bug 24] New: .pid file not set correctly (STOP not working)
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=24 Summary: .pid file not set correctly (STOP not working) Product: syslog-ng Version: unspecified Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: man(a)inspektsecurity.com Type of the Report: --- Estimated Hours: 0.0 Hi. in installed syslog-ng 3.0.1 PE (syslog-ng-premium-edition-3.0.1-linux-i386.run). The installation is on a CentOS 5 system (RedHat spin-off). Although the program itself works fine, the RC scripts STOP command does not work. I traced the problem down to the .pid file not being set correctly. According to the RC script the .pid file shoud be in: PIDFILE=$SYSLOGNG_PREFIX/var/run/syslog-ng.pid resulting in the below when running STOP/RESTART ------------ [root@test-server:~] /etc/init.d/syslog-ng restart -n Restarting syslog-ng: -n Stopping syslog-ng: -n Starting syslog-ng: Error binding socket; addr='AF_INET(0.0.0.0:514)', error='Address already in use (98)' Error initializing source driver; source='tcp_input_syslog', id='tcp_input_syslog#0' Error initializing message pipeline; failed ------------- I found out that the pid file set is actually: /var/run/syslogd.pid So changing the RC script to: PIDFILE=/var/run/syslogd.pid seems to work. br /Marc -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

Timezone problem?!
by Charles Chan 16 Feb '09

16 Feb '09

Recently I use the syslog-ng for log collection environment encountered a time zone problem,hope somebody can help me.Thanks The environment is like this: The apache server(PDT timezone,UTC-8) forword apache log through syslog to the syslog-ng server（CST timezone，UTC＋8）. The server with same time zones no problem between the forward. Syslog-ng test configuration is as follows: destination int_apache (file ( "/ disk3/cookie_logs/pid1 / $ YEAR-$ MONTH-$ DAY / $ (HOST). apache.log" time_zone (-08:00) template ( "$ DAY-$ HOUR $ S_DAY-$ S_HOUR $ R_DAY-$ R_HOUR $ $ ISODATE $ MSGONLY \ n ") template_escape (no));); The original apache log is: 58.240.69.161 - - - [14/Jan/2009: 22:27:47 -0800] "GET / www.example.com/HTTP/1.1" 200 2028 "-" "Mozilla/4.0 ( compatible; MSIE 7.0; Windows NT 5.1) " The syslog-ng server generated log is: 14-22 14-22 14-22 2009-01-14T22 :27:47-08: 00 58.240.69.161 - - - [14/Jan/2009: 22:27:47 -0800] "GET / www.example. com / HTTP/1.1 "200 2028" - "" Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) " Originally generated documents should be kept in /disk3/cookie_logs/pid1/2009-01-14 However, the actual file is stored in the /disk3/cookie_logs/pid1/2009-01-15 It can be seen, in fact, syslog-ng configuration item file() use the $DAY and template() use the $DAY not with a identical Macros. Is this the syslog-ng a bug? or I configuration errors. Who can help me answer you?Thanks!

2 8

Strange template definition problem
by Peter Eckel 14 Feb '09

14 Feb '09

Hi, I am currently using the Open Source edition of syslog-ng version 3.0.1 on RedHat Linux 5.3 and 3.9. So far, there are not too many problems, but one strangeness occurs when defining templates. When I define the template inside the 'destination' configuration, everything works as expected: # + # Definition 1: Works fine. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; destination d_test { file( "/opt/sec/var/log/test" template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ) ); }; log { source(s_local); destination(d_test); }; The log entries look like this: >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 >---------- So far, so good. But when I try global templates, I get a rather strange result: # + # Definition 2: Fails miserably. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; template t_test { template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ); }; destination d_test { file( "/opt/sec/var/log/test" template( t_test ) ); }; log { source(s_local); destination(d_test); }; What I'm getting is this (including marked blank lines): > >---------- >-- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- > >---------- >-- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- > >---------- >-- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 It looks a bit as if all elements of the filled template are output twice: First in reverse order, then again from start in correct order. This is reproducable on several machines. The only difference is the use of the global template in the second case instead of the inline template in the first. Am I making some stupid mistake, or is this a bug in syslog-ng? Best regards and thanks in advance for any suggestions, Peter. -- ------------------------------------------------------------------------- Peter Eckel System- und Netzwerkmanagement Postfach 1463 D-61284 Bad Homburg v.d.H. Tel.: +49 6172 969495 Mobil: +49 172 6812107 Fax: +49 6172 969845 EMail: Peter.Eckel(a)Eckel-EDV.de -------------------------------------------------------------------------

2 4

[patch] Fix build with --disable-ssl
by Joe Shaw 09 Feb '09

09 Feb '09

Hi, Here's a patch to fix building without SSL. Without it, tlstransport.h is included by tlstransport.c and things break because SSLisms are undefined. Thanks, Joe

3 8

1st post on some basic 101 setup
by lance raymond 05 Feb '09

05 Feb '09

Basic scenario is we have 5 web servers in a cluster which I wish to centrally log. I was pointed to syslog-ng and the linuxquestions guys 50+ readings and not 1 response so found this list and will give it a try. Basically all the webservers are running FC5, and rpm -qa shows the following version; syslog-ng-1.6.12-1.fc5 I see some newer versions, not sure if I should/have to, but really just want to log the different vhosts apache logs to a central server. I read a few diff things a few places, and so far have the following conf files. *Server.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; *Client.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { udp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; I restart the services and look for that web.log file and never see anything, so any help is appreciated.

2 2