- syslog-ng - lists.balabit.hu

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

Did as suggested. The errors are now a lot less but now syslog-ng won't start. Here's the error I get [root@soc-nsm-002 init.d]# ./syslog-ng restart Shutting down system logger: [ OK ] Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; syntax error in /etc/syslog-ng/syslog-ng.conf at line 2. # syslog-ng configuration file. @version: 3.0 # # Thi Message: 2 Date: Fri, 1 May 2009 13:15:10 -0700 From: "Ngim, Hyrum CTR ADCF/Arrowpoint" <hyrum.ngim.ctr(a)awps.army.mil> Subject: Re: [syslog-ng] Error when starting syslog-ng 3.0.1 To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <8B986F9B35D04C428A6B98996432C79B05ADE7311E(a)fedmail01.fed.awps.army.mil> Content-Type: text/plain; charset="iso-8859-1" At the very top of your config file, it will need to have: @version: 3.0 You'll just need to remove the # sign.

4 3

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

I'm getting the following error when I start the syslog-ng process. I'm running version 3.0.1 I have tried several different formats for getting it to recognize that fact that I'm running or attempting to run a version 3.0 config. The other problem I'm having is that I'm not getting any local logs. The directory creation and the forwarding of the syslog to the remote system works just fine. Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; Error resolving group; group='log' WARNING: template: the default value for template-escape is changing to 'no' in version 3.0, please update your configuration file accordingly; WARNING: input: sources do not remove new-line characters from messages by default in version 3.0, please add 'no-multi-line' flag to your configuration if you want to retain this functionality; WARNING: file source: default value of follow_freq in file sources is changing in 3.0 to '1' for all files except /proc/kmsg; WARNING: filters do not store matches in macros by default in 3.0, please update your configuration by using an explicit 'store-matches' flag to achieve that; WARNING: template: the meaning of the $MSG/$MESSAGE macros is changing in version 3.0, please prepend a $MSGHDR when upgrading to 3.0 config format; Error resolving user; user='root' Here is the configuration file at /etc/syslog-ng/syslog0ng.conf # syslog-ng configuration file. #@version: 3.0 # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # 20000925 gb(a)sysfive.com # # configuration file for syslog-ng, customized for remote logging # #options { flush_lines (0); # time_reopen (10); # log_fifo_size (1000); # long_hostnames (off); # use_dns (no); # use_fqdn (no); # keep_hostname (yes); # }; options { stats_freq (0); chain_hostnames (no); flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames(off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); perm(0640); group("log"); }; source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # #source s_local { # unix-dgram("/dev/log"); # file("/proc/kmsg" log_prefix("kernel:")); #}; # source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; # network logs come from the local network source src { unix-stream("/dev/log"); internal(); }; source remote { udp(ip ("0.0.0.0")); }; source remote_tcp { tcp(ip ("xxx.xx.xx.xxx")); }; source kernsrc { file("/proc/kmsg"); }; ## Local filters filter f_filter1 { facility(kern); }; filter f_filter2 { level(info) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; # Filters for remote logs added for RSA filter f_paygate { host ("xxx.xxx.xxx.xxx");}; filter f_payweb { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1 { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2 { host ("xxx.xxx.xxx.xxx");}; filter f_roomview { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1a { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2a { host ("xxx.xxx.xxx.xxx");}; #filter f_dc { host ("xxx.xxx.xxx.xxx");}; # Local logs - order DOES matter ! #log { source(s_sys); filter(f_filter1); destination(d_cons); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_auth); }; log { source(s_sys); filter(f_filter4); destination(d_mail); }; log { source(s_sys); filter(f_filter5); destination(d_mlal); }; log { source(s_sys); filter(f_filter6); destination(d_spol); }; log { source(s_sys); filter(f_filter7); destination(d_boot); }; log { source(s_sys); filter(f_filter8); destination(d_cron); }; # Remote logs for RSA Appliance log { source(remote); filter(f_paygate); destination(d_rsa); }; log { source(remote); filter(f_payweb); destination(d_rsa); }; log { source(remote); filter(f_paymarket1); destination(d_rsa); }; log { source(remote); filter(f_paymarket2); destination(d_rsa); }; log { source(remote); filter(f_roomview); destination(d_rsa); }; log { source(remote); filter(f_paymarket1a); destination(d_rsa); }; log { source(remote); filter(f_paymarket2a); destination(d_rsa); }; #log { source(remote); filter(f_dc); destination(d_rsa); }; # log by host log { source(remote); destination(hosts); }; log { source(remote_tcp); destination(hosts); }; # Local destinations destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog"); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; # added for re-direct to RSA Appliance destination d_rsa { udp("xxx.xxx.xxx.xxx" port (514) template("<$PRI> $DATE $HOST $MESSAGE"));}; # automatic host sorting # by current year, month, day, host # files created are also by year, month, day, host, facility # set it up destination hosts { file("/var/syslog/HOSTS/$YEAR/$MONTH/$DAY/$HOST/$YEAR-$MONTH-$DAY-$HOST-$FAC ILITY.log" owner(root) group(root) perm(0655) dir_perm(0 755) create_dirs(yes)); }; Gary L. Bristol CISSP, RHCE University of Oklahoma 200 Felgar St., Suite 226 Norman, OK 73019 405-325-2236 ******************************************** ---------------------------------------------------------------------------- + Ranked 10th in PC Magazine's 2007 Top 20 Wired Campuses + Computerworld 2006 100 Best Places to Work in IT ---------------------------------------------------------------------------- ********************************************************************** This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you **********************************************************************

3 2

[Bug 44] New: name of the file and content of the file contains different timestamp
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=44 Summary: name of the file and content of the file contains different timestamp Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Friendly reminder that this bug needs to be fixed. I can help you with this if you can let me know which source file to look at. >From https://lists.balabit.hu/pipermail/syslog-ng/2009-February/012567.html 2009/2/15 Balazs Scheidler <bazsi at balabit.hu> > On Thu, 2009-02-12 at 09:52 +0800, Charles Chan wrote: > > I use time_zone instead timezone,configuration like this > > =========================================================== > > template apache_template { template("$DAY-$HOUR $S_DAY-$S_HOUR > > $R_DAY-$R_HOUR $MSGONLY\n"); template_escape(no); }; > > destination us_int_apache > > { file("/disk2/apache_logs/$YEAR-$MONTH-$DAY/$HOST.apache.log" > > timezone("PST8PDT") template(apache_template)); }; > > ============================================================ > > service can start,but the problem still exists. > > the file is stored in the /disk2/apache_logs/2009-02-12 > > generated log is: > > 11-17 11-17 11-17 222.69.201.23 - - - [11/Feb/2009:17:37:03 -0800] > > "GET /atmgateway.client.html ................... > > two date string still difference. I use syslog-ng 3.0.1 OSE version,is > > this OSE version problem? > > The PE and OSE versions are the same in this regard. Apart from the > concrete differences listed on our website, I work hard to keep the two > versions in sync. > > So let me try to understand your problem a bit deeper... Sorry for not > having done so previously. Sometimes I just reply without really > thinking through problems in the question. > > hmm.. as I see your problem seems to be that the name of the file and > the _content_ of the file contains a different timestamp. Right? > > This is intentional. > > When expanding the macros in the name of the files, syslog-ng _always_ > uses the server-side, local timezone. The timezone used in the content > of the file can be controlled using the time_zone() option. > > Currently there's no way to control the filename zone offset. The > reasoning behind that was that I already found the timezone related > configuration options too complex, and I didn't see the need so far. > I felt that agregating logs based on the _received timestamp_ is more > trustworthy (because clients are free to specify a timestamp they > feel-like), and that is in local timezone anyway. Also, noone else has > complained previously :) > > That said, I'm willing to add another option to control the timezone > used when expanding the filename, if you really think that solves your > problem. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 2

[Bug 21] New: call shutdown() before closing a socket
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=21 Summary: call shutdown() before closing a socket Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bazsi(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 syslog-ng should call shutdown() before closing a socket to ensure the contents of the socket buffer are emptied. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 23] New: MySQL default configuration causes error on 'pid' column
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=23 Summary: MySQL default configuration causes error on 'pid' column Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mark(a)la-croix.net Type of the Report: bug Estimated Hours: 0.0 When configuring a MySQL destination, the default 'pid' column type is INT. However, if a message comes through with no PID, the SQL command tries to insert an empty quoted string, which causes an error in MySQL 5.0. Example error: Error running SQL query; type='mysql', host='!MY_DB_HOST!', port='', user='syslog-ng', database='syslog-ng', error='1366: Incorrect integer value: \'\' for column \'pid\' at row 1', query='INSERT INTO !MY_LOG_CLIENT_HOST!_2008_09_23 (date, facility, level, host, program, pid, message) VALUES (\'2008-09-23 12:24:35\', \'1\', \'5\', \'!MY_LOG_CLIENT!\', \'root\', \'\', \'test\')' The message is coming from a TCP source from which a large percentage of messages have no PID associated. The workaround is to explicitly set the PID column to VARCHAR so it can handle empty strings, but it would be nice to see the empty PID case gracefully handled. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 34] New: After sighup master doe not write to logs, but clients think the server is writing to the logs
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=34 Summary: After sighup master doe not write to logs, but clients think the server is writing to the logs Product: syslog-ng Version: 2.1.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: michael(a)crowdscience.com QAContact: michael(a)crowdscience.com Type of the Report: --- Estimated Hours: 0.0 We encountered a problem when a sighup is sent to syslo-ng as part of our process to rotate the log file on the syslog-ng master. After the signal is sent and the log file has been rotated syslog-ng creates a new log file, but does not write to it. During this time the syslog-ng clients think that the master is up and continue to send their messages to the server (ie not queuing the messages locally). This resulted in a period of time where messages where not logged and required a stop and start of the syslog-ng service to get the messages logging again. The server is running Solaris Nevada snv_67 X86. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 28] New: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7. 1-RELEASE
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=28 Summary: syslog-ng 2.0.9 is cpu bound after bootup on FreeBSD 7.1-RELEASE Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Other Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: syslog-ng(a)ee.lbl.gov Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=6) --> (https://bugzilla.balabit.com/attachment.cgi?id=6) syslogd lines from /etc.conf I'm using the FreeBSD syslog-ng2 package (2.0.9_2) under 7.1-RELEASE (see rc.conf.txt attachment). I find that syslog-ng is cpu bound following a reboot. This problem occurs on both i386 and amd64 installs. Either stopping and starting ("/etc/rc.d/syslogd restart") or reloading ("kill -HUP `cat /var/run/syslog.pid`") results in normal operation. I used ktrace and it poll() is called followed by four calls to gettimeofday() (see ktrace.txt attachment). gcore and gdb show the loop to be inside glib (see gdb.txt attachment). lsof shows the file descriptor that /dev/console should be on has been revoked (see lsof.txt attachment). I believe something happens early during the bootup process that causes /dev/console to be reconfigured. I think that devd is running when syslog-ng starts up. Forcing the syslogd started to happen after syscons doesn't help. Note: In order to get syslog-ng to work with /etc/rc.d/syslogd the startup script must be tweaked (see syslogd.patch.txt attachment). My syslog-ng.conf is attached (syslog-ng.conf.txt) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 7

[Bug 24] New: .pid file not set correctly (STOP not working)
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=24 Summary: .pid file not set correctly (STOP not working) Product: syslog-ng Version: unspecified Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: man(a)inspektsecurity.com Type of the Report: --- Estimated Hours: 0.0 Hi. in installed syslog-ng 3.0.1 PE (syslog-ng-premium-edition-3.0.1-linux-i386.run). The installation is on a CentOS 5 system (RedHat spin-off). Although the program itself works fine, the RC scripts STOP command does not work. I traced the problem down to the .pid file not being set correctly. According to the RC script the .pid file shoud be in: PIDFILE=$SYSLOGNG_PREFIX/var/run/syslog-ng.pid resulting in the below when running STOP/RESTART ------------ [root@test-server:~] /etc/init.d/syslog-ng restart -n Restarting syslog-ng: -n Stopping syslog-ng: -n Starting syslog-ng: Error binding socket; addr='AF_INET(0.0.0.0:514)', error='Address already in use (98)' Error initializing source driver; source='tcp_input_syslog', id='tcp_input_syslog#0' Error initializing message pipeline; failed ------------- I found out that the pid file set is actually: /var/run/syslogd.pid So changing the RC script to: PIDFILE=/var/run/syslogd.pid seems to work. br /Marc -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

syslog-ng 3.0.1, open source edition, fails write to MySQL DB
by Arnold Wang 29 Apr '09

29 Apr '09

I have installed syslog-ng 3.0.1, open source edition, binary package in RHEL 5.3 system. I'm having trouble to have it writing to MySQL DB using native driver. The current configuration file looks like: # Write the logs to MySQL DB destination dd_mysql { sql(type(mysql) host("localhost") username("syslogadmin") password("syslogadmin") database("syslog") table("logs") columns("host", "facility", "priority", "level", "tag", "datetime", "program", "msg", "seq") values("$HOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$TAG", "$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC", "$PROGRAM", "$MSG", "$SEQ") indexes("seq")); # program("/usr/bin/mysql -usyslogadmin -psyslogadmin syslog" # template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg, seq) # VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG', '$SEQ' );\n") # template-escape(yes)); }; It works if I use the program driver, as currently commented out. The table is configured as following, using the schema defined by php-syslog-ng 2.9.8m. +----------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | host | varchar(128) | YES | MUL | NULL | | | facility | varchar(10) | YES | MUL | NULL | | | priority | varchar(10) | YES | MUL | NULL | | | level | varchar(10) | YES | | NULL | | | tag | varchar(10) | YES | | NULL | | | datetime | datetime | YES | MUL | NULL | | | program | varchar(15) | YES | MUL | NULL | | | msg | text | YES | | NULL | | | seq | bigint(20) unsigned | NO | MUL | 0 | | | counter | int(11) | NO | | 1 | | | fo | datetime | YES | | NULL | | | lo | datetime | YES | | NULL | | +----------+---------------------+------+-----+---------+---------------- What is the problem? Thanks in advance for the help.

3 4

Removing Prefixes from Syslog-ng Messages
by adam.j.brendamour＠accenture.com 29 Apr '09

29 Apr '09

Hello all, I am having issues with syslog-ng adding prefixes to my messages. I currently have a syslog-ng agent sending messages from a log source to a syslog-ng central server. I am then sending the messages from the central server to a 3rd party event management program. Is there a way to forward an unchanged syslog-ng message from the relay server? For example: Message arriving from source: Apr 16 09:31:07 psrdev47 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" Message being forwarded from server: Apr 16 09:31:07 10.56.2.77 125 <0>1 2009-04-16T09:31:02-07:00 psrdev47 - - - - 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" How can I configure the syslog-ng server to relay this unchanged message to our destination? Apr 16 09:31:07 psrdev47 10.56.9.12 - - [16/Apr/2009:09:31:02 -0700] "GET / HTTP/1.1" 302 427 "-" "-" Thanks for your help! Regards, Adam This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

3 4