- syslog-ng - lists.balabit.hu

syslog-ng PID not getting set correctly
by Ngim, Hyrum CTR ADCF/Arrowpoint 04 May '09

04 May '09

We are running the sunfreeware 3.0.1 version of syslog-ng. I'm noticing that the PID is not being set correctly. I saw a bug report but it's not the same as what I'm seeing: https://bugzilla.balabit.com/show_bug.cgi?id=23 Here is what I am experiencing: # ps -ef | grep syslog root 1662 1 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng root 1663 1662 0 07:42:05 ? 0:00 /usr/local/sbin/syslog-ng # cat /var/run/syslog-ng.pid 1663 Shouldn't the PID be 1662? I've also seen inconsistencies when I use the svcadm commands to disable/enable/refresh the syslog-ng service. Many times it won't stop the service properly and the /var/run/syslog-ng.pid file remains in /var/run. When I try to restart syslog-ng, it won't restart until I run an svcadm disable syslog-ng, remove the /var/run/syslog-ng.pid file, then run svcadm enable syslog-ng. Thanks In Advance for any help.

3 3

[Bug 45] New: syslog-ng fails to build with openssl-1.0.0-beta2
by bugzilla＠bugzilla.balabit.com 04 May '09

04 May '09

https://bugzilla.balabit.com/show_bug.cgi?id=45 Summary: syslog-ng fails to build with openssl-1.0.0-beta2 Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 This bug occurs with 3.0.1 as well as with HEAD of git. Configured with --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-ssl --enable-dynamic-linking --enable-ipv6 --disable-sql --with-pidfile-dir=/var/run. gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT cfg-grammar.o -MD -MP -MF .deps/cfg-grammar.Tpo -c -o cfg-grammar.o cfg-grammar.c In file included from /usr/include/openssl/crypto.h:129, from /usr/include/openssl/comp.h:5, from /usr/include/openssl/ssl.h:149, from tlscontext.h:31, from cfg-grammar.y:15: /usr/include/openssl/safestack.h:105: error: expected identifier or ‘(’ before numeric constant make[2]: *** [cfg-grammar.o] Error 1 make[2]: Leaving directory `/tmp/syslog-ng-3.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/syslog-ng-3.0' make: *** [all] Error 2 I know what you're thinking, and no, there's nothing wrong with the openssl headers nor libraries on my system because OpenSSH 5.2p1 and a patched PHP 5.3.0RC1 successfully builds, links and runs with openssl-1.0.0-beta2 on my system. Perhaps some macros used internally by openssl-1.0.0-beta2 have been redefined by syslog-ng? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng OSE and oracle sql() driver
by Crowder, Steven D. 04 May '09

04 May '09

I don't think the first one made it through. According to the page here: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/ It says that the syslog-ng OSE has "Direct database access", however I have run into a few problems with both the precompiled OSE 3.0HEAD package and compiling from source. The binary package simply logs to /var/log/messages that it doesn't know about an "oracl" DBI driver. I thought that maybe I had to setup the ORACLE_HOME and other env vars and it would fix this, but no dice. Then I tried compiling from source (syslog-ng_3.0.1). It complains that it cannot find dbi.h. I know that the the PE version of syslog-ng will allow direct oracle access based on the documentation, but it appears that this is supposed to work in OSE as well based on the page listed above. If this is just an error, and there's no way that the OSE can have "Direct" database access, I understand, I just want to check first. Thanks, Steven

3 3

Tag support in syslog-ng
by ILLES, Marton 03 May '09

03 May '09

Hi, In the recent discussion on db-parser() I mentioned that I would like to add tag support for messages. The whole feature is not 100%, but here is the first phase. I wrote about it in my blog: http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html and here it is: I had some time so I created a tagging framework in syslog-ng. The patches could be found in my syslog-ng git repo (http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=summary) At the current state it is possible to create tags on the fly, attach them to messages and latter match on them in a filter() expression. A tag is simply identified by it's name, obviously these names must be unique. Internally tags are stored as bitmask vector so it is very fast to set/unset or match on them. Though as of now the internal representation limits the number of tags to 8159. I think it ought to be enough for the near future at least. I dare not to say that it ought to be enough for anyone. :) As of now messages could be tagged on sources only and messages are automatically tagged on sources with a special tag representing the source ".source.". Matching on the source this way is much faster than matching on the ${SOURCE} macro as macro matching involves a string comparison or even a regular expression matching while tag matching is a simple bitwise comparison. (I am planning to add automatic tags on sources for the source driver and other hopefully useful information.) Here is an example on how to use it: source s_tcp1 { tcp(ip(192.168.1.1) port(1514) tags("tcp", "router")); }; source s_tcp2 { tcp(ip(192.168.1.2) port(1514) tags("tcp", "windows)); }; source s_udp1 { udp(ip(192.168.1.1) port(1514) tags("udp", "router")); }; #Match messages from s_tcp1 source filter f_tcp1 { tags(".source.s_tcp1"); }; filter f_router { tags("router"); }; #Match on tags "tcp" or "udp" filter f_net { tags("tcp", "udp"); }; Next step is to add tagging support for the db-parser() so messages can be tagged that way and add support for dynamic SQL table scheme for tags. This w ay one can easily match and parse messages and put the parsed information in to different SQL tables. cheers, Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

Did as suggested. The errors are now a lot less but now syslog-ng won't start. Here's the error I get [root@soc-nsm-002 init.d]# ./syslog-ng restart Shutting down system logger: [ OK ] Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; syntax error in /etc/syslog-ng/syslog-ng.conf at line 2. # syslog-ng configuration file. @version: 3.0 # # Thi Message: 2 Date: Fri, 1 May 2009 13:15:10 -0700 From: "Ngim, Hyrum CTR ADCF/Arrowpoint" <hyrum.ngim.ctr(a)awps.army.mil> Subject: Re: [syslog-ng] Error when starting syslog-ng 3.0.1 To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <8B986F9B35D04C428A6B98996432C79B05ADE7311E(a)fedmail01.fed.awps.army.mil> Content-Type: text/plain; charset="iso-8859-1" At the very top of your config file, it will need to have: @version: 3.0 You'll just need to remove the # sign.

4 3

Error when starting syslog-ng 3.0.1
by Bristol, Gary L. 01 May '09

01 May '09

I'm getting the following error when I start the syslog-ng process. I'm running version 3.0.1 I have tried several different formats for getting it to recognize that fact that I'm running or attempting to run a version 3.0 config. The other problem I'm having is that I'm not getting any local logs. The directory creation and the forwarding of the syslog to the remote system works just fine. Starting system logger: Configuration file has no version number, assuming syslog-ng 2.1 format. Please add @version: maj.min to the beginning of the file; WARNING: global: the default value of chain_hostnames is changing to 'no' in version 3.0, please update your configuration accordingly; Error resolving group; group='log' WARNING: template: the default value for template-escape is changing to 'no' in version 3.0, please update your configuration file accordingly; WARNING: input: sources do not remove new-line characters from messages by default in version 3.0, please add 'no-multi-line' flag to your configuration if you want to retain this functionality; WARNING: file source: default value of follow_freq in file sources is changing in 3.0 to '1' for all files except /proc/kmsg; WARNING: filters do not store matches in macros by default in 3.0, please update your configuration by using an explicit 'store-matches' flag to achieve that; WARNING: template: the meaning of the $MSG/$MESSAGE macros is changing in version 3.0, please prepend a $MSGHDR when upgrading to 3.0 config format; Error resolving user; user='root' Here is the configuration file at /etc/syslog-ng/syslog0ng.conf # syslog-ng configuration file. #@version: 3.0 # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # 20000925 gb(a)sysfive.com # # configuration file for syslog-ng, customized for remote logging # #options { flush_lines (0); # time_reopen (10); # log_fifo_size (1000); # long_hostnames (off); # use_dns (no); # use_fqdn (no); # keep_hostname (yes); # }; options { stats_freq (0); chain_hostnames (no); flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames(off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); perm(0640); group("log"); }; source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # #source s_local { # unix-dgram("/dev/log"); # file("/proc/kmsg" log_prefix("kernel:")); #}; # source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; # network logs come from the local network source src { unix-stream("/dev/log"); internal(); }; source remote { udp(ip ("0.0.0.0")); }; source remote_tcp { tcp(ip ("xxx.xx.xx.xxx")); }; source kernsrc { file("/proc/kmsg"); }; ## Local filters filter f_filter1 { facility(kern); }; filter f_filter2 { level(info) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; # Filters for remote logs added for RSA filter f_paygate { host ("xxx.xxx.xxx.xxx");}; filter f_payweb { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1 { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2 { host ("xxx.xxx.xxx.xxx");}; filter f_roomview { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket1a { host ("xxx.xxx.xxx.xxx");}; filter f_paymarket2a { host ("xxx.xxx.xxx.xxx");}; #filter f_dc { host ("xxx.xxx.xxx.xxx");}; # Local logs - order DOES matter ! #log { source(s_sys); filter(f_filter1); destination(d_cons); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_auth); }; log { source(s_sys); filter(f_filter4); destination(d_mail); }; log { source(s_sys); filter(f_filter5); destination(d_mlal); }; log { source(s_sys); filter(f_filter6); destination(d_spol); }; log { source(s_sys); filter(f_filter7); destination(d_boot); }; log { source(s_sys); filter(f_filter8); destination(d_cron); }; # Remote logs for RSA Appliance log { source(remote); filter(f_paygate); destination(d_rsa); }; log { source(remote); filter(f_payweb); destination(d_rsa); }; log { source(remote); filter(f_paymarket1); destination(d_rsa); }; log { source(remote); filter(f_paymarket2); destination(d_rsa); }; log { source(remote); filter(f_roomview); destination(d_rsa); }; log { source(remote); filter(f_paymarket1a); destination(d_rsa); }; log { source(remote); filter(f_paymarket2a); destination(d_rsa); }; #log { source(remote); filter(f_dc); destination(d_rsa); }; # log by host log { source(remote); destination(hosts); }; log { source(remote_tcp); destination(hosts); }; # Local destinations destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog"); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; # added for re-direct to RSA Appliance destination d_rsa { udp("xxx.xxx.xxx.xxx" port (514) template("<$PRI> $DATE $HOST $MESSAGE"));}; # automatic host sorting # by current year, month, day, host # files created are also by year, month, day, host, facility # set it up destination hosts { file("/var/syslog/HOSTS/$YEAR/$MONTH/$DAY/$HOST/$YEAR-$MONTH-$DAY-$HOST-$FAC ILITY.log" owner(root) group(root) perm(0655) dir_perm(0 755) create_dirs(yes)); }; Gary L. Bristol CISSP, RHCE University of Oklahoma 200 Felgar St., Suite 226 Norman, OK 73019 405-325-2236 ******************************************** ---------------------------------------------------------------------------- + Ranked 10th in PC Magazine's 2007 Top 20 Wired Campuses + Computerworld 2006 100 Best Places to Work in IT ---------------------------------------------------------------------------- ********************************************************************** This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you **********************************************************************

3 2

[Bug 44] New: name of the file and content of the file contains different timestamp
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=44 Summary: name of the file and content of the file contains different timestamp Product: syslog-ng Version: 3.0.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ruiyen(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Friendly reminder that this bug needs to be fixed. I can help you with this if you can let me know which source file to look at. >From https://lists.balabit.hu/pipermail/syslog-ng/2009-February/012567.html 2009/2/15 Balazs Scheidler <bazsi at balabit.hu> > On Thu, 2009-02-12 at 09:52 +0800, Charles Chan wrote: > > I use time_zone instead timezone,configuration like this > > =========================================================== > > template apache_template { template("$DAY-$HOUR $S_DAY-$S_HOUR > > $R_DAY-$R_HOUR $MSGONLY\n"); template_escape(no); }; > > destination us_int_apache > > { file("/disk2/apache_logs/$YEAR-$MONTH-$DAY/$HOST.apache.log" > > timezone("PST8PDT") template(apache_template)); }; > > ============================================================ > > service can start,but the problem still exists. > > the file is stored in the /disk2/apache_logs/2009-02-12 > > generated log is: > > 11-17 11-17 11-17 222.69.201.23 - - - [11/Feb/2009:17:37:03 -0800] > > "GET /atmgateway.client.html ................... > > two date string still difference. I use syslog-ng 3.0.1 OSE version,is > > this OSE version problem? > > The PE and OSE versions are the same in this regard. Apart from the > concrete differences listed on our website, I work hard to keep the two > versions in sync. > > So let me try to understand your problem a bit deeper... Sorry for not > having done so previously. Sometimes I just reply without really > thinking through problems in the question. > > hmm.. as I see your problem seems to be that the name of the file and > the _content_ of the file contains a different timestamp. Right? > > This is intentional. > > When expanding the macros in the name of the files, syslog-ng _always_ > uses the server-side, local timezone. The timezone used in the content > of the file can be controlled using the time_zone() option. > > Currently there's no way to control the filename zone offset. The > reasoning behind that was that I already found the timezone related > configuration options too complex, and I didn't see the need so far. > I felt that agregating logs based on the _received timestamp_ is more > trustworthy (because clients are free to specify a timestamp they > feel-like), and that is in local timezone anyway. Also, noone else has > complained previously :) > > That said, I'm willing to add another option to control the timezone > used when expanding the filename, if you really think that solves your > problem. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 2

[Bug 21] New: call shutdown() before closing a socket
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=21 Summary: call shutdown() before closing a socket Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bazsi(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 syslog-ng should call shutdown() before closing a socket to ensure the contents of the socket buffer are emptied. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 23] New: MySQL default configuration causes error on 'pid' column
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=23 Summary: MySQL default configuration causes error on 'pid' column Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mark(a)la-croix.net Type of the Report: bug Estimated Hours: 0.0 When configuring a MySQL destination, the default 'pid' column type is INT. However, if a message comes through with no PID, the SQL command tries to insert an empty quoted string, which causes an error in MySQL 5.0. Example error: Error running SQL query; type='mysql', host='!MY_DB_HOST!', port='', user='syslog-ng', database='syslog-ng', error='1366: Incorrect integer value: \'\' for column \'pid\' at row 1', query='INSERT INTO !MY_LOG_CLIENT_HOST!_2008_09_23 (date, facility, level, host, program, pid, message) VALUES (\'2008-09-23 12:24:35\', \'1\', \'5\', \'!MY_LOG_CLIENT!\', \'root\', \'\', \'test\')' The message is coming from a TCP source from which a large percentage of messages have no PID associated. The workaround is to explicitly set the PID column to VARCHAR so it can handle empty strings, but it would be nice to see the empty PID case gracefully handled. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 34] New: After sighup master doe not write to logs, but clients think the server is writing to the logs
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=34 Summary: After sighup master doe not write to logs, but clients think the server is writing to the logs Product: syslog-ng Version: 2.1.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: michael(a)crowdscience.com QAContact: michael(a)crowdscience.com Type of the Report: --- Estimated Hours: 0.0 We encountered a problem when a sighup is sent to syslo-ng as part of our process to rotate the log file on the syslog-ng master. After the signal is sent and the log file has been rotated syslog-ng creates a new log file, but does not write to it. During this time the syslog-ng clients think that the master is up and continue to send their messages to the server (ie not queuing the messages locally). This resulted in a period of time where messages where not logged and required a stop and start of the syslog-ng service to get the messages logging again. The server is running Solaris Nevada snv_67 X86. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2