- syslog-ng - lists.balabit.hu

login.pdb reworked
by Peter Czanik 05 Nov '10

05 Nov '10

Hello, Attached is a new version of login.pdb (called login2.pdb). It has patterns for many console/terminal/telnet login/logout events. This version should generate one set of name value pairs for each event, and only one. If you use console/login/telnet for logins, plese give it a try and let me know, how it works for you. I found, that there are some slight variations among messages even between different Ubuntu versions, so I'd like to see, how these patterns work on a larger set of Linux distributions, UNIX revisions. Thank you for your help, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

4 6

Can't get program destination to work
by Chuck 05 Nov '10

05 Nov '10

I am running 3.0.4 on a solaris 10 host. I am trying to send critical and emergency events via smtp. I have the following syntax: destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh $HOST" template("<$PRI>$DATE $HOST $MSG\n.\n"); ); }; It doesn't like this syntax, however this is exactly what the documentation shows. I have also tried the following thinking the documentation might be wrong: destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh $HOST"); template("<$PRI>$DATE $HOST $MSG\n.\n"); }; destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh" template("<$PRI>$DATE $HOST $MSG\n.\n"); ); }; destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh"); template("<$PRI>$DATE $HOST $MSG\n.\n"); }; Has anyone done this or know how to send alerts via smtp? Thanks, CC

4 7

Thoughts on patterndb syntax
by Lars Kellogg-Stedman 05 Nov '10

05 Nov '10

I've been playing with 3.2beta1 recently and getting my feet wet with the patterndb support, which I haven't had a chance to work with before. I have a few thoughts regarding the patterndb rule syntax, mostly targeted at making things a little bit easier to work with. - Rule IDs Is there any particular reason why unique IDs were selected as rule identifiers? They're not particularly meaningful to people, and they're hard to talk about. It's much easier to say, "we're suddently seeing lots of matches for ssh-accept-connection" than it is to say, "we're suddenly seeing lots of matches for 4dd5a329-da83-4876-a431-ddcb59c2858c". With the current version of syslog-ng it looks like it's possible to use arbitrary identifiers in place of UUIDs, and that's what I'm doing for my local rulesets. This even makes classification metadata more useful, because .classifier.rule_id=ssh-accept-connection is immediately meaningful, while a UUID is useless unless I go grepping around the database. - Whitespace Log messages tend to be long, which makes them unwieldy in a number of situations. It would be nice if instead of this: <pattern>...some very long log message that makes my life difficult and my patterns hard to read ...</pattern> I could do this: <pattern collapse_whitespace="yes">...some very long log message that has been conveniently wrapped to that it's easier to edit, email, and otherwise work with.</pattern> Specifically, enabling "collapse_whitespace" would transform any sequence of whitespace to a single " ". As proposed here this would be a completely backwords-compatible change because the default behavior would remain the same. - Reusable patterns We use Cisco firewall modules on our network. When these devices log a connection-related message, the source and destination address look something like this: ircs:3610:67.186.94.126/41004 That's: interface:vlan:ip/port Which becomes: @ESTRING:fwsm.src_if::@@NUMBER:fwsm.src_vlan@:@IPv4:fwsm.src_ip@/@NUMBER:fwsm.src_port@ When this occurs throughout the ruleset, and multiple times within a single message, it really lowers the readability of the rules. I wish there was a way to modularize this so that I could create custom types, something like this: <type name="FWSM_ADDRSPEC"> @ESTRING:iface::@@NUMBER:vlan@:@IPv4:ip@/@NUMBER:fwsm.port@ </type> And then do this: <pattern>Accepted src @FWSM_ADDRSPEC:fwsm.src:@ dst @FWSM_ADDRSPEC:fwsm.dst:@</pattern> And get this: fwsm.src.iface fwsm.src.ip fwsm.src.port Etc. Anyway, that's all for now.

5 12

Statistics -- how do I turn them off?
by Liam Kirsher 03 Nov '10

03 Nov '10

My log file is filling up with statistics messages from syslog-ng (v. 3.1.2) , one message every 10 minutes. I tried turning it off by putting stats_freq(0) in the options clause, and restarted the syslog-ng daemon but that seems to have no effect. There must be something obvious I'm missing. -- Liam Kirsher PGP: http://liam.numenet.com/pgp/

3 3

Re: [syslog-ng] Statistics -- how do I turn them off?
by sasanka.sahu＠wipro.com 02 Nov '10

02 Nov '10

Hi Anis, Thanks for giving me INFO. Actually I want to deploy on solaris 10 OS . This is the release verion, bash-3.00# cat /etc/release Solaris 10 10/09 s10s_u8wos_08a SPARC Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 16 September 2009 Am not getting this package "syslog-NG" for solaris OS. Regds, Sasanka -----Original Message----- From: Anisur Rehman(IT) [mailto:Rehman@bankmuscat.com] Sent: Tuesday, November 02, 2010 4:39 PM To: Sasanka Sekhar Sahu (WI01 - Enterprise Services) Subject: RE: [syslog-ng] Statistics -- how do I turn them off? Dear sasanka I will recommend to use RHEL 5 .5 with MYSQL database with frontend Php-Syslog-ng. I am using this combination and it is working fine. If you like to test this option let me know. However you can use HPUX option there is inbuilt syslog-NG available with HPUX version v3. Thanks Anis 0968-99826412 -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of sasanka.sahu(a)wipro.com Sent: Tuesday, November 02, 2010 2:55 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] Statistics -- how do I turn them off? Hi Anybody can tell am using Solaris OS v 10 . Like to install php-syslog-ng GUI tools . where should I get the same as well as provide me some documentation for the same. Sasanka -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Sandor Geller Sent: Tuesday, November 02, 2010 4:12 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Statistics -- how do I turn them off? I'm filtering statistical messages based on the internal source and the following regexp: filter f_internal_statistics { match("^syslog-ng\[[[:digit:]]+\]: STATS") or match ("^syslog-ng\[[[:digit:]]+\]: Log statistics"); }; (for 3.x only the latter match is needed) 600 seconds is the default setting, the timer gets activated only when stats_freq > 0 so stats_freq(0) should turn off the generation of statistical messages completely. Are the logs generated on the same host or are you processing remote logs as well? On Sat, Oct 30, 2010 at 10:17 PM, Liam Kirsher <liamk(a)numenet.com> wrote: > My log file is filling up with statistics messages from syslog-ng (v. > 3.1.2) , one message every 10 minutes. > I tried turning it off by putting stats_freq(0) in the options clause, > and restarted the syslog-ng daemon but that seems to have no effect. > There must be something obvious I'm missing. > > -- > Liam Kirsher > PGP: http://liam.numenet.com/pgp/ > > ________________________________________________________________________ ______ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com

1 0

Syslog-ng on Solaris 9 problem
by Elgin Lorenz 29 Oct '10

29 Oct '10

Hello experts, today we switched our syslog-ng server from linux (CentOS 5.5) to a Solaris 9 box. All went well, until I noticed that syslog-ng obviously changes some of the log entries. Here are the 2 examples I noticed: Message on the local host: Oct 18 14:19:15 xxx kernel: IN=eth1 OUT= MAC=xxx SRC=xxx DST=xxx LEN=235 TOS=0x00 PREC=0x00 TTL=128 ID=119 PROTO=UDP SPT=138 DPT=138 LEN=215 Message on the syslog-ng server: Oct 18 14:19:15 xxx kernel: kernel: IN=eth1 OUT= MAC=xxx SRC=xxx DST=xxx LEN=235 TOS=0x00 PREC=0x00 TTL=128 ID=119 PROTO=UDP SPT=138 DPT=138 LEN=215 (one "kernel: " added) Message on the local host: Oct 18 14:03:06 xxx last message repeated 2 times Message on the syslog-ng server: Oct 18 14:03:06 xxx last: message repeated 2 times (one ":" added) On Linux I never saw this behaviour. Until now I couldn't find the mistake. What am I doing wrong? Did anyone else see something like this? Thanks in advance for your help. Kind regards, Elgin Lorenz -- Elgin Lorenz BTU Cottbus Universitaetsrechenzentrum Tel. 0355 693573 E-Mail lorenz(a)tu-cottbus.de

3 8

Logging all message metadata
by Lars Kellogg-Stedman 28 Oct '10

28 Oct '10

As I work with the classification engine, I wish there were a way for me to log *all* metadata associated with a log message. That is, I would like to record any data parsed out of the message by the parsing engine, as well as all the default metadata that syslog-ng generates about a message. Is there any way to do this other than writing my own output driver? As far as I can tell, all of the available drivers -- including the sql() driver -- require me to explicitly list which attributes I want to log. While my ultimate target would be sending this data into a database, I would be happy if I were able to dump it all to a file in some sort of structured format that I could parse with my own tools (or maybe even feed back into syslog-ng). Thanks, -- Lars

3 8

Compilation problem in Debian (libsyslog-ng dependency)
by Jean F. Mousinho 28 Oct '10

28 Oct '10

Hello, I'm trying to manually build the package in Debian, but I'm getting an error when compiling: ... dh_shlibdeps: Compatibility levels before 4 are deprecated. dpkg-shlibdeps: warning: couldn't find library libdbparser.so needed by debian/syslog-ng/bin/pdbtool (its RPATH is '//lib://lib/syslog-ng'). Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file. To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH. dpkg-shlibdeps: failure: no dependency information found for lib/.libs/libsyslog-ng.so.0 (used by debian/syslog-ng/bin/pdbtool). dh_shlibdeps: command returned error code 512 make: *** [binary-arch] Error 1 dpkg-buildpackage: failure: debian/rules binary gave error exit status 2 ... I've noticed that libdbparser.so is actually located in ./modules/dbparser/.libs/libdbparser.so but the problem seems to be that dpkg-shlibdeps doesn't find dependency information for libsyslog-ng? I don't know much how automake+dpkg-buildpackage work to know how to solve this, even tho I've tried to google for it. Thanks for your time. Jean-François Mousinho

3 3

Having hostname and sender's IP address in header
by Worsham, Michael 28 Oct '10

28 Oct '10

We are trying to streamline the syslog-ng v3.x data from the RHEL server-clients to the RHEL DCS/Syslog server while also providing another destination to a local Tripwire Log Center server. I previously had the version 3.0.8 options (as seen below), which recorded the data files correctly on the DCS server with the correct hostname attribute: options { chain_hostnames(no); time_reopen(10); time_reap(360); log_fifo_size(1000); group(adm); perm(0644); dir_perm(0755); use_dns(yes); use_fqdn(yes); create_dirs(yes); keep_hostname(yes); log_msg_size(32768); stats_freq(0); flush_lines(0); }; Destination (directory and log file structure): destination d_general { file ("/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log" template("$DATE $HOST <$FACILITY.$PRIORITY> $MSGHDR $MSG\n") template_escape(no) ); }; Example: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log However, we found today that Tripwire Log Center (which we are now required to use for log aggregation/reporting needs) doesn't have the ability to do hostname translation (i.e. regex the sender's hostname seen in the datastream and give it an IP address so that it can be tracked). So this means the data file that is being sent from the RHEL clients will need to keep the sender's IP address in the actual data stream. So if I do keep_hostname(no) and still enable the facility.priority template on the DCS server, it will show the IP address rather than the hostname of the reporting client: Oct 25 18:51:12 10.153.13.70 <syslog.err> syslog-ng[2820]: EOF occurred while idle; fd='9' Oct 25 18:51:12 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection broken; fd='9', server='AF_INET(10.153.29.235:514)', time_reopen='10' Oct 25 18:51:22 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection established; fd='9', server='AF_INET(10.153.29.235:514)', local='AF_INET(0.0.0.0:0)' However, this now causes a problem on the Syslog/DCS server with the local log file storage requirements: Old Version: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log New Version: /var/log/syslog/general/10.153.13.70/10.153.13.70-10.25.2010.log NOTE: We don't have a local DNS that all of the servers can be queried against and that putting in nearly 100+ sender hosts in the /etc/hosts really isn't a viable option. How do I satisfy each requirement: 1) Allow the incoming data stream to be saved in a format based on the incoming sender's hostname for the directory structure (ex: "/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log")? 2) Allow the IP address of the sender to be seen in the actual data file once saved to the DCS server (ex: "Oct 25 18:51:12 10.153.13.70 <syslog.err>...")? 3) Is there an option to have the incoming sender's data saved with both the hostname and IP address in the same header format (ex: "Oct 25 18:51:12 CH33Test-WebCO 10.153.13.70 <syslog.err>...")? -- Michael ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are intended solely for the use of the named recipient(s). This email may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, you are prohibited from reviewing, copying, using, disclosing or distributing to others the information in this email and attachments. If you believe you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts of the email or attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

4 10

Error opening log file for writing
by Bill Snow 28 Oct '10

28 Oct '10

I have a openSUSE 11.3 system that I installed syslog-ng on through package management. The intention of the simple config is to get it to write log file messages from one program (pdns) to a separate log file outside of /var/log. No matter what I do with the file and directory permissions, I always get something like: syslog-ng[17864]: Error opening file for writing; filename='/opt/home/user/logs/pdns.log', error='Permission denied (13)' in the log file. syslog-ng is running as root, as follows: # ps xo pid,ppid,user,cmd | grep syslog 17863 1 root supervising syslog-ng 17864 17863 root /sbin/syslog-ng The special config lines in /etc/syslog-ng/syslog-ng.conf look like this: filter f_dyson { match('^pdns'); }; destination dyson { file("/opt/home/user/logs/pdns.log" perm(0644)); }; log { source(src); filter(f_dyson); destination(dyson); }; Googling around indicated a possible problem with capability management, but that is turned off in my version: # syslog-ng --version syslog-ng 3.1.1 Installer-Version: 3.1.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8747d74491eac3fdf5052194f47a68e659633ca9 Compile-Date: Jul 5 2010 13:47:02 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: off Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on I tried explicitly setting the user/group for the destination file in case syslog-ng was trying to write the file with a non-privileged owner in a directory owned by root.... and also tried setting the directory permissions to 777 just in case. Moving the destination to /var/log/pdns.log works. Of course that's not where I want the file. Anybody have an idea of what might be going on here? Thanks, Bill Snow

2 2