- syslog-ng - lists.balabit.hu

AUTO: Shanif Lakhani is out of the office (returning 01/13/2010)
by Shanif Lakhani 05 Jan '10

05 Jan '10

I am out of the office until 01/13/2010. Please contact Robert Holoweckyj and/or Narendra Prasad for any urgent matters. Thanks Note: This is an automated response to your message "syslog-ng Digest, Vol 57, Issue 1" sent on 1/5/10 6:00:02. This is the only notification you will receive while this person is away.

1 0

program_override("kernel") not working in 3.0.4
by Paul B. Henson 05 Jan '10

05 Jan '10

I'm trying to upgrade from 2.1.3 to 3.0.4 under Linux, and am having trouble getting my kernel messages tagged appropriately. My previous configuration had the following: ----- source k_src { pipe("/proc/kmsg" log_prefix("kernel: ")); }; destination iptables { file("/var/log/iptables.log"); }; filter f_iptables { match("iptables:"); }; log { source(k_src); filter(f_iptables); destination(iptables); flags(final); }; destination messages { file("/var/log/messages"); }; log { source(src); source(k_src); destination(messages); }; ----- All of my kernel messages showed up with the "kernel:" prefix, and anything coming from iptables was dropped in a separate log. Based on the documentation, I modified my configuration to the following for 3.0.4: ----- source k_src { file("/proc/kmsg" program_override("kernel")); }; destination iptables { file("/var/log/iptables.log"); }; filter f_iptables { program("^kernel$") and message("^iptables:"); }; log { source(k_src); filter(f_iptables); destination(iptables); flags(final); }; destination messages { file("/var/log/messages"); }; log { source(src); source(k_src); destination(messages); }; ----- However, my kernel messages show up with no prefix. I tried starting up in debug mode: ----- syslog-ng starting up; version='3.0.4' Incoming log entry; line='<6>usb 2-7: USB disconnect, address 8' Filter rule evaluation begins; filter_rule='f_iptables' Filter node evaluation result; filter_result='not-match' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_iptables' Initializing destination file writer; template='/var/log/messages', filename='/var/log/messages' Incoming log entry; line='<4>iptables: IN=eth0 OUT= MAC=0:05:00:10:97:43:00:0SC147.5.2 S=2... E=8TS00 RC0C T= D363POO2' Filter rule evaluation begins; filter_rule='f_iptables' Filter node evaluation result; filter_result='not-match' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_iptables' ----- This didn't help me. Am I doing something wrong? I didn't find any complaints of this nature on the mailing list, which leads me to suspect I'm somehow being stupid, any pointers much appreciated. Thanks... -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson(a)csupomona.edu California State Polytechnic University | Pomona CA 91768

2 2

Chained changing IP addresses to FQDN
by Jason Carr 28 Dec '09

28 Dec '09

Hello, I have a syslog infrastructure using syslog-ng comprising of three tiers: probes that accept messages and forwards them to the broker, a broker that routes and/or duplicates messages to the appropriate storage devices, and storage devices. All forward using the tcp("hostname"); as a destination. Basically it looks like this: machine1 ==udp==> +-------+ +--------+ +-----------+ machine2 ==udp==> | probe | ==tcp==> | broker | ==tcp==> | storage 1 | machine3 ==udp==> +-------+ +--------+ +-----------+ || +-----------+ ++=======tcp==> | storage 2 | +-----------+ The probes rewrite the hostname to the IP address that the packet came from using the chain_hostnames(no) and keep_hostname(no) options. One of the requirements of one specific storage device is to change the IP addresses into hostnames. I'm having difficulty on the storage device switching the IP address in the log message to the FQDN. I have tried all combinations I can think of of chain_hostnames and keep_hostname and cannot seem to get the IP address replaced with the FQDN. Is there a way to replace the IP address with the FQDN while receiving the messages from an intermediary? Thank you, Jason

4 5

troubleshooting and debugging syslog-ng
by Zoltán Pallagi 23 Dec '09

23 Dec '09

Hi, I've sent a new post about it to my blog (http://pzolee.blogs.balabit.com/2009/12/troubleshooting-and-debugging-syslo…) I tried to collect all important things about this issue. I hope this post will make it easier for you to specify the reason of the possible syslog-ng problems and for us to help you. pzolee

1 0

time_sleep in high udp environment?
by Tim Rupp 22 Dec '09

22 Dec '09

Hi, we were debugging message drops on our syslog-ng server (rpm syslog-ng-3.0.5-1.rhel5.x86_64) and found out that if we commented out the time_sleep global option we had set (value of 20) that all the drops went away. Initially we had set this value while reading through the pdf documentation (page 136) figuring that since we had potentially a lot of traffic that this would boost performance. So I guess I'd like to know why we seemed to see the opposite behavior with this option enabled. We don't really do tcp connections, so maybe this option was useless to begin with. Anyways, it made me curious, so if anyone can enlighten me I'd appreciate it. Thanks, Tim

2 1

pdbtool pattern debuging
by ILLES, Marton 22 Dec '09

22 Dec '09

Hi! I have just pushed to my syslog-ng 3.1 git repo some patches to add more detailed debugging/troubleshooting capabilities to patterndb through pdbtool match. Now you can easily see how your patterns matched a given message, which part of the message was matched by literal part of the pattern and which by a parser. All this new feature with a wonderful colorized output if requested, though the colors are still bit ugly... Up to now the easiest way of creating patterns was to do trial & error sequence and try to figure out which part of the pattern was broken. Now you can exactly see where the matching stopped. It is also possible to output the match in parse-able format to be used in scripts or as a backend of some pattern authoring tool. I am still not sure about the colored output nor about the machine parse-able output format, so any feedback, comment, idea or suggestion is very much appreciated. You can see some screen-shots of pdbtool match (with colors) in action on my blog: http://marci.blogs.balabit.com/2009/12/pdbtool-stylists-wanted.html The code is available in my git repo: http://git.balabit.hu/?p=marci/syslog-ng-3.1.git;a=commit;h=c7c3b7b81577507… I did not update the man page so far, but --help works. You should be using the "-D" to enable debuging and "-c" for color output, while "-C" for csv style output. Happy matching! Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0

Memory Exhaust
by blizzards＠libero.it 21 Dec '09

21 Dec '09

Hi all. I'm using syslog-ng 3.0.5. I have a very large configuration, more than 5000 lines, at my site. I write non only to a plain text file, but i redirect all traffic to another syslog-ng server with 10 templetas with udp socket. If i start the service, syslog-ng give me this error: Memory Exhaust at line 3666. If i cut off my configuration at line 3665, the service start correctly. Someone can help me with this issue? Maybe memory leak bug? Gix.

2 1

Sending TCP messages to Syslog-NG server
by James Pirman 18 Dec '09

18 Dec '09

I am having an issue sending TCP messages to a SysLog-NG server running version 3.0.4. For some reason when multiple messages are put into 1 tcp packet, the server queues part of the second packet and then spits it out when it receives the another new message. Has anyone seen this before? Any help would be appreciated. -Jim _________________________________________________________________ Hotmail: Powerful Free email with security by Microsoft. http://clk.atdmt.com/GBL/go/171222986/direct/01/

1 1

Pattern Database first snapshot available
by ILLES, Marton 18 Dec '09

18 Dec '09

Hi, Last week BalaBit made available some 8000 patterns (covering more than 200 applications) for syslog-ng patterndb (or db_parser as you like to call it). The patterns are available under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 (CC by-NC-SA) license. The patterns in their current form are just snapshots of the ongoing effort of providing good quality patterns for various applications. You can download the snapshot of patterns from our website: http://www.balabit.com/downloads/files/patterndb-snapshot/patterndb-2009120… The patterns are partially hand-crafted and also automatically generated from logfiles and from logcheck regexp based database. Some of the patterns also contains example messages which we are using to automatically test the pattern and syslog-ng's db_parser. You can merge the xml files using "pdbtool merge". I would also like to setup a public git repository where anyone interested can follow the patterndb development and can submit patterns or fixes. A patterndb website containing all patterndb related information, links, forums, wikis and other useful documentations is under construction as well. Till than the syslog-ng mailing list a good place for questions, ideas and discussions. As always feedbacks are very welcomed! Happy parsing! Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

3 5

syslog-ng OSE 3.1beta2 released
by Balazs Scheidler 18 Dec '09

18 Dec '09

Dear syslog-ng users, A new beta version of the current syslog-ng feature release has been uploaded to our website. https://www.balabit.com/network-security/syslog-ng/opensource-logging-syste… This is a bugfix only release containing blocker issues found in the past three weeks. Please see the NEWS entry below for more details. Any kind of feedback, be it positive or negative is appreciated. 3.1beta2 Fri, 18 Dec 2009 09:19:39 +0100 Bugfixes: * Fixed two major memory leaks, one on input messages, one in patterndb. * Fixed escaped structured data processing. * Fixed expanding SDATA references in templates which have multiple dots in its name (e.g. when an OID is present in the name of the SD param or SD-ID. * pdbtool now correctly zero pads month and day fields in publish date of the merged patterndb file. * pdbtool won't link to unnecessary libraries, which fixes a compilation problem on Solaris with flex installed. -- Bazsi

1 0