- syslog-ng - lists.balabit.hu

question about pattern database version 1
by Guillaume Rousse 20 Jan '10

20 Jan '10

Hello list. The pattern database format 1 seems to force a mandatory program pattern, wich doesn't exists for some messages, suchas kernel oops and memory exhaustion issues: Jan 19 15:18:24 localhost Call Trace: Jan 19 15:18:24 localhost ntpd invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0 It seems to be corrected in version 2: rulesets can be defined without program pattern where these act as a fail-back rule sets if no program pattern is matching or when message does not have program part I guess that means I'm can't do anything to catch those problems as long as syslog-ng 3.1 OSE is released, right ? -- BOFH excuse #330: quantum decoherence

2 1

filtering messages by context
by Guillaume Rousse 20 Jan '10

20 Jan '10

Hello list. We use hardware-based SLB for our critical services (ldap, radius, dns, ...). It means we have probes testing our services every X seconds, polluting our logs. It's quite easy to catch those traces when they consist of a single message, as for our kerberos servers Jan 20 11:17:02 avron1 kdc[477]: connection closed before end of data after 0 bytes from IPv4:10.202.11.254 to tcp/0 Jan 20 11:17:04 avron1 kdc[477]: connection closed before end of data after 0 bytes from IPv4:10.202.11.254 to tcp/0 Jan 20 11:17:06 avron1 kdc[477]: connection closed before end of data after 0 bytes from IPv4:10.202.11.254 to tcp/0 This simple setup is enough to drop them: destination d_drop { } filter f_slb_kerberos_probe { message("connection closed before end of data"); }; log { source(s_sys); filter(f_kerberos); filter(f_slb_kerberos_probe); destination(d_drop); flags(final); }; However, when those traces actually consist of two messages, as in our LDAP servers, it's a bit more difficult: Jan 20 11:23:04 avron1 slapd[13802]: conn=68089 fd=51 ACCEPT from IP=10.202.11.254:39428 (IP=10.202.11.8:389) Jan 20 11:23:04 avron1 slapd[13802]: conn=68089 fd=51 closed (connection lost) Jan 20 11:23:06 avron1 slapd[13802]: conn=68090 fd=51 ACCEPT from IP=10.202.11.254:39434 (IP=10.202.11.8:389) Jan 20 11:23:06 avron1 slapd[13802]: conn=68090 fd=51 closed (connection lost) I can filter out the first message, using the probe IP adress as criteria, but not the second one: filter f_slb_ldap_probe { message("ACCEPT from IP=10.202.11.254"); }; log { source(s_sys); filter(f_ldap); filter(f_slb_ldap_probe); destination(d_drop); flags(final); }; is there any way to catch the connection id in the first message (68090), so as to filter out any following one refering to the same connection ? -- BOFH excuse #152: My pony-tail hit the on/off switch on the power strip.

2 1

Remove double timestamping
by Marty Sørensen 20 Jan '10

20 Jan '10

Hello .... New user to syslog-ng but still hoping someone can help me with a small config example Im forwarding syslog from my syslog-ng but when it arrives it has double timestamps/hostname: -- Jan 19 11:02:58 cut-hostname 10.229.5.2 32176: Jan 19 11:02:57: %SFF8472-5-THRESHOLD_VIOLATION -- I have addet this info my config: options { use_time_recvd(yes); keep_hostname(no); } also included: destination udp_forward { udp("<my ip>"); }; template fjern { template("$MSGONLY\n"); }; destination cisco_cupe { file("/var/log/cisco_cupe" template("fjern")); }; log { source(s_all); filter(local5); destination(cisco_cupe); destination(udp_forward); }; Cheers Martin

3 5

md5sum or sha1sum files not found .
by Mr. James W. Laferriere 19 Jan '10

19 Jan '10

Hello All , When going to ... http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.0.4/… I am not finding either a md5sum or sha1sum (or greater) file for the contents . Is this due to decission not to or a oversight ? Tia , JimL -- +------------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network&System Engineer | 3237 Holden Road | Give me Linux | | babydr(a)baby-dragons.com | Fairbanks, AK. 99709 | only on AXP | +------------------------------------------------------------------+

5 15

Dear Syslog-Ng, If you can't find in google, try JUSTDIAL.COM
by Vadiraj 18 Jan '10

18 Jan '10

Dear Syslog-Ng,I strongly recommend this website www.justdial.com. It's a world class local search service & I've always found anything I've ever wanted.You can find info on any company, product, or service in over 240 cities in India.You can also call them up 24x7, on phone (69999999), a local call in 240 Indian cities.Ask for anything, you'll get the info on the phone and/or by SMS within 30 secs, and this service is at no cost!For a change, it's an original Indian idea and an Indian company with world class service, and with a vision to spread all over the world.Be a proud Indian and forward this to every Indian you know.Best Wishes,VadirajClick Here to unsubscribe.

1 0

Odd problem after upgrading from 2.1 to 3
by Jerry Bell 18 Jan '10

18 Jan '10

This config worked in syslog-ng 2.1 source NET {udp(port(514)); tcp(port(5140) keep-alive(yes));}; destination A { file("/var/log/A"); }; destination B { file("/var/log/B"); }; filter A { host ("A");}; filter A { host ("A");}; log { source(NET); destination(A); filter(A); }; log { source(NET); destination(B); filter(B); }; worked as expected that is log messages coming from host A went to /var/log/A file and log messages coming from host B went to /var/log/B. The two hostnames are resolved locally in /etc/hosts After upgrading to 3.0.x all the messages duplicated in both files so that messages coming from A are written in /var/log/A AND /var/log/B and vice-versa. Any idea why it does not work in 3? Thank you, Jerry

2 2

Broken TCP connection
by James Pirman 14 Jan '10

14 Jan '10

I am currently having an issue with syslog-ng 3.0.4 where my TCP connection between my client and server is lost throughout the day. By looking at the pcap file from tcpdump I can tell that the TCP connection reset was initiated by the syslog-ng server. The only information that was initially in the log file regarding this disconnection was the following 2 lines: <45>1 2010-01-05T10:29:32.323-06:00 server-db-01 syslog-ng 29213 - [meta sequenceId="2733719"] notice Syslog connection closed; fd='9', client='AF_INET(192.168.27.218:46326)', local='AF_INET(192.168.27.198:20514)' and <46>1 2010-01-05T10:29:32.323-06:00 server-db-01 syslog-ng 29213 - [meta sequenceId="2733720"] info Closing log transport fd; fd='9' In order to get more information, I set the following flags in init.d: "-v -d -t". This did not give me any more information about the TCP disconnect, however I did notice that a lot of my normal messages were preceeded by the following text: <47>1 2010-01-05T10:29:32.323-06:00 server-db-01 syslog-ng 29213 - [meta sequenceId="2733718"] debug Incoming log entry; line= A normal log message then follows the '=' sign. A decent percentage of my messages are preceeded by this throughout the day, but just before the disconnect it appears that all of my messages from server-db-01 are preceeded by the debug line. Any ideas as to what could be going on? I have included my config file below if that helps. Any assistance would be greatly appreciated. -Jim @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { keep_hostname(yes); keep_timestamp(yes); frac_digits(3); }; source all { internal(); syslog(ip("192.168.27.198") port(20514) transport("tcp") log_fetch_limit(100)); }; destination allclientsfile { file("/data/local/Logs/server-$YEAR-$MONTH-$DAY.log" flags(syslog-protocol) flush_timeout(100) create_dirs(yes) dir_owner(jpirman) dir_group(jpirman) owner(jpirman) group(jpirman) template("$PRIORITY $MESSAGE") ); }; destination msgserver { udp("127.0.0.1" port(20515) flush_timeout(100) template("$ISODATE $PROGRAM $PRIORITY $MESSAGE\n")); }; log { source(all); destination(allclientsfile); destination(msgserver);}; _________________________________________________________________ Hotmail: Powerful Free email with security by Microsoft. http://clk.atdmt.com/GBL/go/171222986/direct/01/

3 8

slngctl: syslog-ng debug, trace and statistics utility
by ILLES, Marton 14 Jan '10

14 Jan '10

Hi, PZolee wrote a quite lengthy blog post on troubleshooting and debuging syslog-ng [1] back in December 2009. It popped into my mind that running syslog-ng with debug/trace/verbose option enabled is very handy, but sometimes you do not want to stop syslog-ng just to restart it in a more verbose mode. My idea was to enable syslog-ng to real-time modify these verbosity settings through the control socket which is currently only used for exporting statistical information. I extended the control socket communication to have the ability of modify these logging settings. To make life easier I have created a small utility called slngctl which can be used to communicate with the running syslog-ng process. Using slngctl it also easy to query the statistics collected by syslog-ng. Also on longer term I think other useful small features could be added the slngctl. slngctl commands: marci@octane:$ slngctl Possible commands are: stats Dump syslog-ng statistics verbose Enable/query verbose messages debug Enable/query debug messages trace Enable/query trace messages Querying statistics: marci@octane:$ slngctl stats destination;d_out;;a;processed;0 global;payload_reallocs;;a;processed;0 source;s_tcp;;a;processed;0 global;msg_clones;;a;processed;0 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 center;;queued;a;processed;0 Getting current verbose settings: marci@octane:$ slngctl verbose VERBOSE=1 Enabling trace run-time: marci@octane:$ slngctl trace TRACE=0 marci@octane:$ slngctl trace -s 1 marci@octane:$ slngctl trace TRACE=1 As always you can find the source code in my 3.1 git repository and all feedbacks are very welcome. best, Marton PS: I know slngctl might not be the best name, so I offer a bier to anyone suggesting a better name when visiting us in Budapest, Hungary. :) [1] http://pzolee.blogs.balabit.com/2009/12/troubleshooting-and-debugging-syslo… -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

3 3

requirement for @version directive in 3.x config file
by Paul B. Henson 14 Jan '10

14 Jan '10

Our syslog-ng configuration file is generated automatically, from various different bits and pieces based on what roles and services a system provides. Unfortunately, our current implementation would not allow always ensuring that the "@version" directive is the first line in the generated config file. Version 3 of syslog-ng seems very unhappy if that is not the case. This seems somewhat inflexible, would it be possible to have a command line option specifying the configuration file version so it does not have to be on the very first line of the actual file? Any other suggestions (short of modifying our internal processes :) ) to get verison 3 to be happy with a configuration file without this specific directive as the very first line? Thanks much... -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson(a)csupomona.edu California State Polytechnic University | Pomona CA 91768

3 3

Leaking objects in size-4096 cache in slabinfo
by Doug Warner 12 Jan '10

12 Jan '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm running syslog-ng 3.0.4 OSE on Gentoo and have a central syslog host that accepts about 3Mbps of log traffic via a TCP pipe. We've recently been having oom-killer problems and I finally tracked it down to the fact that we seem to be leaking objects in the size-4096 cache. Now, slabinfo is new to me so I could be interpreting this wrong, but restarting syslog-ng definite reclaims this cache. What data can I provide to help track down this problem? I noticed that there seems to be a similar error in 3.1 beta, reported here: http://thread.gmane.org/gmane.comp.syslog-ng/8544 - -Doug -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFLJ8FqJV36su0A0xIRAizGAKDbVEpHRMhrRgAogUuAh0nCKz9xFwCgtz17 6EN8aU3MxU7bL8Sr4xRx6tI= =1LQg -----END PGP SIGNATURE-----

3 13