October 2009 - syslog-ng - lists.balabit.hu

stupid question: syslogging plaintext non-syslog-format messages unchanged (netconsole)
by Nix 12 Nov '09

12 Nov '09

So I'm trying to get the Linux kernel netconsole working, to diagnose some nasty persistent panics... and after some struggle I've got most of it working, but I can't make syslog-ng (3.0 from git as of tonight) cooperate. The problem seems to be that netconsole is handing syslog-ng completely bare text lines over UDP, and syslog-ng is persistently trying to linebreak them at apparently random space characters, even with the no-parse flag turned on. Some specifics. I'm receiving the netconsole messages with the following (trimmed down from the real 500-line-long config, all global options preserved in this trim, I'm pretty sure this is everything relevant but I haven't tested this on its own yet 'cos it would break syslogging for a fairly large network to actually replace the syslog-ng config with this): ,---- | options | { | keep_hostname(no); | create_dirs(1); | perm(0644); | dir_perm(0755); | log_fifo_size(1000); | use_dns(yes); | dns_cache(yes); | use_fqdn(no); | log_iw_size(5192); | flush_lines(100); | flush_timeout(1000); | stats_freq(3600); | chain_hostnames(yes); | }; | | source netconsole | { | udp(ip(0.0.0.0) port(24183) log_fetch_limit(100) flags(no-parse no-multi-line)); | }; | | # A bare template, for netconsole messages. | template bare | { | template("$MSG\n"); | template_escape(no); | }; | | destination netconsole | { | file("/var/log/$HOST-netconsole.log" | template(bare)); | }; | | log | { | source(netconsole); | destination(netconsole); | }; `---- but this is not enough. Most of the time the same message is received over an ordinary syslog source, too, so I can compare the two. If the syslogged message reads: ,---- | Oct 20 21:17:53 mutilate warning: : [ 6783.237435] pktcdvd: Fixed packets, 32 blocks, Mode-2 disc | Oct 20 21:17:53 mutilate warning: : [ 6783.238541] pktcdvd: write caching control failed | Oct 20 21:17:53 mutilate warning: : [ 6783.238547] pktcdvd: 55 10 00 00 00 00 00 00 14 00 00 00 - sense 05.00.00 (Illegal request) `---- the netconsole message that's logged using the configuration above reads ,---- | [ 6783.237435] pktcdvd: Fixed packets, | 32 blocks, | Mode-2 disc | [ 6783.238541] pktcdvd: write caching control failed | [ 6783.238547] pktcdvd: | 55 | 10 | 00 | 00 | 00 | 00 | 00 | 00 | 14 | 00 | 00 | 00 | - | sense 05.00.00 | (Illegal request) `---- This is obviously not sane. Removing the no-multi-line flag doesn't change anything at all (I only added it in desperation, because I *want* linebreaks in the original message to be reproduced in the output: I just don't want *more* linebreaks than are present in the original output), so that's not it either. I'm logging kernel messages using ,---- | source local | { | unix-stream("/dev/log" flags(no-multi-line)); | internal(); | file("/proc/kmsg" program_override("kernel: ") flags(kernel no-multi-line)); | }; `---- and things hanging off that, without incident: no random linebreaks or anything. So, what am I missing? It's probably incredibly obvious, but I've blown several hours hacking at configs and digging through the syslog-ng source and have seen nothing to indicate what's going on...

2 3

Macro question
by Michael J. Bauer 06 Nov '09

06 Nov '09

I am running syslog-ng 3.0.4 on RHEL 5.4. I have a log message that appears in my logfiles as Oct 28 16:41:22 juniper-router {wan-service-set}[FWNAT]: ASP_NAT_RULE_MATCH: proto 6 (TCP) application: any, ge-0/0/3.2:10.3.13.153:49818 -> 66.249.80.148:80, Match NAT rule-set: , rule: nat-outgoing, term: dynamic-nat There are no carriage returns in that entry; any that appear are artifacts of some mail server or client along the way. I want any log entry containing the string FWNAT to go into a separate file. match(FWNAT) on the filter does just that, but syslog-ng complains that I'm not using value() for efficiency. However, it's not at all clear which macro, if any, actually contains the string FWNAT. I can tell you five that don't: APPLICATION EVENT_SOURCE MESSAGE MSG PROGRAM What macro contains FWNAT? And where can I get a definitive list of macros and specifics on how the value in each is set for each line? The documentation is insufficiently detailed on this subject. Thanks, MJB

3 4

$HOST macro and relay host
by Jason Barrett 06 Nov '09

06 Nov '09

Hi all, I'm relaying log messages from one syslog-ng server to another. At the final stop, the only way I can get the $HOST macro to work is if I enable dns resolution on server 1. Is this by design? Here are the relevant configs: ---------------------------------------------------------------------- syslog-ng server 1 (relays to server 2): chain_hostnames(yes); keep_hostname(yes); use_dns(no); source s_udp { udp(port(514)); }; destination df_udpback { udp("192.168.1.157" port(514)); }; log { source(s_udp); destination(df_udpback); }; ---------------------------------------------------------------------- syslog-ng server 2: chain_hostnames(yes); keep_hostname(yes); use_dns(yes); source s_udp { udp(port(514)); }; destination df_udp { file("/var/log/ics/$HOST/$YEAR/$MONTH/$DAY"); }; log { source(s_udp); destination(df_udp); }; ---------------------------------------------------------------------- Sample log message on server 2: Oct 30 09:35:03 10.12.24.46/10.12.24.46 %ASA-5-111005: 10.28.22.55 end configuration: OK 10.12.24.46 is the correct IP address of the originating host, and $HOST resolves to this IP address. I would prefer $HOST to resolve to the hostname as it exists in the /etc/hosts file. Thanks!

3 6

Re: [syslog-ng] syslog-ng3 with SQL on FreeBSD
by Pal Tamas 03 Nov '09

03 Nov '09

On Wed, Oct 28, 2009 at 12:00:01PM +0100, syslog-ng-request(a)lists.balabit.hu wrote: > Date: Tue, 27 Oct 2009 16:27:31 -0400 > From: "Michael W. Lucas" <mwlucas(a)blackhelicopters.org> > Subject: [syslog-ng] syslog-ng3 with SQL on FreeBSD > To: syslog-ng(a)lists.balabit.hu > Message-ID: > <20091027202731.GA82885(a)bewilderbeast.blackhelicopters.org> > Content-Type: text/plain; charset=us-ascii > > Hi, > > I'm working on getting syslog-ng 3.0.3 to support SQL on FreeBSD, and > hopefully get it added to the FreeBSD package or port. It looks like > something isn't quite right in the configure script, however. > > I have libdbi 0.8.3 installed, but "./configure --enable-sql=yes" > still reports: > > checking for LIBDBI... gnome-config: not found > no Did you installed pkg-config from ports? If not, please install it and re-run configure with the PKG_CONFIG_PATH set to /usr/local/lib/pkgconfig (this is where libdbi installed the dbi.pc config file) > checking for dbi_initialize in -ldbi... no This step will be skipped if library detection using pkg-config detection works. If not, please set include and library detection paths in CFLAGS and LDFLAGS variables before running configure. Ex: CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" ./configure ... Please note this only works in posix shells (bash, /bin/sh), in csh you have to set the variables before running configure. > > I've reviewed /usr/local/include/dbi/dbi.h, and dbi_initialize is in > there. > > Can anyone suggest a fix? If both of these fail, please send in part of config.log from the checking LIBDBI messages to the following text: ## ---------------------- ## ## Running config.status. ## ## ---------------------- ## Regards, -- Pal Tamas/Folti folti(a)balabit.hu

3 5

problem with matching IP address and \d regex operand
by Phil.Newlon＠wendysarbys.com 03 Nov '09

03 Nov '09

I am using this regular expression with Kiwi Syslog to distribute messages to several destinations based on the last number of the third octet (0-4 goes one place, 5-9 goes another). "10\.\d+\.\d*[0-4]\." This doesn't work with syslog-ng, of course, but based on my research of the archives, this should do the same thing because I've escaped the "\d" match("10\.\\d+\.\\d*[0-4]\.") Nope, I get nothing. I've shortened it to just match("10\.\\d+") and still get no matching messages. This sort of works, but gives some unexpected results: match("10\.[0-9]+\.[0-9]*[0-4]\.") The match("10\.[0-9]+\.[0-9]*[0-4]\.") statement resulted in 'true' on this log message. I didn't expect a match on 10.87.48.4 from it because of the '8' as the last number of the third octet not matching '0-4' Oct 29 16:31:20 10.87.48.4 Kiwi_Syslog_Daemon Oct 29 16:31:20 10.87.48.4 MSWinEventLog 0 Security 71000 Thu Oct 29 16:31:17 2009 538 Security pos User Success Audit POS0408748 Logon/Logoff User Logoff: User Name: pos Domain: POS0408748 Logon ID: (0x0,0x4ACB69) Logon Type: 3 42921033 So, I have two questions..... What's wrong with this: match("10\.\\d+\.\\d*[0-4]\.") And why did this match("10\.[0-9]+\.[0-9]*[0-4]\.") match this Oct 29 16:31:20 10.87.48.4 Kiwi_Syslog_Daemon Oct 29 16:31:20 10.87.48.4 MSWinEventLog 0 Security 71000 Thu Oct 29 16:31:17 2009 538 Security pos User Success Audit POS0408748 Logon/Logoff User Logoff: User Name: pos Domain: POS0408748 Logon ID: (0x0,0x4ACB69) Logon Type: 3 42921033 Thanks! Phil Notice: This e-mail message and its attachments are the property of Wendy's/Arby's Group Inc. or one of its subsidiaries and may contain confidential or legally privileged information intended solely for the use of the addressee(s). If you are not an intended recipient, then any use, copying or distribution of this message or its attachments is strictly prohibited. If you received this message in error, please notify the sender and delete this message entirely from your system.

4 6

Seg fault with include file contains an error
by anthony lineham 03 Nov '09

03 Nov '09

Hi Bazsi, I've been investigating a seg fault that occurs when using an include statement in the config file. The error occurs when one of the included files contains an syntax error. The first time syslog-ng is restarted with the bad config, the error is detected and restart is aborted. However, if a second HUP is sent a seg fault occurs. I had a bit of trouble tracking down the cause and I'm still not entirely sure of the mechanism that leads to the crash but I found a couple of apparent problems, which when corrected prevent it. 1. There is a global variable "include_depth" which normally gets decremented back to zero after successful parsing of included config files. However, if an error is detected it stays at its current value and subsequent restarts increment from that point. This doesn't cause the crash, but given enough restarts would lead to overrun or exhaustion of the "include_stack" array. 2. In the case of unsuccessful config parsing certain bits of memory are dealloced but their pointers that not reset. This seems to be what was causing the seg fault but I've found it a bit hard to pin down. 3. There was an off-by-1 error in the included config deinit loop. The following patch fixes the problem, but may not necessarily be the best way to do it - particularly issue 1. Regards, Anthony --- syslog-ng_3.0.1-63-g41f77f5-old/src/cfg-lex.l 2009-09-14 16:02:09.000000000 +1200 +++ syslog-ng_3.0.1-63-g41f77f5/src/cfg-lex.l 2009-09-14 16:15:30.000000000 +1200 @@ -626,17 +626,24 @@ { gint i; - for (i = 0; i < include_depth; i++) + for (i = 0; i <= include_depth; i++) { CfgIncludeLevel *level = &include_stack[i]; if (level->current_file) - g_free(level->current_file); + { + g_free(level->current_file); + level->current_file = NULL; + } g_slist_foreach(level->files, (GFunc) g_free, NULL); g_slist_free(level->files); level->files = NULL; if (level->yybuf) - yy_delete_buffer(level->yybuf); + { + yy_delete_buffer(level->yybuf); + level->yybuf = NULL; + } } + include_depth = 0; }

2 1

syslog-ng3 with SQL on FreeBSD
by Michael W. Lucas 03 Nov '09

03 Nov '09

Hi, I'm working on getting syslog-ng 3.0.3 to support SQL on FreeBSD, and hopefully get it added to the FreeBSD package or port. It looks like something isn't quite right in the configure script, however. I have libdbi 0.8.3 installed, but "./configure --enable-sql=yes" still reports: checking for LIBDBI... gnome-config: not found no checking for dbi_initialize in -ldbi... no I've reviewed /usr/local/include/dbi/dbi.h, and dbi_initialize is in there. Can anyone suggest a fix? Thanks, ==ml -- Michael W. Lucas mwlucas(a)BlackHelicopters.org http://www.MichaelWLucas.com/ Latest book: Cisco Routers for the Desperate, 2nd Edition http://www.CiscoRoutersForTheDesperate.com/

2 1

can two destinations refer to the same file?
by Jason Haar 03 Nov '09

03 Nov '09

Hi there I'm wanting to configure syslog-ng so that I have two different filters that are associated with two different destinations - which have the same filename - but different templates. The issue I have is that we have a range of syslog clients, which may or may not have trustworthy clocks. So I default to ignoring their timestamps and use the syslog servers clock, except for the hosts I know to also use NTP. What I want to confirm is that syslog-ng will happily open /var/log/messages ONCE and will parse the following the way I intend? ie #clocks on unix systems (which are named 'blah-unix-number') are always configured correctly filter f_goodclock { host("-unix-") }; #can't trust everything else filter f_badclock { not host("-unix-") }; destination d_badclock_messages { file("/var/log/messages" template("$R_ISODATE $HOST $FACILITY $PRIORITY $MSG\n") }; destination d_goodclock_messages { file("/var/log/messages" template("$S_ISODATE $HOST $FACILITY $PRIORITY $MSG\n") }; log { ... filter(f_badclock); destination(d_badclock_messages);}; log { ... filter(f_goodclock); destination(d_goodclock_messages);}; Should that work as I hope? Thanks! -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

3 3

Re: [syslog-ng] Re : flags(final)
by Michael J. Bauer 03 Nov '09

03 Nov '09

I've upgraded from syslog-ng 2.1.4 to 3.0.4, in case flags(final) was somehow broken in that version. The upgrade has made no difference: flags(final) still appears to have no effect. I am attempting to shunt all entries containing FWNAT into a separate file (network-address-translation), and keep them out of the default destination (default). The configuration, below, writes the entries to both, despite the presence of flags(final) in a prior log{} statement. My understanding based on reading the documentation and various sample configurations is that flags(final) in one log{} statement should preclude the log entry in question from going to any subsequent log{} statement. Am I misunderstanding how flags(final) works, or should I submit a bug report? If I'm misunderstanding, please let me know if there is a way to get the behavior I'm describing. Thanks, MJB My syslog-ng 3-ified configuration: @version:3.0 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_network_address_translation { file("/var/log/network-address-translation" flush_lines(10) flush_timeout(1000)); }; destination d_default { file("/var/log/default" flush_lines(10) flush_timeout(1000)); }; filter f_network_address_translation { host("hlgn-crtr-01-service") and priority(info) and facility(local2) and match("FWNAT"); }; log { source(s_sys); filter(f_network_address_translation); destination(d_network_address_translation); flags(final); }; log { source(s_sys); destination(d_default); }; Michael J. Bauer wrote: > That flags(final) is in fact part of the log{} line. The destination() > is also part of the log{} line, and appears on a line of its own because > of line wrap. The log{} line is, in one line (and hoping to avoid wrap): > > log { source(s_sys); filter(f_network_address_translation); > destination(d_network_address_translation); flags(final); }; > > The terminating }; is after flags(final);. Neither of the real > destination{} lines contain flags(final). > > MJB > > srainville(a)videotron.ca wrote: > >> The flags(final) goes at the end of the log statement, not the >> destination. >> >> Cheers, >> >> Steve >> ----- Message d'origine ----- >> De: "Michael J. Bauer" <mjbauer(a)eecs.tufts.edu> >> Date: Jeudi, 24 Septembre 2009, 23:00 >> Objet: [syslog-ng] flags(final) >> À: syslog-ng(a)lists.balabit.hu >> >> >>> I think I am misunderstanding what flags(final) is supposed to >>> do. I'm >>> running syslog-ng 2.1.4 on RHEL 5.4 (Tikanga). >>> >>> I have a fairly simple syslog-ng configuration, which I've >>> attached >>> below. I'm trying to pick off individual groups of log >>> entries and put >>> them in their own individual files. I want to ensure that >>> each gets >>> logged exactly once, so I'm using flags(final). I also >>> have a catch-all >>> at the end in case I've missed something, but the ultimate goal >>> is to >>> have that file present, but empty. >>> >>> However, with this configuration, the log entries that appear in >>> d_network_address_translation (/var/log/network-address- >>> translation) >>> also appear in d_default (/var/log/default) despite the presence >>> of >>> flags(final) on an earlier log() line. Should it work this >>> way? If so, >>> what can I do to get the desired behavior? >>> >>> Thanks, >>> MJB >>> >>> options { >>> sync (0); >>> time_reopen (10); >>> log_fifo_size (1000); >>> long_hostnames (off); >>> use_fqdn (no); >>> create_dirs (no); >>> keep_hostname (yes); >>> }; >>> >>> source s_sys { >>> file ("/proc/kmsg" >>> log_prefix("kernel: ")); >>> unix-stream ("/dev/log"); >>> internal(); >>> udp(ip(0.0.0.0) port(514)); >>> }; >>> >>> destination >>> d_network_address_translation { >>> file("/var/log/network-address-translation"); }; >>> destination >>> d_default { file("/var/log/default"); }; >>> >>> filter f_network_address_translation { host("router-service- >>> interface") and >>> priority(info) and >>> facility(local2) and >>> match("FWNAT"); }; >>> >>> log { source(s_sys); >>> filter(f_network_address_translation); >>> >>> destination(d_network_address_translation); flags(final); }; >>> log { source(s_sys); >>> destination(d_default); }; >>> >>> >>> >> ______________________________________________________________________________ >> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> Documentation: >>> http://www.balabit.com/support/documentation/?product=syslog-ng >>> FAQ: http://www.campin.net/syslog-ng/faq.html >>> >>> >> ------------------------------------------------------------------------ >> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> >> >> > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > >

2 1

Syslog-NG interpreting Cisco message 'count' number as $Program
by Patrick Smith 30 Oct '09

30 Oct '09

Hi all, wondering if anyone is familiar with this issue. Have several Cisco switches logging to a syslog-ng server. The syslog-ng server creates a new file for every message as it is interpreting the 'count' item added by the switch as the $program. My configs as follows to demonstrate what I mean: Syslog-ng.conf: destination d_all { file("/var/log/syslog-ng/$HOST/$PROGRAM-$YEAR-$MONTH-$DAY.log"); }; * This has always worked for me before and creates logs for each host with the program prefixed (e.g. snmp-date..., ssl_access_log-date...) But for Cisco switches it writes a new logfile for every message, with the Cisco message number (i.e. log count #) prepended. E.g., for a 6509 switch: -rw-r--r-- 1 syslog-ng wheel 165 Oct 21 11:51 1655156-2009-10-21.log -rw-r--r-- 1 syslog-ng wheel 168 Oct 21 11:51 1655157-2009-10-21.log -rw-r--r-- 1 syslog-ng wheel 147 Oct 21 11:51 1655158-2009-10-21.log -rw-r--r-- 1 syslog-ng wheel 150 Oct 21 11:51 1655159-2009-10-21.log -rw-r--r-- 1 syslog-ng wheel 154 Oct 21 11:52 1655160-2009-10-21.log * where 16551xx is the count of the log message on the switch And each of those files contains one log message, e.g. syslog-ng # cat 1655156-2009-10-21.log Oct 21 11:51:15 6509switch1 1655156: Oct 21 18:51:14.922: %LINK-5-CHANGED: Interface GigabitEthernet3/35, changed state to administratively down For reference the logging on the switches are set as: 6509s: service timestamps log datetime msec logging buffered 20000 debugging logging rate-limit 1000 log-adjacency-changes logging <syslog host IP> logging synchronous 2960s: service timestamps log uptime logging buffered 20000 logging <syslog host IP> logging synchronous I'm thinking I could have a separate destination statement to properly deal with these but I'm wondering if there's any other steps I can take to mitigate it. I have several Cisco firewalls logging to syslog-ng and they do not have the same issue. Thanks for any help. IMPORTANT NOTICE THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary and confidential information intended only for the use of the recipient(s) named above. If you are not the intended recipient, you may not print, distribute, or copy this message or any attachments. If you have received this communication in error, please notify the sender by return e-mail and delete this message and any attachments from your computer. Learn more about Payment Processing's services at www.paypros.com.

3 10