November 2009 - syslog-ng - lists.balabit.hu

Logging only certain logs to a remote syslog server
by Paras Fadte 16 Nov '09

16 Nov '09

Hi, Is it possible to log only a particular logs to a remote syslog server ? For example logging only mysql/postgres logs to a remote host . syslog-ng version used is syslog-ng 1.6.8 Thank you. -Paras

4 6

Re: [syslog-ng] syslog-ng Digest, Vol 55, Issue 18
by Adam Chou 13 Nov '09

13 Nov '09

> > > On Friday 13 of November 2009, Adam Chou wrote: > > > > Just in case you're wondering why I'm trying to build from source, the > > current packages are built for amd64 and I'm running x86_64 on an > > Intel. > > https://lists.balabit.hu/pipermail/syslog-ng/2009-November/013551.html > > HTH > > thanks, that was informative. That link seems to be only the client version, however. Is there a server binary that is also compiled to work with yum's x86_64?

2 1

Can't build syslog-ng RPM
by Adam Chou 13 Nov '09

13 Nov '09

I'm trying to build the syslog-ng RPM and I'm having an issue with the libnet-devel package. I've tried searching for it and I can't find a devel package from Fedora, CentOS, or RHEL. I found the libnet-devel page but the project appears to have forked now so I'm no longer sure if its still good to use. Just in case you're wondering why I'm trying to build from source, the current packages are built for amd64 and I'm running x86_64 on an Intel. Any help with how I can get libnet-devel installed would be much appreciated. Thanks.

3 2

stats_freq ignored in config version 3.0?
by Peter Volkov 12 Nov '09

12 Nov '09

Hi, I've just upgraded syslog-ng up to 3.0.4 and configuration files up to version 3.0. Now I have the following config: ======================================================== @version:3.0 options { stats_freq(4320); }; source src { unix-stream("/dev/log" max-connections(256)); internal(); }; destination messages { file("/var/log/messages"); }; log { source(src); destination(messages); }; ======================================================== The problem I experience is that -- MARK -- is printed every 20 minutes no matter what I put into /etc/syslog-ng/syslog-ng.conf stats_freq option argument: Nov 12 17:56:32 stage4 syslog-ng[25815]: syslog-ng starting up; version='3.0.4' Nov 12 18:16:32 stage4 -- MARK -- Nov 12 18:36:32 stage4 -- MARK -- Does anybody else experience this? What could cause this? I've managed to reproduce this problem on two linux systems with kernel 2.6.27 and 2.6.31. How to debug this kind of problem? Thanks in advance for any hint. -- Peter.

2 2

Re: [syslog-ng] flags(final)
by Michael J. Bauer 12 Nov '09

12 Nov '09

flags(final) is now working with my rewritten match() statements. I wonder if it was a combination of the older version I was initially running and the odd structure of the log entry I was trying to use, but I really am disinclined to try to chase it down. However, in rewriting things, I did discover something else. In cutting and pasting, I had accidentally done this in my config: destination d_dhcpd { file("/var/log/dhcpd" flush_lines(10) flush_timeout(1000)); }; destination d_dhcpd { file("/var/log/named" flush_lines(10) flush_timeout(1000)); }; filter f_dhcpd { match("dhcpd" value(PROGRAM)); }; filter f_named { match("named" value(PROGRAM)); }; log { source(s_sys); filter(f_dhcpd); destination(d_dhcpd); flags(final); }; log { source(s_sys); filter(f_named); destination(d_named); flags(final); }; log { source(s_sys); destination(d_default); flags(fallback); }; Note the log statement containing the non-existent destination(d_named). When I ran "/etc/init.d/syslog-ng reload", it didn't crash, nor did it throw any error where I could see it. What it did do was start logging multiple copies of each line going to destination(d_default), apparently one more line each time I did a reload. I didn't notice what was going on until it was at 8 copies of each line, and it took me until 17 copies of each line to finally give up and run "/etc/init.d/syslog-ng stop; /etc/init.d/syslog-ng start". Once I did that, I got an error and corrected the problem. This is probably an obscure bug, but it may be worth looking at. It was certainly entertaining after the fact. MJB Michael J. Bauer wrote: > I think I am misunderstanding what flags(final) is supposed to do. I'm > running syslog-ng 2.1.4 on RHEL 5.4 (Tikanga). > > I have a fairly simple syslog-ng configuration, which I've attached > below. I'm trying to pick off individual groups of log entries and put > them in their own individual files. I want to ensure that each gets > logged exactly once, so I'm using flags(final). I also have a catch-all > at the end in case I've missed something, but the ultimate goal is to > have that file present, but empty. > > However, with this configuration, the log entries that appear in > d_network_address_translation (/var/log/network-address-translation) > also appear in d_default (/var/log/default) despite the presence of > flags(final) on an earlier log() line. Should it work this way? If so, > what can I do to get the desired behavior? > > Thanks, > MJB > > options { > sync (0); > time_reopen (10); > log_fifo_size (1000); > long_hostnames (off); > use_fqdn (no); > create_dirs (no); > keep_hostname (yes); > }; > > source s_sys { > file ("/proc/kmsg" log_prefix("kernel: ")); > unix-stream ("/dev/log"); > internal(); > udp(ip(0.0.0.0) port(514)); > }; > > destination d_network_address_translation { > file("/var/log/network-address-translation"); }; > destination d_default { file("/var/log/default"); }; > > filter f_network_address_translation { host("router-service-interface") and > priority(info) and > facility(local2) and > match("FWNAT"); }; > > log { source(s_sys); > filter(f_network_address_translation); > destination(d_network_address_translation); > flags(final); }; > log { source(s_sys); > destination(d_default); }; > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > >

2 2

stupid question: syslogging plaintext non-syslog-format messages unchanged (netconsole)
by Nix 12 Nov '09

12 Nov '09

So I'm trying to get the Linux kernel netconsole working, to diagnose some nasty persistent panics... and after some struggle I've got most of it working, but I can't make syslog-ng (3.0 from git as of tonight) cooperate. The problem seems to be that netconsole is handing syslog-ng completely bare text lines over UDP, and syslog-ng is persistently trying to linebreak them at apparently random space characters, even with the no-parse flag turned on. Some specifics. I'm receiving the netconsole messages with the following (trimmed down from the real 500-line-long config, all global options preserved in this trim, I'm pretty sure this is everything relevant but I haven't tested this on its own yet 'cos it would break syslogging for a fairly large network to actually replace the syslog-ng config with this): ,---- | options | { | keep_hostname(no); | create_dirs(1); | perm(0644); | dir_perm(0755); | log_fifo_size(1000); | use_dns(yes); | dns_cache(yes); | use_fqdn(no); | log_iw_size(5192); | flush_lines(100); | flush_timeout(1000); | stats_freq(3600); | chain_hostnames(yes); | }; | | source netconsole | { | udp(ip(0.0.0.0) port(24183) log_fetch_limit(100) flags(no-parse no-multi-line)); | }; | | # A bare template, for netconsole messages. | template bare | { | template("$MSG\n"); | template_escape(no); | }; | | destination netconsole | { | file("/var/log/$HOST-netconsole.log" | template(bare)); | }; | | log | { | source(netconsole); | destination(netconsole); | }; `---- but this is not enough. Most of the time the same message is received over an ordinary syslog source, too, so I can compare the two. If the syslogged message reads: ,---- | Oct 20 21:17:53 mutilate warning: : [ 6783.237435] pktcdvd: Fixed packets, 32 blocks, Mode-2 disc | Oct 20 21:17:53 mutilate warning: : [ 6783.238541] pktcdvd: write caching control failed | Oct 20 21:17:53 mutilate warning: : [ 6783.238547] pktcdvd: 55 10 00 00 00 00 00 00 14 00 00 00 - sense 05.00.00 (Illegal request) `---- the netconsole message that's logged using the configuration above reads ,---- | [ 6783.237435] pktcdvd: Fixed packets, | 32 blocks, | Mode-2 disc | [ 6783.238541] pktcdvd: write caching control failed | [ 6783.238547] pktcdvd: | 55 | 10 | 00 | 00 | 00 | 00 | 00 | 00 | 14 | 00 | 00 | 00 | - | sense 05.00.00 | (Illegal request) `---- This is obviously not sane. Removing the no-multi-line flag doesn't change anything at all (I only added it in desperation, because I *want* linebreaks in the original message to be reproduced in the output: I just don't want *more* linebreaks than are present in the original output), so that's not it either. I'm logging kernel messages using ,---- | source local | { | unix-stream("/dev/log" flags(no-multi-line)); | internal(); | file("/proc/kmsg" program_override("kernel: ") flags(kernel no-multi-line)); | }; `---- and things hanging off that, without incident: no random linebreaks or anything. So, what am I missing? It's probably incredibly obvious, but I've blown several hours hacking at configs and digging through the syslog-ng source and have seen nothing to indicate what's going on...

2 3

syslog-ng on solaris locks up after a while
by Igor Manassypov 12 Nov '09

12 Nov '09

Hello, I am having an issue with a solaris installation of the syslog-ng. It is configured such that all the logs are stored different per-ip folders. This is my centralized logging device, so it is fairly heavily loaded with receiving logs from a few dozen hosts. The syslog-ng process locks up every two to three weeks, with no messages logging to any of the files. The only way of getting it back is kill -9 the process and restart it. Is there any known issue of same sorts and is there any other way around it other than recycling the daemon every night? here is the version info: bash-3.00# syslog-ng --version syslog-ng 3.0.4 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10 Compile-Date: Aug 11 2009 10:44:17 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: on Enable-Sun-Door: on Enable-IPv6: off Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: off Enable-Pcre: on bash-3.00# uname -a SunOS prelude 5.10 Generic_137137-09 sun4v sparc SUNW,T5240 Thanks! -igor Igor Manassypov., M.Eng, P.Eng, CCIE 23032, CCVP Network Architect

3 11

Prefiltering
by Kārlis Repsons 12 Nov '09

12 Nov '09

People, is there currently some good way to filter source and then use it *filtered* for numerous "log" invocations in configuration file? If no, it takes writing the same thing many times...

4 4

syslog-ng 3.0 Administrator Guide, 9th Edition
by Robert Fekete 12 Nov '09

12 Nov '09

Dear syslog-ng users, A new version of the syslog-ng 3.0 Administrator Guide is available on our website. You can read about the changes in detail at the BalaBit Documentation Blog at http://robert.blogs.balabit.com/2009/11/syslog-ng-administrator-guide-9th.h… Kind Regards, Robert Fekete

2 1

rm /var/log/{dmesg,lastlog,wtmp,faillog,..?}?!
by Kārlis Repsons 11 Nov '09

11 Nov '09

If one has set up a dedicated syslog-ng configuration and wants to remove files like those, what would happen? Can anyone list the software entities responsible for writing to those? Has anyone configured a system, which doesn't needs them?

3 3