[syslog-ng] Syslog NG simple filtering not working as expected...
jrhendri at roadrunner.com
jrhendri at roadrunner.com
Wed Oct 2 20:20:28 CEST 2013
Hi,
Not sure if this applies to your specific needs but we do a couple things that sound similar.
First, we find that anything that causes DNS resolution really slows things down.
So we put some common options together like this:
options {
# check if the hostname contains valid characters
check_hostname(yes);
# do not resolve names for speed
use_dns(no);
dns_cache(no);
# keep hostnames to enable related macros
keep_hostname(yes);
# do not track / forward syslog forwarder chain
chain_hostnames(no);
# options related to file and directories
dir_owner("root");
dir_group("log_ro");
owner("root");
group("log_ro");
perm(0640);
dir_perm(0750);
create_dirs(yes);
};
Second - we filter based on netmask since we believe it is easier than any text matching
(should be able to be done with a bitmask on the IP header and not even need to parse the message at all)
#!# filter for specific servers - redacted with fake addressses
filter f_filter1 {
netmask( "192.168.1.1/32" ) or
netmask( "192.168.2.2/32" ) or
netmask( "10.0.1.1/32" ) or
netmask( "10.0.2.2/32" );
};
If this helps - you're welcome.
If not - my apologies :-)
Jim
---- Sandor Geller <Sandor.Geller at morganstanley.com> wrote:
> You should check the incoming traffic syslog-ng gets to see whether it
> is in a known and supported format or something which only resembles
> to syslog traffic so it could happen that syslog-ng doesn't parse it
> properly.
>
> What Bazsi referred to is that processing an IP address (containing
> dots) as a regexp is more expensive than using it for a plain string
> comparision. It's a micro-optimisation though.
>
> spoofing doesn't alter the message part, it changes the source IP
> address of the outgoing packet this is why it works only for UDP.
>
> On Wed, Oct 2, 2013 at 3:24 PM, Paul Hutton <paul_hutton at bigpond.com> wrote:
> > Hi again,
> >
> > Further to this it would appear that the network destination logger doesn't deal well with the syslog format that is being forwarded to it from the syslog-ng host. In particular, the syslog-ng host receives syslogs from the network source which have the fields:
> >
> > "local date" ,"local time", "hostname", "UTC date" ,"UTC time", "message string"
> >
> > The syslog-ng host then forwards to the network destination the above data, but with it prepended by a second entry of "local date","local time" and "source ip address", rather than hostname.
> >
> > I have been using spoof_source (yes) in the destination definition. Will this create the duplicate date and time entry as well as the source ip address?
> >
> > Thanks
> >
> > Paul
> >
> >
> > On 27/09/2013, at 1:26 PM, Paul Hutton wrote:
> >
> >> Hi Bazsi,
> >>
> >> Thanks for your reply. I'm running on an ESX hosted Solaris instance, originally with 1 x 1.8 Ghz CPU and 4 GB RAM. When I enabled diagnostics with syslog-ng-ctl, this pretty much killed the machine, so I'm now running with 2 x 1.8 Ghz CPUs and 8 GB RAM. I only generate a handful of test messages typically 10 to 40, not much of a load.
> >>
> >> With diagnostics enabled, I have seen entries of 50 or 60 in the vmstat kernel thread wait list, simultaneously with a very idle processor which is a bit of a mystery. However with the additional resources, I don't see any problems. I have just done some further testing and vmstat tells me that the testing had no measurable impact on the system. Syslog-ng-ctl stats never reports any dropped messages to the remote destination, rather the network source "src" messages processed always agrees with the destination "Remote_L" messages processed, as well with the local destination log files.
> >>
> >> The firewall filter statements contain nothing other than IPV4 addresses.
> >>
> >> Sorry but I don't understand the distinction that you make between what I am doing with my filter and your suggestion viz:
> >>
> >>> You might want to use a simple substring filter, like so:
> >>>
> >>> filter f_firewalls { host("ipaddress" type(string)); };
> >>
> >>
> >> Otherwise with the latest round of tests I suspect that there is something happening at the ESX layer that is messing me up, and which is now being investigated. If this turns out to be the case then I'm sorry for the noise...
> >>
> >> Thanks and regards
> >>
> >> Paul
> >>
> >> On 27/09/2013, at 1:56 AM, Balazs Scheidler wrote:
> >>
> >>> Hi,
> >>>
> >>> Can it happen that syslog-ng becomes CPU bound when you enable the
> >>> filter? Can you check that using top/vmstat/younameit whether syslog-ng
> >>> is spinning on a CPU?
> >>>
> >>> the host() filter is using regexps, and although your host() pattern
> >>> doesn't seem complex, it was anonimized in the config, so I don't know
> >>> if it is more complex in reality.
> >>>
> >>> You might want to use a simple substring filter, like so:
> >>>
> >>> filter f_firewalls { host("ipaddress" type(string)); };
> >>>
> >>> If you need more than mere substring matching, you might try to use
> >>> "pcre" (if your syslog-ng is compiled with that), or "glob", which
> >>> permits shell-like wildcard matches.
> >>>
> >>> Otherwise I have no clue why that filter reduces perfomance so much, it
> >>> shouldn't, but I don't know how much stress you put to it.
> >>>
> >>> Hope this helps,
> >>> Bazsi
> >>>
> >>>
> >>> On Thu, 2013-09-26 at 23:27 +1000, Paul Hutton wrote:
> >>>> Initially sent to the wrong address....
> >>>>
> >>>> Begin forwarded message:
> >>>>
> >>>>> From: Paul Hutton <paul_hutton at bigpond.com>
> >>>>>
> >>>>> Subject: Syslog NG simple filtering not working as expected...
> >>>>>
> >>>>> Date: 26 September 2013 11:23:15 PM AEST
> >>>>>
> >>>>> To: syslog-ng-request at lists.balabit.hu
> >>>>>
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>>
> >>>>> I am trying to forward network sourced syslogs to a number of
> >>>>> destinations depending on a few simple filters, but am immediately
> >>>>> coming unstuck. My constraint is that all incoming and outgoing
> >>>>> network syslog traffic is udp. My intention in the configuration
> >>>>> code below (apart from logging the localhost) is to:
> >>>>>
> >>>>>
> >>>>> 1. Forward all incoming network sourced syslogs except those
> >>>>> generated by two firewalls to destination Remote_L
> >>>>> 2. Forward incoming network syslogs that are generated by the
> >>>>> firewalls to destination Remote_C (currently commented out)
> >>>>> 3. Forward all incoming network syslogs of severity 0, 1 and 2 to
> >>>>> destinations Remote_D1 and Remote_D2 (currently commented out)
> >>>>>
> >>>>>
> >>>>> In addition all incoming network sourced syslogs are logged locally
> >>>>> to destination d_logger and I've added another local file d_debug
> >>>>> out of curiosity.
> >>>>>
> >>>>>
> >>>>> My problem is that with just the filter enabled at point 1 above and
> >>>>> in the code below, the behaviour of syslog-ng is very erratic,
> >>>>> beyond any reasonable anticipation of the unreliability of udp as
> >>>>> the transport. If I comment out the filter line
> >>>>>
> >>>>>
> >>>>> filter (not_firewalls);
> >>>>>
> >>>>>
> >>>>> in the code below, then test traffic (none of which is generated by
> >>>>> the firewall sources) is received and forwarded correctly as well
> >>>>> as logged locally. If I uncomment the filter, then I have seen test
> >>>>> traffic forwarded correctly on one occasion, but it is mostly the
> >>>>> case that some smaller subset (including none) of the test traffic
> >>>>> is forwarded. In this case, the test traffic is also correctly
> >>>>> written to the local destination d_debug. That is in the same rule
> >>>>> that fails to forward the test traffic.
> >>>>>
> >>>>>
> >>>>> I am running 3.2.4 on Solaris 10 09/10 x86 obtained from the
> >>>>> Sunfreeware site.
> >>>>>
> >>>>>
> >>>>> I have also unashamedly plagiarised configuration code from a couple
> >>>>> of posted sources, so if you recognise your work, thanks for putting
> >>>>> it out there...
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Thanks in advance.
> >>>>>
> >>>>>
> >>>>> Paul
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> @version: 3.2
> >>>>> @include "scl.conf"
> >>>>> options {
> >>>>> stats_level(3);
> >>>>> flush_lines(100);
> >>>>> flush_timeout(10000);
> >>>>> };
> >>>>> ################################################################
> >>>>> # Ensure syslog-ng logs this host similarly to
> >>>>> syslogd #
> >>>>> ################################################################
> >>>>> source s_local {
> >>>>> sun-streams("/dev/log" door("/etc/.syslog_door"));
> >>>>> internal();
> >>>>> };
> >>>>> #
> >>>>> # Set global network source
> >>>>> #
> >>>>> source src {
> >>>>> udp(ip("0.0.0.0") port (514));
> >>>>> };
> >>>>>
> >>>>> ###############################################################
> >>>>> # Local logfiles and remote destinations
> >>>>> setup #
> >>>>> ###############################################################
> >>>>> destination syslog {
> >>>>> file("/var/log/syslog");
> >>>>> };
> >>>>> destination messages {
> >>>>> file("/var/adm/messages");
> >>>>> };
> >>>>> destination loginlog {
> >>>>> file("/var/adm/loginlog");
> >>>>> };
> >>>>> #
> >>>>> # Set remote destinations
> >>>>> #
> >>>>> destination Remote_L {
> >>>>> udp ("10.octet.octet.octetL" port (514) );
> >>>>> };
> >>>>> destination Remote_D1 {
> >>>>> udp ("10.octet.octet.octetD1" port (514) spoof_source
> >>>>> (yes));
> >>>>> };
> >>>>> destination Remote_D2 {
> >>>>> udp ("10.octet.octet.octetD2" port (514) spoof_source
> >>>>> (yes));
> >>>>> };
> >>>>> destination Remote_C {
> >>>>> udp ("10.octet.octet.octetC" port (514) spoof_source (yes));
> >>>>> };
> >>>>> #
> >>>>> # Set local destination for all remote syslog input
> >>>>> #
> >>>>> destination d_logger {
> >>>>> file("/var/log/logger.log");
> >>>>> };
> >>>>> #
> >>>>> # Add another local debug file to check filters are working
> >>>>> #
> >>>>> destination d_debug {
> >>>>> file("/var/log/debug.log");
> >>>>> };
> >>>>> ##############################################################
> >>>>> # Filters for mimicking Solaris
> >>>>> syslogd #
> >>>>> # identifying firewall
> >>>>> syslogs #
> >>>>> # identifying severity levels 0 to
> >>>>> 2 #
> >>>>> ##############################################################
> >>>>> filter f_syslog {
> >>>>> facility(mail) or (facility(daemon) and priority(notice));
> >>>>> };
> >>>>> filter f_messages {
> >>>>> priority(err)
> >>>>> or facility(kern)
> >>>>> or (facility(user) and priority(err))
> >>>>> or (facility(daemon) and priority(notice))
> >>>>> or (facility(mail) and priority(crit));
> >>>>> };
> >>>>> filter f_auth {
> >>>>> facility (auth);
> >>>>> };
> >>>>>
> >>>>> #
> >>>>> # Set Firewall Filters
> >>>>> #
> >>>>> filter firewalls {
> >>>>> host("10.octet.octet.octetfw1") or
> >>>>> host("10.octet.octet.octetfw2");
> >>>>> };
> >>>>> filter not_firewalls {
> >>>>> not (host("10.octet.octet.octetfw1") or
> >>>>> host("10.octet.octet.octetfw2"));
> >>>>> };
> >>>>>
> >>>>> #
> >>>>> # Filter Severity Levels 0, 1 and 2
> >>>>> #
> >>>>> filter f_crit {
> >>>>> level (crit .. emerg);
> >>>>> };
> >>>>> ##############################################################
> >>>>> # Now do some logging #
> >>>>> ##############################################################
> >>>>>
> >>>>> #
> >>>>> # Log local syslog data appropriately
> >>>>> #
> >>>>> log {
> >>>>> source(s_local);
> >>>>> filter(f_syslog);
> >>>>> destination(syslog);
> >>>>> };
> >>>>> log {
> >>>>> source(s_local);
> >>>>> filter(f_messages);
> >>>>> destination(messages);
> >>>>> };
> >>>>> log {
> >>>>> source(s_local);
> >>>>> filter(f_auth);
> >>>>> destination(loginlog);
> >>>>> };
> >>>>>
> >>>>> #
> >>>>> # Log remotely sourced syslogs of all Severity Levels to
> >>>>> remote L system
> >>>>> # but exclude syslogs from the firewalls
> >>>>> #
> >>>>> log {
> >>>>> source (src);
> >>>>> filter (not_firewalls);
> >>>>> destination (Remote_L);
> >>>>> destination (d_debug);
> >>>>> };
> >>>>> #
> >>>>> # Log remotely sourced syslogs of Severity Levels 0 to 2 to remote D
> >>>>> systems
> >>>>> # including syslogs from the firewalls
> >>>>> #
> >>>>> #log {
> >>>>> # source (src);
> >>>>> # filter (f_crit);
> >>>>> # destination (Remote_D1);
> >>>>> # destination (Remote_D2);
> >>>>> #};
> >>>>> #
> >>>>> # Log remotely sourced syslogs from the two firewalls to remote
> >>>>> C system
> >>>>> #
> >>>>> #log {
> >>>>> #source (src);
> >>>>> #filter (firewalls);
> >>>>> # destination (Remote_C);
> >>>>> #};
> >>>>> #
> >>>>> # Log remotely sourced syslogs to a local file
> >>>>> #
> >>>>> log {
> >>>>> source (src);
> >>>>> destination (d_logger);
> >>>>> };
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>> ______________________________________________________________________________
> >>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> >>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>>
> >>>
> >>>
> >>>
> >>> ______________________________________________________________________________
> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> >>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>
> >>
> >
> > ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
More information about the syslog-ng
mailing list