Re: [syslog-ng] Problems With Filter Rules - Using First Rule, Not One Intended

8 May 2008

      Joe;

Many thanks for the fast reply!

My problems lie with the other filters, the ones at the end:

filter F_edge         { 
host("edge*") or host("122.21.*"); };
filter 
F_router       { host("gw*") or host("rtr") or 
host("mmsc"); };
filter F_switch       { 
host("sw*") or host("sw1") or host("sw2"); };
filter 
F_firewall     { host("^fw*") or host("^mlm*-*") or 
host("^cm*"); };
filter 
F_dc           { 
host("^mydc*") or host("^dc*"); };
filter F_accesspoints { host("^melanie*"); 
};
filter F_mailservers  { host("^mail*") or host("^smtpgw*"); 
};
filter F_proxies      { host("^proxygw*"); 
};
filter F_InternetIP   { host("161.17.10.*"); 
};

The above, based on the filter rule for F_mailservers, should place anything coming in from a host named mailserver1, or smtpgw1 into destination D_mailservers, which in turn should save logs into file named /var/log/MyHosts/MailServers/$FULLHOST.log.  Instead I find those logs in /var/log/MyHosts/Switches/$FULLHOST.log (which is really /var/log/MyHosts/Switches/mailserver1.mycorp.net/mailserver1.mycorp.net.log)

I need to figure out a way to write the differences for hosts that begin with pattern xxx (^xxx)? and those with xxx at the end (*xxx) and those with xxx in the middle (*xxx)?, and for the life of me, I can't fifure out why the above is sending into Switches :-(

Sorry if I am confusing this matter, and the reason for the overcomplication on my config is largely due to me being very new at this great tool.

Thanks!

.vp

From: Joe.Fegan@hp.com
To: syslog-ng@lists.balabit.hu
Date: Thu, 8 May 2008 16:08:03 +0000
Subject: Re: [syslog-ng] Problems With Filter Rules - Using First Rule, Not One Intended

Your config seems a bit odd and maybe overly complex. 
Here's how I read it, please let us know if this is how you intend it to 
work.

Your sources:

S_dgram = only messages from local processes that call 
syslog()
S_internal = only internal messages from syslog-ng on this 
node
S_kernel = only messages from the kernel on this 
node
S_tcp = only TCP messages from other nodes
S_udp = only UDP messages from other 
nodes

Your log paths tell syslog-ng to behave as:

log { source(S_udp); 
destination(D_db_mysql); };

All UDP messages from other nodes should be sent to 
mysql.

log { 
source(S_udp); destination(D_sec); };

All UDP messages from other nodes should be sent to 
D_sec.

log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      
filter(F_auth);      destination(D_authlog); 
flags(final); };

All messages matching F_auth and coming either from local 
syslog(), local syslog-ng internal or via TCP from other nodes (but 
not via UDP) should be sent to D_authlog. If you send a message down this 
path then don't bother evaluating any other 
paths.

log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_local7);    
destination(D_bootlog); flags(final); };

All messages matching F_local7 and coming either from 
local syslog(), local syslog-ng internal or via TCP from other nodes 
(but not via UDP) should be sent to D_bootlog. If you send a message down 
this path then don't bother evaluating any other paths.

Then there's a bunch of 
other log paths in the same vein.

From: syslog-ng-bounces@lists.balabit.hu 
[mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of 
wiskbroom@hotmail.com
Sent: 08 May 2008 15:13
To: 
Syslog-ng users' and developers' mailing list
Subject: [syslog-ng] 
Problems With Filter Rules - Using First Rule, Not One 
Intended

Greetings;

My setup works well with one exception, my 
filtering rules contained in my syslog-ng.conf do not appear to work 
properly.  My logs are not lost, instead they end up in a directory which I 
did not intend them to be in. 

Background: I log to a MySql DB, flatfiles 
and finally, to SEC, which parses stuff and takes various actions (almost 
working ;-)

For ease of reading, I will simply add the contents of my 
config file which pertains to just one filter.
Many thanks in advance for taking the time to read and help 
me.

.vp

############
# OPTIONS  
#
############

options
  {

chain_hostnames(no);
    create_dirs 
(yes);
    dir_perm(0755);
    use_dns 
(yes);
    dns_cache(yes);

dns_cache_size(1000);

dns_cache_expire(604800);

keep_hostname(yes);

log_fifo_size(10000);

log_msg_size(8192);

long_hostnames(on);
    perm(0644);

stats(3600);
    sync(0);
    time_reopen 
(10);
    use_dns(yes);

use_fqdn(yes);
  };

############
# 
SOURCES       #
############

source 
S_dgram
  { unix-dgram("/dev/log"); };

source 
S_internal
  { internal(); };

source S_kernel
  { 
file("/proc/kmsg" log_prefix("kernel: ")); };

source S_tcp
  { 
tcp(port(4800) keep-alive(yes) max_connections(100)); };

source S_udp { 
udp(ip("0.0.0.0") port(514)); };

###############
# DEST SQL 
DB         
#
###############

destination D_db_mysql 
{

pipe("/var/log/mysql.pipe"

template("INSERT INTO 
logs

(host, facility, priority, level, tag, datetime, program, 
msg)

VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY 
$HOUR:$MIN:$SEC',

'$PROGRAM', '$MSG' );\n") 
template-escape(yes));
};

##############
# 
DESTINATIONS     #
##############

destination 
D_authlog       { file("/var/log/auth.log"); 
};
destination D_bootlog       { 
file("/var/log/boot.log"); };
destination 
D_debug         { 
file("/var/log/debug"); };
destination 
D_explan        { 
file("/var/log/explanations"); };
destination D_messages   { 
file("/var/log/messages"); };
destination 
D_secure        { file("/var/log/secure"); 
};
destination D_spooler       { 
file("/var/log/spooler"); };
destination 
D_syslog        { file("/var/log/syslog"); 
};
destination 
D_user           { 
file("/var/log/user.log"); };

destination 
D_switch       { 
file("/var/log/MyHosts/Switches/$FULLHOST.log"

perm(0644)); };
destination 
D_edge         { 
file("/var/log/MyHosts/EdgeDevices/$FULLHOST.log"

perm(0644)); };
destination D_firewall     { 
file("/var/log/MyHosts/Firewalls/$FULLHOST.log"

owner(root) group(root) perm(0644) dir_perm(0700) create_dirs(yes)); 
};
destination D_router       { 
file("/var/log/MyHosts/Routers/$FULLHOST.log"

perm(0644)); };
destination D_accesspoints { 
file("/var/log/MyHosts/AccessPoints/$FULLHOST.log"

perm(0644)); };
destination D_mailservers  { 
file("/var/log/MyHosts/MailServers/$FULLHOST.log"

perm(0644)); };

###########
# 
FILTERS        
#
###########

filter      
F_auth         { facility(auth, 
authpriv); };
filter      
F_authpriv     { facility(authpriv); 
};
filter      
F_cron         { facility(cron); 
};
filter      
F_daemon       { facility(daemon); 
};
filter      
F_kern         { facility(kern); 
};
filter      
F_local1       { facility(local1); 
};
filter      
F_local2       { facility(local2); 
};
filter      
F_local3       { facility(local3); 
};
filter      
F_local4       { facility(local4); 
};
filter      
F_local5       { facility(local5); 
};
filter      
F_local6       { facility(local6); 
};
filter      
F_local7       { facility(local7); 
};
filter      
F_lpr          { facility(lpr); 
};
filter      
F_mail         { facility(mail); 
};
filter      F_messages     { 
facility(daemon, kern, user); };
filter      
F_news         { facility(news); 
};
filter      
F_spooler      { facility(uucp,news) and level(crit); 
};
filter      
F_syslog       { not facility(auth, authpriv) and 
not facility(mail); };
filter      
F_user         { facility(user); 
};

filter      
F_crit         { level(crit); 
};
filter      
F_debug        { level(debug); 
};
filter      F_emergency    { 
level(emerg); };
filter      
F_err          { level(err); 
};
filter      
F_info         { level(info); 
};
filter      
F_notice       { level(notice); 
};
filter      
F_warn         { level(warn); 
};

filter F_edge         { 
host("edge*") or host("122.21.*"); };
filter 
F_router       { host("gw*") or host("rtr") or 
host("mmsc"); };
filter F_switch       { 
host("sw*") or host("sw1") or host("sw2"); };
filter 
F_firewall     { host("^fw*") or host("^mlm*-*") or 
host("^cm*"); };
filter 
F_dc           { 
host("^mydc*") or host("^dc*"); };
filter F_accesspoints { host("^melanie*"); 
};
filter F_mailservers  { host("^mail*") or host("^smtpgw*"); 
};
filter F_proxies      { host("^proxygw*"); 
};
filter F_InternetIP   { host("161.17.10.*"); 
};

##############
#    
LOGS                
#
##############

log { source(S_udp); destination(D_db_mysql); 
};

# Send ALL logs to SEC

# log { source(S_dgram); 
source(S_internal); source(S_tcp); filter(F_auth); destination(D_sec); 
};

# log { source(S_udp); source(S_tcp); destination(D_sec); 
};

log { source(S_udp); destination(D_sec); };

###

log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      
filter(F_auth);      destination(D_authlog); 
flags(final); };
log { source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_local7);    
destination(D_bootlog); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_local1);    
destination(D_explan); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_local5);    
destination(D_router); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_messages);  
destination(D_messages); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_authpriv);  
destination(D_secure); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      filter(F_spooler);   
destination(D_spooler); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_kernel);

source(S_tcp);      filter(F_syslog);    
destination(D_syslog); flags(final); };
log { 
source(S_dgram);

source(S_internal);

source(S_tcp);      
filter(F_user);      destination(D_user); flags(final); 
};

log { source(S_dgram);

source(S_internal);

source(S_kernel);

source(S_tcp);                           
destination(D_hosts); flags(final); };

log { source(S_udp); 
filter(F_switch); destination(D_switch); flags(final); };
log { 
source(S_udp); filter(F_firewall); destination(D_firewall); flags(final); 
};
log { source(S_udp); filter(F_router); destination(D_router); 
flags(final); };
log { source(S_udp); filter(F_edge); destination(D_edge); 
flags(final); };
log { source(S_udp); filter(F_dc); destination(D_dc); 
flags(final); };
log { source(S_udp); filter(F_accesspoints); 
destination(D_accesspoints); flags(final); };
log { source(S_udp); 
filter(F_proxies); destination(D_proxies); flags(final); };
log { 
source(S_udp); filter(F_mailservers); destination(D_mailservers); flags(final); 
};

log { source(S_udp); destination(D_udp);};

<!--[if 
!supportLineBreakNewLine]-->
<!--[endif]-->

Re: [syslog-ng] Problems With Filter Rules - Using First Rule, Not One Intended

wiskbroom＠hotmail.com