[syslog-ng] syslog-ng is not reading level/facility properly

[Charlie Reddington]

Hi List,

After my last weeks note, I kept on doing some more digging. Perhaps  
some of this output will aid in finding a solution.

I compiled from source, from the latest stable 2.0 line thinking maybe  
that was part of my problem since I installed via the epel  
repositories. Nope, still having the same problem.

On a remote host, I have a nginx server, that is sending it's access  
logs to local5.notice and it's error logs to local5.crit. In the  
syslog.conf file I have the following entries.

local5.notice    @syslog-ng server
local5.crit     @syslog-ng server

This data is definitely making it to the systlog server. I have done a  
tcpdump and it shows this with the following. Also I have it logging  
to local log files and they are working the way I expect.

$ tcpdump port 514

tcpdump: verbose output suppressed, use -v or -vv for full protocol  
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
16:16:40.725473 IP CLIENTIP.syslog > SYSLOGNGIP.syslog: SYSLOG  
local5.notice, length: 304 <-- coming in as local5.notice as I expect.

Okay so they are hitting the server how I expect, but they aren't  
being filtered the way I expect. Below is my syslog-ng config.

# Syslog-ng config

options {
	use_dns(no);
	keep_hostname(yes);
	long_hostnames(off);
	chain_hostnames(off);
	sync(1);
	log_fifo_size(1024);
	create_dirs(yes);
	perm(0644);
	stats_freq(3600);
	mark_freq(600);
};

#
# Data Sources
#

# External Syslog
source s_external {
	udp(ip("0.0.0.0") port(514) flags(no_parse));
};


#
# Where to put Data
#

# Nginx / Lighttpd / Apache Access Logs
destination d_access_log { file("/data01/syslog-ng/$HOST/ 
access.log"); };

# Nginx / Lighttpd / Apache Error Logs
destination d_error_log { file("/data01/syslog-ng/$HOST/error.log"); };

# Everything else
destination d_log { file("/data01/syslog-ng/$HOST/$FACILITY.$LEVEL"); };

#
# Data Filters
#

# Facility local5
filter f_local5 { facility(local5); };

# Level info
filter f_info { level(info); };
# Level notice
filter f_notice { level(notice); };
# Level warn
filter f_warn { level(warn); };
# Level crit
filter f_crit { level(crit); };
# Level err
filter f_err { level(err); };

#
# Data Logging Locations
#

# Access Logs
log { source(s_external); filter(f_local5); filter(f_notice);  
destination(d_access_log); };

# Error Logs
log { source(s_external); filter(f_local5); filter(f_crit);  
destination(d_error_log); };

# Everything Else
log { source(s_external); destination(d_log); };

Now the only place that ANY messages seem to go, is into /data01/ 
syslog-ng/$HOST/kern.emerg.

Why is everything being marked as kern.emerg?

And lastly here's some debug output that maybe able to shed some light  
to those who know more than I do.

./sbin/syslog-ng -d -v -e --cfgfile=./etc/syslog-ng.conf

Running application hooks; hook='1'
syslog-ng starting up; version='2.0.11'

Incoming log entry; line='<156>logger: REMOTEIP www.domain.com - [08/ 
Sep/2010:17:17:44 -0400] "GET /2010/09/08-7912 HTTP/1.0" 200 83134 "-"  
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 
"\x0a'
Filter rule evaluation begins; filter_rule='f_local5'
Filter node evaluation result; filter_result='not-match',  
filter_type='facility'
Filter rule evaluation result; filter_result='not-match',  
filter_rule='f_local5'
Filter rule evaluation begins; filter_rule='f_local5'
Filter node evaluation result; filter_result='not-match',  
filter_type='facility'
Filter rule evaluation result; filter_result='not-match',  
filter_rule='f_local5'

For some reason this isn't matching, but I'm officially out of  
experience on this to know what to do further. I should also note that  
I'm running splunk on this syslog-ng server and it's able to get my  
access logs properly.

If anyone can throw me a bone here I'd really appreciate it, many  
thanks in advance.

Charlie