I have syslog-ng running on a central logging host. Messages received are stored first in a per-host log and then in a per-facility log. The messages that go into the per-facility logs depend on filters that I define, but EVERYTHING from a host should appear in its per-host logs. I'm finding that many messages from some hosts are appearing in the per-service log but not in the per-host logs. Has anyone else experienced anything like this? Any solutions? TIA Here is my conf file: [aculver@slogr etc]$ cat syslog-ng.conf #$Author: rich $ #$Date: 2005/02/04 14:50:38 $ #$Id: syslog-ng.conf,v 1.8 2005/02/04 14:50:38 rich Exp $ #$Source: /uwo/src/etc/RCS/syslog-ng.conf,v $ #$Locker: $ # # Syslog-ng configuration file. # revised muchly by Andrew Culver <aculver at uwo dot ca> ############################################################### # First, set some global options. options { use_fqdn(yes); use_dns(yes); dns_cache(yes); keep_hostname(no); long_hostnames(off); sync(1); log_fifo_size(1024); owner(syslogda); group(syslogda); perm(0640); dir_owner(syslogda); dir_group(syslogda); dir_perm(0755); create_dirs(yes); stats(0); }; ############################################################### # # Define some sources to gather logging messages from # source s_main { pipe("/proc/kmsg"); unix-stream("/dev/log"); internal(); udp(port(514)); }; ############################################################### # # After that set destinations. # # the standard destination is to have a set of logs per host # ie: /syslogs/hosts/kam.its.uwo.ca/foolog/foolog destination d_default { file("/syslogs/hosts/$FULLHOST/$FACILITY/$FACILITY"); }; # collection of aggregate group logs destination d_console { file("/syslogs/services/console/console"); }; destination d_critical { file("/syslogs/services/critical/critical"); }; destination d_daemon { file("/syslogs/services/daemon/daemon"); }; destination d_login { file("/syslogs/services/login/login"); }; destination d_lpr { file("/syslogs/services/lpr/lpr"); }; destination d_mail { file("/syslogs/services/mail/mail"); }; destination d_misc { file("/syslogs/services/misc/misc"); }; destination d_routers { file("/syslogs/services/routers/routers"); }; destination d_named { file("/syslogs/services/named/named"); }; destination d_dhcp { file("/syslogs/services/dhcp/dhcp"); }; destination d_eventlog { file("/syslogs/services/eventlog/eventlog"); }; destination d_radius { file("/syslogs/services/radius/radius"); }; destination d_firewall { file("/syslogs/services/firewall/firewall"); }; destination d_loadbalancer { file("/syslogs/services/loadbalancer/loadbalancer"); }; destination d_netscreen { file("/syslogs/services/netscreen/netscreen"); }; destination d_ups { file("/syslogs/services/ups/ups"); }; destination d_bluesocket { file("/syslogs/services/bluesocket/bluesocket"); }; destination d_wireless { file("/syslogs/services/wireless/wireless"); }; destination d_dial { file("/syslogs/services/dial/dial"); }; ########################################## # # Here's the filter options. With these rules, we can set which # message go where. # ########################################## # First we'll define some sets of hosts which will be used in other filters # some routers/gateways filter f_hosts_router { host("^nebula\.wireless\.uwo\.ca$") or host("^(gate(1|2)|hse-rt1)\.netmgmt\.uwo\.ca$") or host("^(sup|msfc1|msfc2)\.l1uc00-1\.netmgmt\.uwo\.ca$") or host("^(sup|msfc1|msfc2)\.l1nsc2-2\.netmgmt\.uwo\.pri$"); }; # some firewalls filter f_hosts_firewall { host("^(FW|fw).*\.uwo\.(ca|pri)$") or host("^sgw\.ramp\.its\.uwo\.ca$") or host("^l1nsc2-2fwsm(1|2)\.netmgmt\.uwo\.pri$"); }; # some loadbalancers (.*.lb.its.uwo.ca) filter f_hosts_loadbalancer { host("^.*\.lb\.its\.uwo\.ca$"); }; # some dns servers filter f_hosts_named { host("^(romeo|hurdle)\.its\.uwo\.ca$") or host("^ns(1|2|3)\.uwo\.ca$"); }; # some dhcp servers filter f_hosts_dhcp { host("^(romeo|juliet)\.its\.uwo\.ca$") or host("^nebula\.wireless\.uwo\.ca$"); }; # some windows servers filter f_hosts_windows { host("^(ad4|cogs|elroy|mack|member2|nemo|nitrox|ntpsvcs|ntws(01|10|11|30)|opal|prof|rosie|ruby|winny|winter)\.uwo\.ca$") or host("^(amber|argon|beast|bigmac|cactus|casper|cosmos|crocus|jan|lily|magma|ntts(06|09)|onyx|rogue|storm|topaz|torch|weed1|weed2|weed3)\.its\.uwo\.ca$") or host("^(feb|koigu|manos)\.its\.uwo\.pri$") or host("^(testad1|testad3|smstest|member2|gecko\.frog)\.test\.uwo\.ca$") or host("^ntws(12|13)\.wisg\.its\.uwo\.pri$") or host("^(shop|mill|drill)\.ums\.uwo\.ca$") or host("^spider\.tel\.its\.uwo\.ca$") or host("^(SCCS50|dlink|docock)\.tel\.its\.uwo\.pri$"); }; # some radius servers # obsolete? see f_hosts_dial #filter f_hosts_radius { host("^uwonet-pm(1|2|3|4|5)\.netmgmt\.uwo\.ca$"); }; # some bluesocket servers filter f_hosts_bluesocket { host("^blusck(1|2|3|4)-pro\.wireless\.uwo\.ca$"); }; # some netscreen servers filter f_hosts_netscreen { host("^somenetscreenhost\.uwo\.ca$"); }; # some wireless devices filter f_hosts_wireless { host("^[Ww].*\..*\.wireless\.uwo\.pri$"); }; # some dialpool devices filter f_hosts_dial { host("^uwonet-pm(1|2|3|4|5)\.netmgmt\.uwo\.ca$"); }; ########################################## # Now setup some filters based on facility, level and hosts # routers is actually local 0 on the routers/gateways hosts filter f_routers { facility(local0) and filter(f_hosts_router); }; # console is a big one filter f_console1 { facility(daemon,auth) and level(notice .. emerg); }; # we'll exclude any kern.info from nebula/nexus filter f_console_kern1 { facility(kern) and level(info,warning) and host("^(nebula|nexus)\.wireless\.uwo\.ca$"); }; # kern and syslog except for kern.info from nebula/nexus filter f_console_kern { facility(syslog, kern) and not filter(f_console_kern1); }; # join the above two and also grab any err to emerg or any messages that would go to the routers log filter f_console2 { filter(f_console1) or level(err .. emerg) or filter(f_console_kern) or filter(f_routers); }; # now exclude anything from local 0, 3 and 7. filter f_console { filter(f_console2) and not facility(local0,local3,local7); }; # critical is anything of priority critical and above filter f_critical { level(crit .. emerg); }; # daemon is actually daemon+syslog+cron filter f_daemon { facility(daemon, syslog, cron); }; # login is actually auth and authpriv filter f_login { facility(auth, authpriv); }; # lpr is just itself filter f_lpr { facility(lpr); }; # mail is actually user filter f_mail { facility(mail); }; # misc is actually user filter f_misc { facility(user); }; # named is lactually local1 on named hosts filter f_named { facility(local1) and filter(f_hosts_named); }; # dhcp is actually local2 on dhcp hosts filter f_dhcp { facility(local2) and filter(f_hosts_dhcp); }; # eventlog is actually local3 on windows hosts filter f_eventlog { facility(local3) and filter(f_hosts_windows); }; # radius is actually local6 on radius hosts #filter f_radius { facility(local6) and # filter(f_hosts_radius); }; # firewall is actually local7 on firewall hosts filter f_firewall { facility(local7) and filter(f_hosts_firewall); }; # loadbalancer is actually local6 on loadbalancer hosts filter f_loadbalancer { facility(local6) and filter(f_hosts_loadbalancer); }; # netscreen is actually local7 on netscreen hosts filter f_netscreen { facility(local7) and filter(f_hosts_netscreen); }; # ups is actually local0 on tazmo filter f_ups { facility(local0) and host("^tazmo\.its\.uwo\.ca$"); }; # bluesocket is actually local0 on bluesocket hosts filter f_bluesocket { facility(local0) and filter(f_hosts_bluesocket); }; # wireless is actually local0 on wireless hosts filter f_wireless { facility(local0,local1) and filter(f_hosts_wireless); }; # dial is actually local6 on dial hosts filter f_dial { facility(local6) and filter(f_hosts_dial); }; ############################################################### # # log statements actually send logs somewhere, to a file, across the network, etc # ########################################## # First, log everything to host-specific logs before considering aggregate logs log { source(s_main); destination(d_default); }; ########################################## # Next log the aggregate "services" logs such as mail, routers, console, etc log { source(s_main); filter(f_console); destination(d_console); }; log { source(s_main); filter(f_critical); destination(d_critical); }; log { source(s_main); filter(f_daemon); destination(d_daemon); }; log { source(s_main); filter(f_login); destination(d_login); }; log { source(s_main); filter(f_lpr); destination(d_lpr); }; log { source(s_main); filter(f_mail); destination(d_mail); }; log { source(s_main); filter(f_misc); destination(d_misc); }; log { source(s_main); filter(f_routers); destination(d_routers); }; log { source(s_main); filter(f_named); destination(d_named); }; log { source(s_main); filter(f_dhcp); destination(d_dhcp); }; log { source(s_main); filter(f_eventlog); destination(d_eventlog); }; #log { source(s_main); filter(f_radius); destination(d_radius); }; log { source(s_main); filter(f_firewall); destination(d_firewall); }; log { source(s_main); filter(f_loadbalancer); destination(d_loadbalancer); }; log { source(s_main); filter(f_netscreen); destination(d_netscreen); }; log { source(s_main); filter(f_ups); destination(d_ups); }; log { source(s_main); filter(f_bluesocket); destination(d_bluesocket); }; log { source(s_main); filter(f_wireless); destination(d_wireless); }; log { source(s_main); filter(f_dial); destination(d_dial); }; ###############################################################
On 4/28/05, Andrew Culver <aculver@uwo.ca> wrote:
I have syslog-ng running on a central logging host. Messages received are stored first in a per-host log and then in a per-facility log. The messages that go into the per-facility logs depend on filters that I define, but EVERYTHING from a host should appear in its per-host logs.
I'm finding that many messages from some hosts are appearing in the per-service log but not in the per-host logs. Has anyone else experienced anything like this? Any solutions? TIA
Interesting problem. I notice that your per-host filters use pattern matches, but the per-service logs are matches against the log facility. Perhaps there is something about the messages or the pattern that causes some of the events not to match the host patterns? I have something similar, where I log messages to certain destinations based on the source host, and separately match on priority to send 'critical' messages to an additional destination. To accomplish this, I use a set of exact matches instead of regex patterns, so my filters look like filter f_somehost { host("host") or host("ip") }; Do you see any pattern in time or content or format for events that do not get written to the per-host logfile? Kevin Kadow
participants (2)
-
Andrew Culver
-
Kevin