[syslog-ng]problem with multiply-defined source/filter/destin ation combinations?

[Hamilton, Andrew]

Well.  That was a hefty message.  Anyway, syslog-ng will typically log
messages only once unless you tell it otherwise.  This is the default
behavior.  You may think that by adding another log line that sends the same
message to a different destination you are telling it to do just that.  That
is incorrect.  Once a message is logged it is then forgotten in a sense.
The way to do this is to use multiple destinations on the same line.  (i.e.,
log { source(src); filter(filter1); destination(d1); destination(d2); }; ).
This works well for me, I have been doing it for nearly 4 years.  Syslog-ng
will do exactly what you want to do you just have to tweak it a little.  I
have never done this before but in theory, and I guess someone else from the
list can correct me if I'm wrong, you could write a log line from multiple
sources as well. (i.e., log { source(s1); source(s2); filter(f1);
destination(d1); destination(d2); }; ) etc...  I think you can probably get
the picture.  That might help you clean up your config file a little. I hope
this helps.

Regards,

Drew

-----Original Message-----
From: Jay Goldberg [mailto:jg@open.ch]
Sent: Monday, January 13, 2003 4:34 AM
To: syslog-ng@lists.balabit.hu
Subject: [syslog-ng]problem with multiply-defined
source/filter/destination combinations?


Hi all

I've just encountered a very strange problem with syslog-ng after
modifying our standard syslog-ng.conf file, and hopefully someone
on the group has seen this problem.

Background: We've created a fairly complex system for managing logs
on hosts in various locations.  We have log collectors all running
syslog-ng, and we're now using an Event-to-Syslog package that we
have configured to send Windows Event Log messages to our log
collectors.  The log collectors use an external Perl script which
filters out 'uninteresting' messages and uses our own reliable
encrypted transport mechanism to send the logs to our in-house
management systems.

The Windows messages come in on local3.  The external Perl script
sends messages on local6.  The intention of the configuration is:

  - local6 gets written to its own logfile so that our system doesn't
    get into any message loops if it encounters problems.
  - everything else is written to its appropriate logfile on disk AND
    sent through this external Perl program.

Before modifying our syslog-ng.conf file to send messages to the
external program, we saw all messages written to their appropriate
logfiles on disk.  After modifying the syslog-ng.conf, we see that
the external script is working just fine, but the messages received
on UDP are no longer being written to disk; they are only being sent
to this external Perl script.

I've implemented a workaroud, but I don't know if I've done something
wrong, or if there's perhaps a bug in syslog-ng (we're running 1.5.22
on Solaris 8).

Here's the original syslog-ng.conf:

================== start syslog-ng.conf, ORIGINAL =====================
options { sync (0);
           owner(syslogd);
           group(syslogd);
           time_reopen (10);
           log_fifo_size (1000);
           long_hostnames (off);
           use_dns (no);
           use_fqdn (no);
           create_dirs (no);
           keep_hostname (yes);
         };

source s_sys { sun-streams ("/dev/log" door("/etc/.syslog_door"));
internal(); };
source s_udp { udp(); };

destination d_cons { file("/dev/console"); };
destination d_smsg { file("/dev/sysmsg"); };
destination d_udp  { udp("loghost" port(514)); };
destination d_mlop { usertty("operator"); };
destination d_mlrt { usertty("root"); };
#
#  the following are scripts using cronolog to control logfile rotation
#  only so we can maintain symlinks to the 'traditional' logfile locations
#
destination d_mesg { program("/opt/OSAGslog/bin/c_messages"); };
destination d_sysl { program("/opt/OSAGslog/bin/c_syslog"); };
destination d_mail { program("/opt/OSAGslog/bin/c_maillog"); };
destination d_auth { program("/opt/OSAGslog/bin/c_authlog"); };
destination d_clog { program("/opt/OSAGslog/bin/cronolog"); };

filter f_auth   { facility(auth); };
filter f_cron   { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern   { facility(kern); };
filter f_lpr    { facility(lpr); };
filter f_mail   { facility(mail); };
filter f_news   { facility(news); };
filter f_user   { facility(user); };
filter f_uucp   { facility(uucp); };
filter f_local0 { facility(local0); };
filter f_local1 { facility(local1); };
filter f_local2 { facility(local2); };
filter f_local3 { facility(local3); };
filter f_local4 { facility(local4); };
filter f_local5 { facility(local5); };
filter f_local6 { facility(local6); };
filter f_local7 { facility(local7); };

filter f_mesg   { facility(cron) or facility(daemon) or facility(kern) or
                   facility(lpr) or facility(news) or facility(user) or
                   facility(uucp) or facility(local0) or facility(local1) or
                   facility(local2) or facility(local3) or facility(local4)
or
                   facility(local5) or facility(local7); };

#
#  Log Directives
#
#  We log AUTH to authlog, MAIL to syslog, everything
#  else to messages.
#
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_udp); filter(f_auth); destination(d_auth); };

log { source(s_sys); filter(f_mail); destination(d_sysl); };
log { source(s_udp); filter(f_mail); destination(d_sysl); };

log { source(s_sys); filter(f_mesg); destination(d_mesg); };
log { source(s_udp); filter(f_mesg); destination(d_mesg); };

================== end syslog-ng.conf, ORIGINAL =====================


...here is the addition that we made to the end of the syslog-ng.conf
to send logs to our Perl script called 'gumafilter':

================== start additions for 'gumafilter' =====================

source sg_global {
     sun-streams ("/dev/log" door("/etc/.syslog_door"));
     internal();
     udp(); };

#
#  c_gumalog just uses cronolog to control logfile rotation for
#  messages generated by the 'gumafilter' script.
#
destination dg_filt { program("/opt/OSAGguma/bin/gumafilter"); };
destination dg_log { program("/opt/OSAGslog/bin/c_gumalog"); };

filter fg_local6  { facility(local6); };

filter fg_all  { facility(auth) or facility(mail) or
     facility(cron) or facility(daemon) or facility(kern) or
     facility(syslog) or
     facility(lpr) or facility(news) or facility(user) or
     facility(uucp) or facility(local0) or facility(local1) or
     facility(local2) or facility(local3) or facility(local4) or
     facility(local5) or facility(local7); };

#  This guma sensor accepts remote messages (from guma.conf)
log { source(sg_global); filter(fg_all); destination(dg_filt); };
log { source(sg_global); filter(fg_local6); destination(dg_log); };

================== end additions for 'gumafilter' =====================

The workaround is to add the following line at the bottom of the
syslog-ng.conf file, and I don't know why it's needed!

   log { source(sg_global); filter(f_mesg); destination(d_mesg); };

Without this line, the Perl script 'gumafilter' works correctly, but
none of the logs received on UDP get written to disk in the messages
file, which I would have expected based on the lines:

   log { source(s_sys); filter(f_mesg); destination(d_mesg); };
   log { source(s_udp); filter(f_mesg); destination(d_mesg); };

Any ideas?

...sorry for the long message!

	Thanks in advance,

		Jay

-- 
jay goldberg
.
open systems ag
raeffelstrasse 29
jg@open.ch


_______________________________________________
syslog-ng maillist  -  syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html