Getting Logs in Triplicate
I'm trying to parse cron, mail, and everything else into 3 files but getting 3 copies of each log entry. I also want copies of each log message to be placed in a subdirectory based on each hostname. (So I should have 2 copies of each log message, one in the appropriate master file and then one in the appropriate file under a directory with same name as the source host) Here is my conf: <snip> # Catch All Filter For Mail and Everything Else filter f_master_syslogs { not facility(mail) and not facility(cron); }; filter f_master_maillogs { facility(mail); }; filter f_master_cronlogs { facility(cron); }; destination master_syslogs { file("/var/log/syrrx/master_syslog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination master_maillogs { file("/var/log/syrrx/master_maillog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination master_cronlogs { file("/var/log/syrrx/master_cronlog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination host_logs { file("/var/log/syrrx/$HOST/syslog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination host_mail { file("/var/log/syrrx/$HOST/maillog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); } destination console { file("/dev/tty10"); }; log { source(src); filter(f_console); destination(console); }; destination xconsole { pipe("/dev/xconsole"); }; log { source(src); filter(f_console); destination(xconsole); }; destination root { usertty("root"); }; log { source(src); filter(f_alert); destination(root); }; # Catch All log { source(src); filter(f_master_syslogs); destination(master_syslogs); }; log { source(src); filter(f_master_maillogs); destination(master_maillogs); }; log { source(src); filter(f_master_cronlogs); destination(master_cronlogs); }; # Host Based Log Files log { source(src); filter(f_file-0202); destination(host_logs); }; log { source(src); filter(f_file-0201); destination(host_logs); }; .... log { source(src); filter(f_db-0201_mail); destination(host_mail); }; log { source(src); filter(f_db-0202_mail); destination(host_mail); }; .... Currently I get 3 copies of each log in both places. (the master log and then the one under the host's subdirectory) Any ideas what I missed? ===== Chuck Carson - Sr. Systems Engineer Syrrx, Inc. - www.syrrx.com 10410 Science Center Drive San Diego, CA 92121 Work: 858.622.8528 Fax: 858.550.0526
On Mon, Jan 17, 2005 at 06:52:44AM -0800, Rhugga wrote:
I'm trying to parse cron, mail, and everything else into 3 files but getting 3 copies of each log entry. I also want copies of each log message to be placed in a subdirectory based on each hostname. (So I should have 2 copies of each log message, one in the appropriate master file and then one in the appropriate file under a directory with same name as the source host) ....
Currently I get 3 copies of each log in both places. (the master log and then the one under the host's subdirectory)
Any ideas what I missed?
I want to help but I'm not sure what else is in the config file, go ahead and post the whole thing, or it it's really big post a URL to it. -- Nate "Man is the only animal that blushes. Or needs to." - Samuel Clemens
Getting 5 copies of each message. (was getting only 3 before, but now getting 5 copies of each log message) Here is my entire config file: options { long_hostnames(off); sync(0); }; source src { internal(); unix-dgram("/dev/log"); unix-dgram("/var/lib/dhcp/dev/log"); unix-dgram("/var/lib/named/dev/log"); unix-dgram("/var/lib/ntp/dev/log"); unix-dgram("/var/lib/stunnel/dev/log"); udp(ip("0.0.0.0") port(514)); }; filter f_console { level(warn) and facility(kern) or level(err) and not facility(authpriv); }; filter f_newsnotice { level(notice) and facility(news); }; filter f_newscrit { level(crit) and facility(news); }; filter f_newserr { level(err) and facility(news); }; filter f_news { facility(news); }; filter f_mail { facility(mail); }; filter f_cron { facility(cron); }; filter f_warn { level(warn, err, crit); }; filter f_alert { level(alert); }; filter f_messages { not facility(news, mail); }; filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7); }; filter f_iptables { facility(kern) and match("IN=") and match("OUT="); }; # Everything except mail subsystem logs filter f_db-0201 { host("db-0201") and not facility(mail); }; filter f_db-0202 { host("db-0202") and not facility(mail); }; filter f_db-0203 { host("db-0203") and not facility(mail); }; filter f_db-0204 { host("db-0204") and not facility(mail); }; filter f_db-0205 { host("db-0205") and not facility(mail); }; filter f_web-0201 { host("web-0201") and not facility(mail); }; filter f_web-0202 { host("web-0202") and not facility(mail); }; filter f_web-0205 { host("web-0205") and not facility(mail); }; filter f_web-2206 { host("web-2206") and not facility(mail); }; filter f_back-0202 { host("back-0202") and not facility(mail); }; filter f_file-0201 { host("file-0201") and not facility(mail); }; filter f_file-0202 { host("file-0202") and not facility(mail); }; filter f_syslog { host("syslog") and not facility(mail); }; filter f_hurley { host("hurley") and not facility(mail); }; # Only mail subsystem logs filter f_db-0201_mail { host("db-0201") and facility(mail); }; filter f_db-0202_mail { host("db-0202") and facility(mail); }; filter f_db-0203_mail { host("db-0203") and facility(mail); }; filter f_db-0204_mail { host("db-0204") and facility(mail); }; filter f_db-0205_mail { host("db-0205") and facility(mail); }; filter f_web-0201_mail { host("web-0201") and facility(mail); }; filter f_web-0202_mail { host("web-0202") and facility(mail); }; filter f_web-0205_mail { host("web-0205") and facility(mail); }; filter f_web-2206_mail { host("web-2206") and facility(mail); }; filter f_file-0201_mail { host("file-0201") and facility(mail); }; filter f_file-0202_mail { host("file-0202") and facility(mail); }; filter f_back-0202_mail { host("back-0202") and facility(mail); }; filter f_hurley_mail { host("hurley") and facility(mail); }; filter f_syslog_mail { host("syslog") and facility(mail); }; # Catch All Filter For Mail and Everything Else filter f_master_syslogs { not facility(mail) and not facility(cron); }; filter f_master_maillogs { facility(mail); }; filter f_master_cronlogs { facility(cron); }; destination console { file("/dev/tty10"); }; log { source(src); filter(f_console); destination(console); }; destination xconsole { pipe("/dev/xconsole"); }; log { source(src); filter(f_console); destination(xconsole); }; destination root { usertty("root"); }; log { source(src); filter(f_alert); destination(root); }; # Chuck - 14-JAN-2005 destination master_syslogs { file("/var/log/syrrx/master_syslog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination master_maillogs { file("/var/log/syrrx/master_maillog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination master_cronlogs { file("/var/log/syrrx/master_cronlog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination host_logs { file("/var/log/syrrx/$HOST/syslog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; destination host_mail { file("/var/log/syrrx/$HOST/maillog", owner(root), group(root), perm(0640), dir_perm(0750), create_dirs(yes), ); }; # Catch All log { source(src); filter(f_master_syslogs); destination(master_syslogs); }; log { source(src); filter(f_master_maillogs); destination(master_maillogs); }; log { source(src); filter(f_master_cronlogs); destination(master_cronlogs); }; # Host Based Log Files log { source(src); filter(f_file-0202); destination(host_logs); }; log { source(src); filter(f_file-0201); destination(host_logs); }; log { source(src); filter(f_db-0201); destination(host_logs); }; log { source(src); filter(f_db-0202); destination(host_logs); }; log { source(src); filter(f_db-0203); destination(host_logs); }; log { source(src); filter(f_db-0204); destination(host_logs); }; log { source(src); filter(f_db-0205); destination(host_logs); }; log { source(src); filter(f_web-0201); destination(host_logs); }; log { source(src); filter(f_web-0202); destination(host_logs); }; log { source(src); filter(f_web-0205); destination(host_logs); }; log { source(src); filter(f_web-2206); destination(host_logs); }; log { source(src); filter(f_back-0202); destination(host_logs); }; log { source(src); filter(f_hurley); destination(host_logs); }; log { source(src); filter(f_db-0201_mail); destination(host_mail); }; log { source(src); filter(f_db-0202_mail); destination(host_mail); }; log { source(src); filter(f_db-0203_mail); destination(host_mail); }; log { source(src); filter(f_db-0204_mail); destination(host_mail); }; log { source(src); filter(f_db-0205_mail); destination(host_mail); }; log { source(src); filter(f_web-0201_mail); destination(host_mail); }; log { source(src); filter(f_web-0202_mail); destination(host_mail); }; log { source(src); filter(f_web-0205_mail); destination(host_mail); }; log { source(src); filter(f_web-2206_mail); destination(host_mail); }; log { source(src); filter(f_file-0201_mail); destination(host_mail); }; log { source(src); filter(f_file-0202_mail); destination(host_mail); }; log { source(src); filter(f_syslog_mail); destination(host_mail); }; log { source(src); filter(f_hurley_mail); destination(host_mail); }; destination newscrit { file("/var/log/news/news.crit"); }; log { source(src); filter(f_newscrit); destination(newscrit); }; destination newserr { file("/var/log/news/news.err"); }; log { source(src); filter(f_newserr); destination(newserr); }; destination newsnotice { file("/var/log/news/news.notice"); }; log { source(src); filter(f_newsnotice); destination(newserr); }; destination mail { file("/var/log/mail"); }; log { source(src); filter(f_mail); destination(mail); }; destination localmessages { file("/var/log/localmessages"); }; log { source(src); filter(f_local); destination(localmessages); }; destination messages { file("/var/log/messages"); }; log { source(src); filter(f_messages); destination(messages); }; destination warn { file("/var/log/warn"); }; log { source(src); filter(f_warn); destination(warn); }; I am using "logger -p user.err test message" from a Solaris 9 system. This message is getting logged 5 times to /var/log/syrrx/master_syslogs and also /var/log/syrrx/$HOST/syslog. The syslog system is SLES 9 with all updates. On the Solaris client, here is the syslog config: # BEGIN syslog.conf *.err;kern.notice;auth.notice /dev/sysmsg *.err;kern.debug;daemon.notice;mail.crit /var/adm/messages *.alert;kern.err;daemon.err operator *.alert root *.emerg * mail.debug ifdef(`LOGHOST', /var/log/syslog, @loghost) ifdef(`LOGHOST', , user.err /dev/sysmsg user.err /var/adm/messages user.alert `root, operator' user.emerg * ) # BEGIN RAID Manager additions # DO NOT EDIT from BEGIN above to END below... user.err /dev/console # END RAID Manager additions # To syslog host *.debug @syslog *.info @syslog *.notice @syslog *.warning @syslog *.err @syslog *.crit @syslog *.alert @syslog *.emerg @syslog # END syslog.conf loghost is an alias to localhost and NOT the syslog box. Any ideas? Thx, CC ===== Chuck Carson - Sr. Systems Engineer Syrrx, Inc. - www.syrrx.com 10410 Science Center Drive San Diego, CA 92121 Work: 858.622.8528 Fax: 858.550.0526
On Tue, Jan 18, 2005 at 06:46:33AM -0800, Rhugga wrote: Hi
Getting 5 copies of each message. (was getting only 3 before, but now getting 5 copies of each log message)
I think the main culprit is your syslogd.conf on the Solaris machine:
# To syslog host *.debug @syslog *.info @syslog *.notice @syslog *.warning @syslog *.err @syslog *.crit @syslog *.alert @syslog *.emerg @syslog
syslog.conf(5) on Linux says if you put priority P in a rule everything with priority >= P will be logged. Check your syslog manual. <man page> The behavior of the original BSD syslogd is that all messages of the specified priority and higher are logged according to the given action. </man page> Confirmed this with OpenBSD syslog. If I have *.* @loghost *.debug @loghost messages with priority >= debug are sent over the wire twice. If this is the case you can collapse your above statements to *.* @loghost to get only one copy of each msg.
Here is my entire config file: [...]
Just nitpicking but I think you could collapse most of your syslog-ng.conf if you took out the 'host("xyz")' out of the filters. Since they all go from the same source() to the same destination() with $HOST expansion they don't really accomplish anything. -- Wolfgang Braun, Dipl.-Inform. (FH) <wolfgang.braun@gmx.de> gpg-key: 1024D/4B32CE55 gpg-fingerprint: 7F0F DE82 94A5 B476 0E08 4972 AC95 31A3 4B32 CE55
Solaris doesn't accept *.* notation, the only wildcard can be for the facility. *.err <--- ok cron.* <--- bad Also, when I test with logger, I only get 1 local copy, not 5. I also forgot to mention that the syslog.conf is the last version I tested with, other versions did not use the same notation. So let me understand what you wrote: If I use 'logger -p user.err my test message' your saying this is getting logged at multiple priorities and/or facilies? Or do you mean: user.err /some/file user.crit /some/file Do you mean this syslog config will cause the previous logger statement to log twice? If so, that would make sense, but I don't think the problem I am seeing is caused by this. Ugh, I'm at a loss and the documentation is severely limiting. Any ideas? --- Wolfgang Braun <wolfgang.braun@gmx.de> wrote:
On Tue, Jan 18, 2005 at 06:46:33AM -0800, Rhugga wrote:
Hi
Getting 5 copies of each message. (was getting only 3 before, but now getting 5 copies of each log message)
I think the main culprit is your syslogd.conf on the Solaris machine:
# To syslog host *.debug @syslog *.info @syslog *.notice @syslog *.warning @syslog *.err @syslog *.crit @syslog *.alert @syslog *.emerg @syslog
syslog.conf(5) on Linux says if you put priority P in a rule everything with priority >= P will be logged. Check your syslog manual.
<man page>
The behavior of the original BSD syslogd is that all messages of the specified priority and higher are logged according to the given action.
</man page>
Confirmed this with OpenBSD syslog. If I have
*.* @loghost *.debug @loghost
messages with priority >= debug are sent over the wire twice.
If this is the case you can collapse your above statements to
*.* @loghost
to get only one copy of each msg.
Here is my entire config file: [...]
Just nitpicking but I think you could collapse most of your syslog-ng.conf if you took out the 'host("xyz")' out of the filters. Since they all go from the same source() to the same destination() with $HOST expansion they don't really accomplish anything.
-- Wolfgang Braun, Dipl.-Inform. (FH) <wolfgang.braun@gmx.de> gpg-key: 1024D/4B32CE55 gpg-fingerprint: 7F0F DE82 94A5 B476 0E08 4972 AC95 31A3 4B32 CE55 _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
===== Chuck Carson - Sr. Systems Engineer Syrrx, Inc. - www.syrrx.com 10410 Science Center Drive San Diego, CA 92121 Work: 858.622.8528 Fax: 858.550.0526
On Tue, Jan 18, 2005 at 09:14:49AM -0800, Rhugga wrote:
So let me understand what you wrote:
If I use 'logger -p user.err my test message' your saying this is getting logged at multiple priorities and/or facilies?
No, but it might get logged by different rules like *.err @loghost user.err @loghost This would log your test message twice since syslogd doesn't stop at the first match.
Or do you mean:
user.err /some/file user.crit /some/file
Do you mean this syslog config will cause the previous logger statement to log twice? If so, that would make sense, but I don't think the problem I am seeing is caused by this.
Your example would get logged once I think -- but you got the idea, yes.
Ugh, I'm at a loss and the documentation is severely limiting.
Any ideas?
Try this on your Solaris box: in syslog.conf: user.debug /var/log/experimental user.info /var/log/experimental user.notice /var/log/experimental user.warning /var/log/experimental user.err /var/log/experimental user.crit /var/log/experimental user.alert /var/log/experimental user.emerg /var/log/experimental touch /var/log/experimental kill -HUP syslogd then # logger -p user.debug 'this gets logged once (1)' (matches user.debug) # logger -p user.info 'this gets logged twice (2)' (matches user.debug and user.info) # logger -p user.alert 'this gets logged (7) times' (matches user.debug, ... user.alert) If it doesn't then I'm wrong obviously ;) -- Wolfgang Braun, Dipl.-Inform. (FH) <wolfgang.braun@gmx.de> gpg-key: 1024D/4B32CE55 gpg-fingerprint: 7F0F DE82 94A5 B476 0E08 4972 AC95 31A3 4B32 CE55
: Solaris doesn't accept *.* notation, the only wildcard can be for the facility. : *.err <--- ok : : cron.* <--- bad However, if you use a wildcard for a facility, it will also log everything above that facility. So, *.info Will send all log messages to that stream. - Aakin : : Also, when I test with logger, I only get 1 local copy, not 5. : : I also forgot to mention that the syslog.conf is the last version I tested with, other versions : did not use the same notation. : : So let me understand what you wrote: : : If I use 'logger -p user.err my test message' your saying this is getting logged at multiple : priorities and/or facilies? : : Or do you mean: : : user.err /some/file : user.crit /some/file : : Do you mean this syslog config will cause the previous logger statement to log twice? If so, that : would make sense, but I don't think the problem I am seeing is caused by this. : : Ugh, I'm at a loss and the documentation is severely limiting. : : Any ideas? : : --- Wolfgang Braun <wolfgang.braun@gmx.de> wrote: : : > On Tue, Jan 18, 2005 at 06:46:33AM -0800, Rhugga wrote: : > : > Hi : > : > > Getting 5 copies of each message. (was getting only 3 before, but now : > > getting 5 copies of each log message) : > : > I think the main culprit is your syslogd.conf on the Solaris machine: : > : > > # To syslog host : > > *.debug @syslog : > > *.info @syslog : > > *.notice @syslog : > > *.warning @syslog : > > *.err @syslog : > > *.crit @syslog : > > *.alert @syslog : > > *.emerg @syslog : > : > syslog.conf(5) on Linux says if you put priority P in a rule everything : > with priority >= P will be logged. Check your syslog manual. : > : > <man page> : > : > The behavior of the original BSD syslogd is that all messages of the : > specified priority and higher are logged according to the given : > action. : > : > </man page> : > : > Confirmed this with OpenBSD syslog. If I have : > : > *.* @loghost : > *.debug @loghost : > : > messages with priority >= debug are sent over the wire twice. : > : > If this is the case you can collapse your above statements to : > : > *.* @loghost : > : > to get only one copy of each msg. : > : > : > : > > Here is my entire config file: : > > [...] : > : > Just nitpicking but I think you could collapse most of your : > syslog-ng.conf if you took out the 'host("xyz")' out of the filters. : > Since they all go from the same source() to the same destination() with : > $HOST expansion they don't really accomplish anything. : > : > : > : > -- : > Wolfgang Braun, Dipl.-Inform. (FH) : > <wolfgang.braun@gmx.de> : > gpg-key: 1024D/4B32CE55 : > gpg-fingerprint: 7F0F DE82 94A5 B476 0E08 4972 AC95 31A3 4B32 CE55 : > _______________________________________________ : > syslog-ng maillist - syslog-ng@lists.balabit.hu : > https://lists.balabit.hu/mailman/listinfo/syslog-ng : > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html : > : > : : : ===== : Chuck Carson - Sr. Systems Engineer : Syrrx, Inc. - www.syrrx.com : 10410 Science Center Drive : San Diego, CA 92121 : Work: 858.622.8528 : Fax: 858.550.0526 : _______________________________________________ : syslog-ng maillist - syslog-ng@lists.balabit.hu : https://lists.balabit.hu/mailman/listinfo/syslog-ng : Frequently asked questions at http://www.campin.net/syslog-ng/faq.html :
I notice that logs from Solaris clients are different than those from Linux: Jan 20 05:46:34 syslog syslog-ng[16592]: STATS: dropped 0 Jan 20 05:46:43 ssh-gateway sshd(pam_unix)[4416]: session closed for user logadm Jan 20 05:48:48 db-0201 su: [ID 366847 auth.info] 'su oracle' succeeded for root on /dev/??? In this sampling db-0201 is a Solaris 9 box, syslog is a SLES9 box, and ssh-gateway is a RH9 box. I noticed that the solaris log entry has [ID 366847 auth.info] whereas the linux entry has syslog-ng[16592]. I'm trying to parse these files and store in a database but don't see what exactly these fields are or what is generating them. I am assuming syslog-ng is adding this field since the copy of the log entry in the local /var/adm/messages file does not contain this. So my question is why is it different for Solaris and Linux and can this behavior be changed? The log entry from the Linux box appears to contain the pid appended to the daemon name but the solaris entry lookgs like some kind of internal syslog-ng message id. What is the breakdown of the fields in a syslog-ng log entry? Is this correct? field 1: <timesamp> field 2: <hostname> field 3: <daemon generating log entry> field 4: ? <unknown> field 5: <log content> Thx CC ===== Chuck Carson - Sr. Systems Engineer Syrrx, Inc. - www.syrrx.com 10410 Science Center Drive San Diego, CA 92121 Work: 858.622.8528 Fax: 858.550.0526
On Thu, Jan 20, 2005 at 06:01:20AM -0800, Rhugga wrote:
I am assuming syslog-ng is adding this field since the copy of the log entry in the local /var/adm/messages file does not contain this.
Solaris is adding that, and I don't think you can completely remove this but you can modify it's behavior (you can turn off the message ID, I know that for sure). See https://lists.balabit.hu/pipermail/syslog-ng/2004-August/006329.html -- Nate "Prosperity is the surest breeder of insolence I know." - Samuel Clemens
participants (4)
-
Aakin N. Patel
-
Nate Campi
-
Rhugga
-
Wolfgang Braun