syslog-ng.conf help - syslog-ng writing everything to 'messages' in addition to $HOST.log
All, I'm fairly new to syslog-ng (been using syslogd for many years) and I have a question with the config file syntax. What I'm trying to do is log all remote hosts to /var/log/$HOST.log while keeping the logging host's logs seperate. What I'm seeing is all messages are being written to /var/log/$HOST.log, including the logging system, as well as to /var/log/messages. In a single day, /var/log/messages grows to over 11GB (I'm logging less than 100 devices - Windows servers, routers, and switches.) I haven't quite figured out which part of the config file is causing this to happen, since I'm still going through my growing pains with it. Can someone point me in the right direction with this? This is running on FreeBSD 6.2-RELEASE, and syslog-ng was installed via ports: root@logmeister [~]# pkg_info | grep syslog-ng syslog-ng-2.0.2 A powerful syslogd replacement In /etc/rc.conf I have the following defined: # We want syslog-ng, not the default syslogd: syslog_ng_enable="YES" syslogd_enable="NO" Here's my syslog-ng.conf file (note that I have the swatch portion at the end commented out. I'll figure that out once I get the important issues ironed out.) ------- Begin syslog-ng.conf ------- # /usr/local/etc/syslog-ng.conf # Modified for logmeister (running FreeBSD 6.2-RELEASE SMP) # # options # options { long_hostnames(on); sync(0); use_dns(yes);}; # # sources # source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); udp(); internal(); file("/dev/klog"); }; # # destinations # destination messages { file("/var/log/messages"); }; destination security { file("/var/log/security"); }; destination authlog { file("/var/log/auth.log"); }; destination maillog { file("/var/log/maillog"); }; destination lpd-errs { file("/var/log/lpd-errs"); }; destination xferlog { file("/var/log/xferlog"); }; destination cron { file("/var/log/cron"); }; destination debuglog { file("/var/log/debug.log"); }; destination consolelog { file("/var/log/console.log"); }; destination all { file("/var/log/all.log"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; destination slip { file("/var/log/slip.log"); }; destination ppp { file("/var/log/ppp.log"); }; destination console { file("/dev/console"); }; destination allusers { usertty("*"); }; #destination loghost { udp("loghost" port(514)); }; # Log all remote hosts to their own file: destination remote { file("/var/log/$HOST.log" owner(root) group(wheel) perm(0644) dir_perm(0700) create_dirs(yes)); }; # # log facility filters # filter f_auth { facility(auth); }; filter f_authpriv { facility(authpriv); }; filter f_not_authpriv { not facility(authpriv); }; filter f_console { facility(console); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_ftp { facility(ftp); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_security { facility(security); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; # # log level filters # filter f_emerg { level(emerg); }; filter f_alert { level(alert..emerg); }; filter f_crit { level(crit..emerg); }; filter f_err { level(err..emerg); }; filter f_warning { level(warning..emerg); }; filter f_notice { level(notice..emerg); }; filter f_info { level(info..emerg); }; filter f_debug { level(debug..emerg); }; filter f_is_debug { level(debug); }; # # program filters # filter f_ppp { program("ppp"); }; filter f_slip { program("startslip"); }; # # *.err;kern.warning;auth.notice;mail.crit /dev/console # log { source(src); filter(f_err); destination(console); }; log { source(src); filter(f_kern); filter(f_warning); destination(console); }; log { source(src); filter(f_auth); filter(f_notice); destination(console); }; log { source(src); filter(f_mail); filter(f_crit); destination(console); }; # # *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages # log { source(src); filter(f_notice); filter(f_not_authpriv); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; log { source(src); filter(f_news); filter(f_err); destination(messages); }; # # security.* /var/log/security # log { source(src); filter(f_security); destination(security); }; # # auth.info;authpriv.info /var/log/auth.log log { source(src); filter(f_auth); filter(f_info); destination(authlog); }; log { source(src); filter(f_authpriv); filter(f_info); destination(authlog); }; # # mail.info /var/log/maillog # log { source(src); filter(f_mail); filter(f_info); destination(maillog); }; # # lpr.info /var/log/lpd-errs # log { source(src); filter(f_lpr); filter(f_info); destination(lpd-errs); }; # # ftp.info /var/log/xferlog # log { source(src); filter(f_ftp); filter(f_info); destination(xferlog); }; # # cron.* /var/log/cron # log { source(src); filter(f_cron); destination(cron); }; # # *.=debug /var/log/debug.log # log { source(src); filter(f_is_debug); destination(debuglog); }; # # *.emerg * # log { source(src); filter(f_emerg); destination(allusers); }; # # uncomment this to log all writes to /dev/console to /var/log/console.log # console.info /var/log/console.log # #log { source(src); filter(f_console); filter(f_info); destination(consolelog); }; # # uncomment this to enable logging of all log messages to /var/log/all.log # touch /var/log/all.log and chmod it to mode 600 before it will work # *.* /var/log/all.log # #log { source(src); destination(all); }; # # uncomment this to enable logging to a remote loghost named loghost # *.* @loghost # #log { source(src); destination(loghost); }; # # uncomment these if you're running inn # news.crit /var/log/news/news.crit # news.err /var/log/news/news.err # news.notice /var/log/news/news.notice # #log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; #log { source(src); filter(f_news); filter(f_err); destination(newserr); }; #log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; # # !startslip # *.* /var/log/slip.log # log { source(src); filter(f_slip); destination(slip); }; # # !ppp # *.* /var/log/ppp.log # log { source(src); filter(f_ppp); destination(ppp); }; # Log remote hosts to {hostname}.log ####################### ### Local Machine ##### ####################### # *.err;kern.warning;auth.notice;mail.crit /dev/console log { source(src); filter(f_err); destination(console); }; log { source(src); filter(f_kern); filter(f_warning); destination(console); }; log { source(src); filter(f_auth); filter(f_notice); destination(console); }; log { source(src); filter(f_mail); filter(f_crit); destination(console); }; # *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages log { source(src); filter(f_notice); filter(f_not_authpriv); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; log { source(src); filter(f_news); filter(f_err); destination(messages); }; # security.* /var/log/security log { source(src); filter(f_security); destination(security); }; # auth.info;authpriv.info /var/log/auth.log log { source(src); filter(f_auth); filter(f_info); destination(authlog); }; log { source(src); filter(f_authpriv); filter(f_info); destination(authlog); }; # mail.info /var/log/maillog log { source(src); filter(f_mail); filter(f_info); destination(maillog); }; # cron.* /var/log/cron log { source(src); filter(f_cron); destination(cron); }; # *.=debug /var/log/debug.log log { source(src); filter(f_is_debug); destination(debuglog); }; # *.emerg * log { source(src); filter(f_emerg); destination(allusers); }; ####################### ### Remote Hosts ###### ####################### # *.err;kern.warning;auth.notice;mail.crit log { source(src); filter(f_err); destination(remote); }; log { source(src); filter(f_kern); filter(f_warning); destination(remote); }; log { source(src); filter(f_auth); filter(f_notice); destination(remote); }; log { source(src); filter(f_mail); filter(f_crit); destination(remote); }; # *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err log { source(src); filter(f_notice); filter(f_not_authpriv); destination(remote); }; log { source(src); filter(f_kern); filter(f_debug); destination(remote); }; log { source(src); filter(f_lpr); filter(f_info); destination(remote); }; log { source(src); filter(f_mail); filter(f_crit); destination(remote); }; log { source(src); filter(f_news); filter(f_err); destination(remote); }; # security.* log { source(src); filter(f_security); destination(remote); }; # auth.info;authpriv.info log { source(src); filter(f_auth); filter(f_info); destination(remote); }; log { source(src); filter(f_authpriv); filter(f_info); destination(remote); }; # mail.info log { source(src); filter(f_mail); filter(f_info); destination(remote); }; # cron.* log { source(src); filter(f_cron); destination(remote); }; # *.=debug log { source(src); filter(f_is_debug); destination(remote); }; # *.emerg log { source(src); filter(f_emerg); destination(remote); }; # local.* log { source(src); filter(f_local0); destination(remote); }; log { source(src); filter(f_local1); destination(remote); }; log { source(src); filter(f_local2); destination(remote); }; log { source(src); filter(f_local3); destination(remote); }; log { source(src); filter(f_local4); destination(remote); }; log { source(src); filter(f_local5); destination(remote); }; log { source(src); filter(f_local6); destination(remote); }; log { source(src); filter(f_local7); destination(remote); }; # send all logs to swatch for (near) real-time alerts #log { # source(src); # destination(swatch); #}; ------- End syslog-ng.conf ------- Many Thanks, Kevin Reiter Senior Security Engineer Financial Services, Inc. 21 Harristown Road Glen Rock, New Jersey 07452 (201)652-6000, ext. 588 PGP ID: 0xEE665233 This message may contain confidential or proprietary information and is intended solely for the individual(s) to whom it is addressed. If you are not a named addressee you should not disseminate, distribute or copy this e-mail or act upon the information contained herein. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system.
Hello,
All,
I'm fairly new to syslog-ng (been using syslogd for many years) and I have a question with the config file syntax.
What I'm trying to do is log all remote hosts to /var/log/$HOST.log while keeping the logging host's logs seperate. What I'm seeing is all messages are being written to /var/log/$HOST.log, including the logging system, as well as to /var/log/messages. In a single day, /var/log/messages grows to over 11GB (I'm logging less than 100 devices - Windows servers, routers, and switches.)
I haven't quite figured out which part of the config file is causing this to happen, since I'm still going through my growing pains with it. Can someone point me in the right direction with this?
[ cutting the details ] You're referring to the same source (src) everywhere in your config, so I would like to suggest to remove udp() from this source and move it to a separate one. Then you can use this source for the remote hosts. Something like this: source s_remote { udp(); }; Later in your conffile referring to this source you can differentiate between the local and the remote logs. regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
syslog-ng-bounces@lists.balabit.hu wrote: : Hello, : :: All, :: :: I'm fairly new to syslog-ng (been using syslogd for many :: years) and I have a question with the config file syntax. :: :: What I'm trying to do is log all remote hosts to :: /var/log/$HOST.log while keeping the logging host's logs :: seperate. What I'm seeing is all messages are being written :: to /var/log/$HOST.log, including the logging system, as well :: as to /var/log/messages. In a single day, /var/log/messages :: grows to over 11GB (I'm logging less than 100 devices - :: Windows servers, routers, and switches.) :: :: I haven't quite figured out which part of the config file is :: causing this to happen, since I'm still going through my :: growing pains with it. Can someone point me in the right :: direction with this? : : [ cutting the details ] : : You're referring to the same source (src) everywhere in your : config, so I would like to suggest to remove udp() from this : source and move it to a separate one. Then you can use this : source for the remote hosts. Something like this: : : source s_remote { : udp(); : }; : : Later in your conffile referring to this source you can : differentiate between the local and the remote logs. : : regards, : : Sandor Thanks! That seemed to do the trick. It appears now that each log entry is being duplicated, but I can live with that for the time being. Would that be the result of an extra line I have in there somewhere, or do I need to look forther at the src/destination directives? This message may contain confidential or proprietary information and is intended solely for the individual(s) to whom it is addressed. If you are not a named addressee you should not disseminate, distribute or copy this e-mail or act upon the information contained herein. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system.
Hi,
Thanks! That seemed to do the trick. It appears now that each log entry is being duplicated, but I can live with that for the time being. Would that be the result of an extra line I have in there somewhere, or do I need to look forther at the src/destination directives?
Your log definitions have a lot of duplicate entries as you're defining syslog-like severity ranges (like f_info is an info..emerg range filter, ans do on). So when a message with the err priority arrives then f_info, f_notice, f_warning and f_err will match. When I convert a syslogd conffile to syslog-ng, I usually create a matrix which facility/priority will logged to which destination. Something like this: emerg| alert| crit | err | warn |notice| info | debug kernel| ME mail | ME MA ... (where MA means /var/log/mail.log and ME is /var/log/messages) After this I set up the filters/ destinations, and use this matrix to create the configuration: # kernel.debug /var/log/messages log { source(s_local); filter(f_kernel); filter(f_debug); destination(d_messages); flags(final); }; # mail.crit /var/log/mail.log /var/log/messages log { source(s_local); filter(f_mail); filter(f_crit); destination(d_messages); destination(d_maillog); flags(final); }; # mail.info /var/log/mail.log log { source(s_local); filter(f_mail); filter(f_info); destination(d_maillog); flags(final); }; # don't log mail.debug anywhere else log { source(s_local); filter(f_mail); flags(final); }; Usually I also test the config with a small script which sends logs with every possible facility/severity combinations to check whether the logs end up in the correct destination or not. Of course this isn't the easiest way to configure syslog-ng, just the way I do it :) regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
participants (2)
-
Geller, Sandor (IT)
-
Kevin Reiter