Redirecting logs and UDP issues
All, I am trying to redirect a log file from an NFS mount (AIX server = source and Redhat server = target). The redirect is going to a LogLogic device over UDP, and I am seeing quite a few dropped packets in /var/log/messages. Samples below: Aug 31 07:34:16 lxfwossecp3 syslog-ng[30574]: syslog-ng starting up; version='2.1.4' Aug 31 07:44:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=15', processed='center(received)=15', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=1', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=1', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=4', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 07:54:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=17', processed='center(received)=17', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=2', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=2', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=6', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:04:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=20', processed='center(received)=20', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=4', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=3', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=9', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:14:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=22', processed='center(received)=22', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=2', processed='destination(d_cron)=5', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=4', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=11', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Aug 31 08:24:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=28', processed='center(received)=28', processed='destination(d_messages)=11', processed='destination(d_boot)=0', pro cessed='destination(d_auth)=6', processed='destination(d_cron)=6', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=5', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=17', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0' Here is what my syslog-ng.conf file looks like: # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { time_reopen (10); flush_lines (2000); flush_timeout (20000); log_fifo_size (30000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: # Setup for PeopleSoft logs to transmit to Loglogic source s_file { file("/tmp/APPSRV_current.LOG" flags(no-parse)); }; destination d_messages{ udp("10.13.33.11" port(514) log_fifo_size(900000000)); }; log { source(s_file); destination(d_messages); }; Does anyone see anything wrong with the setup? Could the NFS link be an issue? Any help would be greatly appreciated! Thanks, Jamie
Hi, On Wed, Aug 31, 2011 at 3:33 PM, Aldrich, Jamie S <JSAldrich@pier1.com> wrote:
All,
I am trying to redirect a log file from an NFS mount (AIX server = source and Redhat server = target). The redirect is going to a LogLogic device over UDP, and I am seeing quite a few dropped packets in /var/log/messages. Samples below:
Aug 31 07:34:16 lxfwossecp3 syslog-ng[30574]: syslog-ng starting up; version='2.1.4'
Aug 31 07:44:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=15', processed='center(received)=15', processed='destination(d_messages)=11', processed='destination(d_boot)=0', processed='destination(d_auth)=2', processed='destination(d_cron)=1', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=1', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=4', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0'
May I ask the *second* time why do you think there were dropped messages at all? Do you have any evidence or you just misunderstood the meaning of the above statistical message? Regards, Sandor
Maybe I am misunderstanding the messages. Is it possible to redirect a symlink log file to a server? This is the section in syslog-ng.conf that should be handling this, but does not appear to work. # Setup for PeopleSoft logs to transmit to Loglogic source s_file { file("/tmp/APPSRV_current.LOG" flags(no-parse)); }; destination d_messages{ udp("10.13.33.11" port(514) log_fifo_size(900000000)); }; log { source(s_file); destination(d_messages); }; Jamie -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Sandor Geller Sent: Wednesday, August 31, 2011 9:08 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Redirecting logs and UDP issues Hi, On Wed, Aug 31, 2011 at 3:33 PM, Aldrich, Jamie S <JSAldrich@pier1.com> wrote:
All,
I am trying to redirect a log file from an NFS mount (AIX server = source and Redhat server = target). The redirect is going to a LogLogic device over UDP, and I am seeing quite a few dropped packets in /var/log/messages. Samples below:
Aug 31 07:34:16 lxfwossecp3 syslog-ng[30574]: syslog-ng starting up; version='2.1.4'
Aug 31 07:44:16 lxfwossecp3 syslog-ng[30574]: Log statistics; dropped='udp(10.13.33.11:514)=0', processed='center(queued)=15', processed='center(received)=15', processed='destination(d_messages)=11', processed='destination(d_boot)=0', processed='destination(d_auth)=2', processed='destination(d_cron)=1', processed='destination(d_mlal)=0', processed='destination(d_kern)=0', processed='destination(d_mesg)=1', processed='destination(d_cons)=0', processed='destination(d_spol)= 0', processed='destination(d_mail)=0', processed='source(s_sys)=4', processed='source(s_file)=11', suppressed='udp(10.13.33.11:514)=0'
May I ask the *second* time why do you think there were dropped messages at all? Do you have any evidence or you just misunderstood the meaning of the above statistical message? Regards, Sandor ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hello, On Thu, Sep 1, 2011 at 5:42 PM, Aldrich, Jamie S <JSAldrich@pier1.com> wrote:
Maybe I am misunderstanding the messages. Is it possible to redirect a symlink log file to a server? This is the section in syslog-ng.conf that should be handling this, but does not appear to work.
# Setup for PeopleSoft logs to transmit to Loglogic
source s_file { file("/tmp/APPSRV_current.LOG" flags(no-parse)); };
You're using syslog-ng 2.1.4 where follow_freq() has to get added otherwise syslog-ng will read to the EOF and won't try to read further even when meanwhile new messages were written to the file (and probably it won't even notice when the underlying file has changed, I don't remember from the top of my head). https://bugzilla.balabit.com/show_bug.cgi?id=43 applies to you, using follow_freq() causes that syslog-ng 2.1 might enter a busy loop... Unless you want to patch syslog-ng 2.1.4 it is strongly recommended to migrate to a newer release (3.2.x) which is free of this bug. In addition in 3.2.x releases follow_freq() defaults to 1.
destination d_messages{ udp("10.13.33.11" port(514) log_fifo_size(900000000)); };
That log_fifo_size setting is huge, it controls how many *complete messages* syslog-ng will buffer when it can't send the logs to the given destination. I'd reduce it to a few thousands. regards, Sandor
participants (2)
-
Aldrich, Jamie S
-
Sandor Geller