Syslog-ng writing to files once per minute
I'm hoping this is a really stupid question, but after implementing what I thought to be the fix to this, I can't seem to get it working. I have a number of hosts logging to facility local5, and the destination log file is only updated once per minute (HH:MM.30s to be precise). I've implemented flush_lines(0); and flush_interval(0);, to the global options, but it doesn't seem to have an effect. syslog-ng 3.0.3 My goal is to have logs written to the destination file with minimal latency after syslog-ng receives the message. -- Eric Cables
On Thu, 2010-07-01 at 00:04 -0700, Eric Cables wrote:
I'm hoping this is a really stupid question, but after implementing what I thought to be the fix to this, I can't seem to get it working. I have a number of hosts logging to facility local5, and the destination log file is only updated once per minute (HH:MM.30s to be precise). I've implemented flush_lines(0); and flush_interval(0);, to the global options, but it doesn't seem to have an effect.
syslog-ng 3.0.3
My goal is to have logs written to the destination file with minimal latency after syslog-ng receives the message.
If you don't have flush_lines() > 0 in your configuration, then that's what has to happen. Can you elaborate what you see? how long does it take for syslog-ng to write that message there? -- Bazsi
I am seeing syslog-ng write to the file exactly once per minute, which includes the hundreds of queued messages. On 7/1/10, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Thu, 2010-07-01 at 00:04 -0700, Eric Cables wrote:
I'm hoping this is a really stupid question, but after implementing what I thought to be the fix to this, I can't seem to get it working. I have a number of hosts logging to facility local5, and the destination log file is only updated once per minute (HH:MM.30s to be precise). I've implemented flush_lines(0); and flush_interval(0);, to the global options, but it doesn't seem to have an effect.
syslog-ng 3.0.3
My goal is to have logs written to the destination file with minimal latency after syslog-ng receives the message.
If you don't have flush_lines() > 0 in your configuration, then that's what has to happen.
Can you elaborate what you see? how long does it take for syslog-ng to write that message there?
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Sent from my mobile device -- Eric Cables
On Thu, 2010-07-01 at 10:35 -0700, Eric Cables wrote:
I am seeing syslog-ng write to the file exactly once per minute, which includes the hundreds of queued messages.
this definitely means that syslog-ng is using a non-zero value for flush_lines(). This used to be called "sync_freq" or "sync", but those names were deprecated. Please also note that these can be set on a per-destination basis, but also globally, validate that if you have a global setting, you are overriing at the specific destination. -- Bazsi
Here's the relevant portion of my config: @version: 3.0 options { dir_perm(0755); perm(0644); chain_hostnames(no); keep_hostname(yes); create_dirs(yes); normalize_hostnames(yes); use_fqdn(yes); flush_lines(0); flush_timeout(0); }; # Default local source. source local { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); file("/dev/klog"); udp(ip(127.0.0.1) port(514)); internal(); }; source remote { udp(ip(x.x.x.x) port(514)); tcp(ip(x.x.x.x) port(1470)); }; Also, here's some output to illustrate what I'm seeing: -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log Please let me know if I've implemented the flush_lines() & flush_timeout() values incorrectly. -- Eric Cables On Thu, Jul 1, 2010 at 1:25 PM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Thu, 2010-07-01 at 10:35 -0700, Eric Cables wrote:
I am seeing syslog-ng write to the file exactly once per minute, which includes the hundreds of queued messages.
this definitely means that syslog-ng is using a non-zero value for flush_lines(). This used to be called "sync_freq" or "sync", but those names were deprecated.
Please also note that these can be set on a per-destination basis, but also globally, validate that if you have a global setting, you are overriing at the specific destination.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
I'm not sure if this means anything, but immediately after restarting the syslog-ng daemon, logs seem to update in real-time. This lasts for about 5-10 seconds, and then the update interval drops back to per minute. Here's some output immediately after restarting syslog-ng: -rw-r--r-- 1 root wheel 455017963 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455018571 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455018907 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455019552 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455020305 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455028247 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log As you can see, the first few outputs show the size increase happening frequently, but then reverts back to per-minute updates. -- Eric Cables On Thu, Jul 1, 2010 at 2:50 PM, Eric Cables <ecables@gmail.com> wrote:
Here's the relevant portion of my config: @version: 3.0
options { dir_perm(0755); perm(0644); chain_hostnames(no); keep_hostname(yes); create_dirs(yes); normalize_hostnames(yes); use_fqdn(yes); flush_lines(0); flush_timeout(0); };
# Default local source. source local { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); file("/dev/klog"); udp(ip(127.0.0.1) port(514)); internal(); };
source remote { udp(ip(x.x.x.x) port(514)); tcp(ip(x.x.x.x) port(1470)); };
Also, here's some output to illustrate what I'm seeing: -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log
Please let me know if I've implemented the flush_lines() & flush_timeout() values incorrectly.
-- Eric Cables
On Thu, Jul 1, 2010 at 1:25 PM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Thu, 2010-07-01 at 10:35 -0700, Eric Cables wrote:
I am seeing syslog-ng write to the file exactly once per minute, which includes the hundreds of queued messages.
this definitely means that syslog-ng is using a non-zero value for flush_lines(). This used to be called "sync_freq" or "sync", but those names were deprecated.
Please also note that these can be set on a per-destination basis, but also globally, validate that if you have a global setting, you are overriing at the specific destination.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
FYI, I upgraded to syslog-ng 3.1.1 (from 3.0.3), and this feature seems to be working as expected now. I no longer see 1-minute intervals between updates, and instead I see a constant stream of updates to the expected files. Thanks for the help, and maybe this will provide useful information to others in the future. -- Eric Cables On Thu, Jul 1, 2010 at 3:28 PM, Eric Cables <ecables@gmail.com> wrote:
I'm not sure if this means anything, but immediately after restarting the syslog-ng daemon, logs seem to update in real-time. This lasts for about 5-10 seconds, and then the update interval drops back to per minute.
Here's some output immediately after restarting syslog-ng: -rw-r--r-- 1 root wheel 455017963 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455018571 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455018907 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455019552 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455020305 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455028247 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log -rw-r--r-- 1 root wheel 455035022 Jul 1 15:25 firewalls.log
As you can see, the first few outputs show the size increase happening frequently, but then reverts back to per-minute updates.
-- Eric Cables
On Thu, Jul 1, 2010 at 2:50 PM, Eric Cables <ecables@gmail.com> wrote:
Here's the relevant portion of my config: @version: 3.0
options { dir_perm(0755); perm(0644); chain_hostnames(no); keep_hostname(yes); create_dirs(yes); normalize_hostnames(yes); use_fqdn(yes); flush_lines(0); flush_timeout(0); };
# Default local source. source local { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); file("/dev/klog"); udp(ip(127.0.0.1) port(514)); internal(); };
source remote { udp(ip(x.x.x.x) port(514)); tcp(ip(x.x.x.x) port(1470)); };
Also, here's some output to illustrate what I'm seeing: -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454561970 Jul 1 14:45 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454573725 Jul 1 14:46 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log -rw-r--r-- 1 root wheel 454595899 Jul 1 14:47 firewalls.log
Please let me know if I've implemented the flush_lines() & flush_timeout() values incorrectly.
-- Eric Cables
On Thu, Jul 1, 2010 at 1:25 PM, Balazs Scheidler <bazsi@balabit.hu>wrote:
On Thu, 2010-07-01 at 10:35 -0700, Eric Cables wrote:
I am seeing syslog-ng write to the file exactly once per minute, which includes the hundreds of queued messages.
this definitely means that syslog-ng is using a non-zero value for flush_lines(). This used to be called "sync_freq" or "sync", but those names were deprecated.
Please also note that these can be set on a per-destination basis, but also globally, validate that if you have a global setting, you are overriing at the specific destination.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (2)
-
Balazs Scheidler
-
Eric Cables