[syslog-ng] v3.3 not flushing lines?
Clayton Dukes
cdukes at gmail.com
Fri Jan 13 16:02:39 CET 2012
It's been 3 days, not hours :-)
Here's the user's config file:
#############################################################################
# Default syslog-ng.conf file which collects all local logs into a
# single file called /var/log/messages.
#
@version: 3.3
@include "scl.conf"
source s_local {
system();
internal();
};
source s_remote {
udp(ip(0.0.0.0) port(514));
tcp(ip(0.0.0.0) port(514));
};
destination d_separatedbyhosts {
file("/var/log/syslog-ng/$HOST/messages" owner("root")
group("root") perm(0640) dir_perm(0750) create_dirs(yes)
template("$R_YEAR-$R_MONTH-$R_DAY
$R_HOUR:$R_MIN:$S_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")
template_escape(yes)
);
};
log {
source(s_remote);
# uncomment this line to open port 514 to receive messages
#source(s_network);
destination(d_separatedbyhosts);
};
# <lzconfig> BEGIN LogZilla v3.2 syslog-ng config file
# Please don't remove the lzconfig tags above and at the end of this block.
#
# NOTE:
# You may need to alter your receive buffer if you expect a large amount of
logs!
# Please read
http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0#UDP_Buffers
# Example: udp( so_rcvbuf(1048576));
#
#
###########################################################################################
# BEGIN LogZilla Config for syslog-ng
###########################################################################################
# Last updated on 2011-11-20
###########################################################################################
options {
# long_hostnames(off);
# doesn't actually help on Solaris, log(3) truncates at 1024 chars
log_msg_size(8192);
# buffer just a little for performance
# sync(1); <- Deprecated - use flush_lines() instead
flush_lines(1);
# memory is cheap, buffer messages unable to write (like to loghost)
log_fifo_size(16384);
# Hosts we don't want syslog from
#bad_hostname("^(ctld.|cmd|tmd|last)$");
# The time to wait before a dead connection is reestablished (seconds)
time_reopen(10);
#Use DNS so that our good names are used, not hostnames
use_dns(yes);
dns_cache(yes);
#Use the whole DNS name
use_fqdn(yes);
keep_hostname(yes);
# chain_hostnames(no);
#Read permission for everyone
perm(0644);
# The default action of syslog-ng 1.6.0 is to log a STATS line
# to the file every 10 minutes. That's pretty ugly after a while.
# Change it to every 12 hours so you get a nice daily update of
# # how many messages syslog-ng missed (0).
# stats(43200);
};
# Note: LogZilla will ONLY process log entries in the format below.
# You can't run db_insert.pl on any log file without using this template.
# The reason is that messages vary in composition so the tab delimiters are
# needed to determine the tokens.
destination d_logzilla {
program("/var/www/logzilla/scripts/db_insert.pl"
template("$R_YEAR-$R_MONTH-$R_DAY
$R_HOUR:$R_MIN:$S_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")
template_escape(yes)
);
};
destination df_logzilla {
file("/var/log/logzilla/DEBUG.log"
template("$R_YEAR-$R_MONTH-$R_DAY
$R_HOUR:$R_MIN:$R_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")
template_escape(yes)
);
};
# Tell syslog-ng to log to our new destination
log {
source(s_remote);
destination(d_logzilla);
# Uncomment below and restart syslog-ng for debugging
destination(df_logzilla);
};
# END LogZilla Config for syslog-ng
###########################################################################################
# </lzconfig> END LogZilla v3.2 syslog-ng config file
______________________________________________________________
Clayton Dukes
______________________________________________________________
On Fri, Jan 13, 2012 at 9:54 AM, Patrick Hemmer <syslogng at feystorm.net>wrote:
> Sent: Fri Jan 13 2012 09:45:55 GMT-0500 (EST)
> From: Clayton Dukes <cdukes at gmail.com>
> To: Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> Subject: Re: [syslog-ng] v3.3 not flushing lines?
>
>> 3rd try...anyone?
>>
>> ______________________________**______________________________**__
>>
>> Clayton Dukes
>> ______________________________**______________________________**__
>>
>>
>> On Thu, Jan 12, 2012 at 3:48 PM, Clayton Dukes <cdukes at gmail.com <mailto:
>> cdukes at gmail.com>> wrote:
>>
>> Can anyone help here? I'm out of ideas :-)
>>
>> ______________________________**______________________________**__
>>
>> Clayton Dukes
>> ______________________________**______________________________**__
>>
>>
>>
>> On Wed, Jan 11, 2012 at 8:05 PM, Clayton Dukes <cdukes at gmail.com
>> <mailto:cdukes at gmail.com>> wrote:
>>
>> Hey folks,
>>
>> I have a user experiencing an issue where some of the incoming
>> logs are not getting piped to my perl script until the second
>> time we generate events.
>>
>> I believe this is generally set using flush_lines(1), but it
>> doesn’t seem to be honoring that setting in the config.
>> I also tried adding flush_timeout(900), but that has no affect.
>> In the output below, if I quit the telnet and start it again,
>> the previously missing events are then received (but the
>> current ones are missing) - basically, it doesn't appear to be
>> flushing every single line.
>> Possible other reason: has something changed between v2.x and
>> 3.x where the program() destination would possibly not send an
>> EOF signal - i.e. is the pipe now kept open vs. an individual
>> call to the program each time in v2.x?
>>
>>
>> To verify that this is happening:
>>
>>
>> Term 1:
>> tail -f /tmp/logzilla_import.txt
>> Term 2:
>> /usr/local/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf -Fdv
>> Term 3:
>> telnet 192.168.254.1
>>
>> Term 1 results:
>> 192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST:
>> Telnet2: 1 1 251 1 3751981041 0 2012-01-11 19
>> <tel:2012-01-11%2019>:29:02 2012-01-11 19
>> <tel:2012-01-11%2019>:29:02
>>
>> 192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST:
>> TCP2: Telnet sent WILL ECHO (1) 3751981041 0 2012-01-11 19
>> <tel:2012-01-11%2019>:29:02 2012-01-11 19
>> <tel:2012-01-11%2019>:29:02
>>
>>
>> Term 2 results:
>> Incoming log entry; line='<183>6987: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet sent WILL ECHO (1)'
>> Incoming log entry; line='<183>6988: .Jan 11 2012 19:29:02.284
>> EST: Telnet2: 2 2 251 3'
>> Incoming log entry; line='<183>6989: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet sent WILL SUPPRESS-GA (3)'
>> Incoming log entry; line='<183>6990: .Jan 11 2012 19:29:02.284
>> EST: Telnet2: 80000 80000 253 24'
>> Incoming log entry; line='<183>6991: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet sent DO TTY-TYPE (24)'
>> Incoming log entry; line='<183>6992: .Jan 11 2012 19:29:02.284
>> EST: Telnet2: 10000000 10000000 253 31'
>> Incoming log entry; line='<183>6993: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet sent DO WINDOW-SIZE (31)'
>> Incoming log entry; line='<183>6994: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet received DO ENCRYPTION (38)'
>> Incoming log entry; line='<183>6995: .Jan 11 2012 19:29:02.284
>> EST: TCP2: Telnet sent WONT ENCRYPTION (38) (unimplemented)'
>> Incoming log entry; line='<183>6996: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL ENCRYPTION (38)'
>> Incoming log entry; line='<183>6997: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent DONT ENCRYPTION (38) (unimplemented)'
>> Incoming log entry; line='<183>6998: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received DO SUPPRESS-GA (3)'
>> Incoming log entry; line='<183>6999: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL TTY-TYPE (24)'
>> Incoming log entry; line='<183>7000: .Jan 11 2012 19:29:02.292
>> EST: Telnet2: Sent SB 24 1 '
>> Incoming log entry; line='<183>7001: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL WINDOW-SIZE (31)'
>> Incoming log entry; line='<183>7002: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL TTY-SPEED (32) (refused)'
>> Incoming log entry; line='<183>7003: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent DONT TTY-SPEED (32)'
>> Incoming log entry; line='<183>7004: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL LOCAL-FLOW (33) (refused)'
>> Incoming log entry; line='<183>7005: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent DONT LOCAL-FLOW (33)'
>> Incoming log entry; line='<183>7006: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL LINEMODE (34)'
>> Incoming log entry; line='<183>7007: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent DONT LINEMODE (34) (unimplemented)'
>> Incoming log entry; line='<183>7008: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received WILL NEW-ENVIRON (39)'
>> Incoming log entry; line='<183>7009: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent DONT NEW-ENVIRON (39) (unimplemented)'
>> Incoming log entry; line='<183>7010: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received DO STATUS (5)'
>> Incoming log entry; line='<183>7011: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet sent WONT STATUS (5) (unimplemented)'
>> Incoming log entry; line='<183>7012: .Jan 11 2012 19:29:02.292
>> EST: TCP2: Telnet received DO ECHO (1)'
>> Incoming log entry; line='<183>7013: .Jan 11 2012 19:29:02.292
>> EST: Telnet2: recv SB NAWS 132 63'
>> Incoming log entry; line='<183>7014: .Jan 11 2012 19:29:02.292
>> EST: Telnet2: recv SB 24 0 LINUX'
>> Incoming log entry; line='<183>7015: .Jan 11 2012 19:29:02.493
>> EST: TCP2: Telnet received WILL ENVIRONMENT (36) (refused)'
>> Incoming log entry; line='<183>7016: .Jan 11 2012 19:29:02.493
>> EST: TCP2: Telnet sent DONT ENVIRONMENT (36)'
>>
>>
>>
>>
>>
>>
>> ______________________________**______________________________**__
>>
>> Clayton Dukes
>> ______________________________**______________________________**__
>>
>>
>>
>> This is a mailing list, repeating yourself isnt going to help. Not
> everyone checks it every few hours. Have patience.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20120113/7aa7e162/attachment-0001.htm
More information about the syslog-ng
mailing list