It's been 3 days, not hours :-)<div><br></div><div>Here's the user's config file:</div><div><br></div><div><div>#############################################################################</div><div># Default syslog-ng.conf file which collects all local logs into a</div>
<div># single file called /var/log/messages.</div><div>#</div><div><br></div><div>@version: 3.3</div><div>@include "scl.conf"</div><div><br></div><div>source s_local {</div><div> system();</div><div> internal();</div>
<div>};</div><div><br></div><div>source s_remote {</div><div> udp(ip(0.0.0.0) port(514));</div><div> tcp(ip(0.0.0.0) port(514));</div><div>};</div><div><br></div><div>destination d_separatedbyhosts {</div><div>
file("/var/log/syslog-ng/$HOST/messages" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes)</div><div> template("$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$S_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")</div>
<div> template_escape(yes)</div><div> );</div><div>};</div><div><br></div><div>log {</div><div> source(s_remote);</div><div><br></div><div> # uncomment this line to open port 514 to receive messages</div>
<div> #source(s_network);</div><div> destination(d_separatedbyhosts);</div><div>};</div><div># <lzconfig> BEGIN LogZilla v3.2 syslog-ng config file</div><div># Please don't remove the lzconfig tags above and at the end of this block.</div>
<div>#</div><div># NOTE:</div><div># You may need to alter your receive buffer if you expect a large amount of logs!</div><div># Please read <a href="http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0#UDP_Buffers">http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0#UDP_Buffers</a></div>
<div># Example: udp( so_rcvbuf(1048576));</div><div>#</div><div>#</div><div>###########################################################################################</div><div># BEGIN LogZilla Config for syslog-ng</div>
<div>###########################################################################################</div><div># Last updated on 2011-11-20</div><div>###########################################################################################</div>
<div>options {</div><div> # long_hostnames(off);</div><div> # doesn't actually help on Solaris, log(3) truncates at 1024 chars</div><div> log_msg_size(8192);</div><div> # buffer just a little for performance</div>
<div> # sync(1); <- Deprecated - use flush_lines() instead</div><div> flush_lines(1);</div><div> # memory is cheap, buffer messages unable to write (like to loghost)</div><div> log_fifo_size(16384);</div>
<div> # Hosts we don't want syslog from</div><div> #bad_hostname("^(ctld.|cmd|tmd|last)$");</div><div> # The time to wait before a dead connection is reestablished (seconds)</div><div> time_reopen(10);</div>
<div> #Use DNS so that our good names are used, not hostnames</div><div> use_dns(yes);</div><div> dns_cache(yes);</div><div> #Use the whole DNS name</div><div> use_fqdn(yes);</div><div> keep_hostname(yes);</div>
<div> # chain_hostnames(no);</div><div> #Read permission for everyone</div><div> perm(0644);</div><div> # The default action of syslog-ng 1.6.0 is to log a STATS line</div><div> # to the file every 10 minutes. That's pretty ugly after a while.</div>
<div> # Change it to every 12 hours so you get a nice daily update of</div><div> # # how many messages syslog-ng missed (0).</div><div> # stats(43200);</div><div>};</div><div><br></div><div><br></div><div># Note: LogZilla will ONLY process log entries in the format below. </div>
<div># You can't run <a href="http://db_insert.pl">db_insert.pl</a> on any log file without using this template.</div><div># The reason is that messages vary in composition so the tab delimiters are</div><div># needed to determine the tokens.</div>
<div>destination d_logzilla {</div><div> program("/var/www/logzilla/scripts/<a href="http://db_insert.pl">db_insert.pl</a>"</div><div> template("$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$S_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")</div>
<div> template_escape(yes)</div><div> );</div><div>};</div><div><br></div><div>destination df_logzilla {</div><div> file("/var/log/logzilla/DEBUG.log"</div><div> template("$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$R_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n")</div>
<div> template_escape(yes)</div><div> ); </div><div>};</div><div><br></div><div># Tell syslog-ng to log to our new destination </div><div>log {</div><div> source(s_remote);</div><div> destination(d_logzilla);</div>
<div> # Uncomment below and restart syslog-ng for debugging</div><div> destination(df_logzilla);</div><div>};</div><div># END LogZilla Config for syslog-ng</div><div>###########################################################################################</div>
<div># </lzconfig> END LogZilla v3.2 syslog-ng config file</div></div><div><br></div><div><br></div><div><br clear="all">______________________________________________________________ <br><br>Clayton Dukes<br>______________________________________________________________<br>
<br><br><div class="gmail_quote">On Fri, Jan 13, 2012 at 9:54 AM, Patrick Hemmer <span dir="ltr"><<a href="mailto:syslogng@feystorm.net">syslogng@feystorm.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Sent: Fri Jan 13 2012 09:45:55 GMT-0500 (EST)<br>
From: Clayton Dukes <<a href="mailto:cdukes@gmail.com" target="_blank">cdukes@gmail.com</a>><br>
To: Syslog-ng users' and developers' mailing list <<a href="mailto:syslog-ng@lists.balabit.hu" target="_blank">syslog-ng@lists.balabit.hu</a>><br>
Subject: Re: [syslog-ng] v3.3 not flushing lines?<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
3rd try...anyone?<br>
<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
Clayton Dukes<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
<br></div><div class="im">
On Thu, Jan 12, 2012 at 3:48 PM, Clayton Dukes <<a href="mailto:cdukes@gmail.com" target="_blank">cdukes@gmail.com</a> <mailto:<a href="mailto:cdukes@gmail.com" target="_blank">cdukes@gmail.com</a>>> wrote:<br>
<br>
Can anyone help here? I'm out of ideas :-)<br>
<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
Clayton Dukes<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
<br>
<br>
On Wed, Jan 11, 2012 at 8:05 PM, Clayton Dukes <<a href="mailto:cdukes@gmail.com" target="_blank">cdukes@gmail.com</a><br></div><div><div class="h5">
<mailto:<a href="mailto:cdukes@gmail.com" target="_blank">cdukes@gmail.com</a>>> wrote:<br>
<br>
Hey folks,<br>
<br>
I have a user experiencing an issue where some of the incoming<br>
logs are not getting piped to my perl script until the second<br>
time we generate events.<br>
<br>
I believe this is generally set using flush_lines(1), but it<br>
doesn’t seem to be honoring that setting in the config.<br>
I also tried adding flush_timeout(900), but that has no affect.<br>
In the output below, if I quit the telnet and start it again,<br>
the previously missing events are then received (but the<br>
current ones are missing) - basically, it doesn't appear to be<br>
flushing every single line.<br>
Possible other reason: has something changed between v2.x and<br>
3.x where the program() destination would possibly not send an<br>
EOF signal - i.e. is the pipe now kept open vs. an individual<br>
call to the program each time in v2.x?<br>
<br>
<br>
To verify that this is happening:<br>
<br>
<br>
Term 1:<br>
tail -f /tmp/logzilla_import.txt<br>
Term 2:<br>
/usr/local/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf -Fdv<br>
Term 3:<br>
telnet 192.168.254.1<br>
<br>
Term 1 results:<br>
192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST:<br>
Telnet2: 1 1 251 1 3751981041 0 2012-01-11 19<br></div></div>
<tel:2012-01-11%2019>:29:02 <a href="tel:2012-01-11%2019" value="+12012011119" target="_blank">2012-01-11 19</a><br>
<tel:2012-01-11%2019>:29:02<div class="im"><br>
192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST:<br>
TCP2: Telnet sent WILL ECHO (1) 3751981041 0 2012-01-11 19<br></div>
<tel:2012-01-11%2019>:29:02 <a href="tel:2012-01-11%2019" value="+12012011119" target="_blank">2012-01-11 19</a><br>
<tel:2012-01-11%2019>:29:02<div><div class="h5"><br>
<br>
Term 2 results:<br>
Incoming log entry; line='<183>6987: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet sent WILL ECHO (1)'<br>
Incoming log entry; line='<183>6988: .Jan 11 2012 19:29:02.284<br>
EST: Telnet2: 2 2 251 3'<br>
Incoming log entry; line='<183>6989: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet sent WILL SUPPRESS-GA (3)'<br>
Incoming log entry; line='<183>6990: .Jan 11 2012 19:29:02.284<br>
EST: Telnet2: 80000 80000 253 24'<br>
Incoming log entry; line='<183>6991: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet sent DO TTY-TYPE (24)'<br>
Incoming log entry; line='<183>6992: .Jan 11 2012 19:29:02.284<br>
EST: Telnet2: 10000000 10000000 253 31'<br>
Incoming log entry; line='<183>6993: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet sent DO WINDOW-SIZE (31)'<br>
Incoming log entry; line='<183>6994: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet received DO ENCRYPTION (38)'<br>
Incoming log entry; line='<183>6995: .Jan 11 2012 19:29:02.284<br>
EST: TCP2: Telnet sent WONT ENCRYPTION (38) (unimplemented)'<br>
Incoming log entry; line='<183>6996: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL ENCRYPTION (38)'<br>
Incoming log entry; line='<183>6997: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent DONT ENCRYPTION (38) (unimplemented)'<br>
Incoming log entry; line='<183>6998: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received DO SUPPRESS-GA (3)'<br>
Incoming log entry; line='<183>6999: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL TTY-TYPE (24)'<br>
Incoming log entry; line='<183>7000: .Jan 11 2012 19:29:02.292<br>
EST: Telnet2: Sent SB 24 1 '<br>
Incoming log entry; line='<183>7001: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL WINDOW-SIZE (31)'<br>
Incoming log entry; line='<183>7002: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL TTY-SPEED (32) (refused)'<br>
Incoming log entry; line='<183>7003: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent DONT TTY-SPEED (32)'<br>
Incoming log entry; line='<183>7004: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL LOCAL-FLOW (33) (refused)'<br>
Incoming log entry; line='<183>7005: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent DONT LOCAL-FLOW (33)'<br>
Incoming log entry; line='<183>7006: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL LINEMODE (34)'<br>
Incoming log entry; line='<183>7007: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent DONT LINEMODE (34) (unimplemented)'<br>
Incoming log entry; line='<183>7008: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received WILL NEW-ENVIRON (39)'<br>
Incoming log entry; line='<183>7009: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent DONT NEW-ENVIRON (39) (unimplemented)'<br>
Incoming log entry; line='<183>7010: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received DO STATUS (5)'<br>
Incoming log entry; line='<183>7011: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet sent WONT STATUS (5) (unimplemented)'<br>
Incoming log entry; line='<183>7012: .Jan 11 2012 19:29:02.292<br>
EST: TCP2: Telnet received DO ECHO (1)'<br>
Incoming log entry; line='<183>7013: .Jan 11 2012 19:29:02.292<br>
EST: Telnet2: recv SB NAWS 132 63'<br>
Incoming log entry; line='<183>7014: .Jan 11 2012 19:29:02.292<br>
EST: Telnet2: recv SB 24 0 LINUX'<br>
Incoming log entry; line='<183>7015: .Jan 11 2012 19:29:02.493<br>
EST: TCP2: Telnet received WILL ENVIRONMENT (36) (refused)'<br>
Incoming log entry; line='<183>7016: .Jan 11 2012 19:29:02.493<br>
EST: TCP2: Telnet sent DONT ENVIRONMENT (36)'<br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
Clayton Dukes<br>
______________________________<u></u>______________________________<u></u>__<br>
<br>
<br>
<br>
</div></div></blockquote>
This is a mailing list, repeating yourself isnt going to help. Not everyone checks it every few hours. Have patience.<br>
<br>
</blockquote></div><br></div>