[syslog-ng]Strang behaviour of syslog-ng (and my program)
Antai Ning
aning@jetnet.ca
Thu, 11 Jul 2002 10:51:15 -0400
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C228EA.6808A310
Content-Type: text/plain;
charset="iso-8859-1"
syslog-ng 1.4.11 on Redhat linux 7.2. And here is the syslog-ng globla
option look like. (Sorry I can't post all the configuration because of
obvious reasons :) ) But there is no options set for specific source,
destionation or filter.
options {
use_dns (no); # Don't perform DNS lookups
use_fqdn (no); # Don't use full hostname
create_dirs (yes); # Create any necessary dirs for files
use_time_recvd (yes); # Use our time, not the remote device time
log_fifo_size(10000); # Queue 10000 msgs on a blocked dest
sync(0); # Flush the queues as fast as possible
# Tweak the sync parameter if the disks are
# going wild and thrashing the machine
chain_hostnames(on); # Chain hostname/IP information in log line
};
source s_UDP
{
udp( ip(192.168.1.1) port(514) );
};
destination d_alert
{
pipe ("/usr/local/var/ALERT");
};
filter f_alert
{
match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...")
or match("blah, blah ...");
};
log { source(s_UDP);
filter(f_alert);
destination(d_alert);
};
Thanks,
Andy
-----Original Message-----
From: Michael Earls [mailto:Michael.Earls@chmcc.org]
Sent: Thursday, July 11, 2002 10:13 AM
To: aning@jetnet.ca; syslog-ng@lists.balabit.hu
Subject: RE: [syslog-ng]Strang behaviour of syslog-ng (and my program)
What ver of syslog-ng? and what distro are u running syslog-ng on? what does
your config look like?
michael
>>> aning@jetnet.ca 07/11/02 09:49AM >>>
Just add a little more info, if I send the first 10 messages a little bit
slower, say delay 10 msec in between, then it doesn't happen.
-----Original Message-----
From: Antai Ning [mailto:aning@jetnet.ca]
Sent: Wednesday, July 10, 2002 1:38 PM
To: 'syslog-ng@lists.balabit.hu'
Subject: [syslog-ng]Strang behaviour of syslog-ng (and my program)
Hi Gurus,
I got some strange phenomemon with syslog-ng. Here is what I'm doing.
I configured syslog-ng to forward certain type of messages to a pipe. (say,
ALERT). I wrote a program in perl to read from the pipe and print them out.
The perl program first try to open the pipe and it'll block at the open
function untill syslog-ng has some messages to write to the pipe. Then the
perl program reads using IO::File can_read() function, with a 5 seconds
timeout. So the function looks like this: $sel->can_read(5). Then I send 10
valid messages to syslog-ng from a remote machine by UDP. To my supprise,
the perl program got only 2 of the 10! Then I send another 1 valid message
to this syslog-ng after 5 minutes, and I got not only this message, but also
the 8 I lost last time!
This happens only for the first 10 messages (which includes the opening of
the pipe). Everything is fine after the first round. It looks like it's
related to the pipe opening and messages get queued somewhere until later
messages trigger the queue to flush.
So what's really happening inside? And is there a solution?
Thanks,
Andy
------_=_NextPart_001_01C228EA.6808A310
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 5.50.4916.2300" name=3DGENERATOR></HEAD>
<BODY style=3D"MARGIN-TOP: 2px; FONT: 8pt Tahoma; MARGIN-LEFT: 2px">
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1>syslog-ng 1.4.11 =
on Redhat=20
linux 7.2. And here is the syslog-ng globla option look like<SPAN=20
class=3D650383314-11072002>. </SPAN>(Sorry I can't post all the =
configuration<SPAN=20
class=3D650383314-11072002> because of obvious reasons :) ) But there =
is no=20
options set for specific source, destionation or=20
filter.</SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002></SPAN></FONT></SPAN> </DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>options=20
{<BR> use_dns=20
(no); # Don't perform =
DNS=20
lookups<BR> =
use_fqdn=20
(no); # Don't use full=20
hostname<BR> =
create_dirs=20
(yes); # Create any necessary dirs for=20
files<BR> =
use_time_recvd=20
(yes); # Use our time, not the remote device=20
time<BR> =20
log_fifo_size(10000); # Queue 10000 msgs on a blocked=20
dest<BR> =20
sync(0); &nbs=
p; =20
# Flush the queues as fast as=20
possible<BR> =
=
=20
# Tweak the sync parameter if the disks=20
are<BR>  =
;  =
; =20
# going wild and thrashing the=20
machine<BR> =20
chain_hostnames(on); # Chain hostname/IP information in log=20
line<BR>};<BR></SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>source s_UDP</SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>{ </SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002> =
udp(=20
ip(192.168.1.1) port(514) );</SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>};</SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002> </DIV></SPAN></FONT></SPAN>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>destination=20
d_alert<BR>{<BR> pipe=20
("/usr/local/var/ALERT");<BR>};<BR></SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002>filter f_alert<BR> =20
{<BR> match("blah, blah=20
...")<BR> or match("blah, blah=20
...")<BR> or match("blah, blah=20
...")<BR> or match("blah, blah=20
...")<BR> or match("blah, blah=20
...")<BR> or match("blah, blah=20
...")<BR> or match("blah, blah=20
...")</SPAN></FONT></SPAN></DIV>
<DIV><FONT size=3D1><SPAN=20
class=3D650383314-11072002> =
</SPAN>or=20
match("blah, blah ...")<SPAN =
class=3D650383314-11072002>;</SPAN></FONT></DIV><SPAN=20
class=3D600273114-11072002><SPAN class=3D650383314-11072002>
<DIV><FONT size=3D1>};</FONT></DIV>
<DIV><FONT size=3D1></FONT> </DIV>
<DIV><FONT size=3D1>log { =20
source(s_UDP);<BR> =
=20
filter(f_<SPAN=20
class=3D650383314-11072002>alert</SPAN>);<BR> &nb=
sp; =20
destination(d_<SPAN=20
class=3D650383314-11072002>alert</SPAN>);<BR>};<BR></FONT></SPAN></SPAN>=
</DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002></SPAN></FONT></SPAN> </DIV>
<DIV><SPAN class=3D600273114-11072002><FONT size=3D1><SPAN=20
class=3D650383314-11072002></SPAN></FONT></SPAN> </DIV>
<DIV><SPAN class=3D650383314-11072002><FONT =
size=3D1>Thanks,</FONT></SPAN></DIV>
<DIV><SPAN class=3D650383314-11072002><FONT =
size=3D1></FONT></SPAN> </DIV>
<DIV><SPAN class=3D650383314-11072002><FONT =
size=3D1>Andy</FONT></SPAN></DIV>
<BLOCKQUOTE dir=3Dltr style=3D"MARGIN-RIGHT: 0px">
<DIV class=3DOutlookMessageHeader dir=3Dltr align=3Dleft><FONT =
size=3D2>-----Original=20
Message-----<BR><B>From:</B> Michael Earls=20
[mailto:Michael.Earls@chmcc.org]<BR><B>Sent:</B> Thursday, July 11, =
2002 10:13=20
AM<BR><B>To:</B> aning@jetnet.ca;=20
syslog-ng@lists.balabit.hu<BR><B>Subject:</B> RE: [syslog-ng]Strang =
behaviour=20
of syslog-ng (and my program)<BR><BR></FONT></DIV>
<DIV><FONT size=3D2>What ver of syslog-ng? and what distro are u =
running=20
syslog-ng on? what does your config look like?</FONT></DIV>
<DIV><FONT size=3D2></FONT> </DIV>
<DIV><FONT size=3D2>michael</FONT></DIV>
<DIV><BR><BR>>>> aning@jetnet.ca 07/11/02 09:49AM=20
>>><BR></DIV>
<DIV><SPAN class=3D150044213-11072002><FONT face=3DArial =
color=3D#0000ff size=3D2>Just=20
add a little more info, if I send the first 10 messages a little bit =
slower,=20
say delay 10 msec in between, then it doesn't happen.=20
</FONT></SPAN></DIV>
<BLOCKQUOTE dir=3Dltr style=3D"MARGIN-RIGHT: 0px">
<DIV class=3DOutlookMessageHeader dir=3Dltr align=3Dleft><FONT =
face=3DTahoma=20
size=3D2>-----Original Message-----<BR><B>From:</B> Antai Ning=20
[mailto:aning@jetnet.ca]<BR><B>Sent:</B> Wednesday, July 10, 2002 =
1:38=20
PM<BR><B>To:</B> 'syslog-ng@lists.balabit.hu'<BR><B>Subject:</B>=20
[syslog-ng]Strang behaviour of syslog-ng (and my=20
program)<BR><BR></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN =
class=3D710041517-10072002>Hi=20
Gurus,</SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN =
class=3D710041517-10072002>I=20
got some strange phenomemon with syslog-ng. Here is what I'm=20
doing.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN =
class=3D710041517-10072002>I=20
configured syslog-ng to forward certain type of messages to a pipe. =
(say,=20
ALERT). I wrote a program in perl to read from the pipe and =
print them=20
out. The perl program first try to open the pipe and it'll =
block at the=20
open function untill syslog-ng has some messages to write to the =
pipe.=20
Then the perl program reads using IO::File can_read() =
function,=20
with a 5 seconds timeout. So the function looks like this:=20
$sel->can_read(5). Then I send 10 valid messages to syslog-ng =
from a=20
remote machine by UDP. To my supprise, the perl program got =
only 2 of=20
the 10! Then I send another 1 valid message to this syslog-ng =
after 5=20
minutes, and I got not only this message, but also the 8 =
I lost=20
last time! </SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002>This happens only for the first 10 =
messages (which=20
includes the opening of the pipe). Everything is fine after the =
first round.=20
It looks like it's related to the pipe opening and messages =
get queued=20
somewhere until later messages trigger the queue to=20
flush.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN =
class=3D710041517-10072002>So=20
what's really happening inside? And is there a=20
solution?</SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002>Thanks,</SPAN></FONT></DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002></SPAN></FONT> </DIV>
<DIV><FONT face=3DArial color=3D#0000ff size=3D2><SPAN=20
class=3D710041517-10072002>Andy </SPAN></FONT></DIV>
<BLOCKQUOTE dir=3Dltr style=3D"MARGIN-RIGHT: 0px"><FONT =
face=3DArial=20
color=3D#0000ff=20
size=3D2></FONT> </BLOCKQUOTE></BLOCKQUOTE></BLOCKQUOTE></BODY></HT=
ML>
------_=_NextPart_001_01C228EA.6808A310--