RE: [syslog-ng] flow-control and pipes

27 Jul 2006

      Hi everybody

I think maybe I should be a bit more specifik in my question regarding
flow-control and pipes.

For testing I am reading logs from a file and feeding them into
syslog-ng through a pipe named "from_collect_10_45_1_41_to_syslog.pipe".

Syslog-ng then pipes these logs to a script called archiver.pl through a
pipe called "from_syslog_to_archivar.pipe".

Archiver.pl simply write the logs to files, and at the same time pipes
the logs back to syslog-ng through a pipe called
"from_archivar_to_syslog.pipe".

Last syslog-ng now write these logs to files.

The idea is that the logs going TO  archivar.pl are flow-controlled, and
the logs going FROM archivar.pl are not. I should think that the
flowcontrol would

work in the sence that all logs would arrive to archiver.pl, regardless
of a possible loss of logs after they leave archiver.pl . but this seems
not to be the case.

There is an substantial loss of logs arriving to archive.pl regardless
of the flowcontrol. 

If, on the other hand, i have a script reading from
"from_archivar_to_syslog.pipe" instead of syslog-ng, it all works.

############# OPTIONS
########################################################################
##################

options {

        long_hostnames(off);

        keep_hostname(yes);

        use_dns(no);

        flush_timeout(0);

        flush_lines(1);

        sync(0);

};

############## SOURCES
########################################################################
#################

source collect_10_45_1_41 {

pipe("/opt/logging/logs/data/pipes/from_collect_10_45_1_41_to_syslog.pip
e"  log_iw_size(2) log_fetch_limit(1) );

};

source archivar {

       pipe("/opt/logging/logs/data/pipes/from_archivar_to_syslog.pipe"
log_iw_size(2) log_fetch_limit(1)   );

};

############# DESTINATIONS
########################################################################
#############

destination archivar {

        pipe("/opt/logging/logs/data/pipes/from_syslog_to_archivar.pipe"
log_fifo_size(3)

        template("$FULLHOST/$YEAR/$MONTH/$DAY<CSIS>$DATE $FULLHOST
$MESSAGE\n ")  );

};

destination messages {

       file("/opt/logging/logs/data/pipes/messages" log_fifo_size(3)

       template("$MESSAGE\n")  );

};

############# LOGS
########################################################################
####################

log {

      source(collect_10_45_1_41);

      destination(archivar);

      flags(flow-control, final);

};

log {

      source(archivar);

      destination(messages);

};

I really hope someone can help, since I have tried allmost anything with
no result.

Kind Regards 

Steen Thomsen

softwaredeveloper

<http://www.bridicum.com/> 

<http://www.bridicum.com/>