[syslog-ng]Urgent: Mangled messages in pipe() destinations in versions >1.5.8

Mon, 06 Jan 2003 14:03:28 +0100

Balazs Scheidler wrote:
> 
>>destination d_a { pipe("/tmp/pipe" template("@TAG_A: | $MSG\n")); };
>>destination d_b { pipe("/tmp/pipe" template("@TAG_B: | $MSG\n")); };
>>log { source(..); filter(fa); destination(d_a); };
>>log { source(..); filter(fb); destination(d_b); };
> 
> It is not easy to implement, I'd need to share the destination fd & write
> buffers between multiple, independent destination drivers...

Why to share buffers?
For me it seems, that's only necessary to preserve message
boundaries in all the final output buffers.
So, no buffer handling change is required. Only the content
of the buffers need to be adapted.
This would solve the direct cause of the problem.
And it would increase data integrity, too.
Last but not least, it would solve the problem for all other
destinations (e.g. AF_LOCAL sockets)
But, maybe I'm wrong, because I'm not very familiar with your code.

> I would do something with macro expansion instead, it would only need a
> couple of user changeable variables, like:
> 
 > destination pipe { pipe("/tmp/pipe" template("$ID_TAG: | $MSG\n")); };
 > log { .. variable("ID_TAG", "TAG_A"); destination(pipe); };
 > log { .. variable("ID_TAG", "TAG_B"); destination(pipe); };
 >
> But what happens when multiple destinations match? Are those variables
> combined, or they are cleared for each log rule, or?

Sounds great! This would solve our problem, I think. Thanks!

Ok. Combined vars are not a good idea.
What should happen in the above example, if both log paths match?
I think, the message should logged twice, once with TAG_A and once
with TAG_B.
If this means 'cleared for each log rule', this's the best idea.

Two other (more cosmetically) questions are, what happens in the
examples below? Configuration errors?

destination pipe { pipe("/tmp/pipe" template("$ID_TAG: | $MSG\n")); };
log { .. variable("ID_TAG", "TAG_A"); destination(pipe); };
log { .. variable("ID_TAG_UNUSED", "TAG_B"); destination(pipe); };
   (both log paths matches, but there is a variable, that's
    never referenced in a destination)
and
destination pipe { pipe("/tmp/pipe" template("$T_UNDEF: | $MSG\n")); };
log { .. variable("ID_TAG", "TAG_A"); destination(pipe); };
log { .. variable("ID_TAG", "TAG_B"); destination(pipe); };
   (there is a var reference, that's never defined in a log path)

> my syslog-ng tree makes it possible to match parts of messages and reference
> those as $1 .. $9 in template expansions. But those variables are cleared
> after every match().

Sounds like a very nice feature.
But it's not usable in our context. We need definable tags.
($1..$n references depends on message content)

Thanks a lot for feedback.
-- 
Best regards --Andreas Schulze
                [phone: +49.5246.80.1275, fax: +49.5246.80.2275]