[syslog-ng] Internal error, duplicate configuration elements...

[Balazs Scheidler]

Hi,

the issue is that syslog-ng uses the destination IP address/port
combination as an identifier to recover the queue when reloading syslog-ng,
so it does have some issues (e.g. one borrowing the queue of the other, or
using the same queue after reload). This is not good.

This could be solved with a long-discussed, explicit id() parameter, that
would let the administrator assign a custom ID, which can be made different
for the conflicting drivers, but that hasn't happened yet.

Using junctions and log expressions could help you to reorganize your
configuration in a way that prevent this kind of conflict:

https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improvements/

Hope this helps.


On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh at netacquire.com> wrote:

> This morning Gergely Nagy wrote:
> > David Hauck <davidh at netacquire.com> writes:
> >
> >> Couple things here:
> >> 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
> >
> > Because -s checks for syntax only, and this is not a syntax error, it's
> one level higher.
>
> OK.
>
> >> 2. Other than the error, things appear to function correctly. Why is
> this?
> >
> > That'd be a bit hard to explain without going into low-level details,
> > but I'll try: it works because of luck, mostly. UDP destinations will
> > not trip over each other when used like this. If you'd try that with
> files, all hell would break loose.
>
> OK, thanks, having some understanding of how the low level implementation
> works does help since I don't remember this constraint being discussed in
> the manual/docs (but I may have missed something there?). Initially it also
> wasn't clear that these destinations weren't combined at a high level in
> the implementation so that they, for example, operated within a "pipeline"
> and thereby enforced a "single-writer" paradigm. Either way, and as you
> say, this is required for file destinations, but certainly not for network
> destinations (even in the disconnect case).
>
> In fact, my log() statements with the referenced destinations are complex
> enough that I may need to rely on the multiple network destination
> specification capability anyway since combining these as you suggest would
> make maintenance of the configuration even more difficult (there's a tangle
> of rules/conditions associated with one of the destination log()
> statements).
>
> >> 3. Other blocks do not require that their contents contain unique
> >> statements. For example, I can create filter blocks that have
> >> statements that intersect. Why not destination blocks?
> >
> > Because filters generally don't have state, so they can't conflict
> > with each other, they can work independently. For destinations, that's
> not the case. For example, if you'd have something like this:
> >
> > destination d_1 { file("/var/log/some.log"); udp(...); }; destination
> > d_2 { file("/var/log/some.log"); tcp(...); };
> >
> > That would break horribly, as two threads would try to write to the
> > same file, and would write over each other, resulting in garbage. That
> > doesn't happen with network destinations, but those have different
> > issues. For example, if a network target goes down, and you had
> > duplicates in your config, then they'd notice the target being down
> separately, increasing the chance you'll loose logs. If you had one target,
> and bound it to various sources in a log{} statement instead, then you'd
> only have one instance of the driver.
>
> I'm wondering if there might be other issues associated with multiple
> (common) network destinations (you only mention one as an example)? The
> example you mention isn't relevant in my case. Whether the network
> destination is a combined statement/driver or not the payload includes
> single packets, which are written separately, no? If this is the case it
> wouldn't matter which packet send detected the network error - one or the
> other would get lost in both cases.
>
> Thanks,
> -David
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>


-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140611/7008a29f/attachment.htm