Internal error, duplicate configuration elements...
Hi, I've been playing around with configuring UDP destinations. My syntax checks ('syslog-ng -s') and high-level operation appear to be sound. However, I've noticed the following "internal error" that's issued whenever I start the service: 20140604 09:54:47.780 err syslog(syslog-ng):Internal error, duplicate configuration elements refer to the same persistent config; name='afsocket_dd_connection(dgram,205.159.216.207:514)' The UDP configuration references from 'syslog-ng.conf' are as follows: destination d_NAaudit_Prio { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; destination d_NAmessage_Prio { udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; Does anyone know what this message is referring too? Things seem to work fine (e.g., log statements with these destinations do, ultimately, send the filtered messages to the target system), so I'm wondering if I need to be concerned at all. Thanks, -David
David Hauck <davidh@netacquire.com> writes:
20140604 09:54:47.780 err syslog(syslog-ng):Internal error, duplicate configuration elements refer to the same persistent config; name='afsocket_dd_connection(dgram,205.159.216.207:514)'
The UDP configuration references from 'syslog-ng.conf' are as follows:
destination d_NAaudit_Prio { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; destination d_NAmessage_Prio { udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); };
Does anyone know what this message is referring too?
It refers to having an udp("205.159.216.207" port(514) template(t_NAFormat_Prio)) destination in the config twice: once in d_NAaudit_Prio and once more in d_NAmessage_Prio. This is an error in the config, something which can easily confuse syslog-ng. In general, you should use a certain target in only one destination, and if you want to send to the same destination from multiple sources, or via different filters, use log{} blocks to tie them together. I'd recommend structuring your config differently for the above reasons: instead of describing *what* goes to a particular destination, name them after *where* they go to, and bind the source->filter->destination chain together in a log{} block. For example: destination d_205 { udp(...); }; destination d_na_audit { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); }; log { source(s_all); destination(d_205); filter(f_na_audit); destination(d_na_audit); }; This will send every message from s_all source to the remote server, then filter some, and put them in a file. Hope that helps! -- |8]
First of all, thanks for the response. Please see below. This morning Gergely Nagy, syslog-ng-bounces@lists.balabit.hu wrote:
David Hauck <davidh@netacquire.com> writes:
20140604 09:54:47.780 err syslog(syslog-ng):Internal error, duplicate configuration elements refer to the same persistent config; name='afsocket_dd_connection(dgram,205.159.216.207:514)'
The UDP configuration references from 'syslog-ng.conf' are as follows:
destination d_NAaudit_Prio { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; destination d_NAmessage_Prio { udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); };
Does anyone know what this message is referring too?
It refers to having an udp("205.159.216.207" port(514) template(t_NAFormat_Prio)) destination in the config twice: once in d_NAaudit_Prio and once more in d_NAmessage_Prio. This is an error in the config, something which can easily confuse syslog-ng. In general, you should use a certain target in only one destination, and if you want to send to the same destination from multiple sources, or via different filters, use log{} blocks to tie them together.
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such? 2. Other than the error, things appear to function correctly. Why is this? 3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks? Cheers, -David
I'd recommend structuring your config differently for the above reasons: instead of describing *what* goes to a particular destination, name them after *where* they go to, and bind the source->filter->destination chain together in a log{} block.
For example:
destination d_205 { udp(...); }; destination d_na_audit { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); }; log { source(s_all); destination(d_205); filter(f_na_audit); destination(d_na_audit); };
This will send every message from s_all source to the remote server, then filter some, and put them in a file.
Hope that helps!
David Hauck <davidh@netacquire.com> writes:
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks?
Because filters generally don't have state, so they can't conflict with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this: destination d_1 { file("/var/log/some.log"); udp(...); }; destination d_2 { file("/var/log/some.log"); tcp(...); }; That would break horribly, as two threads would try to write to the same file, and would write over each other, resulting in garbage. That doesn't happen with network destinations, but those have different issues. For example, if a network target goes down, and you had duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver. -- |8]
This morning Gergely Nagy wrote:
David Hauck <davidh@netacquire.com> writes:
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case). In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks?
Because filters generally don't have state, so they can't conflict with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this:
destination d_1 { file("/var/log/some.log"); udp(...); }; destination d_2 { file("/var/log/some.log"); tcp(...); };
That would break horribly, as two threads would try to write to the same file, and would write over each other, resulting in garbage. That doesn't happen with network destinations, but those have different issues. For example, if a network target goes down, and you had duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases. Thanks, -David
Hi, the issue is that syslog-ng uses the destination IP address/port combination as an identifier to recover the queue when reloading syslog-ng, so it does have some issues (e.g. one borrowing the queue of the other, or using the same queue after reload). This is not good. This could be solved with a long-discussed, explicit id() parameter, that would let the administrator assign a custom ID, which can be made different for the conflicting drivers, but that hasn't happened yet. Using junctions and log expressions could help you to reorganize your configuration in a way that prevent this kind of conflict: https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improvements/ Hope this helps. On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh@netacquire.com> wrote:
This morning Gergely Nagy wrote:
David Hauck <davidh@netacquire.com> writes:
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case).
In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks?
Because filters generally don't have state, so they can't conflict with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this:
destination d_1 { file("/var/log/some.log"); udp(...); }; destination d_2 { file("/var/log/some.log"); tcp(...); };
That would break horribly, as two threads would try to write to the same file, and would write over each other, resulting in garbage. That doesn't happen with network destinations, but those have different issues. For example, if a network target goes down, and you had duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases.
Thanks, -David
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
Hi Balazs, On Tuesday, June 10, 2014 11:51 PM, Balazs Scheidler wrote:
Hi,
the issue is that syslog-ng uses the destination IP address/port combination as an identifier to recover the queue when reloading syslog-ng, so it does have some issues (e.g. one borrowing the queue of the other, or using the same queue after reload). This is not good.
Ah, OK, yes, this sounds like a problem.
This could be solved with a long-discussed, explicit id() parameter, that would let the administrator assign a custom ID, which can be made different for the conflicting drivers, but that hasn't happened yet.
Yup, good idea.
Using junctions and log expressions could help you to reorganize your configuration in a way that prevent this kind of conflict:
https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improvem en ts/
Thanks, I'll see what I can do. BTW, would this "duplicate UDP destination" work any better if I had a policy where no re-transmissions occurred (i.e., I don't need any queuing for the purposes of re-transmits when the destinations are down)? If so, how would I configure this?
Hope this helps.
Cheers, -David
On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh@netacquire.com> wrote:
This morning Gergely Nagy wrote: > David Hauck <davidh@netacquire.com> writes: > >> Couple things here: >> 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such? > > Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case).
In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have >> statements that intersect. Why not destination blocks? > > Because filters generally don't have state, so they can't conflict > with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this: > > destination d_1 { file("/var/log/some.log"); udp(...); }; destination > d_2 { file("/var/log/some.log"); tcp(...); }; > > That would break horribly, as two threads would try to write to the > same file, and would write over each other, resulting in garbage. That > doesn't happen with network destinations, but those have different > issues. For example, if a network target goes down, and you had > duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases.
Thanks, -David
__________________________________________________________ ______________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Reorganizing the log configuration with junctions helps in some circumstances. In my case I need to use a different template depending on the source of the message (from a specific file source) so I need to use the same destination IP/port, but defined with a different syslog-ng "destination" specification so that I can use a different template. This results in the same problem that I can't work around by using a different junction/log structure. Evan. On 06/10/2014 11:51 PM, Balazs Scheidler wrote:
Hi,
the issue is that syslog-ng uses the destination IP address/port combination as an identifier to recover the queue when reloading syslog-ng, so it does have some issues (e.g. one borrowing the queue of the other, or using the same queue after reload). This is not good.
This could be solved with a long-discussed, explicit id() parameter, that would let the administrator assign a custom ID, which can be made different for the conflicting drivers, but that hasn't happened yet.
Using junctions and log expressions could help you to reorganize your configuration in a way that prevent this kind of conflict:
https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improvements/
Hope this helps.
On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> wrote:
This morning Gergely Nagy wrote: > David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> writes: > >> Couple things here: >> 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such? > > Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
>> 2. Other than the error, things appear to function correctly. Why is this? > > That'd be a bit hard to explain without going into low-level details, > but I'll try: it works because of luck, mostly. UDP destinations will > not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case).
In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
>> 3. Other blocks do not require that their contents contain unique >> statements. For example, I can create filter blocks that have >> statements that intersect. Why not destination blocks? > > Because filters generally don't have state, so they can't conflict > with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this: > > destination d_1 { file("/var/log/some.log"); udp(...); }; destination > d_2 { file("/var/log/some.log"); tcp(...); }; > > That would break horribly, as two threads would try to write to the > same file, and would write over each other, resulting in garbage. That > doesn't happen with network destinations, but those have different > issues. For example, if a network target goes down, and you had > duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases.
Thanks, -David ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
Hi Evan, On Wednesday, June 11, 2014 8:16 AM, syslog-ng-bounces@lists.balabit.hu wrote:
Reorganizing the log configuration with junctions helps in some circumstances. In my case I need to use a different template depending on the source of the message (from a specific file source) so I need to use the same destination IP/port, but defined with a different syslog-ng "destination" specification so that I can use a different template.
This results in the same problem that I can't work around by using a different junction/log structure.
Yes, interesting. This restriction is turning out to be somewhat unfortunate. I'm still trying to figure out how to wedge this constraint into my configuration... Thanks, -David
Evan.
On 06/10/2014 11:51 PM, Balazs Scheidler wrote:
Hi,
the issue is that syslog-ng uses the destination IP address/port combination as an identifier to recover the queue when reloading syslog-ng, so it does have some issues (e.g. one borrowing the queue of the other, or using the same queue after reload). This is not good.
This could be solved with a long-discussed, explicit id() parameter, that would let the administrator assign a custom ID, which can be made different for the conflicting drivers, but that hasn't happened yet.
Using junctions and log expressions could help you to reorganize your configuration in a way that prevent this kind of conflict:
https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improv em ents/
Hope this helps.
On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> wrote:
This morning Gergely Nagy wrote:
David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> writes:
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case).
In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks?
Because filters generally don't have state, so they can't conflict with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this:
destination d_1 { file("/var/log/some.log"); udp(...); }; destination d_2 { file("/var/log/some.log"); tcp(...); };
That would break horribly, as two threads would try to write to the same file, and would write over each other, resulting in garbage. That doesn't happen with network destinations, but those have different issues. For example, if a network target goes down, and you had duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases.
Thanks, -David
____________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
__________________________________________________________ ____________
________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
A workaround is to use a DNS alias, or different port. The Id generated by syslog-ng contains both. On Jun 11, 2014 5:18 PM, "David Hauck" <davidh@netacquire.com> wrote:
Hi Evan,
On Wednesday, June 11, 2014 8:16 AM, syslog-ng-bounces@lists.balabit.hu wrote:
Reorganizing the log configuration with junctions helps in some circumstances. In my case I need to use a different template depending on the source of the message (from a specific file source) so I need to use the same destination IP/port, but defined with a different syslog-ng "destination" specification so that I can use a different template.
This results in the same problem that I can't work around by using a different junction/log structure.
Yes, interesting. This restriction is turning out to be somewhat unfortunate. I'm still trying to figure out how to wedge this constraint into my configuration...
Thanks, -David
Evan.
On 06/10/2014 11:51 PM, Balazs Scheidler wrote:
Hi,
the issue is that syslog-ng uses the destination IP address/port combination as an identifier to recover the queue when reloading syslog-ng, so it does have some issues (e.g. one borrowing the queue of the other, or using the same queue after reload). This is not good.
This could be solved with a long-discussed, explicit id() parameter, that would let the administrator assign a custom ID, which can be made different for the conflicting drivers, but that hasn't happened yet.
Using junctions and log expressions could help you to reorganize your configuration in a way that prevent this kind of conflict:
https://bazsi.blogs.balabit.com/2012/01/syslog-ng-flexibility-improv em ents/
Hope this helps.
On Fri, Jun 6, 2014 at 4:15 PM, David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> wrote:
This morning Gergely Nagy wrote:
David Hauck <davidh@netacquire.com <mailto:davidh@netacquire.com>> writes:
Couple things here: 1. If this is an error why doesn't 'syslog-ng -s' indicate it as such?
Because -s checks for syntax only, and this is not a syntax error, it's one level higher.
OK.
2. Other than the error, things appear to function correctly. Why is this?
That'd be a bit hard to explain without going into low-level details, but I'll try: it works because of luck, mostly. UDP destinations will not trip over each other when used like this. If you'd try that with files, all hell would break loose.
OK, thanks, having some understanding of how the low level implementation works does help since I don't remember this constraint being discussed in the manual/docs (but I may have missed something there?). Initially it also wasn't clear that these destinations weren't combined at a high level in the implementation so that they, for example, operated within a "pipeline" and thereby enforced a "single-writer" paradigm. Either way, and as you say, this is required for file destinations, but certainly not for network destinations (even in the disconnect case).
In fact, my log() statements with the referenced destinations are complex enough that I may need to rely on the multiple network destination specification capability anyway since combining these as you suggest would make maintenance of the configuration even more difficult (there's a tangle of rules/conditions associated with one of the destination log() statements).
3. Other blocks do not require that their contents contain unique statements. For example, I can create filter blocks that have statements that intersect. Why not destination blocks?
Because filters generally don't have state, so they can't conflict with each other, they can work independently. For destinations, that's not the case. For example, if you'd have something like this:
destination d_1 { file("/var/log/some.log"); udp(...); }; destination d_2 { file("/var/log/some.log"); tcp(...); };
That would break horribly, as two threads would try to write to the same file, and would write over each other, resulting in garbage. That doesn't happen with network destinations, but those have different issues. For example, if a network target goes down, and you had duplicates in your config, then they'd notice the target being down separately, increasing the chance you'll loose logs. If you had one target, and bound it to various sources in a log{} statement instead, then you'd only have one instance of the driver.
I'm wondering if there might be other issues associated with multiple (common) network destinations (you only mention one as an example)? The example you mention isn't relevant in my case. Whether the network destination is a combined statement/driver or not the payload includes single packets, which are written separately, no? If this is the case it wouldn't matter which packet send detected the network error - one or the other would get lost in both cases.
Thanks, -David
____________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
__________________________________________________________ ____________
________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Balazs Scheidler
-
David Hauck
-
Evan Rempel
-
Gergely Nagy