[syslog-ng] [EXTERNAL] Re: Failover/Hi Availability

[Attila Szakacs (aszakacs)]

I see, thanks!

You mentioned that you want to send to Splunk. I don't think writing the same file on a network storage is the way to go. Have you read the blogpost about sending to Splunk with syslog-ng through http event collectors (HEC) [1]? I believe, you can send to any number of HEC instances from any number of syslog-ng servers.

Nonetheless, I am not an expert in Splunk, but they have a Slack channel [2], where I have heard there are syslog-ng related discussions. It might be worth a try.

Cheers,
Attila

[1] https://www.splunk.com/en_us/blog/tips-and-tricks/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splunk.html
[2] https://docs.splunk.com/Documentation/Community/1.0/community/Chat




________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine at nasa.gov>
Sent: Friday, January 15, 2021 4:51 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] [EXTERNAL] Re: Failover/Hi Availability

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


For now it's just an idea for an improved syslog-ng infrastructure to support Splunk.



Not on the same machine but multiple, at least two servers, each with their own syslog-ng instance, receiving data from dozens of hosts (the same hosts) and yes writing to a shared NFS file system.



The only substantial problem I have left in the (as of now) hypothetical scenario is that I need them all to write to the same location so that if one goes down we don't have to do any manual switching of any sort and none of the data fails to be indexed by Splunk.





Thanks,

Mark





From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> On Behalf Of Attila Szakacs (aszakacs)
Sent: Friday, January 15, 2021 03:07
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: [EXTERNAL] Re: [syslog-ng] Failover/Hi Availability



Hi Mark,



Please elaborate a bit more about your architecture. Relevant config parts are also appreciated.



Are there two syslog-ng servers running on the same machine?

Or is it just one syslog-ng instance with two different log sources (on different ports for example), and there is a host which sends to both log sources?

Maybe there are 2 different machines running their own syslog-ng, but using a network shared file system?



Thanks!



Cheers,

Attila





________________________________

From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine at nasa.gov<mailto:mark.faine at nasa.gov>>
Sent: Thursday, January 14, 2021 9:21 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: [syslog-ng] Failover/Hi Availability



CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.



If I have two Syslog-ng servers receiving logs from the same source and writing to the exact same file system destination,  is there a way to do this without running into issues with the two instances clobbering the files or file locking issues?





Thanks,

-Mark




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20210118/af6f3a5b/attachment.html>