I need to have a highly available centralized log server (all Linux). Is there a way to configure syslog-ng to automatically detect when the remote syslog server has gone down so that it can redirect messages elsewhere? Something like this: destination d_tcp1 { tcp("192.168.1.1"); failover(d_tcp2); }; destination d_tcp2 { tcp("192.168.1.2"); }; If not, anyone know of any Linux loggers that will do this? -Jon ----------------------------------------- The information contained in this email message and its attachments is intended only for the private and confidential use of the recipient(s) named above, unless the sender expressly agrees otherwise. Transmission of email over the Internet is not a secure communications medium. If you are requesting or have requested the transmittal of personal data, as defined in applicable privacy laws by means of email or in an attachment to email, you must select a more secure alternate means of transmittal that supports your obligations to protect such personal data. If the reader of this message is not the intended recipient and/or you have received this email in error, you must take no action based on the information in this email and you are hereby notified that any dissemination, misuse or copying or disclosure of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email and delete the original message.
I don't think syslog-ng supports this nor do I know of any other logger that does. Would it be acceptable to have it log to 2 different log servers, that way, you would get a measure of high availability. In any case, if you're using tcp destinations then it will hold the logs until the server becomes available again (up to a point - then any following logs will be lost) You could give your clients massive fifos to try to buy more time (well more log store actually), I think this will make them more resilient to server unavailability as they can cache more logs. Anyone else got any bright ideas on this? -h Hari Sekhon Jonathon Blumenthal wrote:
I need to have a highly available centralized log server (all Linux).
Is there a way to configure syslog-ng to automatically detect when the remote syslog server has gone down so that it can redirect messages elsewhere?
Something like this: destination d_tcp1 { tcp("192.168.1.1"); failover(d_tcp2); }; destination d_tcp2 { tcp("192.168.1.2"); };
If not, anyone know of any Linux loggers that will do this?
-Jon
----------------------------------------- The information contained in this email message and its attachments is intended only for the private and confidential use of the recipient(s) named above, unless the sender expressly agrees otherwise. Transmission of email over the Internet is not a secure communications medium. If you are requesting or have requested the transmittal of personal data, as defined in applicable privacy laws by means of email or in an attachment to email, you must select a more secure alternate means of transmittal that supports your obligations to protect such personal data. If the reader of this message is not the intended recipient and/or you have received this email in error, you must take no action based on the information in this email and you are hereby notified that any dissemination, misuse or copying or disclosure of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email and delete the original message.
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Hari Sekhon wrote:
I don't think syslog-ng supports this nor do I know of any other logger that does.
Would it be acceptable to have it log to 2 different log servers, that way, you would get a measure of high availability.
In any case, if you're using tcp destinations then it will hold the logs until the server becomes available again (up to a point - then any following logs will be lost)
You could give your clients massive fifos to try to buy more time (well more log store actually), I think this will make them more resilient to server unavailability as they can cache more logs.
Anyone else got any bright ideas on this?
-h
There are all kinds of architectures that go a long way to addressing this. One example; Set up a pair of syslog servers that are configures as a high availablity active/passive pair. These servers should share the SAME logging space. This space can be a clustered filessystem which would be best, or could be an network mount. They could even be a single drive/lun/iscsi device. The two servers keep a heartbeat and when one goes down, the other takes over the syslog service and the associated IP address. - Server A - shutdown syslog-ng - Server A - unmount log space (clustered filesystem and network mount don't need this) - Server A - release IP address - Server B - take over IP address - Server B - mount log space - Server B - start syslog-ng This could even happen automatically every 5 minutes so that the situations where some servers can not reach Server A and others can not reach Server B, they would all buffer for 5 minutes and then dump the messages to the server in the next 5 minutes. This does make the log timestamps correct, but out of order on the syslog server :-( Another example is to use 2 syslog servers, but then you have to merge the log files, which may be non trivial if you have lots of log messages (we have 10G each day). Finally, you can wait for someone to implement disk based buffering in syslog-ng. It is on the wish list, but there are only so many hours in a day :-( Evan.
I am new to syslog-ng product which has been working great so far. My question is on filtering unwanted syslog messages during collection. Can I accomplish this using syslog-ng? If so, can some one tell me its possible to filter out messages from specific host or a wildcard (subnet) etc. Thanks Ramesh
Hi there. Yes, syslog-ng has of those features that you want. Check the syslog-ng.conf expanded file. Here you can see those kind of things being used. Using templates it is possible for syslog-ng to create dirs and files based on templates. http://www.campin.net/syslog-ng/expanded-syslog-ng.conf Regards, Bruno. On 3/15/07, Ramesh Uppuluri <ruppuluri@netcordia.com> wrote:
I am new to syslog-ng product which has been working great so far. My question is on filtering unwanted syslog messages during collection. Can I accomplish this using syslog-ng? If so, can some one tell me its possible to filter out messages from specific host or a wildcard (subnet) etc.
Thanks Ramesh _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Hi, Jonathon Blumenthal <jonathon.blumenthal@ironmountain.com> [20070315 10:35:58 -0400]:
I need to have a highly available centralized log server (all Linux).
Is there a way to configure syslog-ng to automatically detect when the remote syslog server has gone down so that it can redirect messages elsewhere?
I submitted a patch that adds multicast support to syslog-ng so that more than one syslog server will receive the messages. Of course this is over UDP though but not a problem for us. The nice side effect of using multicast is that you can from your own workstation 'tune into' the syslog messages and so you do not even need to log into the servers to tail the ends of your log files. If you do go down this path then I recommend you group services to particular multicast group addresses (with a 32 IP address spacing between groups for technical reasons), for example SMTP traffic to 239.192.0.0 whilst IMAP traffic to 239.192.0.32. This means you can easily tune-into the stream you want to, it also makes your syslog-ng.conf file far simpler. I think the approach you are using is slightly wrong, you should not 'failover' but instead send the data to both syslog servers. Its no use having a syslog server crash and burn with all your data on it and the second one having the data from only the point where the other one died.
Something like this: destination d_tcp1 { tcp("192.168.1.1"); failover(d_tcp2); }; destination d_tcp2 { tcp("192.168.1.2"); };
If not, anyone know of any Linux loggers that will do this?
If you need tcp why don't you just use: destination d_tcp { tcp("192.168.1.1"); tcp("192.168.1.2"); }; Cheers Alex
-Jon
----------------------------------------- [snipped excessive disclaimer regarding kitchen sink] _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
participants (6)
-
Alexander Clouter
-
Bruno Vieira
-
Evan Rempel
-
Hari Sekhon
-
Jonathon Blumenthal
-
Ramesh Uppuluri