Alternate logging destination
Hello! I would like to design a centralized logging system with 50 edge nodes and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate destination? For example if the centralized TCP destionation server is down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again. Any ideas? Thanks in advance: Balazs
I can't answer that but if this is a Linux HA-Linux might help you out. It is pretty simple to setup and works well in those types of failovers. http://www.linux-ha.org/ Nick Baronian On 11/6/06, Szeti, Balazs <szeti.balazs@hp.com> wrote:
Hello!
I would like to design a centralized logging system with 50 edge nodes and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate destination? For example if the centralized TCP destionation server is down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
Any ideas?
Thanks in advance: Balazs _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Dear Balazs yes there is a possibility but both syslog srvs will work at the same time which means yr node will send their logs to 2 syslog servers. for further send me the list of those devices ill let u know how to configure them to send logs to multiple srvs for example in MAX TNT access srvs its possible to set 2 logging servers but iam not sure abt this that devices are that mush inteligent that they make their own decisions in case of 1st server down it will re route yr log packet towards another syslog srv. Experience d0es n0t Alwayz Come With 0ld Age. Thanks Good Day, Farhan Ali Khan Network Operations Center CYBERNET "To solve a problem or to reach a goal, you...don't need to know all the answers in advance. But you must have a clear idea of the problem or the goal you want to reach." W. Clement Stone Disclaimer: The information in this email and in any files transmitted with it, is intended only for the addressee and may contain confidential and/or privileged material. Access to this email by anyone else is unauthorized. If you receive this in error, please contact the sender immediately and delete the material from any computer. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is strictly prohibited. Statement and opinions expressed in this e-mail are mine, and do not necessarily reflect those of my employer. ----- Original Message ----- From: Nick Baronian <kvetch@gmail.com> Date: Monday, November 6, 2006 8:40 pm Subject: Re: [syslog-ng] Alternate logging destination
I can't answer that but if this is a Linux HA-Linux might help you out. It is pretty simple to setup and works well in those types of failovers. http://www.linux-ha.org/
Nick Baronian
On 11/6/06, Szeti, Balazs <szeti.balazs@hp.com> wrote:
Hello!
I would like to design a centralized logging system with 50 edge nodes> and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate> destination? For example if the centralized TCP destionation server is down, the edge node syslog-ng may log in to a local file, so the logs> can be reached later manually. When the center server in online again syslog-ng may log online again.
Any ideas?
Thanks in advance: Balazs _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog- ng/faq.html>
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Mon, 2006-11-06 at 16:33 +0100, Szeti, Balazs wrote:
Hello!
I would like to design a centralized logging system with 50 edge nodes and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate destination? For example if the centralized TCP destionation server is down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
you could always do a HA cluster. -- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot
Hello! Do you mean having more central servers? I'm rather affraid of loosing the network connection between the center and the edge node (dedicated private network). So I'd like to have the logs stored locally while the connection is not established. Balazs -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Matt Zagrabelny Sent: Monday, November 06, 2006 4:41 PM To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] Alternate logging destination On Mon, 2006-11-06 at 16:33 +0100, Szeti, Balazs wrote:
Hello!
I would like to design a centralized logging system with 50 edge nodes
and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate
destination? For example if the centralized TCP destionation server is
down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
you could always do a HA cluster. -- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot
On Mon, 2006-11-06 at 16:49 +0100, Szeti, Balazs wrote:
Hello!
Do you mean having more central servers? I'm rather affraid of loosing the network connection between the center and the edge node (dedicated private network). So I'd like to have the logs stored locally while the connection is not established.
i would then just have all the edge nodes log locally (as well as to the central log server) and use a high frequency turnover rate with logrotate to reduce disk usage on the nodes. that would probably be the easiest solution. otherwise you would have to test to see if the log server is alive before sending the logs to it. it would be *much* easier to send the logs to ther server as well as locally than testing if the log server is "alive" and if it isnt log locally.
Balazs
-----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Matt Zagrabelny Sent: Monday, November 06, 2006 4:41 PM To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] Alternate logging destination
On Mon, 2006-11-06 at 16:33 +0100, Szeti, Balazs wrote:
Hello!
I would like to design a centralized logging system with 50 edge nodes
and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate
destination? For example if the centralized TCP destionation server is
down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
you could always do a HA cluster.
-- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2
He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Matt Zagrabelny - mzagrabe@d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot
Hi, Szeti, Balazs <szeti.balazs@hp.com> [20061106 16:33:57 +0100]:
Hello!
I would like to design a centralized logging system with 50 edge nodes and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate destination? For example if the centralized TCP destionation server is down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
Any ideas?
Its all over UDP but I helped add multicast support to to do just this. The network duplicates the syslog messages to each 'core' syslog server so it does not matter if one f the boxes disappears. I'm still pondering about sync'ing/diff'ing the differences[1] however for the effort you would need to put in for a heartbeat system, this solution wins...in my book anyway :) Cheers Alex [1] I don't think its a big problem as you really only need to bear that there could be differences and so should grep both log files for the time frame
Thanks in advance: Balazs _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Thanks for the aswers, but as I wrote before, I'm rather affraid of network connection error (I'll have failover servers in the center, but the network line is a SPOF). Unfortunately syslog-ng doesn't give any response if a destination is unreachable (e.g. the destination file is deleted!). It writes in the internal log if it couldn't connect to destination TCP port on startup, but no error log or negative response when trying to send the log over the "missing" destination (file or TCP). So I can't find out whether my logging was succesfull or not. Balazs -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Alexander Clouter Sent: Monday, November 06, 2006 5:46 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Alternate logging destination Hi, Szeti, Balazs <szeti.balazs@hp.com> [20061106 16:33:57 +0100]:
Hello!
I would like to design a centralized logging system with 50 edge nodes
and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate
destination? For example if the centralized TCP destionation server is
down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
Any ideas?
Its all over UDP but I helped add multicast support to to do just this. The network duplicates the syslog messages to each 'core' syslog server so it does not matter if one f the boxes disappears. I'm still pondering about sync'ing/diff'ing the differences[1] however for the effort you would need to put in for a heartbeat system, this solution wins...in my book anyway :) Cheers Alex [1] I don't think its a big problem as you really only need to bear that there could be differences and so should grep both log files for the time frame
Thanks in advance: Balazs _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Hi, Szeti, Balazs <szeti.balazs@hp.com> [20061106 18:18:46 +0100]:
Thanks for the aswers, but as I wrote before, I'm rather affraid of network connection error (I'll have failover servers in the center, but the network line is a SPOF). Unfortunately syslog-ng doesn't give any response if a destination is unreachable (e.g. the destination file is deleted!). It writes in the internal log if it couldn't connect to destination TCP port on startup, but no error log or negative response when trying to send the log over the "missing" destination (file or TCP). So I can't find out whether my logging was succesfull or not.
What I would do is get each of the end 'nodes' to log to some partition on the local machine and then rsync/scp/ftp/whatever any log files that have not been successfully transferred over every hour/day/week. If you want a *guarentee* system then you usually are comprimising on the 'liveness' of the data on the central machine. If you do not care about an hour lag (or even a day) then I would log locally and transfer the files using a cron job. If you need live data too then you could use a combination of both syslog over the network and this scheduled reliable uploading of your log data. To confirm the otherend got the logs intact you could just md5sum your log files at either end. Cheers Alex
Balazs
-----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Alexander Clouter Sent: Monday, November 06, 2006 5:46 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Alternate logging destination
Hi,
Szeti, Balazs <szeti.balazs@hp.com> [20061106 16:33:57 +0100]:
Hello!
I would like to design a centralized logging system with 50 edge nodes
and one center. It's quiet important to have all the logs even if the center is unreachable. Is there a way to configure syslog-ng to use an alternate
destination? For example if the centralized TCP destionation server is
down, the edge node syslog-ng may log in to a local file, so the logs can be reached later manually. When the center server in online again syslog-ng may log online again.
Any ideas?
Its all over UDP but I helped add multicast support to to do just this. The network duplicates the syslog messages to each 'core' syslog server so it does not matter if one f the boxes disappears.
I'm still pondering about sync'ing/diff'ing the differences[1] however for the effort you would need to put in for a heartbeat system, this solution wins...in my book anyway :)
Cheers
Alex
[1] I don't think its a big problem as you really only need to bear that
there could be differences and so should grep both log files for the time frame
Thanks in advance: Balazs _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
participants (5)
-
Alexander Clouter
-
Farhan .
-
Matt Zagrabelny
-
Nick Baronian
-
Szeti, Balazs