Hi all, I am working on a project to build distributed syslog-ing system for a very lager enterprise with offices all across the globe. Below are the main objectives: 1. Support for primarily network devices, like ciscos, netscreens, junipers etc 2. Minimum or no loss of messages when network fails 3. Central storage of all syslog messages Could you please give me your ideas or point me to any documentation that deals with such designs? Thank you!
the network devices need to be able to log to a remote syslog server, so the requirement falls on the network devices more than it falls on the syslog-ng server. regarding the second point, if one has devices across the globe logging to a central logging server, and the network links between the devices that log and the syslog-ng server is broken, i don't see how this requirement might be met. depending on the network architecture, i might have had each 'region' logging to its own highly available syslog-ng server, and sync/copy those logs to a central server/location once a while. the third requirement might be met by the periodic syncing of the logs from various syslog-ng servers across the 'regions'. i would think it depends largely on how reliable the network links are on the whole network architecture as a whole, and plan with that in consideration. also to note is that remote logging usually happens over UDP, which is unreliable by design. On Thu, 2007-05-24 at 22:37 +0530, Raghu (Lists) wrote:
Hi all,
I am working on a project to build distributed syslog-ing system for a very lager enterprise with offices all across the globe. Below are the main objectives:
1. Support for primarily network devices, like ciscos, netscreens, junipers etc 2. Minimum or no loss of messages when network fails 3. Central storage of all syslog messages
Could you please give me your ideas or point me to any documentation that deals with such designs?
Thank you! _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Hiren Patel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This e-mail and its contents are subject to the Telkom SA Limited e-mail legal notice available at http://www.telkom.co.za/TelkomEMailLegalNotice.PDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Do you need to simply store these syslog messages or do they need to be queried regularly? What is your tolerance for loss of messages when a system or network fails? If you have a 5-minute outage can you afford to lose 5-minutes of logs for devices affected by the outage? Can you afford shared external storeage on your syslog servers so you can build a real HA server pair, or will each server have it's own storage? This affects how you forward and sync data in the event of a syslog server failure. What kind of data size and network bandwidth are we talking about? The issue here is that native syslog forwarding capability works for most cases, but there is potential for loss of messages. If reliability is critical you will need to consider a store-and-forward approach so that logs can be forwarded subsequent to network downtime. Depending on data size and whether you need to query these log files regularly (or if you need to index them), you can build synchronization Perl scripts to sync logs from disparate sources. There are commercial tools you can consider as well but cost is very high for this type of distributed architecture (several hundred thousand dollars USD) On 5/24/07, Raghu (Lists) <raghu.lists1@gmail.com> wrote:
Hi all,
I am working on a project to build distributed syslog-ing system for a very lager enterprise with offices all across the globe. Below are the main objectives:
1. Support for primarily network devices, like ciscos, netscreens, junipers etc 2. Minimum or no loss of messages when network fails 3. Central storage of all syslog messages
Could you please give me your ideas or point me to any documentation that deals with such designs?
Thank you! _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Fri, 2007-05-25 at 01:25 -0700, Tom Le wrote:
There are commercial tools you can consider as well but cost is very high for this type of distributed architecture (several hundred thousand dollars USD)
Can you elaborate on these tools? I would be interested even in private, if you think it is too off-topic here. -- Bazsi
| Tom Le wrote: | > There are commercial tools you can consider as well but cost is very | > high for this type of distributed architecture (several hundred | > thousand dollars USD) | | Bazsi wrote: | Can you elaborate on these tools? I would be interested even in private, | if you think it is too off-topic here. Yes, both Splunk and LogLogic provide distributed and high volume/high performance index, archive, and search capabilities. Splunk even has distributed search, redundancy, and distributed routing logic you can apply for syslog messages. You can try out Splunks for free, 500-MB log size per day limit and some enterprise features disabled. Like I said, these deployments can be expensive vs. what you can do with syslog-ng and some home grown integration. The question really is what exactly do you need? Do you need the ability to query free form text with various compliance reports and alerts available out-of-the-box? Do you need to configure alerts based on specific events or content within the log messages (this then starts become an event monitoring discussion). Do you need specialized reports for specific device types like firewall logs, router logs, Windows logs, etc. Or is centralized collection of logs by itself sufficient (in which case, syslog-ng is all you need). In some large environments, grep'ing and processing GB's or TB's of data is not the answer so you need the high end commercial tools.
On Thu, 2007-05-24 at 22:37 +0530, Raghu (Lists) wrote:
Hi all,
I am working on a project to build distributed syslog-ing system for a very lager enterprise with offices all across the globe. Below are the main objectives:
1. Support for primarily network devices, like ciscos, netscreens, junipers etc 2. Minimum or no loss of messages when network fails 3. Central storage of all syslog messages
Could you please give me your ideas or point me to any documentation that deals with such designs?
First of all, please note that I'm working for BalaBit, so I'm biased. The Open Source Edition of syslog-ng falls short on your second objective: it will start losing messages once its memory based queues are full. However syslog-ng can be a solution, but you need the Premium Edition of syslog-ng. The Premium Edition of syslog-ng is capable of using disk-space as a message queue, so you can get through outages by allocating a large enough chunk of disk-space for syslog-ng to spool messages. The configuration would look like something like this: destination d_central { tcp("1.2.3.4" log_disk_fifo_size(1073741824)); }; This allocates about 1GiB space for destination spooling. [1] If you don't have this amount of space in syslog sources or you are using appliances where installing syslog-ng is not an option, you can use syslog relays. E.g. your infrastructure would be something like this: source1 \ source2 \ source3 |-- relay -- center ... / sourceN / Relays need to be deployed close to the source systems (e.g. on the same physical network, e.g. the probability of loss is low), then relay would be talking to the center using a possibly TLS encrypted TCP connection. You need to size the storage in your relays according to your requirements. Of course you can add another layer of relays (e.g. one dumber device on the same network as the sources, and a larger device somewhere in the middle, collecting several relays' worth of traffic). If you can deploy syslog-ng on the source systems, you could also enable flow-control. [2] [1]Using disk-buffering: http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch07s04.html [2]Flow control: http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch07s03.html -- Bazsi
participants (4)
-
Balazs Scheidler
-
Hiren Patel
-
Raghu (Lists)
-
Tom Le