[syslog-ng]Syslog-ng replay script for centralized syslog data

[Dave Johnson]

Just another thought, (which isn't as easy as the other suggestion) --

* Set up ratelimiting on your remote servers to the central server's
IP and just syslog-ng with tcp to the central server.
  - Make sure you have a decent sized queue on the remote server so
you can queue up packets
  - setting up ratelimiting on linux and getting the results just
right might take some time.

(you can google search for /etc/init.d/cbq scripts)  and make sure you
have class base queueing enabled in your kernel.

---

* You can create another ip on your central server if your going to be
doing admin tasks from that box.  (IE you don't want your ssh to be in
the same ratelimiting rule as the syslog traffic).

* If compression is important (due to the small link size), you could
leverage ssh to do this.

This approach is a little more complicated, but your logs would show up sooner.

Depending on important this data is, you may want the backup ftp/rsync
method anyways...


On Thu, 28 Oct 2004 15:02:33 -0500, Dave Johnson <davejjohnson@gmail.com> wrote:
> You can do it many ways, one way (quick and easy):
> 
> remote nodes <every ten minutes cron>
> log, bzip2 in directory "A"
> run rsyncd for directory "A"
> ---
> central node <every ten minutes +1 minute> <or just do it every 2 mins, etc..>
> run script:
> 1) rsync --bwlimit 9k -u get from remote node's "A"
> 2) bunzip2 files
> 3) cat file into /dev/log (or local platform's way of injecting into syslog)
> ---------
> http://samba.anu.edu.au/rsync/
> 
> 
> 
> On Thu, 28 Oct 2004 12:03:53 -0700 (PDT), LEROY ISAAC
> <lisaac01@yahoo.com> wrote:
> >
> >
> > I have a need to retrieve syslog data from various
> > remote nodes, and the smallest network link to the
> > remote nodes is 19K. The syslog traffic for the link
> > cannot exceed 9K.
> >
> > I plan to setup a configuration which generates new
> > log files every 10 minutes. These files are then
> > compressed, zipped, and transfered to a centralized
> > loghost.
> >
> > The files are then unzipped, uncompressed, and the
> > data is inserted into the syslog-ng data stream on the
> > central syslog-ng host.
> >
> > Is there a script or utility which will accomplish
> > this task? If not, then does any one have any
> > suggestions on products which may accomplish this same
> > task.
> >
> > LeRoy Isaac
> > --- DTrinh71@aol.com wrote:
> >
> > > OK. Thanks.
> > >
> > > So, what does Ray want? Suggestions?
> > >
> > > David
> > >
> >
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >
> >
>