[syslog-ng]Syslog-ng replay script for centralized syslog data

[Dave Johnson]

bill--  

I'm only offering suggestions, and working on a limited subset of
knowledge about your architecture.  Your mileage may vary...


On Thu, 28 Oct 2004 13:52:55 -0700 (PDT), Bill Nash <billn@billn.net> wrote:
> 
> That's a dramatic increase in complexity, however. The strongest lifeform
> is often the simplest. Troubleshooting, or even implementing such a setup,
> may not be feasible and would likely require kernel recompiles to even
> enable the features, depending on the existing implementation. Scale is
> another factor that would make this option less attractive. A QoS option
> would be better implemented at the network level across all tcp/514
> traffic.
> 
> - billn
> 
> 
> 
> On Thu, 28 Oct 2004, Dave Johnson wrote:
> 
> > Just another thought, (which isn't as easy as the other suggestion) --
> >
> > * Set up ratelimiting on your remote servers to the central server's
> > IP and just syslog-ng with tcp to the central server.
> >  - Make sure you have a decent sized queue on the remote server so
> > you can queue up packets
> >  - setting up ratelimiting on linux and getting the results just
> > right might take some time.
> >
> > (you can google search for /etc/init.d/cbq scripts)  and make sure you
> > have class base queueing enabled in your kernel.
> >
> > ---
> >
> > * You can create another ip on your central server if your going to be
> > doing admin tasks from that box.  (IE you don't want your ssh to be in
> > the same ratelimiting rule as the syslog traffic).
> >
> > * If compression is important (due to the small link size), you could
> > leverage ssh to do this.
> >
> > This approach is a little more complicated, but your logs would show up sooner.
> >
> > Depending on important this data is, you may want the backup ftp/rsync
> > method anyways...
> >
> >
> > On Thu, 28 Oct 2004 15:02:33 -0500, Dave Johnson <davejjohnson@gmail.com> wrote:
> >> You can do it many ways, one way (quick and easy):
> >>
> >> remote nodes <every ten minutes cron>
> >> log, bzip2 in directory "A"
> >> run rsyncd for directory "A"
> >> ---
> >> central node <every ten minutes +1 minute> <or just do it every 2 mins, etc..>
> >> run script:
> >> 1) rsync --bwlimit 9k -u get from remote node's "A"
> >> 2) bunzip2 files
> >> 3) cat file into /dev/log (or local platform's way of injecting into syslog)
> >> ---------
> >> http://samba.anu.edu.au/rsync/
> >>
> >>
> >>
> >> On Thu, 28 Oct 2004 12:03:53 -0700 (PDT), LEROY ISAAC
> >> <lisaac01@yahoo.com> wrote:
> >>>
> >>>
> >>> I have a need to retrieve syslog data from various
> >>> remote nodes, and the smallest network link to the
> >>> remote nodes is 19K. The syslog traffic for the link
> >>> cannot exceed 9K.
> >>>
> >>> I plan to setup a configuration which generates new
> >>> log files every 10 minutes. These files are then
> >>> compressed, zipped, and transfered to a centralized
> >>> loghost.
> >>>
> >>> The files are then unzipped, uncompressed, and the
> >>> data is inserted into the syslog-ng data stream on the
> >>> central syslog-ng host.
> >>>
> >>> Is there a script or utility which will accomplish
> >>> this task? If not, then does any one have any
> >>> suggestions on products which may accomplish this same
> >>> task.
> >>>
> >>> LeRoy Isaac
> >>> --- DTrinh71@aol.com wrote:
> >>>
> >>>> OK. Thanks.
> >>>>
> >>>> So, what does Ray want? Suggestions?
> >>>>
> >>>> David
> >>>>
> >>>
> >>> _______________________________________________
> >>> syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> >>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >>>
> >>>
> >>
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >
> _______________________________________________
> 
> 
> syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> 
>