[syslog-ng] poll: web interface for syslog-ng

[Balazs Scheidler]

Thanks for the detailed response. I was thrilled while reading your email
and this confirms that my ongoing efforts were worth it in the last 22
years :)

Thanks

Bazsi


On Wed, Nov 25, 2020 at 5:03 PM John Kristoff <jtk at dataplane.org> wrote:

> On Wed, 25 Nov 2020 14:46:44 +0000
> "Peter Czanik (pczanik)" <Peter.Czanik at oneidentity.com> wrote:
>
> > Which web interface do you use to search / analyze / alert on your
> > logs collected by syslog-ng? Splunk? Elasticsearch? Other? Or still
> > grep/awk?
>
> Hi Peter,
>
> I'm been using syslog-ng since early to mid 2000's, mainly as a
> replacement for stock syslog and more recently rsyslog.  A few years
> back I started down the path of trying Elasticsearch, but never
> finished getting it up and running.  At the time it was too much
> trouble for what was just tinkering at the time.  Maybe one day I'll
> give it another try, but I don't have a large need right now.
>
> It isn't all just grep/awk and for me however.  Originally, one of my
> first uses was on Solaris where I found syslog-ng to be vastly more
> efficient (less CPU for the workload) than the stock daemon.  I grew to
> appreciate what now seem like simple things, the macros to put logs in
> paths and file names expanded by date for instance.  Later I used this
> feature to create circular logs when storage space was important and
> history wasn't (just use the day of the week, overwriting each day's
> file as you go).
>
> My appreciation and usage grew further when I used it as the key
> component of a security sensor network.  Comparing it to rsyslog at the
> time, when it was replacing syslog-ng as the default daemon in distros,
> I found rsyslog to buggy and the documentation very poor in comparison.
> Things have probably changed for rsyslog for the better now, but I've
> never looked back.  Back then I started making heavy use of the the
> pattern matching database feature, which was quite novel then, and
> database integration.
>
> Those two features, coupled with TLS support I was able to remotely log
> from dozens of systems all over the world to a central collector and
> build up my customer tables. Data was logged in files for archival, but
> the things I cared about I parsed and stuck into the database.  This
> project has continued today and I'm still running over 100 nodes this
> way providing various kinds of feeds to the security community. You can
> read more about this project in my article from USENIX's ;login: Fall
> 2018 issue or visit the homepage of the domain from which I'm sending
> this email.
>
> In my network role at a university we have Splunk where some logs go,
> but I personally still to work with raw logs since I rarely need to
> look at them, and when I do it is often to conduct some very
> specific aggregate analysis or exploration that Splunk won't do with
> more work.  I'd just prefer to use unix tools and raw data for those
> rare occasions.
>
> So yes grep/awk (and perl, python, sort, cut and so on), but for my
> DataPlane project it is often SQL queries or custom code to create
> reports from the database that received data directly from syslog-ng.
>
> My standard system build is to always replace whatever syslog daemon
> might be installed with syslog-ng, change the default timestamp settings
> to ts_format(iso) then add other customizations into conf.d/ as I go.
>
> Hope that helps, sorry not much for a web interface user here.
>
> John
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>

-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20201127/ac029df5/attachment.html>