On Wed, 25 Nov 2020 14:46:44 +0000 "Peter Czanik (pczanik)" <Peter.Czanik@oneidentity.com> wrote:
Which web interface do you use to search / analyze / alert on your logs collected by syslog-ng? Splunk? Elasticsearch? Other? Or still grep/awk?
Hi Peter, I'm been using syslog-ng since early to mid 2000's, mainly as a replacement for stock syslog and more recently rsyslog. A few years back I started down the path of trying Elasticsearch, but never finished getting it up and running. At the time it was too much trouble for what was just tinkering at the time. Maybe one day I'll give it another try, but I don't have a large need right now. It isn't all just grep/awk and for me however. Originally, one of my first uses was on Solaris where I found syslog-ng to be vastly more efficient (less CPU for the workload) than the stock daemon. I grew to appreciate what now seem like simple things, the macros to put logs in paths and file names expanded by date for instance. Later I used this feature to create circular logs when storage space was important and history wasn't (just use the day of the week, overwriting each day's file as you go). My appreciation and usage grew further when I used it as the key component of a security sensor network. Comparing it to rsyslog at the time, when it was replacing syslog-ng as the default daemon in distros, I found rsyslog to buggy and the documentation very poor in comparison. Things have probably changed for rsyslog for the better now, but I've never looked back. Back then I started making heavy use of the the pattern matching database feature, which was quite novel then, and database integration. Those two features, coupled with TLS support I was able to remotely log from dozens of systems all over the world to a central collector and build up my customer tables. Data was logged in files for archival, but the things I cared about I parsed and stuck into the database. This project has continued today and I'm still running over 100 nodes this way providing various kinds of feeds to the security community. You can read more about this project in my article from USENIX's ;login: Fall 2018 issue or visit the homepage of the domain from which I'm sending this email. In my network role at a university we have Splunk where some logs go, but I personally still to work with raw logs since I rarely need to look at them, and when I do it is often to conduct some very specific aggregate analysis or exploration that Splunk won't do with more work. I'd just prefer to use unix tools and raw data for those rare occasions. So yes grep/awk (and perl, python, sort, cut and so on), but for my DataPlane project it is often SQL queries or custom code to create reports from the database that received data directly from syslog-ng. My standard system build is to always replace whatever syslog daemon might be installed with syslog-ng, change the default timestamp settings to ts_format(iso) then add other customizations into conf.d/ as I go. Hope that helps, sorry not much for a web interface user here. John