Hi Jim, On Sat, Dec 30, 2017 at 11:27:49AM -0500, Jim Hendrick wrote:

Has anyone added sar data to syslog-ng? I know of a couple "brute-force" ways like running a cron to run the sar command and dump it to a file. But before I do that I wanted to check and see if there was a more elegant solution.

The overall problem would be to run this across 10,000+ servers with minimal need to change configurations. (for example - if there was a need to change the parameters to sar, or change the frequency of the pull).

I guess I was hoping for some add-on that either directly pulled the performance data from the kernel or could include a config that would specify what data elements would be included.

FYI - we do use syslog-ng PE but this seems fairly generic so I thought I'd ask the list.

Given the scale of the problem, pulling "all" data even relatively frequently (say every 1-5 minutes) would result in a huge volume increase in our logging solution (where we pay by the ingested GB...)

Thoughts? Advice?

For this kind of data (system/app metrics), which tends to use astronomical amounts of disk space unprocessed, we use collectd, then pre-aggregate the data (min/max/avg) then push it to Elasticsearch. Query is done either using REST or Grafana. That being said, collection could be done in syslog-ng, much like the pacct driver is reading from the binary file [1]. This could actually be an interesting idea for GSoC :-) Cheers -- [1] https://www.balabit.com/documents/syslog-ng-ose-3.13-guides/en/syslog-ng-ose...