[syslog-ng] syslog-ng stats to ES ?

[Czanik, Péter]

Hi,

If you work with syslog-ng-ctl you can give "jo" ( JSON output:
https://github.com/jpmens/jo ) a try. I only did some basic tests, but it
seems to me that it can turn the output of "syslog-ng-ctl query" into JSON.

Bye,

Peter Czanik (CzP) <peter.czanik at balabit.com>
Balabit / syslog-ng upstream
https://www.balabit.com/blog/author/peterczanik/
https://twitter.com/PCzanik

On Tue, Oct 17, 2017 at 5:20 PM, Scheidler, Balázs <
balazs.scheidler at balabit.com> wrote:

> Difficult, the whole problem is naming of the name value pairs.
>
> The idea behind stats is to generate all name value pairs in one message,
> and this simply does not scale. You are almost certainly interested in a
> set of values or an aggregate of a set, and not everything.
>
> Just set stats-level() to 3, and look at the stats message.
>
> I am not saying its impossible, just that it requires some thought.
>
> On Oct 17, 2017 17:09, "Scot" <scotrn at gmail.com> wrote:
>
>> How about an output modifier ?
>>
>> On Tue, Oct 17, 2017 at 11:02 AM, Scheidler, Balázs <
>> balazs.scheidler at balabit.com> wrote:
>>
>>> Hi,
>>>
>>> the issue with the internal stats() message is that if you have a lot of
>>> counters that message is truncated. Also, it is pretty difficult to parse.
>>>
>>> So I would vote for the "poll syslog-ng-ctl and generate messages"
>>> solution.
>>>
>>> BTW: the internal PE team did something in this area, they created some
>>> sort of internal source that does this polling, but I am not sure how that
>>> works. Possibly there's documentation :)
>>>
>>>
>>> --
>>> Bazsi
>>>
>>> On Tue, Oct 17, 2017 at 4:37 PM, Scot <scotrn at gmail.com> wrote:
>>>
>>>> Doesn't stats_freq() set an interval to log stats to syslog already?
>>>>
>>>> Description: The period between two STATS messages in seconds. STATS are
>>>> log messages sent by syslog-ng, containing statistics about dropped
>>>> log messages. Set to 0to disable the STATS messages.
>>>>
>>>> So
>>>> internal_src -> format > elasticsearch -> syslog-ng_stats index ?
>>>>
>>>> On Mon, Oct 16, 2017 at 11:01 AM, Evan Rempel <erempel at uvic.ca> wrote:
>>>>
>>>>> I have a perl script that collects some stats and logs them to syslog
>>>>> again. The syslog stream gets sent to ES, so they end up there, but as a
>>>>> syslog line, not a specific statistic item for things like grafana.
>>>>>
>>>>>
>>>>> On 10/15/2017 05:57 PM, Scot wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>   Looked around for a few hours and didn't see anything.
>>>>>>
>>>>>> Has anyone worked on sending syslog-ng stats to ES ?
>>>>>> I see several ways I could but wondering if anyone has already. A
>>>>>> push method directly from syslog-ng would be awesome.
>>>>>>
>>>>>> Scot
>>>>>>
>>>>>>
>>>>> ____________________________________________________________
>>>>> __________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation: http://www.balabit.com/support
>>>>> /documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>>>
>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171017/96236cf7/attachment-0001.html>