[syslog-ng] syslog-ng stats to ES ?

[Scheidler, Balázs]

Difficult, the whole problem is naming of the name value pairs.

The idea behind stats is to generate all name value pairs in one message,
and this simply does not scale. You are almost certainly interested in a
set of values or an aggregate of a set, and not everything.

Just set stats-level() to 3, and look at the stats message.

I am not saying its impossible, just that it requires some thought.

On Oct 17, 2017 17:09, "Scot" <scotrn at gmail.com> wrote:

> How about an output modifier ?
>
> On Tue, Oct 17, 2017 at 11:02 AM, Scheidler, Balázs <
> balazs.scheidler at balabit.com> wrote:
>
>> Hi,
>>
>> the issue with the internal stats() message is that if you have a lot of
>> counters that message is truncated. Also, it is pretty difficult to parse.
>>
>> So I would vote for the "poll syslog-ng-ctl and generate messages"
>> solution.
>>
>> BTW: the internal PE team did something in this area, they created some
>> sort of internal source that does this polling, but I am not sure how that
>> works. Possibly there's documentation :)
>>
>>
>> --
>> Bazsi
>>
>> On Tue, Oct 17, 2017 at 4:37 PM, Scot <scotrn at gmail.com> wrote:
>>
>>> Doesn't stats_freq() set an interval to log stats to syslog already?
>>>
>>> Description: The period between two STATS messages in seconds. STATS are
>>> log messages sent by syslog-ng, containing statistics about dropped log
>>> messages. Set to 0to disable the STATS messages.
>>>
>>> So
>>> internal_src -> format > elasticsearch -> syslog-ng_stats index ?
>>>
>>> On Mon, Oct 16, 2017 at 11:01 AM, Evan Rempel <erempel at uvic.ca> wrote:
>>>
>>>> I have a perl script that collects some stats and logs them to syslog
>>>> again. The syslog stream gets sent to ES, so they end up there, but as a
>>>> syslog line, not a specific statistic item for things like grafana.
>>>>
>>>>
>>>> On 10/15/2017 05:57 PM, Scot wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>   Looked around for a few hours and didn't see anything.
>>>>>
>>>>> Has anyone worked on sending syslog-ng stats to ES ?
>>>>> I see several ways I could but wondering if anyone has already. A push
>>>>> method directly from syslog-ng would be awesome.
>>>>>
>>>>> Scot
>>>>>
>>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171017/3ffc40b5/attachment.html>