[syslog-ng] syslog-ng stats to ES ?

Tue Oct 17 20:42:37 UTC 2017

Hi,

Oops, this is also in OSE but found only in the PE documentation. I put
Robert from the documentation team on CC to fix it.

Right now you can read the PE documentation:
https://www.balabit.com/documents/syslog-ng-pe-7.0-guides/en/syslog-ng-pe-guide-admin/html-single/index.html#syslog-ng-ctl.1

Basically: "syslog-ng-ctl query list" will give you a list of available
fields. "syslog-ng-ctl query get '*'" will get all values, "get 'dst.*'"
will get you only values where the name starts with "dst."

Bye,

Peter Czanik (CzP) <peter.czanik at balabit.com>
Balabit / syslog-ng upstream
https://www.balabit.com/blog/author/peterczanik/
https://twitter.com/PCzanik

On Tue, Oct 17, 2017 at 8:44 PM, Scot <scotrn at gmail.com> wrote:

> Where are the query options documented ?   Been looking Google, Balabit
> for an hour.
> man pages have nothing.
>
> On Tue, Oct 17, 2017 at 11:42 AM, Czanik, Péter <peter.czanik at balabit.com>
> wrote:
>
>> Hi,
>>
>> If you work with syslog-ng-ctl you can give "jo" ( JSON output:
>> https://github.com/jpmens/jo ) a try. I only did some basic tests, but
>> it seems to me that it can turn the output of "syslog-ng-ctl query" into
>> JSON.
>>
>> Bye,
>>
>> Peter Czanik (CzP) <peter.czanik at balabit.com>
>> Balabit / syslog-ng upstream
>> https://www.balabit.com/blog/author/peterczanik/
>> https://twitter.com/PCzanik
>>
>> On Tue, Oct 17, 2017 at 5:20 PM, Scheidler, Balázs <
>> balazs.scheidler at balabit.com> wrote:
>>
>>> Difficult, the whole problem is naming of the name value pairs.
>>>
>>> The idea behind stats is to generate all name value pairs in one
>>> message, and this simply does not scale. You are almost certainly
>>> interested in a set of values or an aggregate of a set, and not everything.
>>>
>>> Just set stats-level() to 3, and look at the stats message.
>>>
>>> I am not saying its impossible, just that it requires some thought.
>>>
>>> On Oct 17, 2017 17:09, "Scot" <scotrn at gmail.com> wrote:
>>>
>>>> How about an output modifier ?
>>>>
>>>> On Tue, Oct 17, 2017 at 11:02 AM, Scheidler, Balázs <
>>>> balazs.scheidler at balabit.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> the issue with the internal stats() message is that if you have a lot
>>>>> of counters that message is truncated. Also, it is pretty difficult to
>>>>> parse.
>>>>>
>>>>> So I would vote for the "poll syslog-ng-ctl and generate messages"
>>>>> solution.
>>>>>
>>>>> BTW: the internal PE team did something in this area, they created
>>>>> some sort of internal source that does this polling, but I am not sure how
>>>>> that works. Possibly there's documentation :)
>>>>>
>>>>>
>>>>> --
>>>>> Bazsi
>>>>>
>>>>> On Tue, Oct 17, 2017 at 4:37 PM, Scot <scotrn at gmail.com> wrote:
>>>>>
>>>>>> Doesn't stats_freq() set an interval to log stats to syslog already?
>>>>>>
>>>>>> Description: The period between two STATS messages in seconds. STATS are
>>>>>> log messages sent by syslog-ng, containing statistics about dropped
>>>>>> log messages. Set to 0to disable the STATS messages.
>>>>>>
>>>>>> So
>>>>>> internal_src -> format > elasticsearch -> syslog-ng_stats index ?
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 11:01 AM, Evan Rempel <erempel at uvic.ca>
>>>>>> wrote:
>>>>>>
>>>>>>> I have a perl script that collects some stats and logs them to
>>>>>>> syslog again. The syslog stream gets sent to ES, so they end up there, but
>>>>>>> as a syslog line, not a specific statistic item for things like grafana.
>>>>>>>
>>>>>>>
>>>>>>> On 10/15/2017 05:57 PM, Scot wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>   Looked around for a few hours and didn't see anything.
>>>>>>>>
>>>>>>>> Has anyone worked on sending syslog-ng stats to ES ?
>>>>>>>> I see several ways I could but wondering if anyone has already. A
>>>>>>>> push method directly from syslog-ng would be awesome.
>>>>>>>>
>>>>>>>> Scot
>>>>>>>>
>>>>>>>>
>>>>>>> ____________________________________________________________
>>>>>>> __________________
>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>> /documentation/?product=syslog-ng
>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ____________________________________________________________
>>>>>> __________________
>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>> Documentation: http://www.balabit.com/support
>>>>>> /documentation/?product=syslog-ng
>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ____________________________________________________________
>>>>> __________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation: http://www.balabit.com/support
>>>>> /documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>>>
>>>>>
>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171017/0022931b/attachment.html>