[syslog-ng] syslog-ng for filtering and execute external application

Fri Apr 26 19:03:11 UTC 2019

Hello Péter,

could you send me an example, it's complicated for a beginner. xupnpd2
sends such messages:

run child, pid=1283
exit child, pid=1283
run child, pid=1284
exit child, pid=1284
run child, pid=1285
exit child, pid=1285
run child, pid=1286
exit child, pid=1286
run child, pid=1287
using handler 'hls' for
'http://sochi-strk.ru:1936/strk/strk.stream/playlist.m3u8'
exit child, pid=1287
run child, pid=1289
run child, pid=1290
run child, pid=1291
run child, pid=1293
run child, pid=1292
run child, pid=1294
exit child, pid=1289
exit child, pid=1290

On "exit child, pid=1287" should be executed the shellscript to stop ffmpeg.

Thanks,
Thomas

Am 26.04.2019 um 18:26 schrieb Péter, Kókai:
> Hello,
>
> I think you are looking for *grouping-by*, that can group together multiple
> messages.
> You could use *kv-parse* to parse the exit child, pid=1234 and use *pid* as
> key to group the messages.
>
>
> --
> Kokan
>
> On Fri, Apr 26, 2019 at 5:25 PM Thomas Schmiedl <thomas.schmiedl at web.de>
> wrote:
>
>> Hallo,
>>
>> for a first test, I wrote this small conf, which matches the first
>> stream in
>> https://github.com/clark15b/xupnpd2/blob/master/media/example.m3u.
>>
>> @version: 3.20
>>
>> source s0 { udp(ip(0.0.0.0) port(514)); };
>>
>> destination d0 { file("/home/user/syslog-ng-intel/test"); };
>>
>> log { source(s0); filter { message(".*\[(.*)\].*strk\.stream.*"
>> flags("store-matches")); }; destination(d0); };
>>
>> Is it possible to get the variable value of the stored match from the
>> log for a second log to match "exit child, pid=<variable>"?
>>
>> Thanks,
>> Thomas
>>
>> Am 25.04.2019 um 08:46 schrieb Péter, Kókai:
>>> Hello,
>>>
>>> Assuming the log file looks like this:
>>>
>>> ```
>>> run child, pid = 1715
>>> using handler 'hls' for 'http://localhost/'
>>> exit child, pid = 1715
>>> ```
>>>
>>> Filters and source could be something like this:
>>> ```
>>> @version: 3.20
>>> @include "scl.conf"
>>>
>>> source xupnpd2 {
>>>     file("/tmp/xup" flags(no-parse)); # no-parse needed to set $MESSAGE,
>> if
>>> the file does contain only the above lines it is fine not to parse
>>> };
>>>
>>> log { source(xupnpd2); filter { message("run child"); };
>> destination(d0); };
>>> log { source(xupnpd2); filter { message("exit child"); } ;
>> destination(d0);
>>> };
>>> ```
>>>
>>> If needed syslog-ng could parse, pid and you can transfare more
>> structured
>>> to the program destination.
>>>
>>> --
>>> Kokan
>>>
>>> On Wed, Apr 24, 2019 at 4:15 PM Thomas Schmiedl <thomas.schmiedl at web.de>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> thanks Péter for your reply. Please could you write the filters
>>>>
>>>> 1. for the start (2 lines with a variable 'pid' number):
>>>> run child, pid = 1715
>>>> using handler 'hls' for '... URL ...'
>>>>
>>>> 2. for the stop:
>>>> exit child, pid = 1715
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>> Am 24.04.2019 um 15:05 schrieb Péter, Kókai:
>>>>> Hello,
>>>>>
>>>>> 'I have the question, if I could use syslog-ng to filter the start and
>>>> stop
>>>>> of the stream'
>>>>> Yes, it can filter those messages (disclaimer without seeing those
>>>>> messages).
>>>>>
>>>>> 'and execute an action "ffmpeg stop" and "delete video segments".'
>>>>> It was not something syslog-ng is designed to, but for example you
>> could
>>>>> use program destination to execute arbitrary executable (like s small
>>>>> script to call ffmpeg stop).
>>>>>
>>>>> You could do something like this (not tested) to have separate things
>> to
>>>> do
>>>>> based on start/stop:
>>>>>
>>>>> @version: 3.20
>>>>> source xupnpd2 {
>>>>>      stdin(flags(no-parse));
>>>>> };
>>>>>
>>>>> destination start {
>>>>>      program("/usr/bin/do-start-magic.sh");
>>>>> };
>>>>>
>>>>> destination stop {
>>>>>      program("/usr/bin/do-stop-magic.sh");
>>>>> };
>>>>>
>>>>> log { source(xupnpd2); filter { program("xupnpd2") AND
>> message("start");
>>>> };
>>>>> destination(start); };
>>>>> log { source(xupnpd2); filter { program("xupnpd2") AND message("stop");
>>>> } ;
>>>>> destination(stop); };
>>>>>
>>>>>
>>>>> Each log/event is a single new line to the program stdin, which it
>> should
>>>>> process.
>>>>>
>>>>> --
>>>>> Kokan
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Apr 24, 2019 at 2:36 PM Thomas Schmiedl <
>> thomas.schmiedl at web.de>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I use the mediaserver xupnpd2 (https://github.com/clark15b/xupnpd2)
>> on
>>>>>> the router to show HLS streams on the TV. Unfortunately, the software
>> is
>>>>>> no longer maintained by the original developer (I'm not a developer
>>>>>> myself). The HLS processing would have to be revised.
>>>>>>
>>>>>> My idea is to start a ffmpeg and a local web server on the router, if
>> I
>>>>>> want to display the stream (the script xupnpd.lua will be executed,
>>>>>> which starts ffmpeg etc.). Unfortunately there is not such a script
>> when
>>>>>> terminating the stream (exit ffmpeg etc.).
>>>>>>
>>>>>> xupnpd2 uses a log where you can track the start as well as the ending
>>>>>> of the stream. Since the router has only a small amount of internal
>>>>>> memory to write and analyze a logfile, I have the question, if I could
>>>>>> use syslog-ng to filter the start and stop of the stream and execute
>> an
>>>>>> action "ffmpeg stop" and "delete video segments". I have very little
>>>>>> Linux knowledge, maybe you can help me.
>>>>>>
>>>>>> Here is the excerpt from the xupnpd2 log (Loglevel 8).
>>>>>>
>>>>>> When starting the stream:
>>>>>> run child, pid = 1715
>>>>>> using handler 'hls' for '... URL ...'
>>>>>>
>>>>>> When stopping the stream:
>>>>>> exit child, pid = 1715
>>>>>>
>>>>>> Best regards,
>>>>>> Thomas Schmiedl
>>>>>>
>>>>>>
>>>>
>> ______________________________________________________________________________
>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>> Documentation:
>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>>
>>>>
>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>
>>>
>>>
>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>