[syslog-ng] syslog-ng for filtering and execute external application

Fri Apr 26 16:26:16 UTC 2019

Hello,

I think you are looking for *grouping-by*, that can group together multiple
messages.
You could use *kv-parse* to parse the exit child, pid=1234 and use *pid* as
key to group the messages.

--
Kokan

On Fri, Apr 26, 2019 at 5:25 PM Thomas Schmiedl <thomas.schmiedl at web.de>
wrote:

> Hallo,
>
> for a first test, I wrote this small conf, which matches the first
> stream in
> https://github.com/clark15b/xupnpd2/blob/master/media/example.m3u.
>
> @version: 3.20
>
> source s0 { udp(ip(0.0.0.0) port(514)); };
>
> destination d0 { file("/home/user/syslog-ng-intel/test"); };
>
> log { source(s0); filter { message(".*\[(.*)\].*strk\.stream.*"
> flags("store-matches")); }; destination(d0); };
>
> Is it possible to get the variable value of the stored match from the
> log for a second log to match "exit child, pid=<variable>"?
>
> Thanks,
> Thomas
>
> Am 25.04.2019 um 08:46 schrieb Péter, Kókai:
> > Hello,
> >
> > Assuming the log file looks like this:
> >
> > ```
> > run child, pid = 1715
> > using handler 'hls' for 'http://localhost/'
> > exit child, pid = 1715
> > ```
> >
> > Filters and source could be something like this:
> > ```
> > @version: 3.20
> > @include "scl.conf"
> >
> > source xupnpd2 {
> >    file("/tmp/xup" flags(no-parse)); # no-parse needed to set $MESSAGE,
> if
> > the file does contain only the above lines it is fine not to parse
> > };
> >
> > log { source(xupnpd2); filter { message("run child"); };
> destination(d0); };
> > log { source(xupnpd2); filter { message("exit child"); } ;
> destination(d0);
> > };
> > ```
> >
> > If needed syslog-ng could parse, pid and you can transfare more
> structured
> > to the program destination.
> >
> > --
> > Kokan
> >
> > On Wed, Apr 24, 2019 at 4:15 PM Thomas Schmiedl <thomas.schmiedl at web.de>
> > wrote:
> >
> >> Hello,
> >>
> >> thanks Péter for your reply. Please could you write the filters
> >>
> >> 1. for the start (2 lines with a variable 'pid' number):
> >> run child, pid = 1715
> >> using handler 'hls' for '... URL ...'
> >>
> >> 2. for the stop:
> >> exit child, pid = 1715
> >>
> >> Thanks,
> >> Thomas
> >>
> >> Am 24.04.2019 um 15:05 schrieb Péter, Kókai:
> >>> Hello,
> >>>
> >>> 'I have the question, if I could use syslog-ng to filter the start and
> >> stop
> >>> of the stream'
> >>> Yes, it can filter those messages (disclaimer without seeing those
> >>> messages).
> >>>
> >>> 'and execute an action "ffmpeg stop" and "delete video segments".'
> >>> It was not something syslog-ng is designed to, but for example you
> could
> >>> use program destination to execute arbitrary executable (like s small
> >>> script to call ffmpeg stop).
> >>>
> >>> You could do something like this (not tested) to have separate things
> to
> >> do
> >>> based on start/stop:
> >>>
> >>> @version: 3.20
> >>> source xupnpd2 {
> >>>     stdin(flags(no-parse));
> >>> };
> >>>
> >>> destination start {
> >>>     program("/usr/bin/do-start-magic.sh");
> >>> };
> >>>
> >>> destination stop {
> >>>     program("/usr/bin/do-stop-magic.sh");
> >>> };
> >>>
> >>> log { source(xupnpd2); filter { program("xupnpd2") AND
> message("start");
> >> };
> >>> destination(start); };
> >>> log { source(xupnpd2); filter { program("xupnpd2") AND message("stop");
> >> } ;
> >>> destination(stop); };
> >>>
> >>>
> >>> Each log/event is a single new line to the program stdin, which it
> should
> >>> process.
> >>>
> >>> --
> >>> Kokan
> >>>
> >>>
> >>>
> >>> On Wed, Apr 24, 2019 at 2:36 PM Thomas Schmiedl <
> thomas.schmiedl at web.de>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I use the mediaserver xupnpd2 (https://github.com/clark15b/xupnpd2)
> on
> >>>> the router to show HLS streams on the TV. Unfortunately, the software
> is
> >>>> no longer maintained by the original developer (I'm not a developer
> >>>> myself). The HLS processing would have to be revised.
> >>>>
> >>>> My idea is to start a ffmpeg and a local web server on the router, if
> I
> >>>> want to display the stream (the script xupnpd.lua will be executed,
> >>>> which starts ffmpeg etc.). Unfortunately there is not such a script
> when
> >>>> terminating the stream (exit ffmpeg etc.).
> >>>>
> >>>> xupnpd2 uses a log where you can track the start as well as the ending
> >>>> of the stream. Since the router has only a small amount of internal
> >>>> memory to write and analyze a logfile, I have the question, if I could
> >>>> use syslog-ng to filter the start and stop of the stream and execute
> an
> >>>> action "ffmpeg stop" and "delete video segments". I have very little
> >>>> Linux knowledge, maybe you can help me.
> >>>>
> >>>> Here is the excerpt from the xupnpd2 log (Loglevel 8).
> >>>>
> >>>> When starting the stream:
> >>>> run child, pid = 1715
> >>>> using handler 'hls' for '... URL ...'
> >>>>
> >>>> When stopping the stream:
> >>>> exit child, pid = 1715
> >>>>
> >>>> Best regards,
> >>>> Thomas Schmiedl
> >>>>
> >>>>
> >>
> ______________________________________________________________________________
> >>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>>> Documentation:
> >>>> http://www.balabit.com/support/documentation/?product=syslog-ng
> >>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>
> ______________________________________________________________________________
> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>> Documentation:
> >> http://www.balabit.com/support/documentation/?product=syslog-ng
> >>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>>
> >>
> >>
> ______________________________________________________________________________
> >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >> Documentation:
> >> http://www.balabit.com/support/documentation/?product=syslog-ng
> >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >>
> >>
> >
> >
> >
> ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190426/f6300703/attachment-0001.html>