[syslog-ng] email alert on timeout

[Laszlo Szemere (lszemere)]

Hello Gregg,

 I think you are almost on the right track. A little addition to MARK messages:
 Syslog-ng's destinations will ONLY emit a mark message IF otherwise there will be no message at all from that destination, during a "mark-freq" time period.
 So if there is a message on the Destination, it will reset the "mark-freq" timer, and the interval starts again without sending any mark message. So during a normal work of a busy log path there should be no mark messages at all.

 One more thing: I don't know if it is intentional from you, but you can spare the whole "mark" file logic from your configuration in certain cases, if you use the "internal" mark-mode. Unfortunately I can not give you a direct link, but in the "global options" section of the administration guide: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.24/administration-guide/59#TOPIC-1298095 there is a chapter about "mark-mode"s.

Best regards,
Laci

________________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Gregg Nicholas <gnichola at berriencounty.org>
Sent: Thursday, October 31, 2019 13:56
To: syslog-ng at lists.balabit.hu
Subject: [syslog-ng] email alert on timeout

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


I'm new to syslog-ng OSE. I've been able to follow the samples, and configured an email alert when a syslog message is received.



Now I'm trying to configure an email alert when it has been too long since I've received a particular syslog message.



First, I configured a destination:

                destination d_heartbeat {

                                file("/var/log/heartbeat.log" mark-freq(60) mark-mode(dst-idle) );

                };



Then, I configured a filter and log:

                filter f_heartbeat {

                                match("I am still here" value("MESSAGE) );

                };

                Log { source(s_network); filter(f_heartbeat); destination(d_heartbeat); };



I was pleasantly surprised that this worked. The heartbeat.log file received the message, and 60 seconds later it received the "syslog -- MARK --"



Since I wanted an email alert, I defined a new source, to grab the output of my heartbeat.log as input:



                source s_heartbeat { file(""/var/log/heartbeat.log"); };

                destination d_smtp { smtp(

                                host("10.10.10.25") port(25)

                                from("syslog-ng" "noreply at mydomain.com<mailto:noreply at mydomain.com>")

                                to("me" "myemail at mydomain.com<mailto:myemail at mydomain.com>")

                                subject("host is down")

                                body("no heartbeat received from the program on host\n")

                ); };

                filter f_timeout { match ("MARK --" value("MESSAGE")); };

                log { source(s_heartbeat); filter(f_timeout); destination(d_smtp); };



This seemed to work when I tested it with a single heartbeat followed by timeout, but I’m still misunderstanding something. If it gets more than one message written to heartbeat.log, then the timeout MARK doesn’t happen.



What am I missing? Is there a better way to accomplish this alert?



Thanks,

Gregg

(running syslog-ng v3.19.1-5 on debian 10)