I'm new to syslog-ng OSE. I've been able to follow the samples, and configured an email alert when a syslog message is received. Now I'm trying to configure an email alert when it has been too long since I've received a particular syslog message. First, I configured a destination: destination d_heartbeat { file("/var/log/heartbeat.log" mark-freq(60) mark-mode(dst-idle) ); }; Then, I configured a filter and log: filter f_heartbeat { match("I am still here" value("MESSAGE) ); }; Log { source(s_network); filter(f_heartbeat); destination(d_heartbeat); }; I was pleasantly surprised that this worked. The heartbeat.log file received the message, and 60 seconds later it received the "syslog -- MARK --" Since I wanted an email alert, I defined a new source, to grab the output of my heartbeat.log as input: source s_heartbeat { file(""/var/log/heartbeat.log"); }; destination d_smtp { smtp( host("10.10.10.25") port(25) from("syslog-ng" "noreply@mydomain.com <mailto:noreply@mydomain.com> ") to("me" "myemail@mydomain.com <mailto:myemail@mydomain.com> ") subject("host is down") body("no heartbeat received from the program on host\n") ); }; filter f_timeout { match ("MARK --" value("MESSAGE")); }; log { source(s_heartbeat); filter(f_timeout); destination(d_smtp); }; This seemed to work when I tested it with a single heartbeat followed by timeout, but I'm still misunderstanding something. If it gets more than one message written to heartbeat.log, then the timeout MARK doesn't happen. What am I missing? Is there a better way to accomplish this alert? Thanks, Gregg (running syslog-ng v3.19.1-5 on debian 10)
Hello Gregg, I think you are almost on the right track. A little addition to MARK messages: Syslog-ng's destinations will ONLY emit a mark message IF otherwise there will be no message at all from that destination, during a "mark-freq" time period. So if there is a message on the Destination, it will reset the "mark-freq" timer, and the interval starts again without sending any mark message. So during a normal work of a busy log path there should be no mark messages at all. One more thing: I don't know if it is intentional from you, but you can spare the whole "mark" file logic from your configuration in certain cases, if you use the "internal" mark-mode. Unfortunately I can not give you a direct link, but in the "global options" section of the administration guide: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edit... there is a chapter about "mark-mode"s. Best regards, Laci ________________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Gregg Nicholas <gnichola@berriencounty.org> Sent: Thursday, October 31, 2019 13:56 To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] email alert on timeout CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. I'm new to syslog-ng OSE. I've been able to follow the samples, and configured an email alert when a syslog message is received. Now I'm trying to configure an email alert when it has been too long since I've received a particular syslog message. First, I configured a destination: destination d_heartbeat { file("/var/log/heartbeat.log" mark-freq(60) mark-mode(dst-idle) ); }; Then, I configured a filter and log: filter f_heartbeat { match("I am still here" value("MESSAGE) ); }; Log { source(s_network); filter(f_heartbeat); destination(d_heartbeat); }; I was pleasantly surprised that this worked. The heartbeat.log file received the message, and 60 seconds later it received the "syslog -- MARK --" Since I wanted an email alert, I defined a new source, to grab the output of my heartbeat.log as input: source s_heartbeat { file(""/var/log/heartbeat.log"); }; destination d_smtp { smtp( host("10.10.10.25") port(25) from("syslog-ng" "noreply@mydomain.com<mailto:noreply@mydomain.com>") to("me" "myemail@mydomain.com<mailto:myemail@mydomain.com>") subject("host is down") body("no heartbeat received from the program on host\n") ); }; filter f_timeout { match ("MARK --" value("MESSAGE")); }; log { source(s_heartbeat); filter(f_timeout); destination(d_smtp); }; This seemed to work when I tested it with a single heartbeat followed by timeout, but I’m still misunderstanding something. If it gets more than one message written to heartbeat.log, then the timeout MARK doesn’t happen. What am I missing? Is there a better way to accomplish this alert? Thanks, Gregg (running syslog-ng v3.19.1-5 on debian 10)
Hi Gregg, I tried your configuration, and it works for me. However, while experimenting, I noticed that mark message timer only starts when there is at least one message received. This is probably due that without any message, syslog-ng does not create the file writer. File writers are created on demand, because filename can be template, so syslog-ng might not know the file name without the message. This is what might interfere with your test? To workaround that, you might use example-msg-generator. From 3.20, there is a num() option that limits the number of messages generated. I see you are using 3.19, so unfortunately that means an upgrade. @version: 3.24 log { # source { example-msg-generator(freq(1) template("I am here\n") num(1)); }; source { network(port(5555)); }; destination { file(/tmp/heartbeat.txt mark-freq(5) mark-mode(dst-idle)); }; }; log { source { file(/tmp/heartbeat.txt); }; filter { match ("MARK --" value("MESSAGE")); }; destination { file(/dev/stdout); }; }; results in $ bash -c "sleep 2; ../bin/loggen -S localhost -n 1 5555; sleep 1; ../bin/loggen -S localhost -n 1 5555"& [1] 21679 $ ./syslog-ng -Fe -f ../etc/tmp.conf [2019-10-31T14:30:20.457194] Accepting connections; addr='AF_INET(0.0.0.0:5555)' [2019-10-31T14:30:20.457610] syslog-ng starting up; version='3.24.1.59.g2ab166d' [2019-10-31T14:30:21.162964] Syslog connection accepted; fd='15', client='AF_INET(127.0.0.1:53860)', local='AF_INET(0.0.0.0:5555)' [2019-10-31T14:30:21.163379] Syslog connection closed; fd='15', client='AF_INET(127.0.0.1:53860)', local='AF_INET(0.0.0.0:5555)' average rate = 2.00 msg/sec, count=1, time=0.500232, (average) msg size=256, bandwidth=0.50 kB/sec [2019-10-31T14:30:22.672276] Syslog connection accepted; fd='15', client='AF_INET(127.0.0.1:53862)', local='AF_INET(0.0.0.0:5555)' [2019-10-31T14:30:22.672501] Syslog connection closed; fd='15', client='AF_INET(127.0.0.1:53862)', local='AF_INET(0.0.0.0:5555)' average rate = 2.00 msg/sec, count=1, time=0.500194, (average) msg size=256, bandwidth=0.50 kB/sec Oct 31 14:30:27 furiel -- MARK -- Oct 31 14:30:32 furiel -- MARK -- Oct 31 14:30:37 furiel -- MARK -- Oct 31 14:30:42 furiel -- MARK -- Br, Antal ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Laszlo Szemere (lszemere) <Laszlo.Szemere@oneidentity.com> Sent: Thursday, October 31, 2019 14:30 To: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] email alert on timeout CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello Gregg, I think you are almost on the right track. A little addition to MARK messages: Syslog-ng's destinations will ONLY emit a mark message IF otherwise there will be no message at all from that destination, during a "mark-freq" time period. So if there is a message on the Destination, it will reset the "mark-freq" timer, and the interval starts again without sending any mark message. So during a normal work of a busy log path there should be no mark messages at all. One more thing: I don't know if it is intentional from you, but you can spare the whole "mark" file logic from your configuration in certain cases, if you use the "internal" mark-mode. Unfortunately I can not give you a direct link, but in the "global options" section of the administration guide: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.24%2Fadministration-guide%2F59%23TOPIC-1298095&data=02%7C01%7Cantal.nemes%40oneidentity.com%7C49a647056f144ad1387e08d75e068066%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637081254344960778&sdata=pXsr8zVit8T7SqYm9QRftxv1iRYYzj4V8le17r2FPqM%3D&reserved=0 there is a chapter about "mark-mode"s. Best regards, Laci ________________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Gregg Nicholas <gnichola@berriencounty.org> Sent: Thursday, October 31, 2019 13:56 To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] email alert on timeout CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. I'm new to syslog-ng OSE. I've been able to follow the samples, and configured an email alert when a syslog message is received. Now I'm trying to configure an email alert when it has been too long since I've received a particular syslog message. First, I configured a destination: destination d_heartbeat { file("/var/log/heartbeat.log" mark-freq(60) mark-mode(dst-idle) ); }; Then, I configured a filter and log: filter f_heartbeat { match("I am still here" value("MESSAGE) ); }; Log { source(s_network); filter(f_heartbeat); destination(d_heartbeat); }; I was pleasantly surprised that this worked. The heartbeat.log file received the message, and 60 seconds later it received the "syslog -- MARK --" Since I wanted an email alert, I defined a new source, to grab the output of my heartbeat.log as input: source s_heartbeat { file(""/var/log/heartbeat.log"); }; destination d_smtp { smtp( host("10.10.10.25") port(25) from("syslog-ng" "noreply@mydomain.com<mailto:noreply@mydomain.com>") to("me" "myemail@mydomain.com<mailto:myemail@mydomain.com>") subject("host is down") body("no heartbeat received from the program on host\n") ); }; filter f_timeout { match ("MARK --" value("MESSAGE")); }; log { source(s_heartbeat); filter(f_timeout); destination(d_smtp); }; This seemed to work when I tested it with a single heartbeat followed by timeout, but I’m still misunderstanding something. If it gets more than one message written to heartbeat.log, then the timeout MARK doesn’t happen. What am I missing? Is there a better way to accomplish this alert? Thanks, Gregg (running syslog-ng v3.19.1-5 on debian 10) ______________________________________________________________________________ Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cantal.nemes%40oneidentity.com%7C49a647056f144ad1387e08d75e068066%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637081254344960778&sdata=KZ%2BQryJh8SevPwe3V2dup%2Bgig43DQjDv9vyIGMM%2FzrQ%3D&reserved=0 Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cantal.nemes%40oneidentity.com%7C49a647056f144ad1387e08d75e068066%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637081254344960778&sdata=TlzOnS39Oy6hEnKT8PzskVbVLW4E2FFeC%2FXhYxOfTaA%3D&reserved=0 FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cantal.nemes%40oneidentity.com%7C49a647056f144ad1387e08d75e068066%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637081254344960778&sdata=DIH1%2F789kvu0zotC778v6%2B%2BRv70nN1R7%2B7PvjBUCbvM%3D&reserved=0
Hi Laci, Thanks for your advice. I think that the behavior you described for mark-freq is exactly what I'm trying to accomplish, but it doesn't seem to work. There must be some detail that I'm missing. In my test, I've set mark-freq to 60 seconds for the destination heartbeat.log. When I watch (tail) heartbeat.log, I'm seeing this type of results: Oct 31 08:44:02 192.168.35.1 ...I am still here... Oct 31 08:45:03 syslog -- MARK -- Oct 31 08:45:19 192.168.35.1 ...I am still here... Oct 31 08:46:19 syslog -- MARK – Oct 31 08:46:32 192.168.35.1 ...I am still here... Oct 31 08:46:40 192.168.35.1 ...I am still here... Oct 31 09:43:46 192.168.35.1 ...I am still here... As you can see, the destination is not busy. Shouldn’t a MARK have happened at 08:47:40? The “internal” mark-mode looked a bit complicated, but I’ll read it again. Thanks again, Gregg -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Laszlo Szemere (lszemere) Sent: Thursday, October 31, 2019 09:30 To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] email alert on timeout Hello Gregg, I think you are almost on the right track. A little addition to MARK messages: Syslog-ng's destinations will ONLY emit a mark message IF otherwise there will be no message at all from that destination, during a "mark-freq" time period. So if there is a message on the Destination, it will reset the "mark-freq" timer, and the interval starts again without sending any mark message. So during a normal work of a busy log path there should be no mark messages at all. One more thing: I don't know if it is intentional from you, but you can spare the whole "mark" file logic from your configuration in certain cases, if you use the "internal" mark-mode. Unfortunately I can not give you a direct link, but in the "global options" section of the administration guide: <https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.24/administration-guide/59#TOPIC-1298095> https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edit... there is a chapter about "mark-mode"s. Best regards, Laci
participants (3)
-
Antal Nemes (anemes)
-
Gregg Nicholas
-
Laszlo Szemere (lszemere)