snmptrap handler parses SNMP traps correctly, but the file it reads from is not emptying
Hello - I set up snmptrapd to write to /var/log/snmptrapd.log and syslog-ng is configured to read from it like this ... source { snmptrap(filename("/var/log/snmptrapd.log")); }; ... but /var/log/snmptrapd.log is not getting emptied, so it needs to be truncated every so often to avoid filling up the filesystem. But the problem with truncating /var/log/snmptrapd.log is that we're going to lose traps, since we get a lot of them. When I was testing this syslog-ng snmptrap facility with a few traps a minute, I saw /var/log/snmptrapd.log empty, but with hundreds of traps incoming per second, it does not empty any more. I have what looks like a work-around to the syslog-ng snmptrap handler/parser where ... 1. 1) Snmptrapd 5.7.3 writes to syslog with -Lsd 2. 2) Syslog-ng 3.20 reads from syslog with system-journal() 3. 3) Rewrite the trap $MESSAGE with a series of "substitutions" (subst) into space separated key=value pairs 4. 4) point kv-parser() at $MESSAGE and specify " " as a separator 5. 5) delete the original $MESSAGE block ... but perhaps I didn't have to do that, since perhaps I'm mis-using the snmptrap facility built into syslog-ng, and thus causing /var/log/snmptrapd.log to not empty. Your comments and insights would be appreciated. - Allen -------------------------------------------------------- The information transmitted in this email and any of its attachments is intended only for the person or entity to which it is addressed and may contain information concerning Altice USA and/or its affiliates and subsidiaries that is proprietary, privileged, confidential and/or subject to copyright. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited and may be unlawful. If you received this in error, please contact the sender immediately and delete and destroy the communication and all of the attachments you have received and all copies thereof. --------------------------------------------------------
Just set up logrotate on that file, syslog-ng will read till the end and start the new one if the rotation happens. So instead of truncating, rename it to a new name, and let snmptrapd write to a new file. On Mon, May 20, 2019, 15:55 Allen Pouratian <Allen.Pouratian@alticeusa.com> wrote:
Hello –
I set up snmptrapd to write to */var/log/snmptrapd.log* and syslog-ng is configured to read from it like this …
*source { *
*snmptrap(filename("/var/log/snmptrapd.log")); *
*};*
… but */var/log/snmptrapd.log* is not getting emptied, so it needs to be truncated every so often to avoid filling up the filesystem.
But the problem with truncating */var/log/snmptrapd.log* is that we’re going to lose traps, since we get a lot of them.
When I was testing this syslog-ng snmptrap facility with a few traps a minute, I saw */var/log/snmptrapd.log* empty, but with hundreds of traps incoming per second, it does not empty any more.
I have what looks like a work-around to the syslog-ng snmptrap handler/parser where …
1. 1) Snmptrapd 5.7.3 writes to syslog with -Lsd 2. 2) Syslog-ng 3.20 reads from syslog with system-journal() 3. 3) Rewrite the trap $MESSAGE with a series of “substitutions” (subst) into space separated key=value pairs 4. 4) point kv-parser() at $MESSAGE and specify “ “ as a separator 5. 5) delete the original $MESSAGE block
… but perhaps I didn’t have to do that, since perhaps I’m mis-using the snmptrap facility built into syslog-ng, and thus causing */var/log/snmptrapd.log* to not empty.
Your comments and insights would be appreciated.
- Allen
-------------------------------------------------------- The information transmitted in this email and any of its attachments is intended only for the person or entity to which it is addressed and may contain information concerning Altice USA and/or its affiliates and subsidiaries that is proprietary, privileged, confidential and/or subject to copyright. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited and may be unlawful. If you received this in error, please contact the sender immediately and delete and destroy the communication and all of the attachments you have received and all copies thereof. --------------------------------------------------------
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Balázs – What guarantee is there that I won’t lose traps? 1. Does logrotate make sure snmptrapd has finished writing to "/var/log/snmptrapd.log"? Is the write by snmptrapd 5.7.3+ always atomic for one or more traps? 2. Does syslog-ng 3.20.1 make sure it instantly reads every trap written by snmptrapd to "/var/log/snmptrapd.log" before it’s switched out by logrotate? I apologize if this is basic knowledge I’m not aware of. Best Regards, -Allen From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Scheidler, Balázs Sent: Monday, May 20, 2019 11:55 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] snmptrap handler parses SNMP traps correctly, but the file it reads from is not emptying [External Email] Just set up logrotate on that file, syslog-ng will read till the end and start the new one if the rotation happens. So instead of truncating, rename it to a new name, and let snmptrapd write to a new file. On Mon, May 20, 2019, 15:55 Allen Pouratian <Allen.Pouratian@alticeusa.com<mailto:Allen.Pouratian@alticeusa.com>> wrote: Hello – I set up snmptrapd to write to /var/log/snmptrapd.log and syslog-ng is configured to read from it like this … source { snmptrap(filename("/var/log/snmptrapd.log")); }; … but /var/log/snmptrapd.log is not getting emptied, so it needs to be truncated every so often to avoid filling up the filesystem. But the problem with truncating /var/log/snmptrapd.log is that we’re going to lose traps, since we get a lot of them. When I was testing this syslog-ng snmptrap facility with a few traps a minute, I saw /var/log/snmptrapd.log empty, but with hundreds of traps incoming per second, it does not empty any more. I have what looks like a work-around to the syslog-ng snmptrap handler/parser where … 1. 1) Snmptrapd 5.7.3 writes to syslog with -Lsd 2. 2) Syslog-ng 3.20 reads from syslog with system-journal() 3. 3) Rewrite the trap $MESSAGE with a series of “substitutions” (subst) into space separated key=value pairs 4. 4) point kv-parser() at $MESSAGE and specify “ “ as a separator 5. 5) delete the original $MESSAGE block … but perhaps I didn’t have to do that, since perhaps I’m mis-using the snmptrap facility built into syslog-ng, and thus causing /var/log/snmptrapd.log to not empty. Your comments and insights would be appreciated. - Allen -------------------------------------------------------- The information transmitted in this email and any of its attachments is intended only for the person or entity to which it is addressed and may contain information concerning Altice USA and/or its affiliates and subsidiaries that is proprietary, privileged, confidential and/or subject to copyright. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited and may be unlawful. If you received this in error, please contact the sender immediately and delete and destroy the communication and all of the attachments you have received and all copies thereof. -------------------------------------------------------- ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.balabit.hu_mailman_listinfo_syslog-2Dng&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=-wSgv6P6Tu67nZwvsDpZ8FSGWw3vRdbr9k2k8lOsYN8&e=> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_support_documentation_-3Fproduct-3Dsyslog-2Dng&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=qEw0ZLUxEifTpc9_6CVz8tXvfTCvtmcMX3nvPyriky8&e=> FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_wiki_syslog-2Dng-2Dfaq&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=1ihsxbF1hC0FOjy9_eIGny6fXBoHYk5XPxzJxAB9-Gg&e=> Caution: This email originated outside of Altice USA. Please do not click links or attachments unless you recognize the sender and know the content is safe.
In Unix you can rename an opened file, and any processes having that file open will continue to write to that file. So this is what should happen: * Logrotate moves the current log file to a different name * Logrotate alerts snmptrapd that it needs to write to the new file (usually via SIGHUP) * syslog-ng continues to read the old file (keeping it opened during the process above, e.g. still referencing the old file even if it has been renamed * When eof is reached, syslog-ng checks if there's a different file with the original name. * If there's it switches to it. * The renamed "old" logfile can be compressed/moved away/etc. This handshake ensures that no data is lost. On Tue, May 21, 2019, 17:41 Allen Pouratian <Allen.Pouratian@alticeusa.com> wrote:
Balázs –
What guarantee is there that I won’t lose traps?
1. Does logrotate make sure snmptrapd has finished writing to *"/var/log/snmptrapd.log"? *Is the write by snmptrapd 5.7.3+ always atomic for one or more traps? 2. Does syslog-ng 3.20.1 make sure it instantly reads every trap written by snmptrapd to *"/var/log/snmptrapd.log" *before it’s switched out by logrotate?
I apologize if this is basic knowledge I’m not aware of.
Best Regards,
-Allen
*From:* syslog-ng <syslog-ng-bounces@lists.balabit.hu> * On Behalf Of *Scheidler, Balázs *Sent:* Monday, May 20, 2019 11:55 AM *To:* Syslog-ng users' and developers' mailing list < syslog-ng@lists.balabit.hu> *Subject:* Re: [syslog-ng] snmptrap handler parses SNMP traps correctly, but the file it reads from is not emptying
*[External Email]*
Just set up logrotate on that file, syslog-ng will read till the end and start the new one if the rotation happens.
So instead of truncating, rename it to a new name, and let snmptrapd write to a new file.
On Mon, May 20, 2019, 15:55 Allen Pouratian <Allen.Pouratian@alticeusa.com> wrote:
Hello –
I set up snmptrapd to write to */var/log/snmptrapd.log* and syslog-ng is configured to read from it like this …
*source { *
*snmptrap(filename("/var/log/snmptrapd.log")); *
*};*
… but */var/log/snmptrapd.log* is not getting emptied, so it needs to be truncated every so often to avoid filling up the filesystem.
But the problem with truncating */var/log/snmptrapd.log* is that we’re going to lose traps, since we get a lot of them.
When I was testing this syslog-ng snmptrap facility with a few traps a minute, I saw */var/log/snmptrapd.log* empty, but with hundreds of traps incoming per second, it does not empty any more.
I have what looks like a work-around to the syslog-ng snmptrap handler/parser where …
1. 1) Snmptrapd 5.7.3 writes to syslog with -Lsd 2. 2) Syslog-ng 3.20 reads from syslog with system-journal() 3. 3) Rewrite the trap $MESSAGE with a series of “substitutions” (subst) into space separated key=value pairs 4. 4) point kv-parser() at $MESSAGE and specify “ “ as a separator 5. 5) delete the original $MESSAGE block
… but perhaps I didn’t have to do that, since perhaps I’m mis-using the snmptrap facility built into syslog-ng, and thus causing */var/log/snmptrapd.log* to not empty.
Your comments and insights would be appreciated.
- Allen
-------------------------------------------------------- The information transmitted in this email and any of its attachments is intended only for the person or entity to which it is addressed and may contain information concerning Altice USA and/or its affiliates and subsidiaries that is proprietary, privileged, confidential and/or subject to copyright. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited and may be unlawful. If you received this in error, please contact the sender immediately and delete and destroy the communication and all of the attachments you have received and all copies thereof. --------------------------------------------------------
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.balabit.hu_mailman_listinfo_syslog-2Dng&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=-wSgv6P6Tu67nZwvsDpZ8FSGWw3vRdbr9k2k8lOsYN8&e=> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_support_documentation_-3Fproduct-3Dsyslog-2Dng&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=qEw0ZLUxEifTpc9_6CVz8tXvfTCvtmcMX3nvPyriky8&e=> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_wiki_syslog-2Dng-2Dfaq&d=DwMFaQ&c=j5qhuXVFDG9qAPVzJdE5nsirLGtagR5tRVwNBd9MNeo&r=BBDIJ7y03FqO0tp_wGoZn961RS3d3lkq0jHFc3Knid4&m=uVtE5BY0NtoZSHcV7M9rVxkmtCUx34u93UPnbMDlgco&s=1ihsxbF1hC0FOjy9_eIGny6fXBoHYk5XPxzJxAB9-Gg&e=>
*Caution: This email originated outside of Altice USA. Please do not click links or attachments unless you recognize the sender and know the content is safe.*
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
Allen Pouratian
-
Scheidler, Balázs