"Reinsert" message into syslog-ng via pipe?
Hi Perhaps my idea is absolutely nuts and there is a better way, but I would like to do the following: I receive weblogs in apache-format from my F5 Loadbalancer via syslog-tcp now I would like to use the apache-accesslog-parser on the message to extract some values more easy. I would try to setup following (not a syslog-ng syntax in that description) source: F5 -- via 5140/tcp into syslog-ng syslog-ng: Destination pipe(my-f5-logs) syslog-ng: source(pipe(my-f5-logs)) -> parser(apache-accesslog-parser) -> extract some fields -> write to file The whole runs on the same syslog-ng Is that too complex or a normal way? We are running a syslog-ng OSE 3.23.1 Any clues or a simple "nod thats okay" would be nice cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber@metzler.com www.metzler.com Persönlich haftende Gesellschafter: Harald Illy, Michael Klaus, Friedrich von Metzler, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 27 515 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.
Hi, Unless you want to do something to the logs outside syslog-ng (that is, after you write the logs to the pipe but before you read them back), there is no need to complicate things like that. You can simply create two log paths, where both log paths read the same source, but one uses the pipe destination, the other one the parser and the file destination. Actually, you can do it with one log path if you use an embedded log statement. HTH, Robert ________________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Matthias Gruber <MGruber@metzler.com> Sent: Friday, October 11, 2019 17:03 To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] "Reinsert" message into syslog-ng via pipe? CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hi Perhaps my idea is absolutely nuts and there is a better way, but I would like to do the following: I receive weblogs in apache-format from my F5 Loadbalancer via syslog-tcp now I would like to use the apache-accesslog-parser on the message to extract some values more easy. I would try to setup following (not a syslog-ng syntax in that description) source: F5 -- via 5140/tcp into syslog-ng syslog-ng: Destination pipe(my-f5-logs) syslog-ng: source(pipe(my-f5-logs)) -> parser(apache-accesslog-parser) -> extract some fields -> write to file The whole runs on the same syslog-ng Is that too complex or a normal way? We are running a syslog-ng OSE 3.23.1 Any clues or a simple "nod thats okay" would be nice cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber@metzler.com www.metzler.com<https://nam05.safelinks.protection.outlook.com/?url=www.metzler.com&data=02%7C01%7Crobert.fekete%40oneidentity.com%7Cfdb4d5a6359344d2696708d74e5c2d4f%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637064030129217722&sdata=i1Z6t70sFMKJCwVYiTLxqDWCtVKGF3Q76S49ccmA8gM%3D&reserved=0> Persönlich haftende Gesellschafter: Harald Illy, Michael Klaus, Friedrich von Metzler, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 27 515 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.
On Fri, Oct 11, 2019 at 05:29:54PM +0000, Robert Fekete (rfekete) wrote:
Hi,
Unless you want to do something to the logs outside syslog-ng (that is, after you write the logs to the pipe but before you read them back), there is no need to complicate things like that. You can simply create two log paths, where both log paths read the same source, but one uses the pipe destination, the other one the parser and the file destination. Actually, you can do it with one log path if you use an embedded log statement.
That being said, it would be pretty cool to be able to reinject logs into a source: source s_foo { file("/in"); }; log { source(s_foo); ... destination { reinject(s_foo); }; };
participants (3)
-
Fabien Wernli
-
Matthias Gruber
-
Robert Fekete (rfekete)