I am trying to configure syslog-ng to send multiple json formatted logs to a rabbitmq endpoint. The consumer for the logs checks for a particular field to determine the type of log it is. However, I can't seem to figure out how to send all the different logs as the field it checks is "source" that I add using --pair within the format-json template destination d_amqp_ssh { amqp( vhost("/") host("mozdefqa2.private.scl3.mozilla.com") port(5672) exchange("eventtask") exchange-type("direct") routing-key("eventtask") body("$(format-json --scope nv_pairs --pair category=\"bro\" --pair source=\"ssh\" --pair customendpoint=\" \" --pair tags=\"bro\")") persistent(no) username("guest") password("guest") ); }; So I added another destination with the name d_amqp_conn and it's source field value is "conn" and the amqp endpoint is the same host as the ssh log. But syslog-ng fails to start with this particular setup. It seems to not like having more than one amqp destination. How would something like this typically be solved? -- Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
Hi, 2 solutions: 1. use rewrite rules upstream to set $source then use --pair source=$source 2. use persist-name(myname) in the two destinations config
I like solution #1 and decided to try it, but now I get Compiling p_json sequence [parser] at [/etc/syslog-ng/syslog-ng.conf:1] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:1] Segmentation fault This is using syslog-ng 3.5 and the fault happens in libc-2.19.so. Could it be a bug in the version and if so, should I compile from source with 3.12? On Fri, Nov 3, 2017 at 3:35 PM, Fabien Wernli <wernli@in2p3.fr> wrote:
Hi,
2 solutions:
1. use rewrite rules upstream to set $source then use --pair source=$source
2. use persist-name(myname) in the two destinations config
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
Also, if I comment out the other logs, and only send one log, it works fine, it's when I use multiple that it segfaults. On Fri, Nov 3, 2017 at 4:10 PM, Alicia Smith <asmith@mozilla.com> wrote:
I like solution #1 and decided to try it, but now I get
Compiling p_json sequence [parser] at [/etc/syslog-ng/syslog-ng. conf:1] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:1] Segmentation fault
This is using syslog-ng 3.5 and the fault happens in libc-2.19.so.
Could it be a bug in the version and if so, should I compile from source with 3.12?
On Fri, Nov 3, 2017 at 3:35 PM, Fabien Wernli <wernli@in2p3.fr> wrote:
Hi,
2 solutions:
1. use rewrite rules upstream to set $source then use --pair source=$source
2. use persist-name(myname) in the two destinations config
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
--
Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
-- Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
Hi again, Can you send us your config so we can try to reproduce this behaviour?
Sure! I appreciate the help! I've attached it. On Fri, Nov 3, 2017 at 5:00 PM, Fabien Wernli <wernli@in2p3.fr> wrote:
Hi again,
Can you send us your config so we can try to reproduce this behaviour?
-- Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
Hi, The config looks fine to me. The segfault is clearly a bug. While I'll be able to test this on monday, could you try using a more recent version of syslog-ng?
You might even try the syslog-ng docker images, that should make evaluation and experimentation with newer versions much easier. On Nov 4, 2017 19:25, "Fabien Wernli" <wernli@in2p3.fr> wrote:
Hi,
The config looks fine to me. The segfault is clearly a bug. While I'll be able to test this on monday, could you try using a more recent version of syslog-ng?
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thanks! Also if I were going to build say version 8, because 12 requires librabbitmq4 (and ubuntu 14.04 only has librabbitmq1 available) - how can I compile it without systemd (or the need for mod-journal) I can't upgrade this system (though it is planned for q1 of next year) so any help trying to find the best suitable version with zero systemd or mongodb requirements is appreciated. On Sat, Nov 4, 2017 at 3:19 PM, Scheidler, Balázs < balazs.scheidler@balabit.com> wrote:
You might even try the syslog-ng docker images, that should make evaluation and experimentation with newer versions much easier.
On Nov 4, 2017 19:25, "Fabien Wernli" <wernli@in2p3.fr> wrote:
Hi,
The config looks fine to me. The segfault is clearly a bug. While I'll be able to test this on monday, could you try using a more recent version of syslog-ng?
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Alicia Smith @phrozyn Information Security Engineer asmith@mozilla.com
Hi, journald is optional, like most modules Cheers
Hi, and we would link against the librabbitmq library that was supplied at compile time. -- Bazsi On Mon, Nov 6, 2017 at 8:41 AM, Fabien Wernli <wernli@in2p3.fr> wrote:
Hi,
journald is optional, like most modules
Cheers
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (3)
-
Alicia Smith
-
Fabien Wernli
-
Scheidler, Balázs