Hi, This patch changes the grammar, so it accepts number tokens as well: https://github.com/balabit/syslog-ng/tree/f/java-should-accept-numbers-as-op... testing would be appreciated as this is a "blind" patch, I haven't tried it as I don't personally use elasticsearch/java destinations. The memory corruption problem was already raised in a different thread, but I don't know about any exact outcome from those emails, maybe @lbudai does. -- Bazsi On Thu, Nov 24, 2016 at 3:06 PM, Peter Eckel <lists@eckel-edv.de> wrote:
OK, I got the first one figured out myself ...
flush-limit( 1000 )
does not work.
flush-limit( "1000" )
does, which is inconsistent with the usual syslog-ng.conf behaviour. And setting it to "1" actually fixed the slow propagation to ES for me.
Now I need to see whether the crash still occurs.
Another question that came up: Is it possible to do flushing based on a time interval and not on a message count in the queue? It's rather unsatisfactory to wait for critical messages to appear in ES.
Best regards,
Peter.
On 24 Nov 2016, at 12:28, Peter Eckel <lists@eckel-edv.de> wrote:
I'm currently investigating the Syslog NG -> Elasticsearch 2 destination for a project I'm working on, and started using basically the sample configuration in Peter Czanik's blog article. Thanks, by the way, for the great tutorial.
There is one thing I'm currently struggling with: On my test system I have a fairly low volume of messages, and there seems to be an issue with flushing the cache of Syslog NG to Elasticsearch. To be precise: It doesn't happen. I can easily force a cache flush by reloading Syslog NG, but if I just keep it sitting there it doesn't log anything at all to Elasticsearch (while logs to files, e.g. /var/log/messages, are happening in real time).
I tried configuring the flush-limit() in the elasticsearch2 destination, but the configuration prevents syslog-ng from starting with an error message:
Nov 23 17:30:28 rpm-test syslog-ng[14527]: Error parsing destination, destination plugin flush-limit not found in /etc/syslog-ng/conf.d/elasticsearch.conf at line 9, column 5: Nov 23 17:30:28 rpm-test syslog-ng[14527]: included from /etc/syslog-ng/syslog-ng.conf line 68, column 1 Nov 23 17:30:28 rpm-test syslog-ng[14527]: flush-limit( 1000 ) Nov 23 17:30:28 rpm-test syslog-ng[14527]: ^^^^^^^^^^^ Nov 23 17:30:28 rpm-test syslog-ng[14527]: syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng Nov 23 17:30:28 rpm-test syslog-ng[14527]: mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng
If I remove the flush-limit() line, the config loads without a problem.
[fixed]
After some hours of operation, Syslog NG actually crashed on reload with
a memory corruption issue (might be unlelated):
Nov 23 17:50:14 rpm-test systemd[1]: Started System Logger Daemon. Nov 24 11:04:11 rpm-test systemd[1]: Reloaded System Logger Daemon. Nov 24 11:04:15 rpm-test syslog-ng[31599]: *** Error in
`/usr/sbin/syslog-ng': malloc(): memory corruption (fast): 0x00000000023293df ***
Nov 24 11:04:15 rpm-test syslog-ng[31599]: ======= Backtrace: ========= Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libc.so.6(+0x7b184)[ 0x7f87e29b7184] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libc.so.6(+0x7e877)[ 0x7f87e29ba877] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libc.so.6(__libc_ calloc+0xb4)[0x7f87e29bc2d4] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libglib-2.0.so.0(g_ malloc0+0x17)[0x7f87e39e32c7] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( log_multiplexer_new+0x13)[0x7f87e4529833] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( cfg_tree_new_mpx+0x12)[0x7f87e4521872] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x30136)[0x7f87e4522136] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x2fc11)[0x7f87e4521c11] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x2fb2b)[0x7f87e4521b2b] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x2fc11)[0x7f87e4521c11] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x302a9)[0x7f87e45222a9] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x2fc11)[0x7f87e4521c11] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( cfg_tree_compile_rule+0x35)[0x7f87e4522375] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( cfg_tree_compile+0x4b)[0x7f87e45224cb] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( cfg_tree_start+0x16)[0x7f87e4522576] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( cfg_init+0x16e)[0x7f87e451d58e] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x400de)[0x7f87e45320de] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0(+0x407d7)[0x7f87e45327d7] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libivykis.so.0(+0x3b4f)[0x7f87e2f1cb4f] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libivykis.so.0(+0x5193)[0x7f87e2f1e193] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libivykis.so.0(+0x5810)[0x7f87e2f1e810] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libivykis.so.0(iv_main+0x44)[0x7f87e2f1f7d4] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libsyslog-ng-3.8.so.0( main_loop_run+0x74)[0x7f87e4532734] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /usr/sbin/syslog-ng(main+ 0x1b8)[0x4017b8] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f87e295db15] Nov 24 11:04:15 rpm-test syslog-ng[31599]: /usr/sbin/syslog-ng[0x4018dd]
[status unknown]
OS Version is CentOS 7.2, Syslog NG 5.8.1 (installed from Peter's
repository), Elasticsearch 5.0.1 (installed from Elastic's repo). The machine is a freshly installed system.
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq