[syslog-ng] Insider 2018-12: 3.19 release; optimizing Splunk; Python source; HTTP batch;

[Pal, Laszlo]

Hi,

Is there any real deep-dive document showing why it is actually better to
send from the relay to HEC destination than send simply to a syslog
destination? I was running a pilot project where we pumped firewall logs
from syslog-ng PE 7 relay to Splunk Enterprise on plain syslog/tcp (of
course with disc buffer) and I never experienced any issue in this
scenario, so I've tried to find a detailed document on performance wise
differences between syslog/tcp and HEC destinations

Do you have such a whitepaper?

Thx
L:


On Thu, Dec 13, 2018 at 4:00 PM Czanik, Péter <peter.czanik at balabit.com>
wrote:

> Dear syslog-ng users,
>
> This is the 71st issue of syslog-ng Insider, a monthly newsletter that
> brings you syslog-ng-related news.
>
> NEWS
>
> Version 3.19 of syslog-ng released
> ----------------------------------
>
> Version 3.19 of syslog-ng has been released with plenty of new
> features and bugfixes. Performance of the HTTP destination improved
> thanks to load-balancing to multiple servers. You can use this to send
> the messages to a set of ingestion nodes or indexers of your SIEM
> solution if a single node cannot handle the load. The new Slack
> destination allows you to send alerts to a Slack channel.
>
> Read the complete list of changes at
> https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1.
>
>
> Optimize your Splunk infrastructure using new syslog-ng features
> -------------------------------------------------------
>
> Learn how to use less resources for better performance in Splunk! Many
> people have been using syslog-ng for decades without knowing that it
> receives new features as well as bugfixes. While many Linux utilities
> are practically in maintenance mode, syslog-ng keeps evolving
> constantly. A strong focus in recent years has been on message parsing
> and destination drivers.
>
> After my talk at Suricon, Splunk users explained how they will change
> their syslog-ng configurations to optimize their Splunk
> infrastructure.
>
>
> https://www.syslog-ng.com/community/b/blog/posts/optimize-your-splunk-infrastructure-using-new-syslog-ng-features
>
>
> Python source in syslog-ng
> --------------------------
>
> Using syslog-ng 3.18 and newer releases, you can write new source
> drivers for syslog-ng in Python. While performance is not as good as
> C, you gain flexibility and ease of implementation. There are quite a
> few log sources without a ready to use C API, but with a Python API.
> Using the Python source of syslog-ng you can leverage these.
>
> https://www.syslog-ng.com/community/b/blog/posts/python-source-in-syslog-ng
>
>
> Bulk mode message sending to Elasticsearch with syslog-ng http()
> destination
> ---------------------------------------------------------
>
> Learn how to send log messages in bulk mode to your Elasticsearch
> server with syslog-ng. Bulk mode offers better performance, because it
> sends multiple log messages in a single POST request.
>
>
> https://www.syslog-ng.com/community/b/blog/posts/bulk-mode-message-sending-to-elasticsearch-with-syslog-ng-http-destination
>
>
> WEBINARS
>
> You can watch our past webinars:
>
> * Log ingestion to Splunk HEC:
> https://www.brighttalk.com/webcast/16207/338190
>
> * High performance log streaming to HDFS with syslog-ng:
> https://www.brighttalk.com/webcast/16207/335943
>
>
> Your feedback and news, or tips about the next issue are welcome. To
> read this newsletter online, visit: https://syslog-ng.com/blog/
>
>
> Peter Czanik (CzP) <peter.czanik at balabit.com>
> Balabit (a OneIdentity company) / syslog-ng upstream
> https://syslog-ng.com/community/
> https://twitter.com/PCzanik
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20181213/7464dbbf/attachment-0001.html>