[syslog-ng] Insider 2018-12: 3.19 release; optimizing Splunk; Python source; HTTP batch;

[Scheidler, Balázs]

There was a talk in 2017 .conf where there's an entire slide dedicated to
that. If you look at the video recording you'll see why Splunk engineers
don't recommend doing that.

https://conf.splunk.com/conf-online.html?search=to%20hec%20with%20syslog#/

It's mostly about scalability.

On Thu, Dec 13, 2018 at 4:25 PM Pal, Laszlo <vlad at vlad.hu> wrote:

> Hi,
>
> Is there any real deep-dive document showing why it is actually better to
> send from the relay to HEC destination than send simply to a syslog
> destination? I was running a pilot project where we pumped firewall logs
> from syslog-ng PE 7 relay to Splunk Enterprise on plain syslog/tcp (of
> course with disc buffer) and I never experienced any issue in this
> scenario, so I've tried to find a detailed document on performance wise
> differences between syslog/tcp and HEC destinations
>
> Do you have such a whitepaper?
>
> Thx
> L:
>
>
> On Thu, Dec 13, 2018 at 4:00 PM Czanik, Péter <peter.czanik at balabit.com>
> wrote:
>
>> Dear syslog-ng users,
>>
>> This is the 71st issue of syslog-ng Insider, a monthly newsletter that
>> brings you syslog-ng-related news.
>>
>> NEWS
>>
>> Version 3.19 of syslog-ng released
>> ----------------------------------
>>
>> Version 3.19 of syslog-ng has been released with plenty of new
>> features and bugfixes. Performance of the HTTP destination improved
>> thanks to load-balancing to multiple servers. You can use this to send
>> the messages to a set of ingestion nodes or indexers of your SIEM
>> solution if a single node cannot handle the load. The new Slack
>> destination allows you to send alerts to a Slack channel.
>>
>> Read the complete list of changes at
>> https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1.
>>
>>
>> Optimize your Splunk infrastructure using new syslog-ng features
>> -------------------------------------------------------
>>
>> Learn how to use less resources for better performance in Splunk! Many
>> people have been using syslog-ng for decades without knowing that it
>> receives new features as well as bugfixes. While many Linux utilities
>> are practically in maintenance mode, syslog-ng keeps evolving
>> constantly. A strong focus in recent years has been on message parsing
>> and destination drivers.
>>
>> After my talk at Suricon, Splunk users explained how they will change
>> their syslog-ng configurations to optimize their Splunk
>> infrastructure.
>>
>>
>> https://www.syslog-ng.com/community/b/blog/posts/optimize-your-splunk-infrastructure-using-new-syslog-ng-features
>>
>>
>> Python source in syslog-ng
>> --------------------------
>>
>> Using syslog-ng 3.18 and newer releases, you can write new source
>> drivers for syslog-ng in Python. While performance is not as good as
>> C, you gain flexibility and ease of implementation. There are quite a
>> few log sources without a ready to use C API, but with a Python API.
>> Using the Python source of syslog-ng you can leverage these.
>>
>>
>> https://www.syslog-ng.com/community/b/blog/posts/python-source-in-syslog-ng
>>
>>
>> Bulk mode message sending to Elasticsearch with syslog-ng http()
>> destination
>> ---------------------------------------------------------
>>
>> Learn how to send log messages in bulk mode to your Elasticsearch
>> server with syslog-ng. Bulk mode offers better performance, because it
>> sends multiple log messages in a single POST request.
>>
>>
>> https://www.syslog-ng.com/community/b/blog/posts/bulk-mode-message-sending-to-elasticsearch-with-syslog-ng-http-destination
>>
>>
>> WEBINARS
>>
>> You can watch our past webinars:
>>
>> * Log ingestion to Splunk HEC:
>> https://www.brighttalk.com/webcast/16207/338190
>>
>> * High performance log streaming to HDFS with syslog-ng:
>> https://www.brighttalk.com/webcast/16207/335943
>>
>>
>> Your feedback and news, or tips about the next issue are welcome. To
>> read this newsletter online, visit: https://syslog-ng.com/blog/
>>
>>
>> Peter Czanik (CzP) <peter.czanik at balabit.com>
>> Balabit (a OneIdentity company) / syslog-ng upstream
>> https://syslog-ng.com/community/
>> https://twitter.com/PCzanik
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20181213/8fb50e22/attachment.html>