October 2014 - syslog-ng - lists.balabit.hu

Create Pattern-DB rules
by Justin Kala 20 Oct '15

20 Oct '15

Hello I am trying to create Pattern-DB for the following Authorization messages coming from O/S. Can you help on creating a matching rule . Note: I do not have PATTERN-DB parser utility.I am going to create the db-parser.xml manually and put this rules inside the file. 2014-09-28T14:12:44-04:00 abcdef01-app/abcdef01-app sshd[11019]: [ID 800047 auth.notice] Failed password for root from 100.200.255.01 port 54438 ssh2 2014-09-28T14:03:46-04:00 abcdef01-app/abcdef01-app sshd[27420]: [ID 800047 auth.notice] Failed publickey for root from 100.200.255.02 port 59219 ssh2 2014-09-28T14:08:28-04:00 abcdef01-app/abcdef01-app sshd[3954]: [ID 800047 auth.notice] Failed keyboard-interactive for root from 100.200.255.03 port 65410 ssh2 2014-09-28T14:10:11-04:00 abcdef01-app/abcdef01-app sshd[5222]: [ID 293258 auth.error] libsldap: Status: 49 Mesg: openConnection: simple bind failed - Invalid credentials Thanks & Regards Kaladhar

4 16

I want to forward Windows server 2008 R2 logs to Linux syslog-ng.
by Jason Long 24 Dec '14

24 Dec '14

Hello all. I have a Windows Box that want to forward all Even logs to my Linux box. I install Snare on Windows(172.30.10.19) and configure it to forward logs to Linux and my Linux box receive it properly. When I use " tcpdump udp "port 514" ", Tcpdump show me that Snare sending Logs to Linux but Syslog-ng can't write it to log files :(. I paste my syslog-ng configure : options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; #source s_net { #udp (ip(172.30.10.19) port(514)); #}; source s_net { udp(); }; filter f_openwrt { host("172.30.10.19");}; destination df_openwrt { file("/var/log/winlog/win.log"); }; log { source ( s_net ); filter( f_openwrt ); destination ( df_openwrt ); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: Can you tell me how can I solve this problem? Cheers.

5 8

How can I disable SSLv3 in syslog-ng 3.3.2 client config to sovle CVE-2014-3566(SSLv3 Fallback Vulnerabilit)?
by bluebenben 05 Nov '14

05 Nov '14

Hi guys In my project I am using syslog-ng as syslog client and send log via TLS. We all know that recently there is one new security flaw which is Poodle(CVE-2014-3566 - SSLv3 Fallback Vulnerability) This requires disabling SSLv3 I have checked admin guide of syslog-ng 3.3.2 but I am able to find the option Could you please let me know the way? Alternatively I think I may achieve the object by disable SSLv3 ciphers used by syslog-ng client original ciphers used by us is ALL:!SSLv2:!MEDIUM:!LOW:!EXP:!ADH:!ECDH:!PSK:!MD5:@STRENGTH I may change it to ALL:!SSLv3:!SSLv2:!MEDIUM:!LOW:!EXP:!ADH:!ECDH:!PSK:!MD5:@STRENGTH Bug this will make syslog-ng only supports TLS1.2 and cause negative impact to interoperability Thanks Jason

2 1

Elasticsearch destination
by Russell Fulton 31 Oct '14

31 Oct '14

Hi We are already using the open source version of syslog-ng and I am about to set up some elastic search instances and would much prefer to feed data direct from syslog-ng rather than go through logstash (I already have a heap of patterndb parsers and performance should be way better!) I have spent an hour or so with Google and have found various references to elastic search destination being available but I can find no mention of it in the release notes for 3.6.1. I have also downloaded the the tarball and unpacked it but could not find any evidence of the module , nore is there any mention of it in the manual. As of now what is the recommended way of getting parsed data from OS syslog-ng into ES? Thanks, Russell

6 17

Re: [syslog-ng] Elasticsearch destination
by Jim Hendrick 31 Oct '14

31 Oct '14

If we use syslog-ng, hadoop, redis, elasticsearch and kibana we could call it the SHREK stack ... Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Fabien Wernli <wernli(a)in2p3.fr> Date:10/31/2014 7:43 AM (GMT-05:00) To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] Elasticsearch destination Hi, On Fri, Oct 31, 2014 at 07:05:47AM -0400, Jim Hendrick wrote: > area. That is one of the reasons I am trying to have syslog-ng do as > much as possible so I can remove "L" and only use "E" and "K" :-) yeah, let's call it the SNEK stack, or SNERK when using Riemann too :o) ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

5 4

Re: [syslog-ng] Elasticsearch destination
by Jim Hendrick 31 Oct '14

31 Oct '14

Or reman or "R" .. Hey .. this might actually be a thing! Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Fabien Wernli <wernli(a)in2p3.fr> Date:10/31/2014 7:43 AM (GMT-05:00) To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] Elasticsearch destination Hi, On Fri, Oct 31, 2014 at 07:05:47AM -0400, Jim Hendrick wrote: > area. That is one of the reasons I am trying to have syslog-ng do as > much as possible so I can remove "L" and only use "E" and "K" :-) yeah, let's call it the SNEK stack, or SNERK when using Riemann too :o) ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Re: [syslog-ng] syslog-ng as "shipper" into ELK stack
by Jim Hendrick 28 Oct '14

28 Oct '14

Thanks!I am on vacation this week but I look forward to trying out these suggestions. Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Alexandre Biancalana <biancalana(a)gmail.com> Date:10/06/2014 11:42 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng as "shipper" into ELK stack On Mon, Oct 6, 2014 at 9:36 AM, Radu Gheorghe <radu.gheorghe(a)sematext.com> wrote: Hi Jim, With rabbitmq you have the advantage that you can install the RabbitMQ river and have Elasticsearch pull logs from Rabbit instead of having another [moving] piece pull logs from Rabbit and push them to ES. So you'd have a simpler setup that also makes sure ES isn't overwhelmed (because ES is pulling). Another point that can be an advantage is that with Redis you are limited to the RAM memory available on the machine (all your data need to fit in memory) , with RabbitMQ you have disk persistence that can help in cases where you need to stop ES consumption by any reason. There are some problems with this approach: - the river only runs on one node at a time, which may become a bottleneck - rivers are deprecated (or will be) so the ES side isn't actively maintained. I've seen failover issues (node running the river goes down, another node should start the river but doesn't) which needed river delete + recreate to kick the process in again I didn't know that, I will check it out. Logstash started by recommending RabbitMQ as the queue between two Logstash instances, but now moved to Redis. Apparently the reason is that Redis plays nicely with Logstash, and Rabbit didn't, here's a quote from the guide: "Previous versions of this guide used AMQP via RabbitMQ. Due to the complexity of AMQP as well as performance issues related to the Bunny driver we use, we're now recommending Redis instead." Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Sat, Oct 4, 2014 at 5:09 AM, Jim Hendrick <jrhendri(a)roadrunner.com> wrote: Thanks. Why rabbitmq instead of redis? Is it faster, or does it offer some additional functions? Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Alexandre Biancalana <biancalana(a)gmail.com> Date:10/03/2014 7:01 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng as "shipper" into ELK stack On Thu, Oct 2, 2014 at 9:33 PM, Jim Hendrick <jrhendri(a)roadrunner.com> wrote: Hi, I am working on configuring Elasticsearch, Logstash & Kibana (ELK) to test it as a backend search tool for large volumes of logs. I decided to put Redis in front of Logstash as a "broker" for the incoming logs, and syslog-ng as the "shipper" so it looks like this: syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana I've been using the following: syslog-ng => rabbitmq => elasticsearch syslog-ng + patterndb to parse logs and write then in json format on rabbitmq, after that is just use elasticsearch amqp river to consume the queue. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

4 4

improving SQLite writing performances (WAS: Re: inserts into a sqlite3 database are not delayed)
by Davide Alberani 27 Oct '14

27 Oct '14

On Wed, Oct 15, 2014 at 10:56 AM, Balazs Scheidler <bazsi77(a)gmail.com> wrote: > > It immediately runs the query, flush_lines() controls the transaction size, > which can be enabled using > flags(explicit-commits). Ok, I see and I can confirm noticeable improvements on a system under heavy I/O and CPU load using explicit-commits (or better: I see it gets a lot worse disabling it, since I already had it on ;)) Plus, I'm seeing minor improvements with session_statements set to "PRAGMA synchronous=OFF; PRAGMA count_changes=OFF;" Beside this, has anyone found other ways to improve writing performances using syslog-ng with sqlite? On (physical) systems with high load, I've noticed that the system is much more responsive if the elevator is set to noop or deadline instead of cfq. But I guess this is highly dependent on the specific circumstances of the system. Thanks! -- Davide Alberani <davide.alberani(a)gmail.com> [PGP KeyID: 0x465BFD47] http://www.mimante.net/

2 1

errors running configure for incubator on Ubuntu 12.04
by Russell Fulton 25 Oct '14

25 Oct '14

rful011@secmontst01:~/src/syslog-ng-incubator$ ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane… yes [ snip ] checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking for inline... inline ./configure: line 4534: syntax error near unexpected token `shared' ./configure: line 4534: `LT_INIT(shared dlopen)’ I have looked at configure and config.log to try and figure out what it is actually checking for. ‘inline’ is to generic to google for :( any ideas what is wrong? Russell

2 1

syslog-ng 3.6.1 is released
by Czanik, Péter 22 Oct '14

22 Oct '14

Hi, We are proud to announce that culminating almost a year’s worth of development, syslog-ng 3.6.1 has been released! This is the first stable release from the 3.6 branch, the successor to 3.5, which was originally released in November, 2013. Read more about the highlights of this release at https://czanik.blogs.balabit.com/2014/10/syslog-ng-3-6-1-is-released/ or all the technical details at https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.6.1 Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0