- syslog-ng - lists.balabit.hu

Securing Communications Between syslog-NG Server and syslog Clients
by wiskbroom＠hotmail.com 05 Feb '15

05 Feb '15

Hello; I have several thousand Linux and Solaris clients, some using syslog, some using rsyslog. I intend to use stock syslog client on all of these, this is the easiest approach for my Solaris clients. My server will be syslog-NG, and I was wondering if it is possible, without another app, i.e. stunnel, to secure communications between syslog client and my syslog-NG server; i.e. TLS or SSL? Otherwise, I would have to compile syslog-NG for several platforms, x86, Sparc, three different Linux. My timeframe is quite short. Thank you all!

1 0

syslog-ng 3.6.2 using stomp
by Ole Jørgensen 30 Jan '15

30 Jan '15

Hi, I'm testing the stomp destination using syslog-ng ose 3.6.2, and as the process starts it logs: # /opt/syslog-ng-3.6.2/sbin/syslog-ng -f /opt/syslog-ng-3.6.2/etc/syslog-ng.conf -F -d -v -t 2>&1 | grep -i stomp [2015-01-29T18:29:45.323518] Reading shared object for a candidate module; path='/opt/syslog-ng-3.6.2/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2015-01-29T18:29:45.323629] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2015-01-29T18:29:45.343833] Module loaded and initialized successfully; module='afstomp' [2015-01-29T18:29:45.344725] Compiling d_stomp reference [destination] at [/opt/syslog-ng-3.6.2/etc/syslog-ng.conf:70:2] [2015-01-29T18:29:45.344728] Compiling d_stomp sequence [destination] at [/opt/syslog-ng-3.6.2/etc/syslog-ng.conf:32:1] [2015-01-29T18:29:45.345092] Initializing STOMP destination; host='localhost', port='61613', destination='/topic/syslog' [2015-01-29T18:29:45.345368] Worker thread started; driver='d_stomp#0' [2015-01-29T18:29:45.346854] Stomp connection failed; host='localhost' Relevant parts of my configuration: destination d_stomp { stomp( host("localhost") port(61613) body("testing") ack(yes) retries(9) value-pairs(scope(everything)) ); }; The strange thing is that when I'm starting syslog-ng I am running tcpdump -i lo -s0 -nn -v port 61613, and cannot see any packets. I would expect at least a syn connection since it claims to fail. SELinux does not log any denies during the startup, so I suspect the problem to be elsewhere. If i do a tcpdump -i lo -s0 -nn -v -X and start syslog-ng this is captured: 18:33:26.833330 IP (tos 0x0, ttl 64, id 5652, offset 0, flags [DF], proto TCP (6), length 60) 127.0.0.1.35340 > 127.0.0.1.0: Flags [S], cksum 0xd601 (correct), seq 801587243, win 65495, options [mss 65495,sackOK,TS val 700601919 ecr 0,nop,wscale 7], length 0 0x0000: 4500 003c 1614 4000 4006 26a6 7f00 0001 E..<..@.@.&..... 0x0010: 7f00 0001 8a0c 0000 2fc7 402b 0000 0000 ......../.@+.... 0x0020: a002 ffd7 d601 0000 0204 ffd7 0402 080a ................ 0x0030: 29c2 563f 0000 0000 0103 0307 ).V?........ 18:33:26.833412 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) 127.0.0.1.0 > 127.0.0.1.35340: Flags [R.], cksum 0xb7ce (correct), seq 0, ack 801587244, win 0, length 0 0x0000: 4500 0028 0000 4000 4006 3cce 7f00 0001 E..(..@.@.<..... 0x0010: 7f00 0001 0000 8a0c 0000 0000 2fc7 402c ............/.@, 0x0020: 5014 0000 b7ce 0000 P....... So we get a syn to port 0 and answer with reset. Could there be a port parsing error for stomp in syslog-ng OSE 3.6.2? Any suggestions? Also: syslog-ng spawns another thread to do the stomp connection, any suggestions on how I can strace this thread during startup to see what is actually going on? Would love to know if in fact this connection is made to port 0. Thanks, Ole

4 4

Change JSON Format
by Daniel Neubacher 29 Jan '15

29 Jan '15

Hey there, right now I'm playing around with different json shippers for log files and I'm a bit lost with syslog-ng. I've read the docs but I still don't know how to change the json Fields syslog-ng sends out. In order to get my new Installation approved I have to keep the old field names in mind, for example syslog sends out HOST_FROM but I need source_host. Right now logstash/mutate does the renaming but I don't like to waste performance there. My destination: destination d_logstash_syslog_syslog_new {tcp("consumer.foo.bar" port(6002) template("$(format-json --scope selected_macros --scope nv_pairs)\n") );}; Thanks in advance Daniel

3 3

t-shirt for code: FOSDEM, devconf.cz, scale
by Czanik, Péter 28 Jan '15

28 Jan '15

Hi, Some of you might have already read my blog at https://czanik.blogs.balabit.com/2015/01/syslog-ng-at-fosdem-t-shirt-for-co… I'll be at FOSDEM ( http://fosdem.org ) this weekend and I'd be happy to meet you there and talk about syslog-ng. And if you also happen to send a pull request related to syslog-ng using one of the language bindings (perl/python/lua/java) or in C, I'm also happy to provide you with a syslog-ng t-shirt! If you have a good idea what to code, but can't make it to FOSDEM: don't worry. I'll be in Brno for devconf.cz next week and at Scale in Los Angeles by the end of February. And in a worst case scenario (I'd really prefer to talk to you in person) we can also post the t-shirt for you. The deadline is the 1st of March! See you at one of the conferences! Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

deadlock when logging to SQL destination (PostgreSQL)
by Tomáš Novosad 28 Jan '15

28 Jan '15

Hello to everyone, i'd like to as if someone else experienced such thing before. I have a setup where i'm storing maillog and mailfilter log into PostgreSQL database via syslog-ng. Togethe with that, PostgreSQL is logging via syslog-ng as well. This setup creates some sort of deadlock: after some time of processing (varies) syslog-ng freezes, thus completely blocking the server. It is not possible to do anything on the server as all calls to syslog just blocks. I'm not able to open new SSH session, as login is logged via syslog. Emails do not work as senmail is logging via syslog. These and all other applications just hung, whenever they try to write something into syslog. This deadlock is caused by rather "circular" setup, when syslog writes into Postgres and Postgres logs into syslog. I'm able to reproduce the problem in my testing environment. When Postgres does not log into syslog (only CSV logs are enabled, everything goes just fine. Same thing if syslog-ng stores nessages in Postgres on different host - everything is OK. But in scenario when postgres is on localhost and logs into syslog, i'am experiencing those deadlocks. Does anyone has similar experience? We're regulary using syslog-ng 3.4.6, but i have juste tested it on latest version 3.4.8 - same problem appeared. Postgres is of version 9.2. We're using our own CentOS-based distro (currently CentOS 6). I can provide configs, or reproduce the problem if needed. Thanks for any help or hints. -- Tomáš Novosad LinuxBox.cz, s.r.o. 28. října 168, 709 00 Ostrava tel.: +420 591 166 221 mobil: +420 737 238 655 email: tomas.novosad(a)linuxbox.cz jabber: novosad(a)linuxbox.cz www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis(a)linuxbox.cz

3 7

Confusion on number of filters
by James Lay 20 Jan '15

20 Jan '15

Hey all, Here's what I have...one source: source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514)); file("/proc/kmsg"); }; three destinations: destination d_file { file("/var/log/messages"); }; destination d_syslogserver { udp ("x.x.x.x", port(514)); }; destination d_logontype { udp ("x.x.x.x", port(515)); }; and a few filters: filter f_syslogfilter { message("0x0004") }; filter f_firewall { program("firewall") }; filter f_logontype { message("Logon") }; With the below log entry running with -F -d, I will only get the first 2 of the three filters evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); filter(f_firewall); destination(d_syslogserver); filter(f_logontype); destination(d_logontype); }; but after splitting them out I see all three evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); }; log { source(s_local); filter(f_firewall); destination(d_syslogserver); }; log { source(s_local); filter(f_logontype10); destination(d_logontype10); }; Is there a reason or limit or documentation for this? I can verify that the log entry passed for "Logon" in my testing didn't match the previous two filters. Version is 3.3.11. Thank you. James

3 5

kafka destination
by Czanik, Péter 08 Jan '15

08 Jan '15

Hi, The syslog-ng incubator supports kafka ( http://kafka.apache.org/ ) destination, but it was not enabled in rpm builds due to missing dependencies. In the past few days I packaged librdkafka for rpm (actually fixed the rpm from the upstream developer). What it means, that my syslog-ng incubator packages for openSUSE/SLES12/fedora/RHEL7 now also support the kafka destination. And here I'd like to ask for some help. I plan to test kafka, but for that I'd need some use cases. Why do you use kafka? How do use kafka together with syslog-ng? As usual, openSUSE/SLES12 packages are available at https://build.opensuse.org/project/show/home:czanik:syslog-ng36 and Fedora/RHEL7 packages are available at https://copr.fedoraproject.org/coprs/czanik/syslog-ng36/ If you still use RHEL6 and would like to use kafka, let me know and I'll try to extend also my RHEL6 incubator package with kafka support. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

incubator for RHEL6
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, The syslog-ng incubator is now packaged for RHEL6. Read the details at https://czanik.blogs.balabit.com/2015/01/incubator-packaged-for-rhel6-with-… Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng newsletter in HTML
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, Once upon a time I got many feedback, that the syslog-ng newsletter should be plain text, without any HTML. Insider stayed so ever since. If you would like to read a nice formatted HTML version, it's available at https://insider.blogs.balabit.com/ The last two newsletters were published there this morning. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Reason why syslog-ng service dies frequently
by Nguyen Trung Hieu 05 Jan '15

05 Jan '15

Dear team I have instaled syslog-ng version 3.5.4.1 on Centos 6.5 32 bit for 4 months, recently it dies very frequently, after I restart service it only work for some hours then die again. I have checked but still don't know why. Can anyone help me on this. Thank you. Below is log from my Centos box, I also attached my syslog-ng config file. [root@centos65-x86 ~]# syslog-ng -V syslog-ng 3.5.4.1 Installer-Version: 3.5.4.1 Revision: ssh+git://algernon@git.balabit /var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#4090ee62163780ae68a0c83cfdc23998c904fe97 Compile-Date: Aug 6 2014 11:49:42 Available-Modules: syslogformat,linux-kmsg-format,affile,csvparser,confgen,afamqp,system-source,afsql,dbparser,afstomp,afsocket,afsocket-notls,afmongodb,cryptofuncs,afprog,afuser,basicfuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off [root@centos65-x86 ~]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [root@centos65-x86 ~]# service syslog-ng status syslog-ng dead but subsys locked [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ll /var/lock/subsys/syslog-ng -rw------- 1 root root 0 Dec 31 15:06 /var/lock/subsys/syslog-ng [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ps -ef | grep syslog-ng root 15347 15312 0 09:23 pts/0 00:00:00 grep syslog-ng [root@centos65-x86 ~]# date Thu Jan 1 09:23:49 ICT 2015 [root@centos65-x86 ~]# tail -1 /var/log/messages Dec 31 18:06:47 centos65-x86 syslog-ng[5457]: Log statistics; processed='src.internal(s_local#2)=18', stamp='src.internal(s_local#2)=1420023407', processed='source(s_network)=0', processed='source(s_local)=18', dropped='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=76155', stored='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=91', processed='src.none()=0', stamp='src.none()=0', processed='source(s_snmptrapd)=0', processed='global(payload_reallocs)=20575', processed='global(msg_clones)=0', processed='destination(d_sql_snmp_temp_table)=7697', processed='destination(d_sql_syslog_temp_table)=1420079', dropped='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', stored='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', processed='center(queued)=1435491', processed='destination(d_local)=18', processed='global(sdata_updates)=55857', processed='center(received)=18', processed='destination(d_snmptrap)=7697' [root@centos65-x86 ~]# sestatus SELinux status: disabled [root@centos65-x86 ~]# cat /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted [root@centos65-x86 ~]# service iptables status iptables: Firewall is not running. -- Best regards! ------ Nguyen Trung Hieu Mobile: +84904031032

3 4