- syslog-ng - lists.balabit.hu

PatternDB for a10?
by Daniel Neubacher 20 Feb '15

20 Feb '15

Hey there, I've really become a fan of patterndb and I'm migrating almost all of my grok filters to it. My next log devices to migrate are our a10 ax loadbalancer so I wanted to ask if anyone done this work before me :P Daniel -- Daniel Neubacher, Senior Network Engineer daniel.neubacher(a)xing.com<mailto:daniel.neubacher@xing.com> XING AG Dammtorstraße 30, 20354 Hamburg, Germany Tel. +49 40 419131-28, Fax +49 40 419131-11 Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807 Exec. Board (Vorstand): Dr. Thomas Vollmoeller (Vorsitzender), Ingo Chu, Jens Pape, Timm Richter Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Stefan Winners This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.

2 2

Setting log destination using the MAC address
by Tech Support 16 Feb '15

16 Feb '15

All; I am using syslog-ng to log both the local server and remote clients. Right now I log the clients like so and it works the way it should: destination d_remotehosts { file( "/var/log/HOSTS/$YEAR/$MONTH/$DAY/$HOST/$FACILITY.log" perm( 0644 ) create_dirs( yes ) dir_perm( 0755 ) ); }; However, it is not as helpful when multiple remote clients get recognized as $HOST = 192.168.x.x. Since most remote clients are IP phones, logging using the MAC address instead of the IP address would be much more valuable, but I don't see in the manual where that's possible. Is it possible to log using the MAC address? Any insight at all would be greatly appreciated. Thanks; John Tech Support Tech Support VoIP Business Solutions 240-215-3479 x325 <mailto:fsd@voipbusiness.us> support(a)voipbusiness.us

2 1

Send Apache log to a separate loghost
by Manning, Lee 13 Feb '15

13 Feb '15

In version 1.6.11 of syslog-ng, I need to send Apache logs to a different loghost than the rest of the system logs. Is that possible? If so, how? Our syslog-ng was set up quite some time ago by an admin who is no longer around. I've spent more time than I can afford trying to decipher and attempt to modify his syslog-ng.conf. Assistance from some experts would be greatly appreciated. Thanks, Lee

3 2

syslog-ng at Scale
by Czanik, Péter 10 Feb '15

10 Feb '15

Hi, Two of us from the syslog-ng team will be at Scale this year: http://www.socallinuxexpo.org/scale/13x and I'll also give a presentation about syslog-ng: https://www.socallinuxexpo.org/scale/13x/presentations/syslog-ng-log-collec… To receive a code for a 40% discount at Scale and a free syslog-ng t-shirt at our booth, please register at https://pages.balabit.com/SCALE_discount.html Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

time jumps in logs
by Czanik, Péter 06 Feb '15

06 Feb '15

Hi, I was pointed to a Gentoo syslog-ng bugreport, which shows some interesting time travel: https://bugs.gentoo.org/show_bug.cgi?id=533328 My guess is, that it's something virtualisation related, but even if so, the report says that downgrading to 3.4 solved the problem. Anybody ran into something similar? Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Securing Communications Between syslog-NG Server and syslog Clients
by wiskbroom＠hotmail.com 05 Feb '15

05 Feb '15

Hello; I have several thousand Linux and Solaris clients, some using syslog, some using rsyslog. I intend to use stock syslog client on all of these, this is the easiest approach for my Solaris clients. My server will be syslog-NG, and I was wondering if it is possible, without another app, i.e. stunnel, to secure communications between syslog client and my syslog-NG server; i.e. TLS or SSL? Otherwise, I would have to compile syslog-NG for several platforms, x86, Sparc, three different Linux. My timeframe is quite short. Thank you all!

1 0

syslog-ng 3.6.2 using stomp
by Ole Jørgensen 30 Jan '15

30 Jan '15

Hi, I'm testing the stomp destination using syslog-ng ose 3.6.2, and as the process starts it logs: # /opt/syslog-ng-3.6.2/sbin/syslog-ng -f /opt/syslog-ng-3.6.2/etc/syslog-ng.conf -F -d -v -t 2>&1 | grep -i stomp [2015-01-29T18:29:45.323518] Reading shared object for a candidate module; path='/opt/syslog-ng-3.6.2/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2015-01-29T18:29:45.323629] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2015-01-29T18:29:45.343833] Module loaded and initialized successfully; module='afstomp' [2015-01-29T18:29:45.344725] Compiling d_stomp reference [destination] at [/opt/syslog-ng-3.6.2/etc/syslog-ng.conf:70:2] [2015-01-29T18:29:45.344728] Compiling d_stomp sequence [destination] at [/opt/syslog-ng-3.6.2/etc/syslog-ng.conf:32:1] [2015-01-29T18:29:45.345092] Initializing STOMP destination; host='localhost', port='61613', destination='/topic/syslog' [2015-01-29T18:29:45.345368] Worker thread started; driver='d_stomp#0' [2015-01-29T18:29:45.346854] Stomp connection failed; host='localhost' Relevant parts of my configuration: destination d_stomp { stomp( host("localhost") port(61613) body("testing") ack(yes) retries(9) value-pairs(scope(everything)) ); }; The strange thing is that when I'm starting syslog-ng I am running tcpdump -i lo -s0 -nn -v port 61613, and cannot see any packets. I would expect at least a syn connection since it claims to fail. SELinux does not log any denies during the startup, so I suspect the problem to be elsewhere. If i do a tcpdump -i lo -s0 -nn -v -X and start syslog-ng this is captured: 18:33:26.833330 IP (tos 0x0, ttl 64, id 5652, offset 0, flags [DF], proto TCP (6), length 60) 127.0.0.1.35340 > 127.0.0.1.0: Flags [S], cksum 0xd601 (correct), seq 801587243, win 65495, options [mss 65495,sackOK,TS val 700601919 ecr 0,nop,wscale 7], length 0 0x0000: 4500 003c 1614 4000 4006 26a6 7f00 0001 E..<..@.@.&..... 0x0010: 7f00 0001 8a0c 0000 2fc7 402b 0000 0000 ......../.@+.... 0x0020: a002 ffd7 d601 0000 0204 ffd7 0402 080a ................ 0x0030: 29c2 563f 0000 0000 0103 0307 ).V?........ 18:33:26.833412 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) 127.0.0.1.0 > 127.0.0.1.35340: Flags [R.], cksum 0xb7ce (correct), seq 0, ack 801587244, win 0, length 0 0x0000: 4500 0028 0000 4000 4006 3cce 7f00 0001 E..(..@.@.<..... 0x0010: 7f00 0001 0000 8a0c 0000 0000 2fc7 402c ............/.@, 0x0020: 5014 0000 b7ce 0000 P....... So we get a syn to port 0 and answer with reset. Could there be a port parsing error for stomp in syslog-ng OSE 3.6.2? Any suggestions? Also: syslog-ng spawns another thread to do the stomp connection, any suggestions on how I can strace this thread during startup to see what is actually going on? Would love to know if in fact this connection is made to port 0. Thanks, Ole

4 4

Change JSON Format
by Daniel Neubacher 29 Jan '15

29 Jan '15

Hey there, right now I'm playing around with different json shippers for log files and I'm a bit lost with syslog-ng. I've read the docs but I still don't know how to change the json Fields syslog-ng sends out. In order to get my new Installation approved I have to keep the old field names in mind, for example syslog sends out HOST_FROM but I need source_host. Right now logstash/mutate does the renaming but I don't like to waste performance there. My destination: destination d_logstash_syslog_syslog_new {tcp("consumer.foo.bar" port(6002) template("$(format-json --scope selected_macros --scope nv_pairs)\n") );}; Thanks in advance Daniel

3 3

t-shirt for code: FOSDEM, devconf.cz, scale
by Czanik, Péter 28 Jan '15

28 Jan '15

Hi, Some of you might have already read my blog at https://czanik.blogs.balabit.com/2015/01/syslog-ng-at-fosdem-t-shirt-for-co… I'll be at FOSDEM ( http://fosdem.org ) this weekend and I'd be happy to meet you there and talk about syslog-ng. And if you also happen to send a pull request related to syslog-ng using one of the language bindings (perl/python/lua/java) or in C, I'm also happy to provide you with a syslog-ng t-shirt! If you have a good idea what to code, but can't make it to FOSDEM: don't worry. I'll be in Brno for devconf.cz next week and at Scale in Los Angeles by the end of February. And in a worst case scenario (I'd really prefer to talk to you in person) we can also post the t-shirt for you. The deadline is the 1st of March! See you at one of the conferences! Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

deadlock when logging to SQL destination (PostgreSQL)
by Tomáš Novosad 28 Jan '15

28 Jan '15

Hello to everyone, i'd like to as if someone else experienced such thing before. I have a setup where i'm storing maillog and mailfilter log into PostgreSQL database via syslog-ng. Togethe with that, PostgreSQL is logging via syslog-ng as well. This setup creates some sort of deadlock: after some time of processing (varies) syslog-ng freezes, thus completely blocking the server. It is not possible to do anything on the server as all calls to syslog just blocks. I'm not able to open new SSH session, as login is logged via syslog. Emails do not work as senmail is logging via syslog. These and all other applications just hung, whenever they try to write something into syslog. This deadlock is caused by rather "circular" setup, when syslog writes into Postgres and Postgres logs into syslog. I'm able to reproduce the problem in my testing environment. When Postgres does not log into syslog (only CSV logs are enabled, everything goes just fine. Same thing if syslog-ng stores nessages in Postgres on different host - everything is OK. But in scenario when postgres is on localhost and logs into syslog, i'am experiencing those deadlocks. Does anyone has similar experience? We're regulary using syslog-ng 3.4.6, but i have juste tested it on latest version 3.4.8 - same problem appeared. Postgres is of version 9.2. We're using our own CentOS-based distro (currently CentOS 6). I can provide configs, or reproduce the problem if needed. Thanks for any help or hints. -- Tomáš Novosad LinuxBox.cz, s.r.o. 28. října 168, 709 00 Ostrava tel.: +420 591 166 221 mobil: +420 737 238 655 email: tomas.novosad(a)linuxbox.cz jabber: novosad(a)linuxbox.cz www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis(a)linuxbox.cz

3 7

Confusion on number of filters
by James Lay 20 Jan '15

20 Jan '15

Hey all, Here's what I have...one source: source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514)); file("/proc/kmsg"); }; three destinations: destination d_file { file("/var/log/messages"); }; destination d_syslogserver { udp ("x.x.x.x", port(514)); }; destination d_logontype { udp ("x.x.x.x", port(515)); }; and a few filters: filter f_syslogfilter { message("0x0004") }; filter f_firewall { program("firewall") }; filter f_logontype { message("Logon") }; With the below log entry running with -F -d, I will only get the first 2 of the three filters evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); filter(f_firewall); destination(d_syslogserver); filter(f_logontype); destination(d_logontype); }; but after splitting them out I see all three evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); }; log { source(s_local); filter(f_firewall); destination(d_syslogserver); }; log { source(s_local); filter(f_logontype10); destination(d_logontype10); }; Is there a reason or limit or documentation for this? I can verify that the log entry passed for "Logon" in my testing didn't match the previous two filters. Version is 3.3.11. Thank you. James

3 5

kafka destination
by Czanik, Péter 08 Jan '15

08 Jan '15

Hi, The syslog-ng incubator supports kafka ( http://kafka.apache.org/ ) destination, but it was not enabled in rpm builds due to missing dependencies. In the past few days I packaged librdkafka for rpm (actually fixed the rpm from the upstream developer). What it means, that my syslog-ng incubator packages for openSUSE/SLES12/fedora/RHEL7 now also support the kafka destination. And here I'd like to ask for some help. I plan to test kafka, but for that I'd need some use cases. Why do you use kafka? How do use kafka together with syslog-ng? As usual, openSUSE/SLES12 packages are available at https://build.opensuse.org/project/show/home:czanik:syslog-ng36 and Fedora/RHEL7 packages are available at https://copr.fedoraproject.org/coprs/czanik/syslog-ng36/ If you still use RHEL6 and would like to use kafka, let me know and I'll try to extend also my RHEL6 incubator package with kafka support. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

incubator for RHEL6
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, The syslog-ng incubator is now packaged for RHEL6. Read the details at https://czanik.blogs.balabit.com/2015/01/incubator-packaged-for-rhel6-with-… Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng newsletter in HTML
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, Once upon a time I got many feedback, that the syslog-ng newsletter should be plain text, without any HTML. Insider stayed so ever since. If you would like to read a nice formatted HTML version, it's available at https://insider.blogs.balabit.com/ The last two newsletters were published there this morning. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Reason why syslog-ng service dies frequently
by Nguyen Trung Hieu 05 Jan '15

05 Jan '15

Dear team I have instaled syslog-ng version 3.5.4.1 on Centos 6.5 32 bit for 4 months, recently it dies very frequently, after I restart service it only work for some hours then die again. I have checked but still don't know why. Can anyone help me on this. Thank you. Below is log from my Centos box, I also attached my syslog-ng config file. [root@centos65-x86 ~]# syslog-ng -V syslog-ng 3.5.4.1 Installer-Version: 3.5.4.1 Revision: ssh+git://algernon@git.balabit /var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#4090ee62163780ae68a0c83cfdc23998c904fe97 Compile-Date: Aug 6 2014 11:49:42 Available-Modules: syslogformat,linux-kmsg-format,affile,csvparser,confgen,afamqp,system-source,afsql,dbparser,afstomp,afsocket,afsocket-notls,afmongodb,cryptofuncs,afprog,afuser,basicfuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off [root@centos65-x86 ~]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [root@centos65-x86 ~]# service syslog-ng status syslog-ng dead but subsys locked [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ll /var/lock/subsys/syslog-ng -rw------- 1 root root 0 Dec 31 15:06 /var/lock/subsys/syslog-ng [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ps -ef | grep syslog-ng root 15347 15312 0 09:23 pts/0 00:00:00 grep syslog-ng [root@centos65-x86 ~]# date Thu Jan 1 09:23:49 ICT 2015 [root@centos65-x86 ~]# tail -1 /var/log/messages Dec 31 18:06:47 centos65-x86 syslog-ng[5457]: Log statistics; processed='src.internal(s_local#2)=18', stamp='src.internal(s_local#2)=1420023407', processed='source(s_network)=0', processed='source(s_local)=18', dropped='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=76155', stored='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=91', processed='src.none()=0', stamp='src.none()=0', processed='source(s_snmptrapd)=0', processed='global(payload_reallocs)=20575', processed='global(msg_clones)=0', processed='destination(d_sql_snmp_temp_table)=7697', processed='destination(d_sql_syslog_temp_table)=1420079', dropped='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', stored='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', processed='center(queued)=1435491', processed='destination(d_local)=18', processed='global(sdata_updates)=55857', processed='center(received)=18', processed='destination(d_snmptrap)=7697' [root@centos65-x86 ~]# sestatus SELinux status: disabled [root@centos65-x86 ~]# cat /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted [root@centos65-x86 ~]# service iptables status iptables: Firewall is not running. -- Best regards! ------ Nguyen Trung Hieu Mobile: +84904031032

3 4

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 04 Jan '15

04 Jan '15

Cool. Just read this. Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 10:07 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions SOLVED!! I was editing " /etc/syslog-ng/syslog-ng.conf", while my compiled version was using: " /usr/local/etc/syslog-ng.conf" Apparently, I had two packages installed, one from yum and another one compiled manually. When I made changes to "/usr/local/etc/syslog-ng.conf" files were created with the right permissions. Andrey Smetanin | Technical Consultant | Commerx Computer Systems Inc. 2880 Argentia Road, Unit 1, Mississauga, ON L5N 7X8 O: 905.542.9400 x201 | M: 647.894.6096 | F: 905.542.9410 andrey.smetanin(a)commerx.ca YOUR I.T. SOLUTIONS PARTNER -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Andrey Smetanin Sent: December-30-14 9:51 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/lo

4 3

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 30 Dec '14

30 Dec '14

An odd thought related to the file/file-system permissions. Is this a local drive or network mounted? I've seen problems related to underlying ownership or (lack of) permissions in other scenarios. Also, try modifying the configuration file until you can affect a change. Try a different directory or file system, try a different user or group, if possible try the same configuration file on another system. Once you notice a difference, it should help you understand the situation. Curious, Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 9:51 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='pseudofile.so', module='pseudofile' [2014-12-30T09:34:38.110205] Registering candidate plugin; module='pseudofile', context='destination', name='pseudofile', preference='0' [2014-12-30T09:34:38.110262] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='graphite.so', module='graphite' [2014-12-30T09:34:38.110419] Registering candidate plugin; module='graphite', context='template-func', name='graphite_output', preference='0' [2014-12-30T09:34:38.110517] Finishing include; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.110585] Starting to read include file; filename='/usr/local/share/include/scl/syslogconf/plugin.conf', depth='2' [2014-12-30T09:34:38.110951] Module loaded and initialized successfully; module='confgen' [2014-12-30T09:34:38.110967] Finishing include

1 0

syslog-ng always using default permissions
by Andrey Smetanin 30 Dec '14

30 Dec '14

Here is the part of my configuration file: ... destination d_network { file("/LOGS/fromnetwork" owner('logstash') group('logstash') perm(0660)); }; ... File "/LOGS/fromnetwork" is created with the proper user and group ("logstash"), but permissions are always "0600", regardless whatever I am trying. I don't see any errors while running "syslog-ng -vFe" What else I can troubleshoot? Andrey Smetanin -- This message has been scanned by Commerx Computer Systems Inc. and is believed to be clean.

2 3

I want to forward Windows server 2008 R2 logs to Linux syslog-ng.
by Jason Long 24 Dec '14

24 Dec '14

Hello all. I have a Windows Box that want to forward all Even logs to my Linux box. I install Snare on Windows(172.30.10.19) and configure it to forward logs to Linux and my Linux box receive it properly. When I use " tcpdump udp "port 514" ", Tcpdump show me that Snare sending Logs to Linux but Syslog-ng can't write it to log files :(. I paste my syslog-ng configure : options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; #source s_net { #udp (ip(172.30.10.19) port(514)); #}; source s_net { udp(); }; filter f_openwrt { host("172.30.10.19");}; destination df_openwrt { file("/var/log/winlog/win.log"); }; log { source ( s_net ); filter( f_openwrt ); destination ( df_openwrt ); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: Can you tell me how can I solve this problem? Cheers.

5 8

Re: [syslog-ng] syslog-ng memory usage grows
by Jim Hendrick 24 Dec '14

24 Dec '14

I will certainly try. Send me a link, and Thanks! Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Tibor Benke <ihrwein(a)gmail.com> Date:12/17/2014 5:42 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Cc: wernli(a)in2p3.fr, jrhendri(a)roadrunner.com Subject: Re: [syslog-ng] syslog-ng memory usage grows Hi All, I checked your valgrind log and the redis module and I think the leak is in redis.c:137: redisCommand(self->c, "ping"); This returns a reply object, but it's ignored and never freed. If we make a patch on GitHub, could you test it? Regards, Tibor 2014-12-17 18:36 GMT+01:00 <jrhendri(a)roadrunner.com>: Hi Fabien, Hope the season is treating you well. I ran syslog-ng for 10 minutes (recall, ~7000 EPS incoming) under valgrind and here is the output (attached). It looks like there is some sort of leak as the process from yesterday had grown to over 16GB before I stopped it for the valgrind test. Thanks for the help (as always). And please let me know if I can provide any other data. Jim ---- Fabien Wernli <wernli(a)in2p3.fr> wrote: > Hi Jim, > > On Tue, Dec 16, 2014 at 01:24:28PM -0500, Jim Hendrick wrote: > > I'm not sure what I am asking, other than general advice on: > > - performance using patterndb > > performance is awesome, I wouldn't worry about it > although it would help to have some counters on the parsers in the stats > interface. That bieng said, it could help you to increase the stats level. > > > - performance using redis destination > > I can't comment on that one, sorry > > > - advice on debugging where this memory growth is happening > > Could you run syslog-ng inside `valgrind --tool=memcheck > --trace-children=yes --leak-check=yes`? > > The problem is this will really slow down your instance, but let it run a > few minutes nevertheless and pastebin the result somewhere. > > > As a rough measure - I have a syslog-ng process that has been running for > > less than 3 hours and right now is using 1.52GB of resident memory (shown by > > "top") > > I'm suspecting `format-json` to be the source of the leak, or the flow > control cache. These have been the leak source of our own observations many > times, although many patches have solved the issues. > > > (I would rather parse things in syslog-ng, but I *could* do this all using > > logstash/grok if this proves too much for patterndb at this load) > > if you're worried about performance, I wouldn't even consider to begin thinking about comparing > LS/grok to patterndb > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

3 3

insider 2014-12: incubator 0.4.1 released; anonymization; monitoring;
by Czanik, Péter 18 Dec '14

18 Dec '14

Dear syslog-ng users, This is the 39th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. FEATURED NEWS syslog-ng incubator 0.4.1 released ------------------------ We are proud to announce that version 0.4.1 of syslog-ng incubator is released. It is compatible with syslog-ng version 3.6. Some features previously included in the incubator such as Riemann and Graphite support have been ported to syslog-ng. It also adds many interesting new features: kafka and java destination support, ZMQ source and destination and a grok parser. Ports and packages are available for Fedora, FreeBSD and openSUSE, but with some of the features missing, as dependencies are not yet available packaged. For details, read https://czanik.blogs.balabit.com/2014/12/syslog-ng-incubator-0-4-1-released/ Data Privacy, Anonymization and Log Data ---------------------------------------- Data privacy is becoming a hot topic, especially in the European Union. New features in syslog-ng can be used to become compliant with regulations. Read about encrypting, anonymizing logs and pseudonymization at https://jluby.blogs.balabit.com/2014/12/08/data-privacy-anonymization-and-l… The trinity of monitoring: syslog-ng, Riemann and Elasticsearch -------------------------------------------------------------- BalaBit organizes OpenAcademy in Budapest, Hungary, as a way of continuous learning. Last time we had a guest, Fabien Wernli (on IRC better known as faxmodem), who talked about how syslog-ng can be integrated with Riemann and Elasticsearch. The video starts in Hungarian, but the presentation is in English: https://www.youtube.com/watch?v=Y4uzJm5VLh4 NEW RELEASES syslog-ng 3.6.2: https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.6.2 syslog-ng incubator 0.4.0 & 0.4.1: https://github.com/balabit/syslog-ng-incubator/releases Happy Holidays! Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com To read this newsletter on-line, visit: http://insider.blogs.balabit.com/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng memory usage grows
by Jim Hendrick 17 Dec '14

17 Dec '14

Hi, I am continuing to test syslog-ng as the parser and shipper into an elasticsearch cluster. Right now I have syslog-ng 3.6.1 receiving logs at about ~7000 EPS and running them through a patterndb parser that splits out the (bluecoat proxy) fields into key-value pairs, writing them to a redis destination using format-json. syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 9 2014 19:42:20 Available-Modules: dbparser,json-plugin,afuser,affile,afmongodb,tfgeoip,afprog,redis,afstomp,afsql,afsocket,system-source,afamqp,pseudofile,confgen,afsocket-tls,csvparser,basicfuncs,graphite,syslogformat,afsocket-notls,linux-kmsg-format,cryptofuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off I then use logstash to pull from redis and feed elasticsearch (the thought being that this would provide a buffer for the messages) Over the weekend I had syslog-ng crash (unfortunately no core) but now that I am watching it more closely, I see what appears to be continuous growth in memory usage (which leads me to suppose this was the cause of the crash). I'm not sure what I am asking, other than general advice on: - performance using patterndb - performance using redis destination - advice on debugging where this memory growth is happening As a rough measure - I have a syslog-ng process that has been running for less than 3 hours and right now is using 1.52GB of resident memory (shown by "top") I am including two config files. One has a single source, no filtering, single destination. The other splits logs from three upstream syslog-ng servers thinking this would provide "threading" with multiple patterndb parsers (I have read that patterndb is single threaded). Any thoughts on how to figure out where the memory is going? Or recommendations on whether or not using patterndb in this way is a particularly dumb idea :-) (I would rather parse things in syslog-ng, but I *could* do this all using logstash/grok if this proves too much for patterndb at this load) Thanks! Jim

4 3

Email alerts en syslog-ng 3.3.5
by Roberto Carna 17 Dec '14

17 Dec '14

Dear, I have this syslog platform: - Debian 7 - Syslog-ng 3.3.5.-4 (installed via apt-get) - Php-syslog-ng (graphical interface) - Postfix I need to configure syslog-ng in order to send email alerts/notifications about some specific events. Is it possible to implement with this syslog-ng version and what is the way??? Thanks a lot!!! Roberto

2 4

RFC: debugger
by Scheidler, Balázs 17 Dec '14

17 Dec '14

Hi, I was involved in a number of projects where syslog-ng was used in a relatively complex setup: we wanted to structure incoming log messages and transform them into nice & clean JSON messages. In a number of cases, this required several parsers to get right (csv-parser, db-parser and sometimes regexps) and the resulting configuration was pretty complicated. I've figured, that if we have a domain specific language in the form of the syslog-ng configuration file, we should also have a debugger that makes it easy to find out what went wrong. And this is what I have did, some proof-of-concept code is available on the f/debugger branch on github: https://github.com/balabit/syslog-ng/tree/f/debugger Here's how it works: # start up syslog-ng with the -i option to indicate we # want the debugger enabled $ sbin/syslog-ng -iFe Once syslog-ng starts up, you'll see something like this: Waiting for breakpoint... For now, all processing elements within syslog-ng (called LogPipe) breaks into the debugger. Later, I want to implement real breakpoints, where you can selectively attach breakpoints to elements in the configuration file. # on another console send a log message to syslog-ng: $ logger almafa This is what happens on the syslog-ng side: Breakpoint hit etc/syslog-ng.conf:11:2 11 unix-stream("log"); Dec 15 16:14:05 bazsi: almafa (syslog-ng) At a breakpoint, syslog-ng displays where the break has occurred, quotes the source line and displays the message that is being processed. Now it becomes interactive, you can inspect the message with various commands: (syslog-ng) help syslog-ng interactive console, the following commands are available help, h, or ? Display this help continue or c Continue until the next breakpoint print, p Print the current log message drop, d Drop the current message quit, q Tell syslog-ng to exit (syslog-ng) p $MSG almafa (syslog-ng) p MESSAGE=almafa PROGRAM=bazsi LEGACY_MSGHDR=bazsi: .unix.pid=18703 .unix.uid=1000 .unix.gid=1000 TAGS= (syslog-ng) p "$(format-json --key *)" {"_unix":{"uid":"1000","pid":"18703","gid":"1000"}, ... } With the "continue" command, you can request syslog-ng to proceed to the next LogPipe, where it stops again. "drop" drops the current message, "quit" tells syslog-ng to exit. I would really like to extend this to further ways, like: - real breakpoints, instead of stopping everywhere - inject messages - change messages to make configuration testable - single-step (to the next primitive LogPipe) and step-over (to the next 'real' LogPipe that is found in the configuration file) - interactive REPL to template functions to make it easier to interact with them What do you think? -- Bazsi

2 1

Syslog-ng Drops Logs
by guamar＠IMIST.MA 15 Dec '14

15 Dec '14

Hello, i'm IT technical at CNRSt Morocco. I work on a project to centralize logs with syslog-ng , but so some logs are dropped . I have 10 clients and on each client over 12 website is hosted. thank you to find attached the config file. thanks. Cordialement. ---------------------------------------------------------------------------------------------------------------------------------------------- Thami Guamar Service informatique Institut Marocain de l'Information Scientifique et Technique (IMIST)- Centre National pour la Recherche Scientifique et Technique (CNRST) Angle avenue Allal El Fassi, avenue des FAR,Quartier Hay Ryad B.P. 8027 Rabat-Nations Unies 10102 Rabat, Maroc E-mail :guamar@imist.ma Tél : (212) 5 37 56 99 00/36 Fax : 212 5 37 56 99 01

4 3