- syslog-ng - lists.balabit.hu

deadlock when logging to SQL destination (PostgreSQL)
by Tomáš Novosad 28 Jan '15

28 Jan '15

Hello to everyone, i'd like to as if someone else experienced such thing before. I have a setup where i'm storing maillog and mailfilter log into PostgreSQL database via syslog-ng. Togethe with that, PostgreSQL is logging via syslog-ng as well. This setup creates some sort of deadlock: after some time of processing (varies) syslog-ng freezes, thus completely blocking the server. It is not possible to do anything on the server as all calls to syslog just blocks. I'm not able to open new SSH session, as login is logged via syslog. Emails do not work as senmail is logging via syslog. These and all other applications just hung, whenever they try to write something into syslog. This deadlock is caused by rather "circular" setup, when syslog writes into Postgres and Postgres logs into syslog. I'm able to reproduce the problem in my testing environment. When Postgres does not log into syslog (only CSV logs are enabled, everything goes just fine. Same thing if syslog-ng stores nessages in Postgres on different host - everything is OK. But in scenario when postgres is on localhost and logs into syslog, i'am experiencing those deadlocks. Does anyone has similar experience? We're regulary using syslog-ng 3.4.6, but i have juste tested it on latest version 3.4.8 - same problem appeared. Postgres is of version 9.2. We're using our own CentOS-based distro (currently CentOS 6). I can provide configs, or reproduce the problem if needed. Thanks for any help or hints. -- Tomáš Novosad LinuxBox.cz, s.r.o. 28. října 168, 709 00 Ostrava tel.: +420 591 166 221 mobil: +420 737 238 655 email: tomas.novosad(a)linuxbox.cz jabber: novosad(a)linuxbox.cz www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis(a)linuxbox.cz

3 7

Confusion on number of filters
by James Lay 20 Jan '15

20 Jan '15

Hey all, Here's what I have...one source: source s_local { unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514)); file("/proc/kmsg"); }; three destinations: destination d_file { file("/var/log/messages"); }; destination d_syslogserver { udp ("x.x.x.x", port(514)); }; destination d_logontype { udp ("x.x.x.x", port(515)); }; and a few filters: filter f_syslogfilter { message("0x0004") }; filter f_firewall { program("firewall") }; filter f_logontype { message("Logon") }; With the below log entry running with -F -d, I will only get the first 2 of the three filters evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); filter(f_firewall); destination(d_syslogserver); filter(f_logontype); destination(d_logontype); }; but after splitting them out I see all three evaluated: log { source(s_local); filter(f_syslogfilter); destination(d_file); }; log { source(s_local); filter(f_firewall); destination(d_syslogserver); }; log { source(s_local); filter(f_logontype10); destination(d_logontype10); }; Is there a reason or limit or documentation for this? I can verify that the log entry passed for "Logon" in my testing didn't match the previous two filters. Version is 3.3.11. Thank you. James

3 5

kafka destination
by Czanik, Péter 08 Jan '15

08 Jan '15

Hi, The syslog-ng incubator supports kafka ( http://kafka.apache.org/ ) destination, but it was not enabled in rpm builds due to missing dependencies. In the past few days I packaged librdkafka for rpm (actually fixed the rpm from the upstream developer). What it means, that my syslog-ng incubator packages for openSUSE/SLES12/fedora/RHEL7 now also support the kafka destination. And here I'd like to ask for some help. I plan to test kafka, but for that I'd need some use cases. Why do you use kafka? How do use kafka together with syslog-ng? As usual, openSUSE/SLES12 packages are available at https://build.opensuse.org/project/show/home:czanik:syslog-ng36 and Fedora/RHEL7 packages are available at https://copr.fedoraproject.org/coprs/czanik/syslog-ng36/ If you still use RHEL6 and would like to use kafka, let me know and I'll try to extend also my RHEL6 incubator package with kafka support. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

incubator for RHEL6
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, The syslog-ng incubator is now packaged for RHEL6. Read the details at https://czanik.blogs.balabit.com/2015/01/incubator-packaged-for-rhel6-with-… Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng newsletter in HTML
by Czanik, Péter 06 Jan '15

06 Jan '15

Hi, Once upon a time I got many feedback, that the syslog-ng newsletter should be plain text, without any HTML. Insider stayed so ever since. If you would like to read a nice formatted HTML version, it's available at https://insider.blogs.balabit.com/ The last two newsletters were published there this morning. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Reason why syslog-ng service dies frequently
by Nguyen Trung Hieu 05 Jan '15

05 Jan '15

Dear team I have instaled syslog-ng version 3.5.4.1 on Centos 6.5 32 bit for 4 months, recently it dies very frequently, after I restart service it only work for some hours then die again. I have checked but still don't know why. Can anyone help me on this. Thank you. Below is log from my Centos box, I also attached my syslog-ng config file. [root@centos65-x86 ~]# syslog-ng -V syslog-ng 3.5.4.1 Installer-Version: 3.5.4.1 Revision: ssh+git://algernon@git.balabit /var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#4090ee62163780ae68a0c83cfdc23998c904fe97 Compile-Date: Aug 6 2014 11:49:42 Available-Modules: syslogformat,linux-kmsg-format,affile,csvparser,confgen,afamqp,system-source,afsql,dbparser,afstomp,afsocket,afsocket-notls,afmongodb,cryptofuncs,afprog,afuser,basicfuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off [root@centos65-x86 ~]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [root@centos65-x86 ~]# service syslog-ng status syslog-ng dead but subsys locked [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ll /var/lock/subsys/syslog-ng -rw------- 1 root root 0 Dec 31 15:06 /var/lock/subsys/syslog-ng [root@centos65-x86 ~]# [root@centos65-x86 ~]# [root@centos65-x86 ~]# ps -ef | grep syslog-ng root 15347 15312 0 09:23 pts/0 00:00:00 grep syslog-ng [root@centos65-x86 ~]# date Thu Jan 1 09:23:49 ICT 2015 [root@centos65-x86 ~]# tail -1 /var/log/messages Dec 31 18:06:47 centos65-x86 syslog-ng[5457]: Log statistics; processed='src.internal(s_local#2)=18', stamp='src.internal(s_local#2)=1420023407', processed='source(s_network)=0', processed='source(s_local)=18', dropped='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=76155', stored='dst.sql(d_sql_syslog_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOG_TEMP)=91', processed='src.none()=0', stamp='src.none()=0', processed='source(s_snmptrapd)=0', processed='global(payload_reallocs)=20575', processed='global(msg_clones)=0', processed='destination(d_sql_snmp_temp_table)=7697', processed='destination(d_sql_syslog_temp_table)=1420079', dropped='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', stored='dst.sql(d_sql_snmp_temp_table#0,oracle,,,PTUD_SYSLOG,SLG_LOGS_SNMPTRAP_TEMP)=0', processed='center(queued)=1435491', processed='destination(d_local)=18', processed='global(sdata_updates)=55857', processed='center(received)=18', processed='destination(d_snmptrap)=7697' [root@centos65-x86 ~]# sestatus SELinux status: disabled [root@centos65-x86 ~]# cat /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted [root@centos65-x86 ~]# service iptables status iptables: Firewall is not running. -- Best regards! ------ Nguyen Trung Hieu Mobile: +84904031032

3 4

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 04 Jan '15

04 Jan '15

Cool. Just read this. Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 10:07 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions SOLVED!! I was editing " /etc/syslog-ng/syslog-ng.conf", while my compiled version was using: " /usr/local/etc/syslog-ng.conf" Apparently, I had two packages installed, one from yum and another one compiled manually. When I made changes to "/usr/local/etc/syslog-ng.conf" files were created with the right permissions. Andrey Smetanin | Technical Consultant | Commerx Computer Systems Inc. 2880 Argentia Road, Unit 1, Mississauga, ON L5N 7X8 O: 905.542.9400 x201 | M: 647.894.6096 | F: 905.542.9410 andrey.smetanin(a)commerx.ca YOUR I.T. SOLUTIONS PARTNER -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Andrey Smetanin Sent: December-30-14 9:51 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/lo

4 3

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 30 Dec '14

30 Dec '14

An odd thought related to the file/file-system permissions. Is this a local drive or network mounted? I've seen problems related to underlying ownership or (lack of) permissions in other scenarios. Also, try modifying the configuration file until you can affect a change. Try a different directory or file system, try a different user or group, if possible try the same configuration file on another system. Once you notice a difference, it should help you understand the situation. Curious, Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 9:51 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='pseudofile.so', module='pseudofile' [2014-12-30T09:34:38.110205] Registering candidate plugin; module='pseudofile', context='destination', name='pseudofile', preference='0' [2014-12-30T09:34:38.110262] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='graphite.so', module='graphite' [2014-12-30T09:34:38.110419] Registering candidate plugin; module='graphite', context='template-func', name='graphite_output', preference='0' [2014-12-30T09:34:38.110517] Finishing include; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.110585] Starting to read include file; filename='/usr/local/share/include/scl/syslogconf/plugin.conf', depth='2' [2014-12-30T09:34:38.110951] Module loaded and initialized successfully; module='confgen' [2014-12-30T09:34:38.110967] Finishing include

1 0

syslog-ng always using default permissions
by Andrey Smetanin 30 Dec '14

30 Dec '14

Here is the part of my configuration file: ... destination d_network { file("/LOGS/fromnetwork" owner('logstash') group('logstash') perm(0660)); }; ... File "/LOGS/fromnetwork" is created with the proper user and group ("logstash"), but permissions are always "0600", regardless whatever I am trying. I don't see any errors while running "syslog-ng -vFe" What else I can troubleshoot? Andrey Smetanin -- This message has been scanned by Commerx Computer Systems Inc. and is believed to be clean.

2 3

I want to forward Windows server 2008 R2 logs to Linux syslog-ng.
by Jason Long 24 Dec '14

24 Dec '14

Hello all. I have a Windows Box that want to forward all Even logs to my Linux box. I install Snare on Windows(172.30.10.19) and configure it to forward logs to Linux and my Linux box receive it properly. When I use " tcpdump udp "port 514" ", Tcpdump show me that Snare sending Logs to Linux but Syslog-ng can't write it to log files :(. I paste my syslog-ng configure : options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; #source s_net { #udp (ip(172.30.10.19) port(514)); #}; source s_net { udp(); }; filter f_openwrt { host("172.30.10.19");}; destination df_openwrt { file("/var/log/winlog/win.log"); }; log { source ( s_net ); filter( f_openwrt ); destination ( df_openwrt ); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: Can you tell me how can I solve this problem? Cheers.

5 8