December 2014 - syslog-ng - lists.balabit.hu

Create Pattern-DB rules
by Justin Kala 20 Oct '15

20 Oct '15

Hello I am trying to create Pattern-DB for the following Authorization messages coming from O/S. Can you help on creating a matching rule . Note: I do not have PATTERN-DB parser utility.I am going to create the db-parser.xml manually and put this rules inside the file. 2014-09-28T14:12:44-04:00 abcdef01-app/abcdef01-app sshd[11019]: [ID 800047 auth.notice] Failed password for root from 100.200.255.01 port 54438 ssh2 2014-09-28T14:03:46-04:00 abcdef01-app/abcdef01-app sshd[27420]: [ID 800047 auth.notice] Failed publickey for root from 100.200.255.02 port 59219 ssh2 2014-09-28T14:08:28-04:00 abcdef01-app/abcdef01-app sshd[3954]: [ID 800047 auth.notice] Failed keyboard-interactive for root from 100.200.255.03 port 65410 ssh2 2014-09-28T14:10:11-04:00 abcdef01-app/abcdef01-app sshd[5222]: [ID 293258 auth.error] libsldap: Status: 49 Mesg: openConnection: simple bind failed - Invalid credentials Thanks & Regards Kaladhar

4 16

Upcoming changes in unofficial packaging
by Gergely Nagy 23 Feb '15

23 Feb '15

Hi! I started packaging syslog-ng 3.6.1 for Debian and Ubuntu, to provide binaries at the usual[0] location. In the process, I intend to simplify and clean up the packaging considerably. This also means dropping support for a few older platforms. [0]: http://asylum.madhouse-project.org/projects/debian/ So beginning with 3.6.1, powerpc is going to be dropped, along with Debian Squeeze, and anything prior to Ubuntu Trusty (14.04) (mind you, I may need to add back Precise (12.04), for Travis CI). Furthermore, for Debian Wheezy, one will likely need to use the Debian backports repository[1] along with mine. [1]: http://backports.debian.org/Instructions/ I expect packages to be ready in a few days. -- |8]

3 4

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 04 Jan '15

04 Jan '15

Cool. Just read this. Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 10:07 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions SOLVED!! I was editing " /etc/syslog-ng/syslog-ng.conf", while my compiled version was using: " /usr/local/etc/syslog-ng.conf" Apparently, I had two packages installed, one from yum and another one compiled manually. When I made changes to "/usr/local/etc/syslog-ng.conf" files were created with the right permissions. Andrey Smetanin | Technical Consultant | Commerx Computer Systems Inc. 2880 Argentia Road, Unit 1, Mississauga, ON L5N 7X8 O: 905.542.9400 x201 | M: 647.894.6096 | F: 905.542.9410 andrey.smetanin(a)commerx.ca YOUR I.T. SOLUTIONS PARTNER -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Andrey Smetanin Sent: December-30-14 9:51 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/lo

4 3

Re: [syslog-ng] syslog-ng always using default permissions
by Jim Hendrick 30 Dec '14

30 Dec '14

An odd thought related to the file/file-system permissions. Is this a local drive or network mounted? I've seen problems related to underlying ownership or (lack of) permissions in other scenarios. Also, try modifying the configuration file until you can affect a change. Try a different directory or file system, try a different user or group, if possible try the same configuration file on another system. Once you notice a difference, it should help you understand the situation. Curious, Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Andrey Smetanin <Andrey.Smetanin(a)commerx.ca> Date:12/30/2014 9:51 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng always using default permissions Here is the version: ----------------------------------------------------------------------------------- [root@logmonitor ~]# syslog-ng -V syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 12 2014 15:08:34 Available-Modules: afsocket,syslogformat,afsocket-notls,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off ---------------------------------------------------------------------- Sorry, didn’t understand the last question about “later release”. Is there any way to see in the logs, debug messages...whatever...in which syslog-ng is making the decision what permissions to use? When I run debug mode all I can see is the messages about compiling config file.. You can see below, where I started syslog-ng in debug verbose mode, waited for a first log message and shut it down with CTL+C : ---------------------------------------------------------------------------------------------- [root@logmonitor ~]# systemctl stop syslog-ng [root@logmonitor ~]# ps aux | grep syslog-ng root 14070 0.0 0.0 110208 900 pts/0 R+ 09:34 0:00 grep --color=auto syslog-ng [root@logmonitor ~]# syslog-ng -vFed [2014-12-30T09:34:38.103631] Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' [2014-12-30T09:34:38.103935] Global value changed; define='scl-root', value='/usr/local/share/include/scl' [2014-12-30T09:34:38.103966] Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' [2014-12-30T09:34:38.104013] Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104431] Module loaded and initialized successfully; module='system-source' [2014-12-30T09:34:38.104447] Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' [2014-12-30T09:34:38.104510] Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.104603] Reading path for candidate modules; path='/usr/local/lib/syslog-ng' [2014-12-30T09:34:38.104659] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket.so', module='afsocket' [2014-12-30T09:34:38.104900] Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.104911] Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.104919] Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104927] Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.104934] Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.104941] Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.104968] Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.104982] Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.104990] Registering candidate plugin; module='afsocket', context='source', name='udp', preference='0' [2014-12-30T09:34:38.104997] Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.105004] Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.105017] Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.105024] Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.105031] Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.105044] Registering candidate plugin; module='afsocket', context='source', name='network', preference='0' [2014-12-30T09:34:38.105053] Registering candidate plugin; module='afsocket', context='destination', name='network', preference='0' [2014-12-30T09:34:38.105060] Registering candidate plugin; module='afsocket', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.105136] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslog-ng-crypto.so', module='syslog-ng-crypto' [2014-12-30T09:34:38.105333] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='syslogformat.so', module='syslogformat' [2014-12-30T09:34:38.105488] Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' [2014-12-30T09:34:38.105504] Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' [2014-12-30T09:34:38.105558] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' [2014-12-30T09:34:38.105727] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' [2014-12-30T09:34:38.105737] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' [2014-12-30T09:34:38.105744] Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105751] Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' [2014-12-30T09:34:38.105758] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' [2014-12-30T09:34:38.105765] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' [2014-12-30T09:34:38.105781] Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' [2014-12-30T09:34:38.105789] Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' [2014-12-30T09:34:38.106087] Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' [2014-12-30T09:34:38.106096] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' [2014-12-30T09:34:38.106102] Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' [2014-12-30T09:34:38.106109] Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' [2014-12-30T09:34:38.106115] Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' [2014-12-30T09:34:38.106121] Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' [2014-12-30T09:34:38.106134] Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' [2014-12-30T09:34:38.106142] Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' [2014-12-30T09:34:38.106148] Registering candidate plugin; module='afsocket-notls', context='source', name='systemd-syslog', preference='0' [2014-12-30T09:34:38.106214] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='affile.so', module='affile' [2014-12-30T09:34:38.106454] Registering candidate plugin; module='affile', context='source', name='file', preference='0' [2014-12-30T09:34:38.106470] Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' [2014-12-30T09:34:38.106497] Registering candidate plugin; module='affile', context='destination', name='file', preference='0' [2014-12-30T09:34:38.106527] Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' [2014-12-30T09:34:38.106611] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afprog.so', module='afprog' [2014-12-30T09:34:38.106846] Registering candidate plugin; module='afprog', context='source', name='program', preference='0' [2014-12-30T09:34:38.106857] Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' [2014-12-30T09:34:38.106908] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afuser.so', module='afuser' [2014-12-30T09:34:38.107058] Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' [2014-12-30T09:34:38.107116] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afamqp.so', module='afamqp' [2014-12-30T09:34:38.107301] Registering candidate plugin; module='afamqp', context='destination', name='amqp', preference='0' [2014-12-30T09:34:38.107367] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afmongodb.so', module='afmongodb' [2014-12-30T09:34:38.107665] Registering candidate plugin; module='afmongodb', context='destination', name='mongodb', preference='0' [2014-12-30T09:34:38.107756] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='csvparser.so', module='csvparser' [2014-12-30T09:34:38.107907] Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' [2014-12-30T09:34:38.107965] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='confgen.so', module='confgen' [2014-12-30T09:34:38.108141] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='system-source.so', module='system-source' [2014-12-30T09:34:38.108161] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' [2014-12-30T09:34:38.108319] Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' [2014-12-30T09:34:38.108359] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='basicfuncs.so', module='basicfuncs' [2014-12-30T09:34:38.108525] Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' [2014-12-30T09:34:38.108536] Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' [2014-12-30T09:34:38.108544] Registering candidate plugin; module='basicfuncs', context='template-func', name='or', preference='0' [2014-12-30T09:34:38.108551] Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' [2014-12-30T09:34:38.108870] Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' [2014-12-30T09:34:38.108879] Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' [2014-12-30T09:34:38.108887] Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' [2014-12-30T09:34:38.108894] Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' [2014-12-30T09:34:38.108901] Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' [2014-12-30T09:34:38.108908] Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' [2014-12-30T09:34:38.108915] Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' [2014-12-30T09:34:38.108922] Registering candidate plugin; module='basicfuncs', context='template-func', name='padding', preference='0' [2014-12-30T09:34:38.108929] Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' [2014-12-30T09:34:38.108942] Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' [2014-12-30T09:34:38.108950] Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' [2014-12-30T09:34:38.108957] Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' [2014-12-30T09:34:38.108964] Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' [2014-12-30T09:34:38.108971] Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' [2014-12-30T09:34:38.108978] Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' [2014-12-30T09:34:38.108985] Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' [2014-12-30T09:34:38.108992] Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' [2014-12-30T09:34:38.109035] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' [2014-12-30T09:34:38.109261] Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' [2014-12-30T09:34:38.109373] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='dbparser.so', module='dbparser' [2014-12-30T09:34:38.109739] Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' [2014-12-30T09:34:38.109821] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='afstomp.so', module='afstomp' [2014-12-30T09:34:38.109988] Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' [2014-12-30T09:34:38.110050] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='pseudofile.so', module='pseudofile' [2014-12-30T09:34:38.110205] Registering candidate plugin; module='pseudofile', context='destination', name='pseudofile', preference='0' [2014-12-30T09:34:38.110262] Reading shared object for a candidate module; path='/usr/local/lib/syslog-ng', fname='graphite.so', module='graphite' [2014-12-30T09:34:38.110419] Registering candidate plugin; module='graphite', context='template-func', name='graphite_output', preference='0' [2014-12-30T09:34:38.110517] Finishing include; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' [2014-12-30T09:34:38.110585] Starting to read include file; filename='/usr/local/share/include/scl/syslogconf/plugin.conf', depth='2' [2014-12-30T09:34:38.110951] Module loaded and initialized successfully; module='confgen' [2014-12-30T09:34:38.110967] Finishing include

1 0

syslog-ng always using default permissions
by Andrey Smetanin 30 Dec '14

30 Dec '14

Here is the part of my configuration file: ... destination d_network { file("/LOGS/fromnetwork" owner('logstash') group('logstash') perm(0660)); }; ... File "/LOGS/fromnetwork" is created with the proper user and group ("logstash"), but permissions are always "0600", regardless whatever I am trying. I don't see any errors while running "syslog-ng -vFe" What else I can troubleshoot? Andrey Smetanin -- This message has been scanned by Commerx Computer Systems Inc. and is believed to be clean.

2 3

I want to forward Windows server 2008 R2 logs to Linux syslog-ng.
by Jason Long 24 Dec '14

24 Dec '14

Hello all. I have a Windows Box that want to forward all Even logs to my Linux box. I install Snare on Windows(172.30.10.19) and configure it to forward logs to Linux and my Linux box receive it properly. When I use " tcpdump udp "port 514" ", Tcpdump show me that Snare sending Logs to Linux but Syslog-ng can't write it to log files :(. I paste my syslog-ng configure : options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; #source s_net { #udp (ip(172.30.10.19) port(514)); #}; source s_net { udp(); }; filter f_openwrt { host("172.30.10.19");}; destination df_openwrt { file("/var/log/winlog/win.log"); }; log { source ( s_net ); filter( f_openwrt ); destination ( df_openwrt ); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: Can you tell me how can I solve this problem? Cheers.

5 8

Re: [syslog-ng] syslog-ng memory usage grows
by Jim Hendrick 24 Dec '14

24 Dec '14

I will certainly try. Send me a link, and Thanks! Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Tibor Benke <ihrwein(a)gmail.com> Date:12/17/2014 5:42 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Cc: wernli(a)in2p3.fr, jrhendri(a)roadrunner.com Subject: Re: [syslog-ng] syslog-ng memory usage grows Hi All, I checked your valgrind log and the redis module and I think the leak is in redis.c:137: redisCommand(self->c, "ping"); This returns a reply object, but it's ignored and never freed. If we make a patch on GitHub, could you test it? Regards, Tibor 2014-12-17 18:36 GMT+01:00 <jrhendri(a)roadrunner.com>: Hi Fabien, Hope the season is treating you well. I ran syslog-ng for 10 minutes (recall, ~7000 EPS incoming) under valgrind and here is the output (attached). It looks like there is some sort of leak as the process from yesterday had grown to over 16GB before I stopped it for the valgrind test. Thanks for the help (as always). And please let me know if I can provide any other data. Jim ---- Fabien Wernli <wernli(a)in2p3.fr> wrote: > Hi Jim, > > On Tue, Dec 16, 2014 at 01:24:28PM -0500, Jim Hendrick wrote: > > I'm not sure what I am asking, other than general advice on: > > - performance using patterndb > > performance is awesome, I wouldn't worry about it > although it would help to have some counters on the parsers in the stats > interface. That bieng said, it could help you to increase the stats level. > > > - performance using redis destination > > I can't comment on that one, sorry > > > - advice on debugging where this memory growth is happening > > Could you run syslog-ng inside `valgrind --tool=memcheck > --trace-children=yes --leak-check=yes`? > > The problem is this will really slow down your instance, but let it run a > few minutes nevertheless and pastebin the result somewhere. > > > As a rough measure - I have a syslog-ng process that has been running for > > less than 3 hours and right now is using 1.52GB of resident memory (shown by > > "top") > > I'm suspecting `format-json` to be the source of the leak, or the flow > control cache. These have been the leak source of our own observations many > times, although many patches have solved the issues. > > > (I would rather parse things in syslog-ng, but I *could* do this all using > > logstash/grok if this proves too much for patterndb at this load) > > if you're worried about performance, I wouldn't even consider to begin thinking about comparing > LS/grok to patterndb > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

3 3

insider 2014-12: incubator 0.4.1 released; anonymization; monitoring;
by Czanik, Péter 18 Dec '14

18 Dec '14

Dear syslog-ng users, This is the 39th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. FEATURED NEWS syslog-ng incubator 0.4.1 released ------------------------ We are proud to announce that version 0.4.1 of syslog-ng incubator is released. It is compatible with syslog-ng version 3.6. Some features previously included in the incubator such as Riemann and Graphite support have been ported to syslog-ng. It also adds many interesting new features: kafka and java destination support, ZMQ source and destination and a grok parser. Ports and packages are available for Fedora, FreeBSD and openSUSE, but with some of the features missing, as dependencies are not yet available packaged. For details, read https://czanik.blogs.balabit.com/2014/12/syslog-ng-incubator-0-4-1-released/ Data Privacy, Anonymization and Log Data ---------------------------------------- Data privacy is becoming a hot topic, especially in the European Union. New features in syslog-ng can be used to become compliant with regulations. Read about encrypting, anonymizing logs and pseudonymization at https://jluby.blogs.balabit.com/2014/12/08/data-privacy-anonymization-and-l… The trinity of monitoring: syslog-ng, Riemann and Elasticsearch -------------------------------------------------------------- BalaBit organizes OpenAcademy in Budapest, Hungary, as a way of continuous learning. Last time we had a guest, Fabien Wernli (on IRC better known as faxmodem), who talked about how syslog-ng can be integrated with Riemann and Elasticsearch. The video starts in Hungarian, but the presentation is in English: https://www.youtube.com/watch?v=Y4uzJm5VLh4 NEW RELEASES syslog-ng 3.6.2: https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.6.2 syslog-ng incubator 0.4.0 & 0.4.1: https://github.com/balabit/syslog-ng-incubator/releases Happy Holidays! Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com To read this newsletter on-line, visit: http://insider.blogs.balabit.com/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng memory usage grows
by Jim Hendrick 17 Dec '14

17 Dec '14

Hi, I am continuing to test syslog-ng as the parser and shipper into an elasticsearch cluster. Right now I have syslog-ng 3.6.1 receiving logs at about ~7000 EPS and running them through a patterndb parser that splits out the (bluecoat proxy) fields into key-value pairs, writing them to a redis destination using format-json. syslog-ng 3.6.1 Installer-Version: 3.6.1 Revision: Compile-Date: Dec 9 2014 19:42:20 Available-Modules: dbparser,json-plugin,afuser,affile,afmongodb,tfgeoip,afprog,redis,afstomp,afsql,afsocket,system-source,afamqp,pseudofile,confgen,afsocket-tls,csvparser,basicfuncs,graphite,syslogformat,afsocket-notls,linux-kmsg-format,cryptofuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off I then use logstash to pull from redis and feed elasticsearch (the thought being that this would provide a buffer for the messages) Over the weekend I had syslog-ng crash (unfortunately no core) but now that I am watching it more closely, I see what appears to be continuous growth in memory usage (which leads me to suppose this was the cause of the crash). I'm not sure what I am asking, other than general advice on: - performance using patterndb - performance using redis destination - advice on debugging where this memory growth is happening As a rough measure - I have a syslog-ng process that has been running for less than 3 hours and right now is using 1.52GB of resident memory (shown by "top") I am including two config files. One has a single source, no filtering, single destination. The other splits logs from three upstream syslog-ng servers thinking this would provide "threading" with multiple patterndb parsers (I have read that patterndb is single threaded). Any thoughts on how to figure out where the memory is going? Or recommendations on whether or not using patterndb in this way is a particularly dumb idea :-) (I would rather parse things in syslog-ng, but I *could* do this all using logstash/grok if this proves too much for patterndb at this load) Thanks! Jim

4 3

Email alerts en syslog-ng 3.3.5
by Roberto Carna 17 Dec '14

17 Dec '14

Dear, I have this syslog platform: - Debian 7 - Syslog-ng 3.3.5.-4 (installed via apt-get) - Php-syslog-ng (graphical interface) - Postfix I need to configure syslog-ng in order to send email alerts/notifications about some specific events. Is it possible to implement with this syslog-ng version and what is the way??? Thanks a lot!!! Roberto

2 4