- syslog-ng - lists.balabit.hu

Log to syslog file, filter from fifo
by Lay, James 09 Dec '11

09 Dec '11

Topic says it. I think I have a pretty simple layout: log { source(s_local); filter(f_firewall); destination(d_file); destination(fifo); }; Can I change this to: log { source(s_local); destination(d_file); filter(f_firewall); destination(fifo); }; ? Will this log to file, then filter, then go to fifo? Thanks and hope this isn't a silly question...I read this from the admin guide: Log statements are processed in the order they appear in the configuration file, thus the order of log paths may influence what happens to a message, especially when using filters and log flags. So I'm hoping that is the case. Thank you. James

6 8

Re: [syslog-ng] Log only one host
by tokie＠tiscali.it 09 Dec '11

09 Dec '11

Tks for reply, I tried but don't work! More specific.. I wish that all the servers in my network, logging into a specific file of the same syslog server. Now all log all!! I have the same result in different file(100.log, 101.log, and so on) tks Tokie p.s.: netmask("a.b.c. d/32") or netmask("a.b.c.d/255.255.255.255") ??? ----------------------------------------------------------------------------------------- Hi, On Fri, Dec 9, 2011 at 10:29 AM, Tokie <tokie(a)tiscali.it> wrote: > Hi to all, > my question is very simple: > How do I to log only one a specific ip address?? > I tried filtering with the Host, Host_From, SOURCEIP, etc without any > positive result. > > What is the correct syntax? > Could you help me?? Try using the netmask filter. filter f_host_a_b_c_d { netmask("a.b.c.d/32"); }; Regards, Sandor End of syslog-ng Digest, Vol 80, Issue 14 ***************************************** E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

1 0

syslog-ng 3.3.3 repeatedly writes same message to local file when forwarding enabled
by Dave Haywood 09 Dec '11

09 Dec '11

Hi, I have a problem with syslog-ng 3.3.3. When I have forwarding enabled to a remote syslog server (via UDP) syslog-ng repeatedly writes the same message(s) to the log file and only stops when the disk is full. Using tcpdump on the remote server, I don't see any data arrive from the syslog-ng server so forwarding is not working either. When I remove the forwarding part of the config file the local file is written correctly (ie once). If I remove the local file part from the config file and only enable the forwarding, I see syslog-ng take all the CPU time. I never see any syslog messages arrive at the remote syslog server. I tried: 1) disabling IPv6 - no change 2) running outside the chroot jail - no change 3) running as userid root - no change Does anyone have any idea what would cause this? Debug info below. The environment is: RedHat AS 4.8 (linux 2.6.9-89.ELsmp) on vmware ESXi 4.1.0 All required software built and installed in /usr/local/ : eventlog_0.2.12.tar.gz gettext-0.18.1.1.tar.gz glib-2.29.90.tar.bz2 libdbi-0.8.4.tar.gz libdbi-drivers-0.8.3.tar.gz libffi-3.0.9.tar.gz libnet-0.10.11.tar.gz pkg-config-0.26.tar.gz Python-2.7.2.tar.bz2 zlib-1.2.5.tar.bz2 syslog-ng_3.3.3.tar.gz syslog-ng is running chroot() in directory /data as user syslogng:sysadmins and listens on port 1514. iptables redirects any incoming port 514 traffic to 1514. The required /usr/local/ directories are mounted (-o bind) under /data. syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#master#d199a1980be6b23fe24189e86a882812288e292c Compile-Date: Dec 8 2011 17:46:40 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql Available-Modules: convertfuncs,affile,afmongodb,dummy,basicfuncs,csvparser,confgen,afsql,syslogformat,afuser,afsocket,afprog,afsocket-notls,dbparser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: off Config file: @version: 3.3 source s_udp { udp(ip("0.0.0.0") port(1514)); }; destination file1 { file("/log/network.log" owner(syslogng)group(sysops) perm(0640) flags(no-multi-line)); }; destination NeDi { udp("192.168.0.7" port(514)); }; log { source(s_udp); destination(file1); }; # enabling the line below breaks logging to the file above log { source(s_udp); destination(NeDi); }; Debug: # /usr/local/sbin/syslog-ng --cfgfile=/usr/local/etc/syslog-ng.conf --chroot=/data --user=syslogng --group=sysadmins --persist-file=/log/syslog-ng.persist --foreground --process-mode=foreground --stderr --debug nanosleep() is not accurate enough to introduce minor stalls on the reader side, multi-threaded performance may be affected; Trying to open module; module='affile', filename='/usr/local/lib/syslog-ng/libaffile.so' Trying to open module; module='afprog', filename='/usr/local/lib/syslog-ng/libafprog.so' Trying to open module; module='afsocket', filename='/usr/local/lib/syslog-ng/libafsocket.so' Trying to open module; module='afuser', filename='/usr/local/lib/syslog-ng/libafuser.so' Trying to open module; module='basicfuncs', filename='/usr/local/lib/syslog-ng/libbasicfuncs.so' Trying to open module; module='csvparser', filename='/usr/local/lib/syslog-ng/libcsvparser.so' Trying to open module; module='dbparser', filename='/usr/local/lib/syslog-ng/libdbparser.so' Trying to open module; module='syslogformat', filename='/usr/local/lib/syslog-ng/libsyslogformat.so' Trying to open module; module='afsql', filename='/usr/local/lib/syslog-ng/libafsql.so' Syslog connection established; fd='8', server='AF_INET(192.168.0.7:514)', local='AF_INET(0.0.0.0:0)' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.3.3' Incoming log entry; line='<189>41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)' Incoming log entry; line='<189>Dec 9 08:41:24 6500-1 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' ....forever....

2 2

Log only one host
by Tokie 09 Dec '11

09 Dec '11

Hi to all, my question is very simple: How do I to log only one a specific ip address?? I tried filtering with the Host, Host_From, SOURCEIP, etc without any positive result. What is the correct syntax? Could you help me?? Thanks Tokie

2 1

syslog-ng Insider - December 2011
by Peter Czanik 08 Dec '11

08 Dec '11

Dear syslog-ng users, This is the 9th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS syslog-ng 3.3.3 is released! ---------------------------- A new version of syslog-ng is released! There are no new features to announce, but most problems reported since 3.3.1 should be fixed by now! Thank you for all of those, who helped us to hunt bugs with detailed reports and many testing! The release of 3.3.3 brought many new users to the latest syslog-ng version, which helped to uncover some more bugs in advanced configurations. Until a new release arrives, please check the git tree ( git://git.balabit.hu/bazsi/syslog-ng-3.3 ) and the mailing list archives, if your problem is addressed by a patch. Sources are available at http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.3 Binary packages are available are available for several Linux distributions. Please check availability at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… syslog-ng and patterns ---------------------- Patterndb is one of the most important features of syslog-ng, still not many people are using it. So we are very happy to see, that patterndb was the focus point in many recent syslog-ng mailing list threads. First of all, thanks for Evan Rempel for providing many useful ideas and feedback about patterndb on the syslog-ng mailing list. ELSA (Enterprise Log and Search Archive), which uses patterndb heavily, had some major updates recently, which make it a lot more easy to install on a couple of different systems. It is available at http://code.google.com/p/enterprise-log-search-and-archive/ We plan to use CEE (http://cee.mitre.org <http://cee.mitre.org/>) for our patterns in the long term. But even until this standard is available, please share your patterns in any form to lower the entry barrier for your fellow syslog-ng users. If you send them to the list or directly to me, I'll make them available at http://git.balabit.hu/?p=czanik/syslog-ng-patterndb.git;a=summary syslog-ng and the journal ------------------------- There’s an ongoing project to create a new logging subsystem for Linux, called the journal, by Lennart Poettering of PulseAudio & systemd fame. It is implemented as a core component of systemd, thus has a good chance to be integrated to all distributions that carry systemd. Since syslog-ng is also in the logging sphere, the logical question arises: how does this new project affect syslog-ng in the long run? For the answer, read Bazsi's blog at http://bazsi.blogs.balabit.com/2011/12/syslog-ng-and-the-journal/ OTHER SHORT NEWS * An interesting article about extracting useful information from log messages was published in Free BSD Magazine (not only for BSD users :-) ): http://www.balabit.com/support/documentation/BSD_12_2011%20-%20BalaBit_sing… where you also can read about several up-to-date topics, like "Rolling Your Own FreeBSD Kernel", "Hardening BSD with Security Levels" and so on. The whole Free BSD Magazine can be downloaded at http://bsdmag.org <http://bsdmag.org/> NEW RELEASES * syslog-ng OSE 3.3.3: http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.3 ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Syslog-ng is not sending messages line by line
by Alptugay 06 Dec '11

06 Dec '11

Hello all, I have a python script which prints 30 lines. And my syslog-ng is listening this script and writing it's messages to a stream. But the application which is listening to the stream recieves all the lines in one line. To be more clear my source program is basically like this: while i<30: print "cccc dddd" i=i+1 But my application listening to the stream gets a line like this: [[u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:1 1 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 c ccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd' ], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13 >Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 1 1:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 d dos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>De c 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:4 7:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'] .... etc But it should recieve one line from the stream for one print command of the source program. Btw I am using freeBSD _____________________________________________________________________ Register now! Use this rare opportunity! http://partners.labris.eu

2 1

Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu 10.10
by Luis Pugoy 05 Dec '11

05 Dec '11

> > Message: 3 > Date: Mon, 05 Dec 2011 10:10:40 +0100 > From: Gergely Nagy <algernon(a)balabit.hu> > Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu > 10.10 > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <87aa77l6y7.fsf(a)algernon.balabit> > Content-Type: text/plain > > Luis Pugoy <lpugoy(a)insynchq.com> writes: > > > Does anyone have any insight into this? The segfault is happening more > > often now. :( > > I'd suggest either upgrading to 3.3 from my repo[1], or if an upgrade is > not feasible, then if you could compile 3.1 with debug symbols, get it > to drop a core, and do a backtrace, that would help tremendously. > > [1]: http://asylum.madhouse-project.org/projects/debian/ > > To do the latter, the following steps should work: > > $ apt-get source syslog-ng > $ sudo apt-get build-dep syslog-ng > $ cd syslog-ng-* > $ DEB_BUILD_OPTIONS="debug nostrip" dpkg-buildpackage -us -uc -rfakeroot > $ sudo dpkg -i ../syslog-ng_*.deb > > Then add --enable-core to SYSLOGNG_OPTS in /etc/default/syslog-ng, and > when it segfaults, it'll make a core in (as far as I remember) /. Once > that happened, do something like this: > > $ gdb -c /core /usr/bin/syslog-ng > (gdb) thread apply all bt full > > This should give us a few hints. > > -- > |8] > I installed a newer version of the PCRE library. That seems to have worked for now. If it happens again I will be sure to follow your instructions, thanks.

1 0

syslog-ng 3.1.2 segmentation fault in Ubuntu 10.10
by Luis Pugoy 05 Dec '11

05 Dec '11

Hello, Our syslog-ng server is lately raising a segfault every day or so. The error is of the form "segfault at 7f8677df0338 ip 00007f8676fa8e9d sp 00007fff1d2ec280 error 4 in libpcre.so.3.12.1[7f8676f96000+30000]." Does anyone know what this means? Thanks.

3 3

Question on syslog-ng filtering and performance - host vs. netmask
by jrhendri＠maine.rr.com 03 Dec '11

03 Dec '11

Hi, I am using syslog-ng to do log collection and "routing" to different destinations based on the origin of the log. I used to (still do in some cases) filter like this: filter f_one_list { host("10.0.0.1") or host("10.0.2.1") or ... host("10.3.4.5"); } Since (too) many log sources break the RFC and put all kinds of strangeness in the "header", I started using netmask instead: filter f_another_list { netmask("10.0.0.1/32") or netmask("10.0.2.1/32") or ... netmask("10.3.4.5/32"); } My question is this - Is there a measurable performance difference from one to another? "intuitively" it would seem doing the netmask check should be faster than parsing the syslog header, but I thought I'd ask here. (currently I have some servers processing 4000+ messages per second with no real problem, but the CPUs are clearly busy. BTW - I am using 3.2.4 at the moment (looking at moving to the multi-threaded newer versions in the future) Thanks! Jim Hendrick

3 2

syslog-ng log server stops listening on network
by Paul Muther 02 Dec '11

02 Dec '11

Hello all, I have a syslog-ng log server running 3.2.4 listening on a TCP port 4514. We are not a high volume environment (server processes on average between 10 and 20 MPS). Twice now the server has just stopped accepting traffic on the TCP port causing all the clients to store messages. There seems to be no explanation for the event and the local service is running and processing all local logs for the server no problem. I feel like this might be an IP port conflict but I just want to make sure I'm not missing anything. Thanks much, Paul M

2 1