- syslog-ng - lists.balabit.hu

Re: [syslog-ng] Log only one host
by tokie＠tiscali.it 13 Dec '11

13 Dec '11

As you can see the device gives a valid syslog header. Destination of Firewall in fw.log: # tailf fw.log Dec 13 13:58:47 192.168.1.254/192. 168.1.254 Fw-Vigor2955: 149:13:59.970 lan->wan @S:R=13:1 p 192.168.1.8,19679 -> 82.200.234.214,33244 PR UDP len 20 131 Destination of Access Point .241 in ap241.log: # tailf ap241.log Dec 13 13:58:47 192.168.1.254/192.168.1.254 Fw-Vigor2955: 149:13:59.770 lan- >wan @S:R=13:1 p 192.168.1.8,19679 -> 94.244.30.85,63708 PR UDP len 20 131 It writes the same thing!! My configuration is the following: source udp { udp(); }; destination d_fw { file("/**/**/fw.log" create_dirs(yes) fsync(yes)); }; filter f_fw { netmask ("192.168.1.254/32"); ##try to log one host }; log { source(udp); destination(d_fw); filter(f_fw); }; Therefore, how I do?? Thanks Tokie E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

2 1

R: syslog-ng Digest, Vol 80, Issue 18
by tokie＠tiscali.it 13 Dec '11

13 Dec '11

As you can see the device gives a valid syslog header. Destination of Firewall in fw.log: # tailf fw.log Dec 13 13:58:47 192.168.1.254/192. 168.1.254 Fw-Vigor2955: 149:13:59.970 lan->wan @S:R=13:1 p 192.168.1.8,19679 -> 82.200.234.214,33244 PR UDP len 20 131 Destination of Access Point .241 in ap241.log: # tailf ap241.log Dec 13 13:58:47 192.168.1.254/192.168.1.254 Fw-Vigor2955: 149:13:59.770 lan- >wan @S:R=13:1 p 192.168.1.8,19679 -> 94.244.30.85,63708 PR UDP len 20 131 It writes the same thing!! My configuration is the following: source udp { udp(); }; destination d_fw { file("/**/**/fw.log" create_dirs(yes) fsync(yes)); }; filter f_fw { netmask ("192.168.1.254/32"); ## try for only host }; log { source(udp); destination(d_fw); filter(f_fw); }; Therefore, how can I do?? Thanks Tokie ------------------------------ Date: Mon, 12 Dec 2011 19: 10:30 -0500 From: Jim <jrhendri(a)maine.rr.com> Subject: Re: [syslog-ng] Log only one host To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <1323735030.16055.1. camel@ubuntu> Content-Type: text/plain; charset="UTF-8" Couple things to note: - not all syslogs give a valid syslog header (e.g. <date> <time> <host> <program>:<message>) - name resolution costs cycles (may not be an issue) - know the difference between $HOST (parsed from the syslog header) and $HOST_FROM (the origin of the packet) Enjoy! On Mon, 2011-12-12 at 12:05 +0100, Thomas Wollner wrote: > Hello, > > to separate the destination logfiles for each host, use the following > > destination d_file_foreach_host { > file("/var/log/$FULLHOST.log"); > } > > > log { > source(s_all); > destination (d_file_foreach_host), > }; > > hope it helps, > regards, > > Tom > > > > Zitat von "tokie(a)tiscali.it" <tokie(a)tiscali.it>: > > >> Try using the netmask filter. > > > >> filter f_host_a_b_c_d { > >> netmask > > ("a.b.c.d/32"); > >> }; > > > > Tks for reply, > > I tried but don't work! > > More > > specific: > > I wish that all devicies in my network, logging into a > > specific file on syslog server. > > > > Now all files log all devicies!! > > I > > have the same result in different > > file(100.log, 101.log, and so on) > > > > > > Must I use iptables's match?? How?? > > > > tks > > Tokie > > > > p.s.: > > netmask ("a.b.c. > > > > d/32") or netmask("a.b.c.d/255.255.255.255") ??? E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

1 0

Log only one host
by tokie＠tiscali.it 13 Dec '11

13 Dec '11

> Try using the netmask filter. > filter f_host_a_b_c_d { > netmask ("a.b.c.d/32"); > }; Tks for reply, I tried but don't work! More specific: I wish that all devicies in my network, logging into a specific file on syslog server. Now all files log all devicies!! I have the same result in different file(100.log, 101.log, and so on) Must I use iptables's match?? How?? tks Tokie p.s.: netmask("a.b.c. d/32") or netmask("a.b.c.d/255.255.255.255") ??? ----Messaggio originale---- Da: syslog-ng-request(a)lists.balabit.hu Data: 10/12/2011 12.00 A: <syslog-ng(a)lists.balabit.hu> Ogg: syslog-ng Digest, Vol 80, Issue 15 Send syslog-ng mailing list submissions to syslog-ng@lists. balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists. balabit.hu You can reach the person managing the list at syslog-ng- owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: syslog-ng 3.3.3 repeatedly writes same message to local file when forwarding enabled (Dave Haywood) 2. Re: Log only one host (tokie(a)tiscali.it) 3. Re: Log to syslog file, filter from fifo (Balazs Scheidler) 4. [Bug 146] pdbtool match does not display tags (bugzilla(a)bugzilla.balabit.com) ---------------------------------------------------------------------- Message: 1 Date: Fri, 09 Dec 2011 11:22:24 +0000 From: Dave Haywood <tla(a)oak.selfip.net> Subject: Re: [syslog-ng] syslog-ng 3.3.3 repeatedly writes same message to local file when forwarding enabled To: Syslog-ng users' and developers' mailing list <syslog-ng@lists. balabit.hu> Cc: Sandor Geller <Sandor.Geller(a)morganstanley.com> Message- ID: <4EE1EF70.1060001(a)oak.selfip.net> Content-Type: text/plain; charset=ISO-8859-1 On 09/12/2011 09:53, Sandor Geller wrote: > Sounds like messages sent to 192.168.0.7 are feeded back to syslog-ng > so there is a logging loop. Is this address local? When not then there > is a chance that the packet filter rule isn't correct. Thanks! You were right, the issue was with the iptables rule. I was trying to capture traffic from localhost to port 514 and redirect it to 1514 using NAT table OUTPUT. I use this for testing every facility / severity combination during install. But I didn't specify a destination host (of the local IP address); I only specified the port. This meant and traffic forwarded to a remote host is redirected by iptables back to the localhost, causing a loop. Thanks for the help :) > > On Fri, Dec 9, 2011 at 10:34 AM, Dave Haywood <tla(a)oak.selfip. net> wrote: >> Hi, >> >> I have a problem with syslog-ng 3.3.3. When I have forwarding enabled to a remote syslog server (via UDP) syslog-ng repeatedly writes the same message(s) to the log file and only stops when the disk is full. Using tcpdump on the remote server, I don't see any data arrive from the syslog-ng server so forwarding is not working either. >> >> When I remove the forwarding part of the config file the local file is written correctly (ie once). If I remove the local file part from the config file and only enable the forwarding, I see syslog- ng take all the CPU time. I never see any syslog messages arrive at the remote syslog server. >> >> I tried: >> 1) disabling IPv6 - no change >> 2) running outside the chroot jail - no change >> 3) running as userid root - no change >> >> Does anyone have any idea what would cause this? Debug info below. >> >> The environment is: >> >> RedHat AS 4.8 (linux 2.6.9-89.ELsmp) on vmware ESXi 4.1.0 >> >> All required software built and installed in /usr/local/ : >> >> eventlog_0.2.12.tar.gz >> gettext-0.18.1.1.tar.gz >> glib-2.29.90.tar.bz2 >> libdbi-0.8.4.tar.gz >> libdbi-drivers-0.8.3. tar.gz >> libffi-3.0.9.tar.gz >> libnet-0.10.11.tar.gz >> pkg-config- 0.26.tar.gz >> Python-2.7.2.tar.bz2 >> zlib-1.2.5.tar.bz2 >> syslog- ng_3.3.3.tar.gz >> >> syslog-ng is running chroot() in directory /data as user E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

3 2

error while loading shared libraries
by Johnson, Chris (HP TippingPoint Roseville) 12 Dec '11

12 Dec '11

I'm getting the following error: /usr/local/sbin/syslog-ng: error while loading shared libraries: /usr/local/lib/libsyslog-ng-3.3.3.so: ELF file OS ABI invalid When I try to run syslog-ng (also occurs when I run pbdtool) pdbtool: error while loading shared libraries: /usr/local/lib/libsyslog-ng-3.3.3.so: ELF file OS ABI invalid I'm building on one box (RedHat), and running on another (not sure what flavor of Linux, might be CentOS). Any pointers to what I can set during the build phase that will allow it to run? Thanks, Chris ---------------------------------------- Christopher Johnson chris.johnson3(a)hp.com HP Software - Security Product Group ----------------------------------------

2 4

[Bug 146] New: pdbtool match does not display tags
by bugzilla＠bugzilla.balabit.com 09 Dec '11

09 Dec '11

https://bugzilla.balabit.com/show_bug.cgi?id=146 Summary: pdbtool match does not display tags Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: bug Estimated Hours: 0.0 When using the pdbtool match command the tags of a message are not displayed. % pdbtool match -M "Fri Nov 25 00:00:29.618 2011: Accepted and connected to 172.20.102.38 hermes0080 <c0n350>" -p mmfs.xml --program=mmfs MESSAGE=Fri Nov 25 00:00:29.618 2011: Accepted and connected to 172.20.102.38 hermes0080 <c0n350> PROGRAM=mmfs .classifier.class=security .classifier.rule_id=463e34dd-f946-4f25-9cd2-9c00e25eba4c .net.destination.address=172.20.102.38 .net.destination.name=hermes0080 .mmfs.node=c0n350 application=mmfs -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

Log to syslog file, filter from fifo
by Lay, James 09 Dec '11

09 Dec '11

Topic says it. I think I have a pretty simple layout: log { source(s_local); filter(f_firewall); destination(d_file); destination(fifo); }; Can I change this to: log { source(s_local); destination(d_file); filter(f_firewall); destination(fifo); }; ? Will this log to file, then filter, then go to fifo? Thanks and hope this isn't a silly question...I read this from the admin guide: Log statements are processed in the order they appear in the configuration file, thus the order of log paths may influence what happens to a message, especially when using filters and log flags. So I'm hoping that is the case. Thank you. James

6 8

Re: [syslog-ng] Log only one host
by tokie＠tiscali.it 09 Dec '11

09 Dec '11

Tks for reply, I tried but don't work! More specific.. I wish that all the servers in my network, logging into a specific file of the same syslog server. Now all log all!! I have the same result in different file(100.log, 101.log, and so on) tks Tokie p.s.: netmask("a.b.c. d/32") or netmask("a.b.c.d/255.255.255.255") ??? ----------------------------------------------------------------------------------------- Hi, On Fri, Dec 9, 2011 at 10:29 AM, Tokie <tokie(a)tiscali.it> wrote: > Hi to all, > my question is very simple: > How do I to log only one a specific ip address?? > I tried filtering with the Host, Host_From, SOURCEIP, etc without any > positive result. > > What is the correct syntax? > Could you help me?? Try using the netmask filter. filter f_host_a_b_c_d { netmask("a.b.c.d/32"); }; Regards, Sandor End of syslog-ng Digest, Vol 80, Issue 14 ***************************************** E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

1 0

syslog-ng 3.3.3 repeatedly writes same message to local file when forwarding enabled
by Dave Haywood 09 Dec '11

09 Dec '11

Hi, I have a problem with syslog-ng 3.3.3. When I have forwarding enabled to a remote syslog server (via UDP) syslog-ng repeatedly writes the same message(s) to the log file and only stops when the disk is full. Using tcpdump on the remote server, I don't see any data arrive from the syslog-ng server so forwarding is not working either. When I remove the forwarding part of the config file the local file is written correctly (ie once). If I remove the local file part from the config file and only enable the forwarding, I see syslog-ng take all the CPU time. I never see any syslog messages arrive at the remote syslog server. I tried: 1) disabling IPv6 - no change 2) running outside the chroot jail - no change 3) running as userid root - no change Does anyone have any idea what would cause this? Debug info below. The environment is: RedHat AS 4.8 (linux 2.6.9-89.ELsmp) on vmware ESXi 4.1.0 All required software built and installed in /usr/local/ : eventlog_0.2.12.tar.gz gettext-0.18.1.1.tar.gz glib-2.29.90.tar.bz2 libdbi-0.8.4.tar.gz libdbi-drivers-0.8.3.tar.gz libffi-3.0.9.tar.gz libnet-0.10.11.tar.gz pkg-config-0.26.tar.gz Python-2.7.2.tar.bz2 zlib-1.2.5.tar.bz2 syslog-ng_3.3.3.tar.gz syslog-ng is running chroot() in directory /data as user syslogng:sysadmins and listens on port 1514. iptables redirects any incoming port 514 traffic to 1514. The required /usr/local/ directories are mounted (-o bind) under /data. syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#master#d199a1980be6b23fe24189e86a882812288e292c Compile-Date: Dec 8 2011 17:46:40 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql Available-Modules: convertfuncs,affile,afmongodb,dummy,basicfuncs,csvparser,confgen,afsql,syslogformat,afuser,afsocket,afprog,afsocket-notls,dbparser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: off Config file: @version: 3.3 source s_udp { udp(ip("0.0.0.0") port(1514)); }; destination file1 { file("/log/network.log" owner(syslogng)group(sysops) perm(0640) flags(no-multi-line)); }; destination NeDi { udp("192.168.0.7" port(514)); }; log { source(s_udp); destination(file1); }; # enabling the line below breaks logging to the file above log { source(s_udp); destination(NeDi); }; Debug: # /usr/local/sbin/syslog-ng --cfgfile=/usr/local/etc/syslog-ng.conf --chroot=/data --user=syslogng --group=sysadmins --persist-file=/log/syslog-ng.persist --foreground --process-mode=foreground --stderr --debug nanosleep() is not accurate enough to introduce minor stalls on the reader side, multi-threaded performance may be affected; Trying to open module; module='affile', filename='/usr/local/lib/syslog-ng/libaffile.so' Trying to open module; module='afprog', filename='/usr/local/lib/syslog-ng/libafprog.so' Trying to open module; module='afsocket', filename='/usr/local/lib/syslog-ng/libafsocket.so' Trying to open module; module='afuser', filename='/usr/local/lib/syslog-ng/libafuser.so' Trying to open module; module='basicfuncs', filename='/usr/local/lib/syslog-ng/libbasicfuncs.so' Trying to open module; module='csvparser', filename='/usr/local/lib/syslog-ng/libcsvparser.so' Trying to open module; module='dbparser', filename='/usr/local/lib/syslog-ng/libdbparser.so' Trying to open module; module='syslogformat', filename='/usr/local/lib/syslog-ng/libsyslogformat.so' Trying to open module; module='afsql', filename='/usr/local/lib/syslog-ng/libafsql.so' Syslog connection established; fd='8', server='AF_INET(192.168.0.7:514)', local='AF_INET(0.0.0.0:0)' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.3.3' Incoming log entry; line='<189>41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)' Incoming log entry; line='<189>Dec 9 08:41:24 6500-1 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' Incoming log entry; line='<189>Dec 9 08:41:24 localhost 41609: Dec 9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a' ....forever....

2 2

Log only one host
by Tokie 09 Dec '11

09 Dec '11

Hi to all, my question is very simple: How do I to log only one a specific ip address?? I tried filtering with the Host, Host_From, SOURCEIP, etc without any positive result. What is the correct syntax? Could you help me?? Thanks Tokie

2 1

syslog-ng Insider - December 2011
by Peter Czanik 08 Dec '11

08 Dec '11

Dear syslog-ng users, This is the 9th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS syslog-ng 3.3.3 is released! ---------------------------- A new version of syslog-ng is released! There are no new features to announce, but most problems reported since 3.3.1 should be fixed by now! Thank you for all of those, who helped us to hunt bugs with detailed reports and many testing! The release of 3.3.3 brought many new users to the latest syslog-ng version, which helped to uncover some more bugs in advanced configurations. Until a new release arrives, please check the git tree ( git://git.balabit.hu/bazsi/syslog-ng-3.3 ) and the mailing list archives, if your problem is addressed by a patch. Sources are available at http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.3 Binary packages are available are available for several Linux distributions. Please check availability at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… syslog-ng and patterns ---------------------- Patterndb is one of the most important features of syslog-ng, still not many people are using it. So we are very happy to see, that patterndb was the focus point in many recent syslog-ng mailing list threads. First of all, thanks for Evan Rempel for providing many useful ideas and feedback about patterndb on the syslog-ng mailing list. ELSA (Enterprise Log and Search Archive), which uses patterndb heavily, had some major updates recently, which make it a lot more easy to install on a couple of different systems. It is available at http://code.google.com/p/enterprise-log-search-and-archive/ We plan to use CEE (http://cee.mitre.org <http://cee.mitre.org/>) for our patterns in the long term. But even until this standard is available, please share your patterns in any form to lower the entry barrier for your fellow syslog-ng users. If you send them to the list or directly to me, I'll make them available at http://git.balabit.hu/?p=czanik/syslog-ng-patterndb.git;a=summary syslog-ng and the journal ------------------------- There’s an ongoing project to create a new logging subsystem for Linux, called the journal, by Lennart Poettering of PulseAudio & systemd fame. It is implemented as a core component of systemd, thus has a good chance to be integrated to all distributions that carry systemd. Since syslog-ng is also in the logging sphere, the logical question arises: how does this new project affect syslog-ng in the long run? For the answer, read Bazsi's blog at http://bazsi.blogs.balabit.com/2011/12/syslog-ng-and-the-journal/ OTHER SHORT NEWS * An interesting article about extracting useful information from log messages was published in Free BSD Magazine (not only for BSD users :-) ): http://www.balabit.com/support/documentation/BSD_12_2011%20-%20BalaBit_sing… where you also can read about several up-to-date topics, like "Rolling Your Own FreeBSD Kernel", "Hardening BSD with Security Levels" and so on. The whole Free BSD Magazine can be downloaded at http://bsdmag.org <http://bsdmag.org/> NEW RELEASES * syslog-ng OSE 3.3.3: http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.3 ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Syslog-ng is not sending messages line by line
by Alptugay 06 Dec '11

06 Dec '11

Hello all, I have a python script which prints 30 lines. And my syslog-ng is listening this script and writing it's messages to a stream. But the application which is listening to the stream recieves all the lines in one line. To be more clear my source program is basically like this: while i<30: print "cccc dddd" i=i+1 But my application listening to the stream gets a line like this: [[u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:1 1 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 c ccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd' ], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13 >Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 1 1:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 d dos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>De c 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:4 7:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'], [u'<13>Dec 6 11:47:11 ddos1 cccc dddd'] .... etc But it should recieve one line from the stream for one print command of the source program. Btw I am using freeBSD _____________________________________________________________________ Register now! Use this rare opportunity! http://partners.labris.eu

2 1

Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu 10.10
by Luis Pugoy 05 Dec '11

05 Dec '11

> > Message: 3 > Date: Mon, 05 Dec 2011 10:10:40 +0100 > From: Gergely Nagy <algernon(a)balabit.hu> > Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu > 10.10 > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <87aa77l6y7.fsf(a)algernon.balabit> > Content-Type: text/plain > > Luis Pugoy <lpugoy(a)insynchq.com> writes: > > > Does anyone have any insight into this? The segfault is happening more > > often now. :( > > I'd suggest either upgrading to 3.3 from my repo[1], or if an upgrade is > not feasible, then if you could compile 3.1 with debug symbols, get it > to drop a core, and do a backtrace, that would help tremendously. > > [1]: http://asylum.madhouse-project.org/projects/debian/ > > To do the latter, the following steps should work: > > $ apt-get source syslog-ng > $ sudo apt-get build-dep syslog-ng > $ cd syslog-ng-* > $ DEB_BUILD_OPTIONS="debug nostrip" dpkg-buildpackage -us -uc -rfakeroot > $ sudo dpkg -i ../syslog-ng_*.deb > > Then add --enable-core to SYSLOGNG_OPTS in /etc/default/syslog-ng, and > when it segfaults, it'll make a core in (as far as I remember) /. Once > that happened, do something like this: > > $ gdb -c /core /usr/bin/syslog-ng > (gdb) thread apply all bt full > > This should give us a few hints. > > -- > |8] > I installed a newer version of the PCRE library. That seems to have worked for now. If it happens again I will be sure to follow your instructions, thanks.

1 0

syslog-ng 3.1.2 segmentation fault in Ubuntu 10.10
by Luis Pugoy 05 Dec '11

05 Dec '11

Hello, Our syslog-ng server is lately raising a segfault every day or so. The error is of the form "segfault at 7f8677df0338 ip 00007f8676fa8e9d sp 00007fff1d2ec280 error 4 in libpcre.so.3.12.1[7f8676f96000+30000]." Does anyone know what this means? Thanks.

3 3

Question on syslog-ng filtering and performance - host vs. netmask
by jrhendri＠maine.rr.com 03 Dec '11

03 Dec '11

Hi, I am using syslog-ng to do log collection and "routing" to different destinations based on the origin of the log. I used to (still do in some cases) filter like this: filter f_one_list { host("10.0.0.1") or host("10.0.2.1") or ... host("10.3.4.5"); } Since (too) many log sources break the RFC and put all kinds of strangeness in the "header", I started using netmask instead: filter f_another_list { netmask("10.0.0.1/32") or netmask("10.0.2.1/32") or ... netmask("10.3.4.5/32"); } My question is this - Is there a measurable performance difference from one to another? "intuitively" it would seem doing the netmask check should be faster than parsing the syslog header, but I thought I'd ask here. (currently I have some servers processing 4000+ messages per second with no real problem, but the CPUs are clearly busy. BTW - I am using 3.2.4 at the moment (looking at moving to the multi-threaded newer versions in the future) Thanks! Jim Hendrick

3 2

syslog-ng log server stops listening on network
by Paul Muther 02 Dec '11

02 Dec '11

Hello all, I have a syslog-ng log server running 3.2.4 listening on a TCP port 4514. We are not a high volume environment (server processes on average between 10 and 20 MPS). Twice now the server has just stopped accepting traffic on the TCP port causing all the clients to store messages. There seems to be no explanation for the event and the local service is running and processing all local logs for the server no problem. I feel like this might be an IP port conflict but I just want to make sure I'm not missing anything. Thanks much, Paul M

2 1

syslog-ng 3.3.3 rewrite question regarding cisco IOS Messages
by Thomas Wollner 02 Dec '11

02 Dec '11

Hello List, I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message.. I found the following example on the list: ... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); }; log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); }; But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages. Sample log line (written with template $R_ISODATE $HOST $MSG): 2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down I`m using syslog-ng 3.3.3 debian package from madhouse. How I can rewrite my messages to filter the timestamp in the message field? any ideas? Any help is higly welcome, thanks in advance, Tom ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

4 18

patterndb skewing timestamps
by Clements, Frank 01 Dec '11

01 Dec '11

I've been working a bit over the last few weeks with patterndb - specifically correlating messages. I just noticed that when log messages are correlated, patterndb is skewing the timestamps from the first message (possibly others in between as well) by a few seconds. While, this may not seem like a big problem it results in logs that do not align property in history. A little on the configuration. I have two patterns to match on receiving the second I generate a new message with the format: logHost=${HOST} mapStart=${S_UNIXTIME}@2 mapStop=${S_UNIXTIME} protocol=${PROGRAM} insideAddr=${.dict.insideAddr}@1 insidePort=${.dict.insidePort}@1 outsideAddr=${.dict.outsideAddr}@1 outsidePort=${.dict.outsidePort}@1 destAddr=${.dict.destAddr}@2 destPort=${.dict.destPort}@2 My original log lines are as follows: Dec 1 08:39:41 AX2600 UC: e0a8636e:a16c->5f2c65b3:a16c to 8f2c77ca:a1 Dec 1 08:48:06 AX2600 UF: e0a8636e:a16c->5f2c65b3:a16c The generated log line is: logHost=RHOSTNAME mapStart=1322747067 mapStop=1322747286 protocol=U insideAddr=e0a8636e insidePort=a16c outsideAddr=5f2c65b3 outsidePort=a16c destAddr=80a8650c destPort=3f According to the docs S_ represents the log message time - so that means mapStart should line up, but it does not: 1322747067 == Thu Dec 01 2011 08:44:27. The stop message does not suffer from this issue. Anyone have a workaround for this? I tried using R_UNIXTIME@2 instead, but that is also skewed. -- Frank Clements

1 0

An interesting assert in 3.3.1
by Gergely Nagy 01 Dec '11

01 Dec '11

A while ago I noticed that my syslog-ng 3.3 segfaults every time I boot up my desktop machine at home (but then it gets restarted by the supervisor), and once its booted up, I can't reproduce the problem anymore. I assume it has something to do with the system being in a reasonably early state, or the load being particularly high. While I didn't have time to debug it further, I'd like to record the current state of investigation here: Core was generated by `/usr/sbin/syslog-ng -p /var/run/syslog-ng.pid --enable-core'. Program terminated with signal 6, Aborted. #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007fd927cc9680 in *__GI_abort () at abort.c:92 #2 0x00007fd928ae6e31 in g_assertion_message (domain=<optimized out>, file=<optimized out>, line=<optimized out>, func=0x7fd929609dc0 "log_writer_work_perform", message=0x7fd920007a60 "assertion failed: ((self->super.flags & PIF_INITIALIZED) != 0)") at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1358 #3 0x00007fd928ae73d0 in g_assertion_message_expr (domain=0x0, file=0x7fd929609acb "../../../lib/logwriter.c", line=128, func=0x7fd929609dc0 "log_writer_work_perform", expr=<optimized out>) at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1369 #4 0x00007fd9295d3570 in log_writer_work_perform (s=0x1d68080) at ../../../lib/logwriter.c:128 #5 0x00007fd9295d3897 in log_writer_io_flush_output (s=0x1d68080) at ../../../lib/logwriter.c:198 #6 0x00007fd9295ff986 in iv_run_active_list (active=0x7fff3da27700) at ../../../../../lib/ivykis/lib/iv_main.c:226 #7 0x00007fd9295ffb0d in iv_main () at ../../../../../lib/ivykis/lib/iv_main.c:269 #8 0x00007fd9295d8082 in main_loop_run () at ../../../lib/mainloop.c:722 #9 0x0000000000401a22 in main (argc=1, argv=0x7fff3da27878) at ../../../syslog-ng/main.c:260 (gdb) The particular writer is a program destination (sshguard), and at that point, it's probably not running yet. (For the record, I don't have threaded(yes) in my config at the moment). I'll look into this further, as time permits. -- |8]

1 1

ELSA Google Group
by Martin Holste 01 Dec '11

01 Dec '11

Just a quick note for those of you using or interested in ELSA, I created a Google Group for it at https://groups.google.com/group/enterprise-log-search-and-archive . Still feel free to email me directly for help, but I created the group so that the questions and answers might be visible to others searching on Google for help. Thanks, Martin

1 0

Bug in rewrite logic, 3.3.3
by Dave Rawks 30 Nov '11

30 Nov '11

I've got two different log paths defined that have the same rewrite included in them. The second log path always causes the rewrite to overwrite the incorrect macro value. > ######################## > # Destinations > ######################## > > destination dn_json { > tcp( "logstash" > port(10514) > so_keepalive(yes) > template("$(format_json --pair \@timestamp=\"$ISODATE\" message=\"$MESSAGE\" priority=\"$PRIORITY\" severity=\"$SEVERITY\" facility=\"$FACILITY\" program=\"$PROGRAM\" pid=\"$PID\" date=\"$R_DATE\" host=\"$HOST\" )\n") > ); > }; > > ######################## > #Rewrites > ######################## > > rewrite r_no_pid{ > set("0", value("PID") condition("$PID" =="")); > }; > > ######################## > # Log paths > # order matters if you use "flags(final);" to mark the end of processing in a > # "log" statement. ORDER MATTERS! > ####################### > > > Send all log events on to logstash > log { > source(s_remote); > rewrite(r_no_pid); > destination(dn_json); > }; > > log { > source(s_netloghost); > rewrite(r_no_pid); > destination(df_netloghost); > destination(dn_opennms); > destination(dn_json); > }; If the above configuration is used the events that pass through the second log path have their MESSAGE macro overwritten as "0" instead of their PID macro as is defined in the r_no_pid rewrite rule. The events that pass through the first logpath behave as expected. If I comment out the first logpath then the second behaves as expected. -Dave

2 1

distro URLs for syslog-ng
by Peter Czanik 29 Nov '11

29 Nov '11

Hello, Please take a look at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… It has now information for many Linux distributions, where up-to-date syslog-ng package information is available, unofficial repositories with latest and/or backported syslog-ng, bugreporting, etc. Let me know, if you have info or links to additional distributions or repositories! Bye, -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 147] New: missing share/xsd/patterndb-4.xsd schema definition
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=147 Summary: missing share/xsd/patterndb-4.xsd schema definition Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Is OSE 3.3.3 I downloaded the source and installed, but only go schema definitions for database formats 1,2 and 3. The manual for Syslog-ng 3.3.3 has a secion on "What's new in the syslog-ng pattern database format V4" so I can only assume that the version 4 schema should be present. Can you provide one? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 148] New: .classifier. unknown not set when patterndb unable to classify message
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=148 Summary: .classifier.unknown not set when patterndb unable to classify message Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 filter f_tag_unknown { tags(".classifier.unknown"); }; Using this filter results in no messages going to a destination, but using filter f_tag_unknown { not tags(".classifier.system"); }; logs all of the unclassified messages. The problem is that I am going to have a lot more than just .classifier.system in my pattern database, so this is not a viable workaround. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 145] New: Comilation on Solaris 10 with gcc 3.4.3 fails
by bugzilla＠bugzilla.balabit.com 29 Nov '11

29 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=145 Summary: Comilation on Solaris 10 with gcc 3.4.3 fails Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Solaris Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: andyh(a)nominet.org.uk Type of the Report: bug Estimated Hours: 0.0 Hi, Trying to compile 3.3.3 from source on Solaris 10 with gcc 3.4.4 and get this: libtool: compile: /usr/sfw/bin/gcc -DHAVE_CONFIG_H -I. -I.. -D_GNU_SOURCE -I../lib/include -I../lib/include -I../lib -I../modules/include -Wall -m64 -mcpu=v9 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event.lo -MD -MP -MF .deps/iv_event.Tpo -c iv_event.c -fPIC -DPIC -o iv_event.o iv_event.c: In function `iv_event_run_pending_events': iv_event.c:74: error: unrecognizable insn: (insn 80 79 81 2 (set (reg:DI 139) (lo_sum:DI (reg:DI 139) (unspec:DI [ (symbol_ref:DI ("__tls") [flags 0xa] <var_decl fefbb480 __tls>) ] 0))) -1 (nil) (nil)) iv_event.c:74: internal compiler error: in extract_insn, at recog.c:2083 Please submit a full bug report, with preprocessed source if appropriate. See <URL:http://gcc.gnu.org/bugs.html> for instructions. It appears (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21613) that there may be a workaround, but I'm not a developer and can't understand hwo to apply the workaround -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Problems with custom patterndb
by Clements, Frank 29 Nov '11

29 Nov '11

Hello Everyone! I've been trying to get some custom patterns put together to do some log correlation and I'm having one hell of a time getting a working pattern. I think I need a second, third, forth set of eyes on this ... Any help is appreciated! Message: Nov 25 12:02:27 GENERATED NAT-UDP-C: 192.168.107.132:12260 -> 207.44.101.104:12260 to 48.70.67.223:940 Pattern: @IPvANY:.dict.insideAddr@:@NUMBER:.dict.insidePort@ -> @IPvANY:.dict.outsideAddr@:@NUMBER:.dict.outsidePort@ to @IPvANY:.dict.destAddr@:@NUMBER:.dict.destPort@ I've looked at a few examples from the community patterns, but nothing in this stands out as being "wrong". Thanks - Frank W Clements

2 4