- syslog-ng - lists.balabit.hu

Re: [syslog-ng] Encoding provlem when writing to postgres
by ml＠bortal.de 24 Oct '08

24 Oct '08

Hi, > The nightly snapshot went to the 2.1 directory, but now I've changed > that. Tomorrow the 3.0 sources should appear in the nightly snapshot > directory: > > http://www.balabit.com/downloads/files/syslog-ng/sources/3.0/src-snapshot i am getting the following error: -------------------------------------------- make[1]: Leaving directory `/usr/local/src/syslog-ng-2.9.0+20081017' syslog-db:/usr/local/src/syslog-ng-2.9.0+20081017# make make all-recursive make[1]: Entering directory `/usr/local/src/syslog-ng-2.9.0+20081017' Making all in src make[2]: Entering directory `/usr/local/src/syslog-ng-2.9.0+20081017/src' if gcc -DHAVE_CONFIG_H -I. -I. -I.. -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/local/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT afsql.o -MD -MP -MF ".deps/afsql.Tpo" -c -o afsql.o afsql.c; \ then mv -f ".deps/afsql.Tpo" ".deps/afsql.Po"; else rm -f ".deps/afsql.Tpo"; exit 1; fi afsql.c: In function afsql_dd_init: afsql.c:713: error: server_mode undeclared (first use in this function) afsql.c:713: error: (Each undeclared identifier is reported only once afsql.c:713: error: for each function it appears in.) afsql.c:738: error: too many arguments to function log_queue_new afsql.c:741: warning: implicit declaration of function log_queue_restore_queue afsql.c:746: warning: implicit declaration of function log_queue_get_filename afsql.c:746: warning: passing argument 2 of evt_tag_str makes pointer from integer without a cast afsql.c: In function afsql_dd_deinit: afsql.c:849: warning: implicit declaration of function log_queue_save_queue make[2]: *** [afsql.o] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-2.9.0+20081017/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/syslog-ng-2.9.0+20081017' make: *** [all] Error 2 Any hint? Thanks, Mario

2 2

Who use syslog-ng for Solaris 10
by Shlayen, Yakov (CAT) 23 Oct '08

23 Oct '08

Hello, Please, let me know who use syslog-ng2.1.1OSE for Solaris 10 and reside in Atlanta or east coast of US. I'll be very appreciate if somebody contact with me. Best regards, Yakov (Jake) Shlayen Unix System Administrator Cox AutoTrader/AutoMart 5909 Peachtree Dunwoody Road Atlanta, GA 30328 Email:yakov.shlayen@autotraderpub.com Phone: 678-645-8227 Mobile: 678-481-3219

1 0

Re: [syslog-ng] UPDATE: sysutils/syslog-ng
by Brian A. Seklecki 17 Oct '08

17 Oct '08

Damnit! I've been waiting for my 4.4 CDs since July/August and you beat me to the update by 8 days. :) I did mine as a new port, based on the old one, as some people may want to run the 1.6.x bitrot version. http://people.collaborativefusion.com/~seklecki/openbsd44_port_syslog-ng209… # syslog-ng -V syslog-ng 2.0.9 # uname -a OpenBSD cauldron-obsd-44stable-i386 4.4 GENERIC.MP#844 i386 Lets get this bad boy tested and updated before 4.5 is released! ~BAS # tar cvf openbsd44_port_syslogng209_etlog027.tar eventlog/ syslog-ng2/ eventlog eventlog/Makefile eventlog/distinfo eventlog/pkg eventlog/pkg/DESCR eventlog/pkg/PLIST eventlog/pkg/PFRAG.shared syslog-ng2 syslog-ng2/CVS syslog-ng2/CVS/Repository syslog-ng2/CVS/Entries syslog-ng2/CVS/Tag syslog-ng2/pkg syslog-ng2/pkg/CVS syslog-ng2/pkg/CVS/Repository syslog-ng2/pkg/CVS/Entries syslog-ng2/pkg/CVS/Tag syslog-ng2/pkg/DESCR syslog-ng2/pkg/PLIST syslog-ng2/pkg/PLIST.orig syslog-ng2/pkg/MESSAGE syslog-ng2/Makefile syslog-ng2/distinfo syslog-ng2/files syslog-ng2/files/syslog-ng.conf syslog-ng2/patches syslog-ng2/patches/patch-tests_loggen_loggen_c # tar tzvf /usr/ports/packages/i386/all/syslog-ng-2.0.9.tgz -r--r--r-- 1 root wheel 721 Oct 16 11:46 +CONTENTS -r--r--r-- 1 root wheel 568 Oct 16 11:46 +DESC -r--r--r-- 1 root wheel 216 Oct 16 11:46 +DISPLAY -r--r--r-- 1 root bin 5899 Oct 16 11:46 man/man5/syslog-ng.conf.5 -r--r--r-- 1 root bin 3268 Oct 16 11:46 man/man8/syslog-ng.8 -r-xr-xr-x 1 root bin 1277148 Oct 16 11:46 sbin/syslog-ng -r--r--r-- 1 root bin 8837 Oct 16 11:46 share/examples/syslog-ng.conf -r-xr-xr-x 1 root bin 10948 Oct 16 11:46 bin/loggen # make install ===> Verifying specs: glib-2.0 c glib-2.0 ===> found glib-2.0.1600.2 c.48.0 ===> Installing syslog-ng-2.0.9 from /usr/ports/packages/i386/all/ syslog-ng-2.0.9: complete --- syslog-ng-2.0.9 ------------------- Add the following to rc.local(8): if [ -x /usr/local/sbin/syslog-ng -a -f /etc/syslog-ng.conf ]; then /usr/local/sbin/syslog-ng echo "syslog-ng2" fi Thanks for shopping balabit.hu and company. On Wed, 2008-10-08 at 21:34 +0200, Steven Mestdagh wrote: > this requires the new sysutils/eventlog port sent earlier. > > please test/comment/ok. > > Index: Makefile > =================================================================== > RCS file: /cvs/ports/sysutils/syslog-ng/Makefile,v > retrieving revision 1.8 > diff -u -r1.8 Makefile > --- Makefile 6 Oct 2008 16:09:13 -0000 1.8 > +++ Makefile 8 Oct 2008 19:29:28 -0000 > @@ -2,50 +2,39 @@ > > COMMENT= syslogd replacement > > +DISTNAME= syslog-ng-2.1.1 > CATEGORIES= sysutils > > -DISTNAME= syslog-ng-1.6.8 > -LIBOL= libol-0.3.16 > +MAINTAINER= Steven Mestdagh <steven(a)openbsd.org> > > -MAINTAINER= Jakob Schlyter <jakob(a)openbsd.org> > +HOMEPAGE = http://www.balabit.com/network-security/syslog-ng/ > > -MASTER_SITE_SYSLOGNG=\ > - http://www.balabit.com/downloads/files/syslog-ng/sources/ > +# GPL v2 > +PERMIT_PACKAGE_CDROM = Yes > +PERMIT_PACKAGE_FTP = Yes > +PERMIT_DISTFILES_CDROM = Yes > +PERMIT_DISTFILES_FTP = Yes > > -MASTER_SITES= ${MASTER_SITE_SYSLOGNG:=1.6/src/} > -MASTER_SITES0= ${MASTER_SITE_SYSLOGNG:=libol/0.3/} > +MASTER_SITES = http://www.balabit.com/downloads/files/syslog-ng/sources/2.1/src/ > > -HOMEPAGE= http://www.balabit.com/products/syslog_ng/ > +WANTLIB = c iconv intl wrap > > -DISTFILES= ${DISTNAME}.tar.gz \ > - ${LIBOL}.tar.gz:0 > +LIB_DEPENDS = glib-2.0::devel/glib2 \ > + evtlog::sysutils/eventlog > > -# GPL > -PERMIT_PACKAGE_CDROM= Yes > -PERMIT_PACKAGE_FTP= Yes > -PERMIT_DISTFILES_CDROM= Yes > -PERMIT_DISTFILES_FTP= Yes > -WANTLIB= c > +CONFIGURE_STYLE = gnu > +CONFIGURE_ARGS += --enable-tcp-wrapper > > -CONFIGURE_STYLE= gnu > -CONFIGURE_ARGS+= --with-libol=${WRKDIR}/${LIBOL} \ > - --enable-tcp-wrapper > +DOC = ${PREFIX}/share/doc/syslog-ng/ > +EXAMPLES = ${PREFIX}/share/examples/syslog-ng/ > > -DOC= ${PREFIX}/share/doc/syslog-ng > -EXAMPLES= ${PREFIX}/share/examples/syslog-ng > - > -pre-configure: > - cp -f ${PORTSDIR}/infrastructure/db/config.guess \ > - ${PORTSDIR}/infrastructure/db/config.sub ${WRKDIR}/${LIBOL} > - cd ${WRKDIR}/${LIBOL}; ./configure ; ${MAKE} > +post-extract: > + tar -C ${WRKBUILD} -xzf ${WRKSRC}/doc/reference/syslog-ng.html.tar.gz > > post-install: > ${INSTALL_DATA_DIR} ${DOC} > - ${INSTALL_DATA} ${WRKSRC}/doc/sgml/*.{ps,sgml,txt} ${DOC} > - (cd ${DOC} ;\ > - tar xzf ${WRKSRC}/doc/sgml/syslog-ng.html.tar.gz ;\ > - ln -s book1.html syslog-ng.html/index.html ) > + ${INSTALL_DATA} ${WRKBUILD}/syslog-ng.html/index.html ${DOC} > ${INSTALL_DATA_DIR} ${EXAMPLES} > - ${INSTALL_DATA} ${WRKSRC}/doc/syslog-ng.conf.sample ${EXAMPLES} > + ${INSTALL_DATA} ${WRKSRC}/doc/examples/syslog-ng.conf.sample ${EXAMPLES} > > .include <bsd.port.mk> > Index: distinfo > =================================================================== > RCS file: /cvs/ports/sysutils/syslog-ng/distinfo,v > retrieving revision 1.7 > diff -u -r1.7 distinfo > --- distinfo 5 Apr 2007 17:26:13 -0000 1.7 > +++ distinfo 8 Oct 2008 19:29:28 -0000 > @@ -1,10 +1,5 @@ > -MD5 (libol-0.3.16.tar.gz) = Hym+P0vN21svPZZeePBABg== > -MD5 (syslog-ng-1.6.8.tar.gz) = /7rX6ObcvjhYILj/uiO2Ig== > -RMD160 (libol-0.3.16.tar.gz) = ts8Hi0sH/EyHm/SicZpto+lvT8Y= > -RMD160 (syslog-ng-1.6.8.tar.gz) = EcZ5miRCaYao/ZdDxjQH4ums+mc= > -SHA1 (libol-0.3.16.tar.gz) = bW4e+U2Lbdnd2OkkUMXky54wGIo= > -SHA1 (syslog-ng-1.6.8.tar.gz) = cIEq8c7jIXlwlcryoK32CvyBpwA= > -SHA256 (libol-0.3.16.tar.gz) = aL69o59D/V+hO0ARqRxAsmhP4mKvKkCeKC99mn0o7J4= > -SHA256 (syslog-ng-1.6.8.tar.gz) = PIQf2JWZ/7dwzfKERCaYDXXcPasS4PcH5Mu1GTf2El4= > -SIZE (libol-0.3.16.tar.gz) = 345231 > -SIZE (syslog-ng-1.6.8.tar.gz) = 383589 > +MD5 (syslog-ng-2.1.1.tar.gz) = f/7jARSKZ41cGLGRDLe+HQ== > +RMD160 (syslog-ng-2.1.1.tar.gz) = H/iIJPTV1D3CxjmK6Am8tKas8As= > +SHA1 (syslog-ng-2.1.1.tar.gz) = EeX7CzsrnxzgJlGXVVCcMkz0orQ= > +SHA256 (syslog-ng-2.1.1.tar.gz) = is0YN/wS8OZfns/7jFU794OjUGKKzKPCkyvy9FCfpIU= > +SIZE (syslog-ng-2.1.1.tar.gz) = 420883 > Index: patches/patch-Makefile_in > =================================================================== > RCS file: patches/patch-Makefile_in > diff -N patches/patch-Makefile_in > --- /dev/null 1 Jan 1970 00:00:00 -0000 > +++ patches/patch-Makefile_in 8 Oct 2008 19:29:28 -0000 > @@ -0,0 +1,12 @@ > +$OpenBSD$ > +--- Makefile.in.orig Mon Oct 6 19:27:00 2008 > ++++ Makefile.in Mon Oct 6 19:27:05 2008 > +@@ -132,7 +132,7 @@ sbindir = @sbindir@ > + sharedstatedir = @sharedstatedir@ > + sysconfdir = @sysconfdir@ > + target_alias = @target_alias@ > +-SUBDIRS = src tests doc contrib solbuild tgzbuild > ++SUBDIRS = src doc contrib solbuild tgzbuild > + > + EXTRA_DIST = debian/README.Debian debian/copyright debian/syslog-ng.conf.example debian/syslog-ng.files debian/syslog-ng.logrotate debian/syslog-ng.postrm \ > + debian/changelog debian/changelog.in debian/rules debian/syslog-ng.default debian/syslog-ng.init debian/syslog-ng.logrotate.example debian/syslog-ng.preinst \ > Index: patches/patch-src_Makefile_in > =================================================================== > RCS file: patches/patch-src_Makefile_in > diff -N patches/patch-src_Makefile_in > --- patches/patch-src_Makefile_in 19 Jun 2004 22:16:23 -0000 1.1 > +++ /dev/null 1 Jan 1970 00:00:00 -0000 > @@ -1,12 +0,0 @@ > -$OpenBSD: patch-src_Makefile_in,v 1.1 2004/06/19 22:16:23 jakob Exp $ > ---- src/Makefile.in.orig 2004-05-14 16:43:43.000000000 +0200 > -+++ src/Makefile.in 2004-06-19 23:48:58.000000000 +0200 > -@@ -73,7 +73,7 @@ YACC = @YACC@ > - YFLAGS = @YFLAGS@ > - MAKE_CLASS = @MAKE_CLASS@ > - > --SUBDIRS = . tests > -+SUBDIRS = . > - > - dist_h_files = syslog-ng.h cfgfile.h sources.h center.h filters.h destinations.h log.h afinter.h affile.h afsocket.h afuser.h afstreams.h afprogram.h afremctrl.h utils.h syslog-names.h nscache.h macros.h > - > Index: patches/patch-src_macros_c > =================================================================== > RCS file: patches/patch-src_macros_c > diff -N patches/patch-src_macros_c > --- patches/patch-src_macros_c 19 Jun 2004 22:16:23 -0000 1.1 > +++ /dev/null 1 Jan 1970 00:00:00 -0000 > @@ -1,15 +0,0 @@ > -$OpenBSD: patch-src_macros_c,v 1.1 2004/06/19 22:16:23 jakob Exp $ > ---- src/macros.c.orig 2004-06-19 23:54:01.000000000 +0200 > -+++ src/macros.c 2004-06-20 00:00:25.000000000 +0200 > -@@ -115,7 +115,10 @@ format_tzofs(char *dest, size_t left) > - (timezone < 0 ? -timezone : timezone) / 3600, > - (timezone % 3600) / 60); > - #else > -- length = strftime(dest, left -1, "%z", tm); > -+ time_t unixtime; > -+ tzset(); > -+ time(&unixtime); > -+ length = strftime(dest, left -1, "%z", localtime(&unixtime)); > - #endif > - return length; > - } > Index: pkg/PLIST > =================================================================== > RCS file: /cvs/ports/sysutils/syslog-ng/pkg/PLIST,v > retrieving revision 1.5 > diff -u -r1.5 PLIST > --- pkg/PLIST 19 Sep 2005 13:35:17 -0000 1.5 > +++ pkg/PLIST 8 Oct 2008 19:29:28 -0000 > @@ -1,27 +1,9 @@ > @comment $OpenBSD: PLIST,v 1.5 2005/09/19 13:35:17 jakob Exp $ > @man man/man5/syslog-ng.conf.5 > @man man/man8/syslog-ng.8 > -sbin/syslog-ng > +@bin sbin/syslog-ng > share/doc/syslog-ng/ > -share/doc/syslog-ng/syslog-ng.html/ > -share/doc/syslog-ng/syslog-ng.html/book1.html > -share/doc/syslog-ng/syslog-ng.html/destinations.html > -share/doc/syslog-ng/syslog-ng.html/index.html > -share/doc/syslog-ng/syslog-ng.html/intro.html > -share/doc/syslog-ng/syslog-ng.html/logpath.html > -share/doc/syslog-ng/syslog-ng.html/msgroute.html > -share/doc/syslog-ng/syslog-ng.html/reference.html > -share/doc/syslog-ng/syslog-ng.html/tuning.html > -share/doc/syslog-ng/syslog-ng.html/x172.html > -share/doc/syslog-ng/syslog-ng.html/x361.html > -share/doc/syslog-ng/syslog-ng.html/x728.html > -share/doc/syslog-ng/syslog-ng.html/x768.html > -share/doc/syslog-ng/syslog-ng.html/x900.html > -share/doc/syslog-ng/syslog-ng.html/x906.html > -share/doc/syslog-ng/syslog-ng.html/x97.html > -share/doc/syslog-ng/syslog-ng.ps > -share/doc/syslog-ng/syslog-ng.sgml > -share/doc/syslog-ng/syslog-ng.txt > +share/doc/syslog-ng/index.html > share/examples/syslog-ng/ > @sample ${SYSCONFDIR}/syslog-ng/ > share/examples/syslog-ng/syslog-ng.conf.sample > -- Brian A. Seklecki <bseklecki(a)collaborativefusion.com> Collaborative Fusion, Inc.

2 1

Re: [syslog-ng] wip/syslog-ng2: 2.0.0 -> 2.0.9
by Brian A. Seklecki 16 Oct '08

16 Oct '08

Actually here's the updated distinfo from pkglint(8). Also should have included the syslog-ng@ crew to let them know about the update to pkgsrc-wip.sf.net ~BAS ----------- # more distinfo $NetBSD$ SHA1 (syslog-ng-2.0.9.tar.gz) = f7ea7bf6640d8ae2d745d70a883a7aa37f5bcd9d RMD160 (syslog-ng-2.0.9.tar.gz) = a937b39c7592e57533aa9f164127bef73c4dec0e Size (syslog-ng-2.0.9.tar.gz) = 378779 bytes SHA1 (patch-aa) = e69031ddeb20cec2c43edd8b03ec1cefc26a68c4 On Sat, 11 Oct 2008, Brian A. Seklecki wrote: > > Seems okay: > > /var/log/messages: > 2008 Oct 11 20:35:13 -04:00 metrotitan [syslog-ng][17211] [syslog] > [notice] syslog-ng[17211]: syslog-ng starting up; version='2.0.9' > > There's just one small bug which Bazsi says is fixed in Trunk/-rHEAD: > > https://lists.balabit.hu/pipermail/syslog-ng/2008-July/011880.html > (small patch at end of msg) > > Diff: > > root@metrotitan:/usr/pkgsrc/wip/syslog-ng2# cvs diff -u > cvs diff: Diffing . > Index: Makefile > =================================================================== > RCS file: /cvsroot/pkgsrc-wip/wip/syslog-ng2/Makefile,v > retrieving revision 1.3 > diff -u -r1.3 Makefile > --- Makefile 26 May 2008 13:59:47 -0000 1.3 > +++ Makefile 12 Oct 2008 00:36:04 -0000 > @@ -1,7 +1,7 @@ > # $NetBSD$ > # > > -PKGVERSION= 2.0.0 > +PKGVERSION= 2.0.9 > DISTNAME= syslog-ng-${PKGVERSION} > PKGNAME= syslog-ng2-${PKGVERSION} > > Index: distinfo > =================================================================== > RCS file: /cvsroot/pkgsrc-wip/wip/syslog-ng2/distinfo,v > retrieving revision 1.1.1.1 > diff -u -r1.1.1.1 distinfo > --- distinfo 30 Jun 2007 11:04:50 -0000 1.1.1.1 > +++ distinfo 12 Oct 2008 00:36:04 -0000 > @@ -1,5 +1,5 @@ > $NetBSD$ > > -SHA1 (syslog-ng-2.0.0.tar.gz) = 11a00e49f51056873d9889b8f2a62733b71bd040 > -RMD160 (syslog-ng-2.0.0.tar.gz) = ed2c3ff8eea834c0c2fdae03ce2f9068a6d24128 > -Size (syslog-ng-2.0.0.tar.gz) = 337553 bytes > +SHA1 (syslog-ng-2.0.9.tar.gz) = f7ea7bf6640d8ae2d745d70a883a7aa37f5bcd9d > +RMD160 (syslog-ng-2.0.9.tar.gz) = a937b39c7592e57533aa9f164127bef73c4dec0e > +Size (syslog-ng-2.0.9.tar.gz) = 378779 bytes > > > --------------------------------- > > $ more patch-aa > --- tests/loggen/loggen.c.orig 2008-10-11 20:42:13.000000000 -0400 > +++ tests/loggen/loggen.c 2008-10-11 20:42:36.000000000 -0400 > @@ -22,6 +22,8 @@ > > #define MAX_MESSAGE_LENGTH 8192 > > +#define AI_ADDRCONFIG 0x00000020 > + > #define USEC_PER_SEC 10e6 > #define MIN(a, b) ((a) < (b) ? (a) : (b)) > > > > l8*

1 1

Re: [syslog-ng] Encoding provlem when writing to postgres
by ml＠bortal.de 16 Oct '08

16 Oct '08

Hi Bazsi, > > where can i get syslog-ng 3.0? > > > > I cant find it on > > http://www.balabit.com/downloads/files/syslog-ng/sources/ > > I've just sent out an announcement that it's available as a git > repository. > > No formal releases have been made yet. I don't need a formal release announcment :) A snapshot or svn/cvs would be great. I just need that enconding support stuff :) Thanks, Mario

2 1

Re: [syslog-ng] Destination program behavior
by Gaetan Lord 16 Oct '08

16 Oct '08

>> My script is meant to run once and exit. Could it be the problem. > >Yes absolutely. You cannot exit after each log message, that is not permitted and explains why you are getting the SIGPIPE errors. You have to run in a loop, reading and processing >messages from stdin until you get an eof. See old postings to this list. Here's a pointer to one that gives an outline bash script that you can build upon. > >https://lists.balabit.hu/pipermail/syslog-ng/2008-March/011512.html Thank for the information. The script didn't work at first, at least under SuSE 10 with the native syslog-ng version 1.6.8 I compile and tried with version 2.0.9, and it work. :) I do have one more question for you. For now, I cannot replace the current syslog-ng V1.6.8. Can I run a second instance of syslog-ng, in this case version 2.0.9. The second one will have a different configuration file, and basically only run my destination program rule. Thanks again -- Helen Hayes - "Age is not important unless you're a cheese."

1 0

About Syslog-NG performance
by Yu Watanabe 15 Oct '08

15 Oct '08

Hello all, I am planning to build a syslog server using syslog-ng v 2.0.5. However, I am very concerned with the how much performace could syslog-ng can handle. Following is the system specification I am using. OS : RHEL 4 or RHEL 5 CPU : Xeon(R) E5205 (1.86GHz)×2 Memory : 4 GB I would like get an advice with the performace of the syslog-ng. Does any one has a testimonial with this? Yu Watanabe

2 2

Re: [syslog-ng] Encoding provlem when writing to postg res
by Paul Robert Marino 15 Oct '08

15 Oct '08

I've seen this with other web based apps. It usualy happens when tere is a control character in the text unfortunitly most text editors will simply ignore the character so it makes it hard to diagnose. If this is only happening with email related messages (which is not un-common) you may need to handle it at your mail server. Specificlty in the perl script that apears to be generating the message. Here is a regex I've used in the past in webforms where people would often copy and past fields from outlook that helped with the same error s/\P{IsPrint}//g this regex will strip out any characters that are not unicode printable. -----Original Message----- From: ml(a)bortal.de Subj: [syslog-ng] Encoding provlem when writing to postg res Date: Wed Oct 15, 2008 3:53 am Size: 1K To: syslog-ng(a)lists.balabit.hu Hello List, i get the following error when i try to write to postres: Error running SQL query; type='pgsql', host='XXX', port='', user='XXX', database='syslogng', error='ERROR: invalid byte sequence for encoding "UTF8": 0xe96461\x0aHINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".\x0a', query='INSERT INTO msg_table (msg_rcv_time, msg_sent_time, hostname, msg_facility, msg_priority, msg_text) VALUES (\'2008-10-14T19:59:46+02:00\', \'2008-10-14T19:59:46+02:00\', \'foo\', \'mail\', \'warning\', \'mimedefang.pl[3497]: Message contains more than one Subject: header: Bulletin d\\\'\'info d\\\'\'Enseigner.TV et la fiche p\xffdagogique de l\\\'\'\xffmission du mois \xff 14 octobre 208 --> Bulletin d\\\'\'info d\\\'\'Enseigner.TV et la fiche p\xffdagogique de l\\\'\'\xffmission du mois \xff 14 octobre 208\')' How can i fix this? I am running postgresql-8.1 with Debian 4.0. Thanks, Mario ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

Encoding provlem when writing to postgres
by ml＠bortal.de 15 Oct '08

15 Oct '08

Hello List, i get the following error when i try to write to postres: Error running SQL query; type='pgsql', host='XXX', port='', user='XXX', database='syslogng', error='ERROR: invalid byte sequence for encoding "UTF8": 0xe96461\x0aHINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".\x0a', query='INSERT INTO msg_table (msg_rcv_time, msg_sent_time, hostname, msg_facility, msg_priority, msg_text) VALUES (\'2008-10-14T19:59:46+02:00\', \'2008-10-14T19:59:46+02:00\', \'foo\', \'mail\', \'warning\', \'mimedefang.pl[3497]: Message contains more than one Subject: header: Bulletin d\\\'\'info d\\\'\'Enseigner.TV et la fiche p\xffdagogique de l\\\'\'\xffmission du mois \xff 14 octobre 208 --> Bulletin d\\\'\'info d\\\'\'Enseigner.TV et la fiche p\xffdagogique de l\\\'\'\xffmission du mois \xff 14 octobre 208\')' How can i fix this? I am running postgresql-8.1 with Debian 4.0. Thanks, Mario

2 1

syslog-ng 3.0 OSE and PE Release Date?
by Christian Lammers 14 Oct '08

14 Oct '08

Hello, I've compiled the 3.0 OSE successful and get it working. But also, it would be nice to test the Premium Edition in Version 3.0. I've read on the homepage, that it should be released in Q2 2008? I'm currently writing my bachelor thesis about "Audit proof logging", and in my opinion the syslog-ng 3.0 is the only professional tool to conform compliance requirements. Maybe, there is some Beta Version available? Thanks in advance, -- ------------------------------------------------------------------------ Christian Lammers Thinking Objects GmbH Tel. +49 71188770-301 Lilienthalstraße 2/1 Fax +49 711 88770-449 70825 Korntal/Stuttgart christian.lammers(a)to.com http://www.to.com Geschäftsführer: Markus Klingspor Sitz und Amtsgericht Stuttgart, HRB 19769 ------------------------------------------------------------------------

2 1

syslog-ng Agent, 3.0.3 release schedule (hatarido: 2008.10.22)
by Balazs Scheidler 14 Oct '08

14 Oct '08

hali, Rogtonzott megbeszelesunkon a kovetkezokben maradtunk: Memo: ----- * Zolinak szuksege van 1 napra egyedul, hogy az Agent olyan allapotba jusson, hogy a tobbi tesztelot is ra lehessen engedni (most nem kuld egyaltalan XP-n) * Szukseges funkcionalis tesztek: * 32/64 bit tesztek, telepitovel, kulonbozo kombinaciok, 2 nap * Vista funkcionalis teszt: 2nap * Hibajegybol most kb 20 van, amit tesztelni kell, feltetelezzuk, hogy a funkcionalis tesztekbol tovabbi 20 esik ki, igy 40 hibajegy tesztelesevel szamolunk, ami sacc/kb 6 embernap, ami viszont parhuzamosithato * Ghost meg csinalja az egy-ket nyitott hibajegyet, ha barmi a tesztelest blokkolo dolog van, akkor azt oldja meg inkabb * Abban maradtunk, hogy PZolin kivul MAndras es Lorant is syslog-ng agent-et fog tesztelni, Lorant alapvetoen holnaptol, MAndras matol * Szukseg van egy tesztkornyezetre, ld. todo TODO summary: ------------- * PZoli: ma elokesziti a tobbi tesztelest, a mindenkit blokkolo hibak kivasalasa * Ghost: ha van PZoli altal bejelentett blokkolo hiba, akkor az, ha nincs, akkor a meg nyitott hibajegyek * MAndras: * ma: tesztkornyezet osszerakasa, ami egy olyan Windows, ami: * minden audit esemeny bekapcsolva * IIS beallit, es fut, es logol * ISA beallit, es fut, es logol (ebben Scott/Ghost rendelkezhet infoval) * holnaptol: hibajegyteszt, esetleg szcenario teszt (PZoli megmondja) * Lorant: holnaptol hibajegyteszt, esetleg szcenraio teszt (PZoli megmondja) Abban maradtunk, hogy jovo het szerdara (okt. 22), az osszes uj funkcio letesztelve, hibajavitva lesz egy binarisunk, csak a release teszt lesz vissza. -- Bazsi

1 1

Benchmarking my syslog-ng infrastructure
by Francis Lacoste 14 Oct '08

14 Oct '08

Hello all, I am currently configuring my infrastructure to throw all logs to a dedicated syslog-ng server. I would like to benchmark it to know how many messages per seconds it can accept before it starts dropping. Is there any tools available ? Thanks. *Francis Lacoste*

2 1

FW: syslog-ng not working
by Ramprasad 13 Oct '08

13 Oct '08

Hi List, Do you have any suggestions for the below mentioned issue? Thanks & Regards, Ramprasad B ________________________________ From: Ramprasad Sent: Wednesday, October 08, 2008 12:43 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: syslog-ng not working Hi list, I have the following configuration in /etc/syslog-ng/syslog-ng.conf file on OpenSuSE 11.0 64 bit system. # cat /etc/syslog-ng/syslog-ng.conf options { long_hostnames(off); sync(0); perm(0640); stats(3600); }; source src { internal(); unix-stream("/dev/log"); unix-dgram("/dev/log"); }; destination d_filepath_10 { file("/tmp/syslog" fsync(yes)); }; filter f_filter_10 {level(debug .. emerg); }; log { source(src); filter(f_filter_10); destination(d_filepath_10); }; I don't see any log messaged being logged to /tmp/syslog file. Though, I restarted the syslog daemon, there is not effect. I went through some syslog-ng examples available on internet such as the one below, but still I don't see syslog-ng logging to the specified file - /tmp/syslog # cat /etc/syslog-ng/syslog-ng.conf options { keep_hostname(yes); long_hostnames(off); sync(1); log_fifo_size(1024); }; source src { unix-stream("/dev/log"); internal(); udp(); tcp(port(5140) keep-alive(yes)); }; destination syslog { file("/var/log/syslog"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/tmp/syslog"); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; log { source(src); filter(f_messages); destination(messages); }; Please help us on this. Thanks & Regards, Ramprasad B

1 0

Destination program behavior
by Gaetan Lord 13 Oct '08

13 Oct '08

Hi, I try to define a log parser. The idea is to monitor every lines for specific pattern and run action command. My config lines are destination syslog_parser { program("/usr/local/bin/test_syslog"); }; log { source(src); destination(syslog_parser); }; To test syslog-ng, I wrote a basic shell script, which only contain /bin/echo "This is a test" | /usr/bin/wall and exit This should issue a "wall" command on every lines written in the syslog I test it with the command "logger", and this do not work. I have the following syslog-ng error when I start it. Oct 10 13:04:00 minime1 syslog-ng[5470]: io.c: do_write: write() failed (errno 32), Broken pipe Oct 10 13:04:00 minime1 syslog-ng[5470]: pkt_buffer::do_flush(): Error flushing data I'm currently running version 1.6 from SLES10, syslog-ng-1.6.8-20.18 for IA64 I red the documentation provided by the package, and saw the following NOTE: the program is executed once at startup, and kept running until SIGHUP or exit. The reason is to prevent starting up a large number of programs for messages, which would imply an easy DoS. My script is meant to run once and exit. Could it be the problem. Is there any way to achieve what I want. I'm aware about the DoS, but in this case, there it's not a problem. Thank you -- Spike Milligan - "All I ask is the chance to prove that money can't make me happy."

2 1

Does syslog-ng support triggers?
by Luís Miguel Silva 11 Oct '08

11 Oct '08

Hello everyone, I was wondering if syslog-ng supports triggers (based on keywords OR time events). Ie: i want to be able to call an application if syslog-ng detects the same log message came from the same host x times in y minutes. We are suffering a lot of attacks against our webmail servers and would like to use this to try and trigger an alarm against brute force connections. Thanks in advance, -- Luís Miguel Ferreira da Silva Qualidade e Segurança CICA - FEUP GSM: +351 912671471

3 3

Re: [syslog-ng] Log to postgres
by ml＠bortal.de 09 Oct '08

09 Oct '08

Hello List, i got syslog-ng compiled and now i get those errors: (is there maybe a table dump for postgres i could use?) What do there errors mean? How can i debug them? -------------------------------------------------------------------- ~# syslog-ng -F -v -f /etc/syslog-ng.conf -e syslog-ng starting up; version='2.1.1' Database thread started; Syslog connection accepted; from='AF_INET(<some-ip>:51708)', to='AF_INET(<some-ip>:514)' Initializing destination file writer; template='/var/log/messages', filename='/var/log/messages' Syslog connection accepted; from='AF_INET(<some-ip>:54521)', to='AF_INET(<some-ip>:514)' Syslog connection accepted; from='AF_UNIX(anonymous)', to='AF_UNIX(/dev/log)' EOF occurred while reading; fd='26' Closing log reader fd; fd='26' Syslog connection accepted; from='AF_UNIX(anonymous)', to='AF_UNIX(/dev/log)' EOF occurred while reading; fd='26' Closing log reader fd; fd='26' Syslog connection accepted; from='AF_INET(<some-ip>:54887)', to='AF_INET(<some-ip>:514)' EOF occurred while reading; fd='26' Closing log reader fd; fd='26' Syslog connection accepted; from='AF_INET(<some-ip>:64789)', to='AF_INET(<some-ip>:514)' Syslog connection accepted; from='AF_UNIX(anonymous)', to='AF_UNIX(/dev/log)' Syslog connection accepted; from='AF_UNIX(anonymous)', to='AF_UNIX(/dev/log)' EOF occurred while reading; fd='27' Closing log reader fd; fd='27' EOF occurred while reading; fd='28' Closing log reader fd; fd='28' Syslog connection accepted; from='AF_INET(<some-ip>:4622)', to='AF_INET(<some-ip>:514)'

4 3

(no subject)
by n r 08 Oct '08

08 Oct '08

hi every body My syslog-ng server is on debian eth. I woudl like to know how can I push on my server: - my client /var/log/dkpg.log, /varl/log/samba/ "host".log, /var/log/samba/wtmp and utmp directory ? thanks nr _________________________________________________________________ Appelez vos amis de PC à PC -- C'EST GRATUIT http://get.live.com/messenger/overview

2 3

syslog-ng not working
by Ramprasad 08 Oct '08

08 Oct '08

Hi list, I have the following configuration in /etc/syslog-ng/syslog-ng.conf file on OpenSuSE 11.0 64 bit system. # cat /etc/syslog-ng/syslog-ng.conf options { long_hostnames(off); sync(0); perm(0640); stats(3600); }; source src { internal(); unix-stream("/dev/log"); unix-dgram("/dev/log"); }; destination d_filepath_10 { file("/tmp/syslog" fsync(yes)); }; filter f_filter_10 {level(debug .. emerg); }; log { source(src); filter(f_filter_10); destination(d_filepath_10); }; I don't see any log messaged being logged to /tmp/syslog file. Though, I restarted the syslog daemon, there is not effect. I went through some syslog-ng examples available on internet such as the one below, but still I don't see syslog-ng logging to the specified file - /tmp/syslog # cat /etc/syslog-ng/syslog-ng.conf options { keep_hostname(yes); long_hostnames(off); sync(1); log_fifo_size(1024); }; source src { unix-stream("/dev/log"); internal(); udp(); tcp(port(5140) keep-alive(yes)); }; destination syslog { file("/var/log/syslog"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/tmp/syslog"); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; log { source(src); filter(f_messages); destination(messages); }; Please help us on this. Thanks & Regards, Ramprasad B

1 0

Re: [syslog-ng] Reload does not free up RAM
by Evan Rempel 07 Oct '08

07 Oct '08

Balazs Scheidler wrote: > On Mon, 2008-10-06 at 11:38 -0700, Evan Rempel wrote: >> Balazs Scheidler wrote: >>> On Sun, 2008-10-05 at 13:49 -0700, Evan Rempel wrote: >>>> Balazs Scheidler wrote: >>>>>>>>>> We have cases where a syslog-ng buffers quite a bit of data (2GB or so) before >>>>>>>>>> it is able to flush the data to the destination. Currently, the only way to "free up" that RAM >>>>>>>>>> is to restart syslog-ng, which means that some log messages are lost during the restart >>>>>>>>>> window. >>>>> Both PE and OSE frees up disk/memory when possible. If they don't, then >>>>> that's probably a leak. >> No, it isn't a memory leak as the process can be months old and only grows when >> it buffers messages. >> >>>>> So if your syslog-ng process is 2GB in size, and you didn't specify an >>>>> overly large log_fifo_size(), it's probably because of a leak. >> >> >>>> I have a large log_fifo_size() but the process starts small, grows when it needs >>>> to buffer, and does NOT get small again when the destination is available. >>> Hmm... that might be related to the malloc implementation. Which >>> platform are you using? >>> >>>>>> 2. The buffer space should be thrown away only after the message >>>>>> has been delivered to the destination. >>>>> I see, this is what should happen now. >> >> >>>> This does not appear to be happening. >>>> I am using OSE 2.0.8 Have there been memory fixes/changes since then? >>> Not that I know of. >>> >>>> Is there a way to get buffer count statistics out? How many messages are >>>> in each fifo buffer? >>> In the experimental 3.0 branch, yes there is. In current 2.0, I'm afraid >>> there's none. >>> >>> Is there a limit on the growth of syslog-ng? I mean if it gets large >>> once and then empties its buffers, it stays large, but does it start >>> growing again, or it stays steadily at the size it was before? >> I have not specifically tested this scenario, but we do track processes >> and memory usage, and it appears that once the space is allocated, it does >> not grow unless a larger set of messages gets buffered. >> >> >>> Again, the platform matters a lot, how the local malloc implementation >>> takes care about fragmentation, whether it shrinks the process once >>> memory is freed. >> This is running on Redhat AS 3 (2.4 kernel) and Redhat AS 4 (2.6 kernel) >> >> On the Redhat 4 system the glibc packages installed are >> >> glibc-common-2.3.4-2.36 >> glibc-devel-2.3.4-2.36 >> glibc-headers-2.3.4-2.36 >> glib2-2.4.7-1 >> glibc-kernheaders-2.4-9.1.100.EL >> > > hmm.. this version should be all right, at least in my experience glibc > tries to release memory back to the system once the application freed it > using free(). > > Maybe it is some kind of memory fragmentation issue. How easy is this to > reproduce? Does it require months of running, or you have a simple > recipe to create it? We are encountering this fairly easily in production, so I should be able to verify a recipe. It will take me a couple of days to "get to it", but will report back. -- Evan Rempel erempel(a)uvic.ca Senior Programmer Analyst 250.721.7691 University Systems, University of Victoria

1 0

syslog-ng dropping packet
by Francis Lacoste 07 Oct '08

07 Oct '08

Hello all, I'm experiencing packet losses with syslog-ng, I know that there's some tweak that can be done to kernel to handle a bigger buffer... Anyone could tell me what can I tweak with sysctl so it will stop dropping ? Thanks. ------- Francis Lacoste

3 4

Re: [syslog-ng] Reload does not free up RAM
by Evan Rempel 07 Oct '08

07 Oct '08

Balazs Scheidler wrote: > On Sun, 2008-10-05 at 13:49 -0700, Evan Rempel wrote: >> Balazs Scheidler wrote: >>>>>>>> We have cases where a syslog-ng buffers quite a bit of data (2GB or so) before >>>>>>>> it is able to flush the data to the destination. Currently, the only way to "free up" that RAM >>>>>>>> is to restart syslog-ng, which means that some log messages are lost during the restart >>>>>>>> window. >>> Both PE and OSE frees up disk/memory when possible. If they don't, then >>> that's probably a leak. No, it isn't a memory leak as the process can be months old and only grows when it buffers messages. >>> >>> So if your syslog-ng process is 2GB in size, and you didn't specify an >>> overly large log_fifo_size(), it's probably because of a leak. >> >> I have a large log_fifo_size() but the process starts small, grows when it needs >> to buffer, and does NOT get small again when the destination is available. > > Hmm... that might be related to the malloc implementation. Which > platform are you using? > >>>> 2. The buffer space should be thrown away only after the message >>>> has been delivered to the destination. >>> I see, this is what should happen now. >> >> This does not appear to be happening. >> I am using OSE 2.0.8 Have there been memory fixes/changes since then? > > Not that I know of. > >> Is there a way to get buffer count statistics out? How many messages are >> in each fifo buffer? > > In the experimental 3.0 branch, yes there is. In current 2.0, I'm afraid > there's none. > > Is there a limit on the growth of syslog-ng? I mean if it gets large > once and then empties its buffers, it stays large, but does it start > growing again, or it stays steadily at the size it was before? I have not specifically tested this scenario, but we do track processes and memory usage, and it appears that once the space is allocated, it does not grow unless a larger set of messages gets buffered. > Again, the platform matters a lot, how the local malloc implementation > takes care about fragmentation, whether it shrinks the process once > memory is freed. This is running on Redhat AS 3 (2.4 kernel) and Redhat AS 4 (2.6 kernel) On the Redhat 4 system the glibc packages installed are glibc-common-2.3.4-2.36 glibc-devel-2.3.4-2.36 glibc-headers-2.3.4-2.36 glib2-2.4.7-1 glibc-kernheaders-2.4-9.1.100.EL > The problem is, that if the local malloc implementation is wrong the > only thing I could do is to try to use a different allocator (e.g. use > GNU malloc on your platform), but that's not a simple change. Understood. -- Evan Rempel

2 1

syslog-ng 2.1: upstream libdbi does NOT provide pkgconfig .pc file
by Wolfram Schlich 07 Oct '08

07 Oct '08

Hi, syslog-ng 2.1 depends on libdbi >= 0.8.0 with --enable-sql. Unfortunately, it assumes that a pkg-config .pc file for libdbi is in place which the upstream libdbi just does not include. It seems that there is a balabit-modified version of libdbi available (libdbi8_0.8.2bb2-2.tar.gz) which was patched to include a dbi-0.8.pc file. What's the reason for that? Was upstream asked to include that .pc file in their regular releases? What other changes were made to the upstream libdbi to form the balabit version? This is kinda maintenance and dependency hell so far :( Please clear up the situation so I can take care of building a Gentoo package for 2.1 *including* SQL support :) Thanks! -- Regards, Wolfram Schlich <wschlich(a)gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/

2 4

Regexp question
by Mártha Csaba 06 Oct '08

06 Oct '08

Hello I use this regexp: match ="FlashLog: ([0-9a-zA-Z]+)-([0-9a-zA-Z]+)-([0-9]{4})-([0-9]{2})- ([0-9]{2})-([0-9]{2})-([0-9]{2})-([0-9a-zA-Z]+)"); and this df_out: destination df_out { file("/var/log/client/flashlog/${1}/${8}_.txt"); }; I have /var/log/client/flashlog/_.txt file. What is the problem? Best Regards: csaba

1 0

Inline Compression
by Aaron Wiebe 06 Oct '08

06 Oct '08

Greetings, I've noticed in 1.6.x that there are tags already built into the configuration file to support inline compression of file output (but they complain if used). Is there any plan to support this? Are there reasons this hasn't been implemented yet? Given my knowledge of the libgzip, I can't imagine its terribly hard. The ability to write logs compressed would be -very- useful. -Aaron

2 3

Re: [syslog-ng] Reload does not free up RAM
by Evan Rempel 06 Oct '08

06 Oct '08

Balazs Scheidler wrote: >>>>>> We have cases where a syslog-ng buffers quite a bit of data (2GB or so) before >>>>>> it is able to flush the data to the destination. Currently, the only way to "free up" that RAM >>>>>> is to restart syslog-ng, which means that some log messages are lost during the restart >>>>>> window. > > Both PE and OSE frees up disk/memory when possible. If they don't, then > that's probably a leak. > > So if your syslog-ng process is 2GB in size, and you didn't specify an > overly large log_fifo_size(), it's probably because of a leak. I have a large log_fifo_size() but the process starts small, grows when it needs to buffer, and does NOT get small again when the destination is available. >> 2. The buffer space should be thrown away only after the message >> has been delivered to the destination. > > I see, this is what should happen now. This does not appear to be happening. I am using OSE 2.0.8 Have there been memory fixes/changes since then? Is there a way to get buffer count statistics out? How many messages are in each fifo buffer? Evan.

2 1