- syslog-ng - lists.balabit.hu

R: Re: I: Re: R: Re: Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

ameconi@ubuntu:/usr/local/sbin$ ./syslog-ng --version ./syslog-ng: error while loading shared libraries: libsyslog-ng-3.3.2.so: cannot open shared object file: No such file or directory ameconi@ubuntu:/usr/local/sbin$ >----Messaggio originale---- >Da: algernon(a)balabit.hu >Data: 16/11/2011 19.56 >A: "andrea.meconi(a)libero.it"<andrea.meconi(a)libero.it> >Cc: "Syslog-ng users' and developers' mailing list"<syslog-ng(a)lists.balabit. hu> >Ogg: Re: [syslog-ng] I: Re: R: Re: Compiling syslog-ng on Ubuntu. > >"andrea.meconi(a)libero.it" <andrea.meconi(a)libero.it> writes: > >>>I'm trying to compile 3.3.2. > >It's installed under /usr/local, not /usr. So you won't find it under >/usr/sbin, try /usr/local/sbin instead. > >-- >|8] > >

3 2

I: Re: R: Re: Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

>----Messaggio originale---- >Da: andrea.meconi(a)libero.it >Data: 16/11/2011 19.30 >A: <andrea.meconi(a)libero.it> >Ogg: R: Re: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. > >I'm trying to compile 3.3.2. >Andrea > >>----Messaggio originale---- >>Da: algernon(a)balabit.hu >>Data: 16/11/2011 17.17 >>A: "Syslog-ng users' and developers' mailing list"<syslog-ng(a)lists.balabit. >hu> >>Cc: "andrea.meconi(a)libero.it"<andrea.meconi(a)libero.it> >>Ogg: Re: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >>"andrea.meconi(a)libero.it" <andrea.meconi(a)libero.it> writes: >> >>> I'm using >>> >>> sudo ./configure --enable-spoof-source >>> >>> than make and make install. >>> Sorry, but I'm not able to see any errors! >> >>Could you post the full output of make install then? >> >>Issue the following command: >> sudo make install 2>&1 | tee make.log >> >>And attach the resulting make.log. >> >>Which version of syslog-ng are you trying to compile, by the way? >> >>-- >>|8] >> >> > >

2 1

R: RE: RE: RE: R: Re: Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

ameconi@ubuntu:/usr/sbin$ ./syslog-ng version -bash: ./syslog-ng: No such file or directory ameconi@ubuntu:/usr/sbin$ >----Messaggio originale---- >Da: james.lay(a)wincofoods.com >Data: 16/11/2011 19.39 >A: <andrea.meconi(a)libero.it> >Ogg: RE: RE: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. > >Ah good! Well...the .o files and that syslog-ng file that show below mean that it has compiled and it's there in the source at least. Now...let's try this....do a: > >"cd /usr/sbin/" > >And then: > >"./syslog-ng --version" > >What does that show you? > >James > >> -----Original Message----- >> From: andrea.meconi(a)libero.it [mailto:andrea.meconi@libero.it] >> Sent: Wednesday, November 16, 2011 11:37 AM >> To: Lay, James >> Subject: R: RE: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >> James, >> I'm trying to compile 3.3.2 >> Anyway, attached you can find config.log >> >> ameconi@ubuntu:/usr/local/src/syslog-ng-3.3.2/syslog-ng$ ls -l >> total 164 >> -rw-r--r-- 1 root root 7231 2011-09-23 12:09 main.c >> -rw-r--r-- 1 root root 20500 2011-11-15 17:55 main.o >> -rw-r--r-- 1 root root 22625 2011-11-15 17:49 Makefile >> -rw-r--r-- 1 root root 898 2011-10-26 15:54 Makefile.am >> -rw-r--r-- 1 root root 21875 2011-11-13 13:24 Makefile.in >> -rwxr-xr-x 1 root root 5691 2011-11-15 17:55 syslog-ng >> -rwxr-xr-x 1 root root 44336 2011-11-15 17:55 syslog-ng-ctl >> -rw-r--r-- 1 root root 6946 2011-06-10 10:30 syslog-ng-ctl.c >> -rw-r--r-- 1 root root 15528 2011-11-15 17:55 syslog-ng-ctl.o >> -rw-r--r-- 1 root root 1809 2011-02-24 09:14 wrapper.c >> ameconi@ubuntu:/usr/local/src/syslog-ng-3.3.2/syslog-ng$ >> >> Many many thanks. >> Andrea >> >> >----Messaggio originale---- >> >Da: james.lay(a)wincofoods.com >> >Data: 16/11/2011 19.25 >> >A: <andrea.meconi(a)libero.it> >> >Ogg: RE: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> > >> >Hrmm... >> > >> >In your syslog-ng-3.3.1 directory you should have a config.log file. Can you >> send that to me? Also, in that syslog-ng-3.3.1 directory there's a directory >> called syslog-ng. Go there and do a "ls"...should show something like this: >> > >> >Makefile Makefile.am Makefile.in main.c main.o syslog-ng* syslog-ng- >> ctl* syslog-ng-ctl.c syslog-ng-ctl.o wrapper.c >> > >> >What do you have in your syslog-ng directory? >> > >> >James >> > >> >> -----Original Message----- >> >> From: andrea.meconi(a)libero.it [mailto:andrea.meconi@libero.it] >> >> Sent: Wednesday, November 16, 2011 11:19 AM >> >> To: Lay, James >> >> Subject: R: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >> >> >> Yes James. >> >> First configure, than make and make install. >> >> Andrea >> >> >> >> >----Messaggio originale---- >> >> >Da: james.lay(a)wincofoods.com >> >> >Data: 16/11/2011 17.14 >> >> >A: <andrea.meconi(a)libero.it>, <syslog-ng(a)lists.balabit.hu> >> >> >Ogg: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >> > >> >> > >> >> > >> >> >> -----Original Message----- >> >> >> From: syslog-ng-bounces(a)lists.balabit.hu >> >> >[mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of >> >> >> andrea.meconi(a)libero.it >> >> >> Sent: Wednesday, November 16, 2011 9:08 AM >> >> >> To: syslog-ng(a)lists.balabit.hu >> >> >> Subject: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >> >> >> >> >> >> >> >> Hello James. >> >> >> I'm using >> >> >> >> >> >> sudo ./configure --enable-spoof-source >> >> >> >> >> >> than make and make install. >> >> >> Sorry, but I'm not able to see any errors! >> >> >> Andrea >> >> > >> >> > >> >> >Andrea, >> >> > >> >> >You are running sudo make install yes? I would recommend running >> >> >./configure --enable-spoof-source, then make, then sudo make install. >> >> > >> >> >James >> >> > >> >> >> > >> > > >

1 0

I: RE: RE: R: Re: Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

>----Messaggio originale---- >Da: andrea.meconi(a)libero.it >Data: 16/11/2011 19.36 >A: <james.lay(a)wincofoods.com> >Ogg: R: RE: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. > >James, >I'm trying to compile 3.3.2 >Anyway, attached you can find config.log > >ameconi@ubuntu:/usr/local/src/syslog-ng-3.3.2/syslog-ng$ ls -l >total 164 >-rw-r--r-- 1 root root 7231 2011-09-23 12:09 main.c >-rw-r--r-- 1 root root 20500 2011-11-15 17:55 main.o >-rw-r--r-- 1 root root 22625 2011-11-15 17:49 Makefile >-rw-r--r-- 1 root root 898 2011-10-26 15:54 Makefile.am >-rw-r--r-- 1 root root 21875 2011-11-13 13:24 Makefile.in >-rwxr-xr-x 1 root root 5691 2011-11-15 17:55 syslog-ng >-rwxr-xr-x 1 root root 44336 2011-11-15 17:55 syslog-ng-ctl >-rw-r--r-- 1 root root 6946 2011-06-10 10:30 syslog-ng-ctl.c >-rw-r--r-- 1 root root 15528 2011-11-15 17:55 syslog-ng-ctl.o >-rw-r--r-- 1 root root 1809 2011-02-24 09:14 wrapper.c >ameconi@ubuntu:/usr/local/src/syslog-ng-3.3.2/syslog-ng$ > >Many many thanks. >Andrea > >>----Messaggio originale---- >>Da: james.lay(a)wincofoods.com >>Data: 16/11/2011 19.25 >>A: <andrea.meconi(a)libero.it> >>Ogg: RE: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >> >>Hrmm... >> >>In your syslog-ng-3.3.1 directory you should have a config.log file. Can you >send that to me? Also, in that syslog-ng-3.3.1 directory there's a directory >called syslog-ng. Go there and do a "ls"...should show something like this: >> >>Makefile Makefile.am Makefile.in main.c main.o syslog-ng* syslog-ng- >ctl* syslog-ng-ctl.c syslog-ng-ctl.o wrapper.c >> >>What do you have in your syslog-ng directory? >> >>James >> >>> -----Original Message----- >>> From: andrea.meconi(a)libero.it [mailto:andrea.meconi@libero.it] >>> Sent: Wednesday, November 16, 2011 11:19 AM >>> To: Lay, James >>> Subject: R: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >>> >>> Yes James. >>> First configure, than make and make install. >>> Andrea >>> >>> >----Messaggio originale---- >>> >Da: james.lay(a)wincofoods.com >>> >Data: 16/11/2011 17.14 >>> >A: <andrea.meconi(a)libero.it>, <syslog-ng(a)lists.balabit.hu> >>> >Ogg: RE: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >>> > >>> > >>> > >>> >> -----Original Message----- >>> >> From: syslog-ng-bounces(a)lists.balabit.hu >>> >[mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of >>> >> andrea.meconi(a)libero.it >>> >> Sent: Wednesday, November 16, 2011 9:08 AM >>> >> To: syslog-ng(a)lists.balabit.hu >>> >> Subject: [syslog-ng] R: Re: Compiling syslog-ng on Ubuntu. >>> >> >>> >> >>> >> Hello James. >>> >> I'm using >>> >> >>> >> sudo ./configure --enable-spoof-source >>> >> >>> >> than make and make install. >>> >> Sorry, but I'm not able to see any errors! >>> >> Andrea >>> > >>> > >>> >Andrea, >>> > >>> >You are running sudo make install yes? I would recommend running >>> >./configure --enable-spoof-source, then make, then sudo make install. >>> > >>> >James >>> > >>> >> >> > >

1 0

R: Re: Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

Hello James. I'm using sudo ./configure --enable-spoof-source than make and make install. Sorry, but I'm not able to see any errors! Andrea

3 2

Compiling syslog-ng on Ubuntu.
by andrea.meconi＠libero.it 16 Nov '11

16 Nov '11

Hello.I'm a begineer.I need to recompile syslog-ng with option spoof-source on Ubuntu 8.04 starting from a minimal cd installation.Make install terminates without errors and installation.Many thanks for your help.Regards.Andrea

2 1

Referencing earlier message
by Hery Fanomezantsoa 16 Nov '11

16 Nov '11

When using correlating message with syslog-ng 3.3.1 i get only the value in the last matching rule but not the earlier. Here is my patterndb.xml <?xml version='1.0' encoding='UTF-8'?> > <patterndb version='3' pub_date='2011-11-07'> > <ruleset name='ecelerity' id='12345678'> > <pattern>ecelerity</pattern> > <rules> > <rule provider='me' id='123475980' class='system' > context-scope='program' context-id='${MSG.UID}' > context-timeout='10'> > <patterns> > > <pattern>@ESTRING:LOG.UTC:|@@ESTRING:LOG.UID:|@ORCPTS|@ANYSTRING:LOG.VAL:@ > </pattern> > </patterns> <examples> > <example> > <test_message > program="ecelerity">1319550976|c0a80a3c-b7f6c6d000002063-1f-4ea6c0004833|ORCPTS|s.andriamampianina@ > ***.**</test_message> > <test_values> > <test_value name="LOG.UTC">1319550976</test_value> > <test_value > name="LOG.UID">c0a80a3c-b7f6c6d000002063-1f-4ea6c0004833</test_value> > > <test_value name="LOG.VAL">s.andriamampianina@ > ***.**</test_value> > </test_values> > </example> > </examples> > </rule> > <rule provider='me' id='123475981' class='system' > context-id='${MSG.UID}'> > <patterns> > > <pattern>@ESTRING:LOG.UTC:|@@ESTRING:LOG.UID:|@SENDER|@ANYSTRING:LOG.VAL:@ > </pattern> > </patterns> > <actions> > <action> > <message> > <values> > <value name="MESSAGE">From ${LOG.VAL}@1 to ${LOG.VAL}@2. > </value> > <value name="TRIGGER">yes</value> > </values> > </message> > </action> > </actions> > </rule> > </rules> > </ruleset> > </patterndb> > And the message i get is "from *****(a)**.** to ." Where did I get wrong?

2 2

syslog-ng 3.3.2 seg faulting sending message via tcp
by Patrick H. 16 Nov '11

16 Nov '11

Syslog-ng 3.3.2 (obtained from http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.2/source) is segfaulting when it tries to send a message to a TCP destination. I'm asking here instead of a bug report because I didnt see a 3.3.2 release announcement, so not sure if this is the final build or not. Anyway, gdb backtrace shows: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x40f67940 (LWP 29440)] 0x00002b047a812261 in log_pipe_queue (s=0x12d0d94f, msg=0x12d04400, path_options=0x40f64150) at logpipe.h:288 288 s->queue(s, msg, path_options, s->queue_data); (gdb) where #0 0x00002b047a812261 in log_pipe_queue (s=0x12d0d94f, msg=0x12d04400, path_options=0x40f64150) at logpipe.h:288 #1 0x00002b047a8121b2 in log_multiplexer_queue (s=0x12cd7e60, msg=0x12d04400, path_options=0x40f642d0, user_data=0x0) at logmpx.c:125 #2 0x00002b047a80ce2c in log_pipe_queue (s=0x12cd7e60, msg=0x12d04400, path_options=0x40f642d0) at logpipe.h:288 #3 0x00002b047a80cddb in log_pipe_forward_msg (self=0x12d0d810, msg=0x12d04400, path_options=0x40f642d0) at logpipe.h:275 #4 0x00002b047a80cd79 in log_filter_pipe_queue (s=0x12d0d810, msg=0x12d04400, path_options=0x40f642d0, user_data=0x0) at filter.c:698 #5 0x00002b047a812264 in log_pipe_queue (s=0x12d0d810, msg=0x12d04400, path_options=0x40f642d0) at logpipe.h:288 #6 0x00002b047a8121b2 in log_multiplexer_queue (s=0x12cd7e00, msg=0x12d04400, path_options=0x40f64550, user_data=0x0) at logmpx.c:125 #7 0x00002b047a82cd20 in log_pipe_queue (s=0x12cd7e00, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:288 #8 0x00002b047a82cccf in log_pipe_forward_msg (self=0x12d127e0, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:275 #9 0x00002b047a82cc8a in log_source_group_queue (s=0x12d127e0, msg=0x12d04400, path_options=0x40f64550, user_data=0x0) at sgroup.c:102 #10 0x00002b047a81ee1f in log_pipe_queue (s=0x12d127e0, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:288 #11 0x00002b047a81edce in log_pipe_forward_msg (self=0x12d125a0, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:275 #12 0x00002b047a81ee32 in log_pipe_queue (s=0x12d125a0, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:292 #13 0x00002b047a81edce in log_pipe_forward_msg (self=0x12cfb920, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:275 #14 0x00002b047a81ee32 in log_pipe_queue (s=0x12cfb920, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:292 #15 0x00002b047a81edce in log_pipe_forward_msg (self=0x12cd8800, msg=0x12d04400, path_options=0x40f64550) at logpipe.h:275 #16 0x00002b047a81eccc in log_source_queue (s=0x12cd8800, msg=0x12d04400, path_options=0x40f645f0, user_data=0x0) at logsource.c:290 #17 0x00002b047a81c969 in log_pipe_queue (s=0x12cd8800, msg=0x12d04400, path_options=0x40f645f0) at logpipe.h:288 #18 0x00002b047a81c919 in log_reader_handle_line (self=0x12cd8800, line=0x12d09800 "<22>Nov 14 02:48:00 postfix/smtpd[23425]: connect from slider.dev.usa.net[165.212.101.17]\n", length=89, saddr=0x0) at logreader.c:503 #19 0x00002b047a81cadd in log_reader_fetch_log (self=0x12cd8800) at logreader.c:561 #20 0x00002b047a81be25 in log_reader_work_perform (s=0x12cd8800) at logreader.c:115 #21 0x00002b047a824648 in main_loop_io_worker_job_start (self=0x12cd8a98) at mainloop.c:367 #22 0x00002b047a86ae96 in iv_work_thread_got_event (_thr=0x12cdb1f0) at iv_work.c:113 #23 0x00002b047a869e17 in iv_event_run_pending_events (_dummy=0x0) at iv_event.c:67 #24 0x00002b047a86a3db in iv_event_raw_got_event (_this=0x40f67858) at iv_event_raw.c:82 #25 0x00002b047a866945 in iv_run_active_list (active=0x40f64c20) at iv_main.c:219 #26 0x00002b047a866b83 in iv_main () at iv_main.c:269 #27 0x00002b047a86b290 in iv_work_thread (_thr=0x12cdb1f0) at iv_work.c:196 #28 0x00002b047a86b81a in iv_thread_handler (_thr=0x12cda400) at iv_thread.c:100 #29 0x00000039820064a7 in start_thread () from /lib64/libpthread.so.0 #30 0x00000039818d3c2d in clone () from /lib64/libc.so.6 Config: @version: 3.3 options { time_reopen(1); use_dns(no); use_fqdn(yes); keep_hostname(yes); normalize_hostnames(yes); create_dirs(yes); dir_perm(0755); perm(0644); mark_freq(0); threaded(yes); }; @include "scl.conf" source s_local { system(); internal(); }; filter f_mail { facility(mail); }; destination d_central { tcp('syslog.usa.net' port(514) log_fifo_size(50000)); }; log { source(s_local); filter(f_mail); destination(d_central); };

3 5

[Bug 111] New: RPM build errors: Installed (but unpackaged) file(s) found:
by bugzilla＠bugzilla.balabit.com 14 Nov '11

14 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=111 Summary: RPM build errors: Installed (but unpackaged) file(s) found: Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: owen.mann(a)interactivedata.com Type of the Report: bug Estimated Hours: 0.0 when attempting `rpmbuild -tb syslog-ng_3.2beta1.tar.gz`, the compilation completes, but: [...] Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/syslog-ng-root error: Installed (but unpackaged) file(s) found: /bin/pdbtool /bin/update-patterndb /etc/syslog-ng/modules.conf /etc/syslog-ng/scl.conf /lib/libsyslog-ng.la /lib/libsyslog-ng.so /lib/libsyslog-ng.so.0 /lib/libsyslog-ng.so.0.0.0 /lib/syslog-ng/libaffile.la /lib/syslog-ng/libaffile.so /lib/syslog-ng/libafprog.la /lib/syslog-ng/libafprog.so /lib/syslog-ng/libafsocket-notls.la /lib/syslog-ng/libafsocket-notls.so /lib/syslog-ng/libafsocket-tls.la /lib/syslog-ng/libafsocket-tls.so /lib/syslog-ng/libafsocket.so /lib/syslog-ng/libafuser.la /lib/syslog-ng/libafuser.so /lib/syslog-ng/libbasicfuncs.la /lib/syslog-ng/libbasicfuncs.so /lib/syslog-ng/libconfgen.la /lib/syslog-ng/libconfgen.so /lib/syslog-ng/libconvertfuncs.la /lib/syslog-ng/libconvertfuncs.so /lib/syslog-ng/libcsvparser.la /lib/syslog-ng/libcsvparser.so /lib/syslog-ng/libdbparser.la /lib/syslog-ng/libdbparser.so /lib/syslog-ng/libdummy.la /lib/syslog-ng/libdummy.so /lib/syslog-ng/libsyslogformat.la /lib/syslog-ng/libsyslogformat.so /sbin/syslog-ng-ctl /share/include/scl/pacct/plugin.conf /share/include/scl/syslogconf/README /share/include/scl/syslogconf/convert-syslogconf.awk /share/include/scl/syslogconf/plugin.conf /share/include/scl/system/generate-system-source.sh /share/include/scl/system/plugin.conf /share/xsd/patterndb-1.xsd /share/xsd/patterndb-2.xsd /share/xsd/patterndb-3.xsd [...] If these files are added to the %files section in syslog-ng.spec, the build completes successfully. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

syslog-ng 3.3.2 has been released
by devel＠balabit.hu 14 Nov '11

14 Nov '11

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.2 SUMMARY : new stable release DATE : Nov 13, 2011 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.2) has been released. For latest fixes in the 3.3.x feature branch you are recommended to upgrade to this version. CHANGES: 3.3.2 Sat, 12 Nov 2011 20:48:53 +0100 Bugfixes: ========= * Fixed a segmentation fault on reload when using the same rewrite rule from multiple log paths. * Fixed a segmentation fault when processing a reload request in case an existing tcp() source is removed from the config and there are open connections. * Fixed a possible segmentation fault in the scalable queue implementation, which happens in case a destination is slower to process messages than syslog-ng would like to send them. * Fixed a possible file() destination issue that could cause syslog-ng to omit data or to write garbage to the log file in case the kernel reports that only a smaller portion of the actual write request could be accomplished. * Fixed an "internal error duplicate config element" error during reload due to an invalid bugfix applied for 3.3.1. Older beta versions of 3.3 were not affected. * Fixed a memory leak that causes macro based file destinations to leak their queue when destination files are closed due to time-reap(). * Fixed the handling of the condition() option for rewrite rules. * Fixed a race condition in value-pairs support, potentially causing heap corruption problems when $(format-json) is used in threaded mode. * Fixed a memory leak in value-pairs template function argument parsing, fixing a leak if $(format-json) is used. * Repeated definitions of source, destination, filter, rewrite, parser and block elements are not allowed by default anymore. These are reported as configuration errors unless @define allow-config-dups 1 is specified in the configuration file. * Fixed pdbtool error reporting in "pdbtool test" to make it easier to understand what went wrong. * Added an SQL connection health check in case an INSERT failed. This way syslog-ng handles SQL server timeouts better. * Fixed support for systemd socket activation. Previously such sockets were not set to non-blocking mode, causing syslog-ng to hang. * Fixed the filter() function in the filter expression to work also when used as a part of an AND or OR construct. * Allow the sql() destination to operate even without an indexes() option. That parameter was meant to be optional, but it wasn't. * Fixed compilation issues if no OpenSSL is present. * Fixed a minor memory leak in the usertty() driver that can increase memory usage on every reload. (The username() parameter wasn't properly freed on reload). * Fixed a minor memory leak in the sql() driver that can increase the memory usage on every reload (indexes() parameter wasn't properly freed on reload). Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Anthony Lineham (Allied Telesys) Arkadiusz Miśkiewicz (PLD Linux) Attila Szalay (BalaBit) Balazs Scheidler (BalaBit) Balint Kovacs (BalaBit) Daniel Ankers Dave Rawks (Pandora) Enrico Scholz (Technische Universitat Chemnitz) Eric Lindvall (Seven Scale) Erik Maciejewski Gergely Nagy (BalaBit) Hery Fanomezantsoa Jack Nagel Jakub Jankowski (SuperHost.pl) Leonid Isaev (Indiana University) Martin Holste Michael Sterrett (Gentoo) Patrick H. Peter Czanik (BalaBit) Viktor Juhasz (BalaBit) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Logs after "\n" are getting truncated
by Swati4 M/TVM/TCS 14 Nov '11

14 Nov '11

Hi All, When logging multiline messages (simple strings with in-line newlines "\n") our logs only show the first line (ie) before "\n" is encountered and rest of the line after the first newline character("\n") is truncated. Example: Code Snippet: HS_MAP_INFO_LOG("[ %s ] Resource [ %d ] Sensor Number: %2d, Sensor Type: %d, Event Category: %d, Assertion: %d, \n" "Event State: %d, Previous State: %d, OEM Code: %d, Sensor Specific: %d\n", eidStr, pEvt->Source, thisSensorEvent->SensorNum, thisSensorEvent->SensorType, thisSensorEvent->EventCategory, thisSensorEvent->Assertion, thisSensorEvent->EventState, thisSensorEvent->PreviousState, thisSensorEvent->Oem, thisSensorEvent->SensorSpecific); Expected Output on Log: May 4 11:32:34 base0-0-0-1-0-11-1 shelfHs[5255]: 5257: sHs[Info]: 1304523154.708143: hsMapLog.cpp(47): hsMapHpiShelfHsBUtils.cpp(240): [ 0:0:9:0:11:1 ] Resource [ 68 ] Sensor Number: 9, Sensor Type: 2, Event Category: 1, Assertion: 1, Event State: 16, PreviousState: 0, OEM Code: 0, Sensor Specific: 0 Actual Output populated on Log: May 4 10:18:37 base0-0-0-1-0-11-1 shelfHs[4539]: 4542: sHs[Info]: 1304518717.501982: hsMapLog.cpp(47): hsMapHpiShelfHsBUtils.cpp(240): [ 0:0:9:0:11:1 ] Resource [ 98 ] Sensor Number: 9, Sensor Type: 2, Event Category: 1, Assertion: 1, So,if anyone know some kind of fix/patch for resolving this issue, please let me know. Your help would be highly appreciated. Thanks & regards Swati. =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

4 7

[PATCH] gprocess: Fix --no-caps handling when building with hardening options.
by Gergely Nagy 13 Nov '11

13 Nov '11

When both glib and syslog-ng is built with hardening options enabled, overwriting a string pointer with a FALSE boolean does not have the expected results: the string will not become NULL. In this case, --no-caps will end up with a segfault, because process_opts.caps is non-NULL, and points to junk. To fix this, introduce a new function that sets process_opts.caps to NULL explicitly, and use this function as a callback, instead of assuming that a FALSE boolean will have the expected results. Reported-By: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/gprocess.c | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-) diff --git a/lib/gprocess.c b/lib/gprocess.c index 2ccaffe..aaefbab 100644 --- a/lib/gprocess.c +++ b/lib/gprocess.c @@ -1440,6 +1440,14 @@ g_process_process_mode_arg(const gchar *option_name G_GNUC_UNUSED, const gchar * return TRUE; } +static gboolean +g_process_process_no_caps(const gchar *option_name G_GNUC_UNUSED, const gchar *value G_GNUC_UNUSED, + gpointer data G_GNUC_UNUSED, GError *error) +{ + process_opts.caps = NULL; + return TRUE; +} + static GOptionEntry g_process_option_entries[] = { { "foreground", 'F', G_OPTION_FLAG_REVERSE, G_OPTION_ARG_NONE, &process_opts.mode, "Do not go into the background after initialization", NULL }, @@ -1450,7 +1458,7 @@ static GOptionEntry g_process_option_entries[] = { "gid", 0, G_OPTION_FLAG_HIDDEN, G_OPTION_ARG_STRING, &process_opts.group, NULL, NULL }, { "chroot", 'C', 0, G_OPTION_ARG_STRING, &process_opts.chroot_dir, "Chroot to this directory", "<dir>" }, { "caps", 0, 0, G_OPTION_ARG_STRING, &process_opts.caps, "Set default capability set", "<capspec>" }, - { "no-caps", 0, G_OPTION_FLAG_REVERSE, G_OPTION_ARG_NONE, &process_opts.caps, "Disable managing Linux capabilities", NULL }, + { "no-caps", 0, G_OPTION_FLAG_NO_ARG, G_OPTION_ARG_CALLBACK, g_process_process_no_caps, "Disable managing Linux capabilities", NULL }, { "pidfile", 'p', 0, G_OPTION_ARG_STRING, &process_opts.pidfile, "Set path to pid file", "<pidfile>" }, { "enable-core", 0, 0, G_OPTION_ARG_NONE, &process_opts.core, "Enable dumping core files", NULL }, { "fd-limit", 0, 0, G_OPTION_ARG_INT, &process_opts.fd_limit_min, "The minimum required number of fds", NULL }, -- 1.7.7.2

3 4

Possible memleak in 3.3 HEAD
by Jakub Jankowski 12 Nov '11

12 Nov '11

Hi, I think I'm having a memleak in 3.3's HEAD. My production system, which receives ~300 messages/s throughout the day (so ~26.4 million messages daily) leaks ~700MB of memory per day. I did some investigation, here are the details: # syslog-ng -V syslog-ng 3.3.0beta2 Installer-Version: 3.3.0beta2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#no_branch#d6eea6a1130f8343b6600976d9f2319f9ed86c88 Compile-Date: Sep 9 2011 16:32:59 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql Available-Modules: syslogformat,affile,afmongodb,basicfuncs,csvparser,dbparser,afsql,convertfuncs,afprog,confgen,afsocket,afuser,afsocket-tls,dummy Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: off # My build is patched with: - https://github.com/algernon/libmongo-client/commit/93c5cff05feec5d7f421acfd… (so it compiles on rhel5) - https://lists.balabit.hu/pipermail/syslog-ng/2011-April/016375.html (I need this feature) I don't think any of these is the reason, though. My test configuration is basically this: $ cat /etc/syslog-ng/syslog-ng.conf @version: 3.3 options { time_reap (1); log_fifo_size (100000); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); log_msg_size(16384); owner ("root"); group ("root"); perm (0644); dir_owner ("root"); dir_group ("root"); dir_perm (0755); }; source s_www { syslog(ip(0.0.0.0) port(515) flags(no-multi-line) log_iw_size(6000) max-connections(30) tags("src.syslog", "src.port.515", "www", "www.accesslog")); }; filter f_svc_www_accesslog { tags("www.accesslog"); }; parser p_apache_split_vhost_access { csv-parser( columns("APACHE.ACCESS.VHOST", "APACHE.ACCESS.LOG_MESSAGE") flags(greedy) delimiters(" ") ); }; parser dbp_apache_split_vhost_access { db-parser( file("/etc/syslog-ng/pdb1.xml") ); }; destination d_access_split { file("/logs/$S_YEAR/$S_MONTH/$S_DAY/${APACHE.ACCESS.VHOST:-default}-access.log-${S_YEAR}${S_MONTH}${S_DAY}" template("${APACHE.ACCESS.LOG_MESSAGE}\n")); }; log { source(s_www); filter(f_svc_www_accesslog); parser(dbp_apache_split_vhost_access); destination(d_access_split); flags(final); }; $ Those two parsers (csv-parser and db-parser) are doing exactly the same, they split my Apache HTTPd access logs into two parts: first column (which is the vhost name) and the rest of the line. I have them both because I needed to know if that's a leak in any particular parser type. (Apparently not) I'm not very experienced with valgrind, but I did some tests, and what I've found is that the number of "definitely lost" blocks depends on the number of files created by d_access_split destination. So I took 10000 lines of some real httpd logs and did two tests: 1) replace real vhost names with completely random strings, to maximize number of files created by d_access_split 2) replace real vhost names with constant string, so my destination always writes to the same file. And this is what I got, using "G_SLICE=always-malloc valgrind --leak-check=full -v --show-reachable=yes --track-origins=yes --log-file=valgrind.log syslog-ng -F --no-caps -p /var/run/syslog-ng.pid" 1) random files: ==22416== 3,675,952 bytes in 9,989 blocks are definitely lost in loss record 578 of 578 ==22416== at 0x4021EC2: calloc (vg_replace_malloc.c:418) ==22416== by 0x40FF57D: g_malloc0 (in /lib/libglib-2.0.so.0.1200.3) ==22416== by 0x4052899: log_queue_fifo_new (logqueue-fifo.c:438) ==22416== by 0x4046057: log_dest_driver_acquire_queue_method (driver.c:153) ==22416== by 0x47081D1: affile_dw_init (driver.h:178) ==22416== by 0x4707C41: affile_dd_open_writer (logpipe.h:239) ==22416== by 0x405C258: main_loop_call (mainloop.c:145) ==22416== by 0x4707A6B: affile_dd_queue (affile.c:1127) ==22416== by 0x40451C8: log_dest_group_queue (logpipe.h:288) ==22416== by 0x404B9DD: log_multiplexer_queue (logpipe.h:288) ==22416== by 0x404EC16: log_parser_queue (logpipe.h:288) ==22416== by 0x4047473: log_filter_pipe_queue (logpipe.h:288) ==22416== ==22416== LEAK SUMMARY: ==22416== definitely lost: 3,676,156 bytes in 9,992 blocks ==22416== indirectly lost: 6,541 bytes in 56 blocks ==22416== possibly lost: 16,967 bytes in 117 blocks ==22416== still reachable: 76,908 bytes in 3,286 blocks ==22416== suppressed: 0 bytes in 0 blocks 2) single file (constant filename): ==19460== 368 bytes in 1 blocks are definitely lost in loss record 525 of 577 ==19460== at 0x4021EC2: calloc (vg_replace_malloc.c:418) ==19460== by 0x40FF57D: g_malloc0 (in /lib/libglib-2.0.so.0.1200.3) ==19460== by 0x4052899: log_queue_fifo_new (logqueue-fifo.c:438) ==19460== by 0x4046057: log_dest_driver_acquire_queue_method (driver.c:153) ==19460== by 0x47081D1: affile_dw_init (driver.h:178) ==19460== by 0x4707C41: affile_dd_open_writer (logpipe.h:239) ==19460== by 0x405C258: main_loop_call (mainloop.c:145) ==19460== by 0x4707A6B: affile_dd_queue (affile.c:1127) ==19460== by 0x40451C8: log_dest_group_queue (logpipe.h:288) ==19460== by 0x404B9DD: log_multiplexer_queue (logpipe.h:288) ==19460== by 0x404EC16: log_parser_queue (logpipe.h:288) ==19460== by 0x4047473: log_filter_pipe_queue (logpipe.h:288) (...) ==19460== LEAK SUMMARY: ==19460== definitely lost: 572 bytes in 4 blocks ==19460== indirectly lost: 6,541 bytes in 56 blocks ==19460== possibly lost: 12,919 bytes in 106 blocks ==19460== still reachable: 76,908 bytes in 3,286 blocks ==19460== suppressed: 0 bytes in 0 blocks Could anyone proficient in syslog-ng source code take a look on this? Can I do anything else to help you diagnose this? (I can provide complete valgrind output logs if this helps.) Also, I'll try to reproduce this on other versions later this week, unless someone fixes this earlier. :) Cheers, -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

7 43

RFC: value-pairs key rewrite framework, part N+1
by Gergely Nagy 12 Nov '11

12 Nov '11

Hi! After my former mails (see the thread starting at http://thread.gmane.org/gmane.comp.syslog-ng/11355/focus=11421) I'd like to do a recap, and ask for comments, as I came up with a few new ideas. The purpose of the value-pairs key rewrite framework is to make it possible to apply various transformations to the keys we selected with VP. So that we can add or remove prefixes, replace parts of the string, and so on. As of this writing, code to do this exists on one of my branches, but it hasn't been touched for a while, and I planned to update it in the near future. And that's when it dawned on me, that perhaps the syntax isn't all that great. To show why I think that, let's see an example first: value-pairs( scope("everything") rekey( add-prefix(".secevt" "events") add-prefix(".classifier" "syslog-ng") shift(".sdata.*" 1) replace("." "_") ) ); This will add an "events" prefix to each key that starts with ".secevt", so that ".secevt.verdict" becomes "events.secevt.verdict"; similary, ".classifier.class" becomes "syslog-ng.classifier.class"; keys that match '.sdata.*' get shifted to the right, removing the dot. And all remaining dots at the begininng of a key will get replaced by an underscore instead. This kinda makes sense, and I could even massage the syntax into format-json: $(format-json --scope everything --rekey --add-prefix .secevt=events --addprefix .classifier=syslog-ng --shift .sdata.*=1 --replace .=_ --end-rekey) However, this syntax has the downside of transformations being global: I can't choose subset of my keys, and apply a list of transformations on those and only those. Once I made a transformation, any transformations in the list afterwards will see the transformed key. So I can't easily say: "take all the keys starting with '.sdata.', shift them 6 chars, then replace any key names that start with 'win' with 'lose', and finally prefix them with 'whatever'". With the current syntax, that's next to impossible to do sanely. It's also not all that intuitive.. So I came up with a different syntax: wiring rekey into the key() option of value-pairs! That way, we already selected a subset to work on, and the transformations would apply to only those. (This could be combined with the global syntax aswell, though) So it'd look something like this: value-pairs( scope("everything") key(".secevt.*" rekey(add-prefix("events"))) key(".classifier.*" rekey(add-prefix("syslog-ng"))) key(".sdata.*" rekey(shift(1))) key(".*" rekey(replace ("." "_"))) ); This would achieve the exact same effect as the example above, but with a clearer syntax, perhaps. It would also mean that key rewriting can be described at the same place where the key is selected to begin with. The downside of this is that it'd be a bit harder to come up with a syntax for format-json that mimics the config file syntax. So, instead of trying to do that and end up with something horrible, I have another proposal: lets make value-pairs a top-level citizen, so that it joins the ranks of filter{} and rewrite{} and the like! That way, we could turn the following ugly thing: destination d_structured { mongodb( value-pairs( scope("everything") key(".secevt.*" rekey(add-prefix("events"))) key(".classifier.*" rekey(add-prefix("syslog-ng"))) key(".sdata.*" rekey(shift(1))) key(".*" rekey(replace ("." "_"))) ) ); file("/var/log/structured.json" template("$(format-json <repeat the above stuff, but with format-json syntax>)\n")); }; Into this beauty: valuepairs vp_example { scope(everything); key(".secevt.*" rekey(add-prefix("events"))); key(".classifier.*" rekey(add-prefix("syslog-ng"))); key(".sdata.*" rekey(shift(1))); key(".*" rekey(replace ("." "_"))); }; destination d_structured { mongodb(value-pairs(vp_example)); file("/var/log/structured.json" template("$(format-json --with-config vp_example)\n")); }; This would have the nice consequence of not having to keep two parsers in sync: format-json would only have a --with-config option, and nothing else. So, in the end, this whole boils down to two questions: * What do you think about allowing (or even moving) rekey() inside key() options? * What do you think about introducing a top-level valuepairs element, and dropping the format-json argument parsing stuff? -- |8]

2 5

[PATCH] resolve_sockaddr: Fix domain stripping on initial lookup.
by Gergely Nagy 12 Nov '11

12 Nov '11

When looking up a hostname, we were only setting the 'positive' bool during lookup, so it was only set when the host already existed in the cache. When it didn't, and we just put it there right after the failed lookup, the bool remained false. This resulted in the domain part not being stripped even when use_fqdn(no) was set. The fix implemented in this patch is dead simple: if we managed to resolve a host, we set the bool to TRUE. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/misc.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/lib/misc.c b/lib/misc.c index 80f5877..82c2d1b 100644 --- a/lib/misc.c +++ b/lib/misc.c @@ -211,7 +211,10 @@ resolve_sockaddr(gchar *result, gsize *result_len, GSockAddr *saddr, gboolean us hp = gethostbyaddr(addr, addr_len, saddr->sa.sa_family); hname = (hp && hp->h_name) ? hp->h_name : NULL; - + + if (hname) + positive = TRUE; + if (use_dns_cache && hname) { /* resolution success, store this as a positive match in the cache */ -- 1.7.7.1

2 1

syslog-ng 3.2.4 MySQL connection loss Server has gone away
by Erik Maciejewski 12 Nov '11

12 Nov '11

Hello, I have been researching an issue with syslog-ng 3.2.4 (both the Linux glibc2.3.6 and platform independent compiled from source versions) and MySQL on CentOS 5.6 in which syslog-ng tries to insert a log messages using a dead TCP or unix socket connection to a MySQL database. The issue results in the max attempt to insert a log message and then the message subsequently dropped. This seems to occur regularly in predominately low message volume environments, but has the potential to affect all environments using a MySQL (or other db) destination. The root cause of the issue seems to be directly related to the health of the connection between syslog-ng and MySQL and can be affected by the "wait_timeout" value used by MySQL to kill off inactive connections. I want to provide justification for implementing a fix to syslog-ng as I believe manipulating a, possibly tuned, environment variable for MySQL would seem like the wrong approach to take for remedial action. I feel that many MySQL instances are implemented in shared environments and are themselves shared by many applications. New connections to a MySQL instance are generally regarded as low cost and, it would seem, more often than not never used in a persisted fashion when supporting distributed applications. That being said, many times the "wait_timeout" value is set very low so that the MySQL instance can effectively control the number of simultaneous connections. By taking somewhat of a black-box approach to the issue (I'm just starting to explore syslog-ng), I noticed that there doesn't seem to be any health checking of the TCP or unix socket connection in the SQL destination implementation after the initial connection is made (most of the db actions being abstracted away by libdbi). Even if a syslog-ng database thread is suspended and reactivated due to error, there does not seem to be any attempt to check the health of the database connection. I have implemented a fix in my compiled version that will check a database connection's health and attempt to reconnect upon discovering a dead connection. I would like to know the policies/procedures/best practices for submitting such issues and/or fixes for discussion, review, and implementation. If this is a known issue or I am just wrong, all the better! Thank you guys for keeping this project going and I'm looking forward to implementing syslog-ng in a production environment in the very near future! Regards, Erik Maciejewksi

2 7

Variables in PatternDB - what am I doing wrong?
by Daniel Ankers 12 Nov '11

12 Nov '11

Hi, I've got the following as part of my ruleset:  <rule id='def42cfb-4364-4cfd-bbf1-cdf4f0bc3c88' class='lowpri' context-id="badauth-${badauth.errorsrc}" context-timeout='600' context-scope='host'> <patterns> <pattern>%TCP-6-BADAUTH: No MD5 digest from @IPvANY:badauth.errorsrc:@(@NUMBER::@) to @IPvANY:badauth.errordst:@(@NUMBER::@) (RST)</pattern> </patterns> <examples> <example> <test_message program='patternize'>%TCP-6-BADAUTH: No MD5 digest from 195.66.XXX.XX(179) to 195.66.YYYY.YY(56561) (RST)</test_message> </example> <example> <test_message program='patternize'>%TCP-6-BADAUTH: No MD5 digest from 2001:7F8:X::abcd:1(179) to 2001:7F8:X::1234:1(21296) (RST)</test_message> </example> </examples> <actions> <action trigger='match'> <message> <values> <value name='MESSAGE'>$(if ("${badauth.errorsrc}@1" eq "${badauth.errorsrc}@2") "Duplicate" "First") BGP Authentication Error to neighbour ${badauth.errorsrc}@1 - last one was ${badauth.errorsrc}@2</value> </values> </message> </action> </actions> </rule>  When I feed example messages in from a file, I expect to see something like the following: Nov 4 10:40:54 syslog First BGP Authentication Error to neighbour 2001:7F8:X::abcd:1 - last one was Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 2001:7F8:X::abcd:1 - last one was 2001:7F8:X::abcd:1 Nov 4 10:40:54 syslog First BGP Authentication Error to neighbour 195.66.Y.Z - last one was Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z Instead I see: Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 2001:7F8:X::abcd:1 - last one was Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 2001:7F8:X::abcd:1 - last one was 2001:7F8:4::abcd:1 Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z Nov 4 10:40:54 syslog Duplicate BGP Authentication Error to neighbour 195.66.Y.Z - last one was 195.66.Y.Z What am I getting wrong with the "if" function? The background to this is that the first of these messages from any host is useful information - however they are sent every 10 seconds and I'm trying to suppress all but the first message for each neighbour. I've tried several different ways of doing this including rate-limiting on the <action> but none of them have worked the way I expected. Thanks, Dan Daniel Ankers

2 2

Problems writing to files with different templates/ownership in 3.x
by Brian De Wolf 12 Nov '11

12 Nov '11

Hello, I'm using syslog-ng as a remote syslog server for several of our internal groups. As such, when a syslog arrives, I use the source address to determine the ownership of the log file. Here is how I'm currently doing this: source net { udp(ip(0.0.0.0), port(514) flags(no-multi-line)); }; filter examplefilter { netmask("10.0.0.0/255.255.0.0"); }; filter example2filter { netmask("192.168.0.0/255.255.0.0"); }; destination examplelog { file("/var/log/remote/$HOST.log" group(examplegroup)); }; destination example2log { file("/var/log/remote/$HOST.log" group(example2group)); }; log { source(net); filter(examplefilter); destination(examplelog); flags(final); }; log { source(net); filter(example2filter); destination(example2log); flags(final); }; This worked fine in 1.6.x and appears to work fine in 3.2.x, but when logrotate pokes syslog-ng I get lines like this: syslog-ng: Internal error, duplicate configuration elements refer to the same persistent config; name='affile_dd_writers(/var/log/remote/$HOST.log)' Is there a better way to do this which only produces one reference to "/var/log/remote/$HOST.log"? Also, I'm using a similar set of filters and destinations to apply different templates to text that is being written to a single file. This also produces the error, but I am more concerned that the combined file might experience issues if the config isn't supposed to have multiple destinations be the same location. Is there a way to use filters to apply templates while still only having one destination defined? Could my current setup result in lost log lines? Thanks, Brian

2 1

[PATCH (3.3)] value-pairs: Fix a small memory leak in the commandline parser.
by Gergely Nagy 12 Nov '11

12 Nov '11

Whenever we parse a command-line, we construct a new argv array, to prepend a zeroth element for glibs option parser. We correctly free'd it on the error path, but never on the success path. This lead to a small memory leak every time value_pairs_new_from_cmdline() was called (that is, at the moment, each $(format-json) template function leaked a little bit). This patch fixes this issue by modifying the error handling a little: instead of freeing everything and returning NULL, we only free the value-pairs structure, and set it to NULL. The rest of the freeing is left up to the common parts afterwards, which now correctly frees the constructed argv array too. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/value-pairs.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/value-pairs.c b/lib/value-pairs.c index 803e14c..b5a95af 100644 --- a/lib/value-pairs.c +++ b/lib/value-pairs.c @@ -473,11 +473,10 @@ value_pairs_new_from_cmdline (GlobalConfig *cfg, if (!g_option_context_parse (ctx, &argc, &argv, error)) { value_pairs_free (vp); - g_option_context_free (ctx); - g_free (argv); - return NULL; + vp = NULL; } g_option_context_free (ctx); + g_free (argv); return vp; } -- 1.7.7.2

1 0

Loganalyzer
by Peter Czanik 11 Nov '11

11 Nov '11

Hello, Recently I found many e-mail and forum posts looking for a web GUI for syslog-ng. One of them, Loganalyzer is getting more popular, as a completely free web application. There is just one catch, it was not designed for syslog-ng originally. So I did some research and tests, and using my instructions quite easy to get started with it. Of course Loganalyzer has some rough edges, but it works for me on a number of different systems. For a step by step tutorial to get started, read my blog at http://czanik.blogs.balabit.com/2011/02/loganalyzer/ Have a nice weekend! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 3

Syslog_ng 3.0.7 issues running in Central server mode on AIX 7.1
by Rakesh Rajasekharan 11 Nov '11

11 Nov '11

Hi, I am facing some challenges configuring the syslog_ng( 3.0.7) for central logging on AIX. The subsystem becomes in-operative while trying to set it up for central logging. But otherwise while tryng to log the messages on the same machine it runs fine... Below are the logs 1. Logging on the same host. # /opt/soe/local/syslog_ng/etc/init.d/syslog-ng.d restart 0513-044 The syslog_ng Subsystem was requested to stop. 0513-059 The syslog_ng Subsystem has been started. Subsystem PID is 9043968. # ^C # lssrc -a |grep -i syslog syslog_ng local 9043968 active syslogd ras inoperative Now ,after I change the conf file to the central server one... # cp syslog-ng.conf.server-sample syslog-ng.conf # /opt/soe/local/syslog_ng/etc/init.d/syslog-ng.d restart 0513-044 The syslog_ng Subsystem was requested to stop. 0513-059 The syslog_ng Subsystem has been started. Subsystem PID is 3342368. # lssrc -a |grep -i syslog syslogd ras inoperative syslog_ng local inoperative Below is are the contents for myu conf file whiel trying to set up for central logging. # cat syslog-ng.conf.server-sample @version: 3.0 source s_remote { tcp(); }; destination d_clients { file("/var/adm/messages-ng.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; source s_local { internal();unix-dgram("/dev/log"); }; destination d_local { file("/var/log/syslog-ng.log" owner("root") group("adm") perm(0640)); }; log { source(s_local); destination(d_local); }; destination d_messages { file("/var/log/messages-ng" owner("root") group("adm") perm(0640)); }; log { source(s_local); destination(d_messages); }; Please let me know if somehting in he conf file does not look fine and might be causing the susbsytem to turn inoperative. Because If i revert back to my old config file the subsystem again becomes active. Regards, Rakesh

1 1

syslog-ng Insider - November 2011
by Peter Czanik 11 Nov '11

11 Nov '11

Dear syslog-ng users, This is the 8th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS syslog-ng 3.3.2 is about to be released! ---------------------------------------- A new version of syslog-ng is about to be released! There are no new features to announce, but all problems reported since 3.3.1 should be fixed by now! To make it the best syslog-ng ever, please test it to make sure, that all your problems are fixed. Sources are available in git or as a snapshot: * git://git.balabit.hu/bazsi/syslog-ng-3.3 * http://packages.madhouse-project.org/syslog-ng/3.3/3.3.2/syslog-ng-3.3.2-HE… Binary packages are available are available for several Linux distributions: * openSUSE: http://download.opensuse.org/repositories/home:/czanik:/syslog-ng33/ * Debian and Ubuntu: http://asylum.madhouse-project.org/projects/debian/ syslog-ng and CEE ----------------- The latest syslog-ng release, version 3.3 can be used to implement part of the “CEE over syslog” standard. BalaBit’s patterndb <http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…> technology was able to extract information from syslog messages already for a long time. With this release JSON output was added, meaning the extracted information can be output as JSON data. What it means in practice, that syslog-ng is able to parse log messages, and output the extracted fields in the form required by CEE. To see, how it works, check http://czanik.blogs.balabit.com/2011/10/cee-and-syslog-ng/ Development of syslog-ng 3.4 started ------------------------------------ While 3.3 was just released, development of 3.4 is already started. The first version of a JSON parser is already merged ( https://github.com/bazsi/syslog-ng-3.4/commit/e5569687bba2551c89a78faee55bc… ). There are some pending fixes and enhancements, which add boolean, array and nested JSON parsing ( https://github.com/algernon/syslog-ng/commits/feature/3.4/json/parser ). Value-pairs key rewrite is work in progress ( https://github.com/algernon/syslog-ng/commits/feature/3.4/value-pairs/rekey ) and nested JSON output is also planned. The above features among others help us to better support CEE. With key rewriting we could use a “.cee.” prefix in CEE related patterns and rewrite it later. It also makes parsing of messages possible. All the current code is available for testing in Algernon's 3.4 sandbox project: https://github.com/algernon/syslog-ng/tree/sandbox/3.4 To download it, use git: $ git clone -b sandbox/3.4 git@github.com:algernon/syslog-ng <mailto:git@github.com:algernon/syslog-ng> OTHER SHORT NEWS * An interesting article about centralized syslog server in Linux Journal: http://www.linuxjournal.com/content/creating-centralized-syslog-server NEW RELEASES * syslog-ng OSE 3.2.5: http://www.balabit.com/downloads/files/syslog-ng/sources/3.2.5 WHITE PAPERS A longer paper about the “Future of logging tools”, which also provides some background information about HSRL, as used in syslog-ng: http://andrea.blogs.balabit.com/files/2011/10/HSRL_backgrounder_english_fin… ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

3.3.x conditional rewrite bug
by Martin Holste 11 Nov '11

11 Nov '11

The following will not work in the 3.3.x branch: filter f_rewrite_cisco_program { match('^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}): ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); }; It will process a few lines and then hang indefinitely. It will process ok if the condition is taken off of the filter, so I am assuming this is a PCRE issue, though I found it made no difference to remove the PCRE requirement and use a simpler matching expression. All works perfectly in 3.2.4.

3 4

Quick filter question
by Lay, James 10 Nov '11

10 Nov '11

Hey all! Real quick...trying to filter OUT firewall hits that have say...169.254. Will this do the trick? filter f_firewall { not program (firewall flags(ignore-case)); and not message("169\.254\.[0-9]+\.[0-9]+"); }; Thanks all. James

4 10

Syslog-ng under cygwin.
by andrea.meconi＠libero.it 10 Nov '11

10 Nov '11

Hello. Does anyone know the step by step to compile syslog-ng under Cygwin? I'm running Cygwin on Windows 2008 R2 64 bit and I need option spoof-source enabled. I'm trying this guide, http://mathieu.carbou.free.fr/wiki/index.php?title=How_to_compile_Libnet_un… without success. Thanks. Regards. Andrea

1 0