- syslog-ng - lists.balabit.hu

Syslog-ng OSE Question.
by Malkin, Vlad 02 Dec '13

02 Dec '13

Hi, Is it possible to have syslog-ng client to invoke an external (to it) software to acknowledge when each individual log statement is sent to the server? Background: logs should not be lost. The system needs to store logs internally until they are transmitted to the server (use of TCP transport provides a level of reliable delivery). After each log statement is transmitted by syslog-ng client, it needs to be removed from internal store to preserve allocated space. Thank you for your help! --Vlad

5 6

syslog-ng 3.5.2 has been released
by devel＠balabit.hu 29 Nov '13

29 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.5.2 SUMMARY : new stable release DATE : Nov 29, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.5.2) has been released. For latest fixes in the 3.5.x feature branch you are recommended to upgrade to this version. CHANGES: 3.5.2 Fri, 29 Nov 2013 17:09:40 +0100 This is the first bug-fix release for the 3.5.x series, with a number of significant fixes over the original 3.5.1 release. Upgrading to this release is highly recommended. Bugfixes ======== * The system() source has been fixed on Solaris and HP-UX: it correctly recognises the system on Solaris, and uses /dev/log (and not /dev/pipe) on HP-UX. * The system() source has much stricter requirements to select /dev/kmsg on Linux, so it won't choose it over /proc/kmsg unless the kernel really does support it. * The syslog-format() module has been fixed to only set the PROGRAM to "kernel" if all of the previous conditions apply, and the message is a local message. * The linux-kmsg-format parser (and system() on Linux 3.5+ as a consequence) will correctly calculate the timestamp of kernel messages on 32-bit systems now. * The sun-streams() source has been fixed, it compiles and works again. * A very rare and short race condition has been fixed in the pipe() source and destination modules, which could leave the pipe available with elevated permissions for a short time. * The SQL, MongoDB and SMTP modules were fixed to not cause a segmentation fault when loaded before the socket module. * A crash was fixed in the no-tls build of the socket destination driver. * It is now possible to specify the host() and port() parameters for the redis() destination. * Various stability fixes, such as: - A use-after-free condition was corrected in the queue backlog acknowledgement code. - Always use setsid() when detaching, even if we're not detaching from a TTY. - Memory leaks were fixed in the persist state handling code. - Changing stats-level() and reloading syslog-ng produced a crash, which has been fixed. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Lecuyer Balazs Scheidler <bazsi(a)balabit.hu> Daniel Gados <gdani(a)balabit.hu> Gergely Nagy <algernon(a)balabit.hu> Howard Tsai <htsai(a)skytap.com> Imre Lazar <imre(a)balabit.hu> Jakub Wilk <jwilk(a)debian.org> Laszlo Budai <lbudai(a)balabit.hu> Martin Bagge / brother <brother(a)bsnet.se> Matyas Koszik <koszik(a)atw.hu> Oscar Muñoz Otto Berger <otto(a)bergerdata.de> Viktor Juhasz <jviktor(a)balabit.hu> Viktor Tusa <tusa(a)balabit.hu> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

syslog-ng 3.4.6 has been released
by devel＠balabit.hu 27 Nov '13

27 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.6 SUMMARY : new stable release DATE : Nov 27, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.6) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.6 Wed, 27 Nov 2013 16:21:47 +0100 This is the fifth bug-fix release for the 3.4.x series, with a number of significant fixes over the 3.4.5 release. Upgrading to this release is highly recommended. Bugfixes ======== * The system() source has been fixed on Solaris and HP-UX: it correctly recognises the system on Solaris, and uses /dev/log (and not /dev/pipe) on HP-UX. * The sun-streams() source has been fixed, it compiles and works again. * A very rare and short race condition has been fixed in the pipe() source and destination modules, which could leave the pipe available with elevated permissions for a short time. * The SQL, MongoDB and SMTP modules were fixed to not cause a segmentation fault when loaded before the socket module. * The syslog-format() module has been fixed to only set the PROGRAM to "kernel" if all of the previous conditions apply, and the message is a local message. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Lecuyer Balazs Scheidler <bazsi(a)balabit.hu> Gergely Nagy <algernon(a)balabit.hu> Imre Lazar <imre(a)balabit.hu> Laszlo Budai <lbudai(a)balabit.hu> Matyas Koszik <koszik(a)atw.hu> Oscar Muñoz DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Mongodb
by Nilo Cabeza Diaz 27 Nov '13

27 Nov '13

how send log to mongodb??? this is what I do in syslog-ng.conf and does nothing for me destination d_dw_v3_mongo { mongodb( host("localhost") port(27017) database("syslog") collection("messages") value-pairs( key("ISODATE") key("PRIORITY") pair("svc_id" "${.SDATA.syslog(a)0.svcid}") pair("tid" "${.SDATA.syslog(a)0.tid}") pair("act_id" "${.SDATA.syslog(a)0.id}") pair("act_eid" "${.SDATA.syslog(a)0.eid}") key("MESSAGE") key("SDATA") key("HOST") ); };

2 1

Error of parser
by Nilo Cabeza Diaz 26 Nov '13

26 Nov '13

hello: look this error Failed to start Syslog-NG server : * Starting system logging syslog-ng syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted' Error parsing config, syntax error, unexpected KW_DESTINATION, expecting ';' in /etc/syslog-ng/syslog-ng.conf at line 94, column 1: destination d_mongodb { ^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng ...fail!

2 1

syslog-ng 3.6 rebase
by Balazs Scheidler 25 Nov '13

25 Nov '13

Hi, After integrating everything I could, I would like to rebase the current master to merge fixups into the base patches. This is the last such rebase in 3.6. You can see the rebased series in the "next" branch, I intend to reset the master branch to that history tomorrow evening. the HEAD of both branches are identical, only the history differs. In the meantime, you can submit patches either the current "master" or "next" branches, I'll integrate stuff to "next" myself. Thanks. -- Bazsi

1 1

[Bug 253] New: Segfault when reloading configuration
by bugzilla＠bugzilla.balabit.com 21 Nov '13

21 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=253 Summary: Segfault when reloading configuration Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: gonzalo.paniagua+slng1(a)acquia.com Type of the Report: bug Estimated Hours: 0.0 syslog-ng version: 3.4.3 built from sources syslog-ng -V: syslog-ng 3.4.3 Installer-Version: 3.4.3 Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.4#no_branch#64d670f3cbfb90769f3c7f0fdd9c70bb9136ec5b Compile-Date: Sep 11 2013 05:33:29 Available-Modules: confgen,afsocket-tls,csvparser,affile,system-source,syslogformat,afuser,dbparser,afsocket-notls,basicfuncs,cryptofuncs,afsocket,afprog Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: on On rare occasions (I can't reproduce it reliably) syslog-ng segfaults when reloading (syslog-ng-ctl reload). I have a core dump file that gave me this stack trace: #0 0x00007fbd30102131 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fbd30101d76 in strdup () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007fbd2f868b19 in evt_tag_str (tag=0x7fbd309a42f0 "encoding", value=0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>) at evttags.c:76 #3 0x00007fbd3097918c in log_proto_server_options_validate (options=<optimized out>) at logproto-server.c:138 #4 0x00007fbd3097d191 in log_proto_server_validate_options (self=<optimized out>) at logproto-server.h:82 #5 log_reader_init (s=0x14a6a30) at logreader.c:638 #6 0x00007fbd2e9d5b07 in log_pipe_init (cfg=0x0, s=0x14a6a30) at ../../lib/logpipe.h:253 #7 log_pipe_init (cfg=0x0, s=0x14a6a30) at afsocket-source.c:763 #8 afsocket_sc_init (s=0x14a68f0) at afsocket-source.c:149 #9 0x00007fbd2e9d67fd in log_pipe_init (cfg=0x0, s=0x14a68f0) at ../../lib/logpipe.h:253 #10 log_pipe_init (cfg=0x0, s=0x14a68f0) at afsocket-source.c:763 #11 afsocket_sd_init (s=0x1489630) at afsocket-source.c:548 #12 0x00007fbd2e9d9719 in afunix_sd_init (s=<optimized out>) at afunix-source.c:194 #13 0x00007fbd3096d27a in log_pipe_init (cfg=<optimized out>, s=0x1489630) at logpipe.h:253 #14 cfg_tree_start (self=0x1c15c48) at cfg-tree.c:1064 #15 0x00007fbd309685de in cfg_init (cfg=0x1c15ae0) at cfg.c:220 #16 0x00007fbd30984faf in main_loop_reload_config_apply () at mainloop.c:501 #17 0x00007fbd3096d699 in control_connection_reload (self=0x1d09e90, command=<optimized out>) at control.c:150 #18 0x00007fbd3096dae8 in control_connection_io_input (s=0x1d09e90) at control.c:242 #19 control_connection_io_input (s=0x1d09e90) at control.c:172 #20 0x00007fbd309a106c in iv_fd_poll_and_run (st=0x1482580, to=<optimized out>) at iv_fd.c:163 #21 0x00007fbd309a1aac in iv_main () at iv_main_posix.c:117 #22 0x00007fbd309859fb in main_loop_run () at mainloop.c:736 #23 0x000000000040142b in main (argc=1, argv=0x7fffd8212a58) at main.c:267 When at frame #5, I got this: (gdb) print *self->proto->options $19 = {destroy = 0, initialized = -1, encoding = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, convert = 0xffffffffffffffff, max_msg_size = -1, max_buffer_size = 0, init_buffer_size = 0} I could not find anywhere in the code where initialized is set to -1, so I assume this is some kind of memory corruption. Let me know if you need anything else from me. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 16

[Bug 226] New: 3.3.8: iv_tls_user_register: called after iv_init
by bugzilla＠bugzilla.balabit.com 21 Nov '13

21 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=226 Summary: 3.3.8: iv_tls_user_register: called after iv_init Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: bug Estimated Hours: 0.0 When building with libivykis library (internal or external) starting syslog-ng 3.3.8 ends up with: # /sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf iv_tls_user_register: called after iv_init System is Linux 3.7.10, glibc 2.17. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

redis destination bug
by Otto Berger 21 Nov '13

21 Nov '13

Hi, found a small bug in the redis destination module. I actually dont know where to file a report for it, maybe someone here knows... In modules/redis/redis-parser.c the host and port assignments are missing: { "host", KW_HOST }, { "port", KW_PORT }, without these it is not possible to connect to redis servers other than the default (localhost). The redis destination works very well btw (tested at a quite large deployment). cheers Otto -- www.bergerdata.de

2 2

Re: [syslog-ng] [review request] 3.6queue/f/stats-reset
by Jim Hendrick 20 Nov '13

20 Nov '13

If desired, could a "zero-counters" be added to syslog-ng-ctl ? Not critical but maybe useful for troubleshooting. Jim Sent from my Verizon Wireless 4G LTE Smartphone -------- Original message -------- From: Gergely Nagy <algernon(a)balabit.hu> Date: 11/20/2013 8:47 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Cc: syslog-ng-dev-l(a)balabit.hu Subject: Re: [syslog-ng] [review request] 3.6queue/f/stats-reset Balazs Scheidler <bazsi77(a)gmail.com> writes: > On Mon, 2013-11-18 at 18:22 +0100, Gergely Nagy wrote: >> Evan Rempel <erempel(a)uvic.ca> writes: >> >> > Perhaps the same logic can be used to identify those counters that have expired meaning that the stats-lifetime() has been exceeded,but >> > instead of dropping the stat counter at that time, the counter is just flagged for dropping. >> > >> > Then it becomes the job of the statistics logging, based on the stats-freq() value, to log AND drop those counters that >> > have been flagged as having exceeded the stats-lifetime(). >> > >> > This way a counter is never dropped until it has exceeded the >> > stats-lifetime() AND has been logged. >> >> That makes a lot of sense, but syslog-ng-ctl stats should not count (or >> perhaps only with a flag), precisely to avoid the scenario you outlined >> too. >> > > This really makes sense, however my issue is that stats-freq() is often > set to 0 which disables the Log statistics message currently. Relying > the cleanup mechanism on that would disable cleanup altogether in a lot > of setups. > > hmm... maybe we should use stats-freq() if that's nonzero and > stats-lifetime() when it is zero. This way the cleanup mechanism would > kick in, without the risk of losing messages. > > syslog-ng-ctl stats wouldn't prune counters. > > This way, stats-freq() would be the one which actually clears counters, > we get rid of an additional iteration over the set of counters. > > This paves the way for an improved log statistics message that could be > sent into graphite/logstash/whatever graphing application via standard > syslog-ng destination drivers, and all this without the risk of losing > counters. > > What do you think? Sounds very good to me. This is a big step towards what I need, and is simple enough to understand too. -- |8] ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

2 1