- syslog-ng - lists.balabit.hu

[ANN] syslog-ng 3.5 and 3.6 release plans
by Gergely Nagy 24 Jun '14

24 Jun '14

Hi! I described my plans for syslog-ng maintainance last October[1], and promised that I will give ample warning when 3.6 is about to hit beta. Here's the warning: I plan to release either an alpha, or a beta of 3.6 in early September (I will aim for beta, with an alpha in August), at which point, the 3.3 branch will no longer be maintained, and will be removed from my Debian repository[2] too. At that point, 3.4 will become a maintainance release, 3.5 will be stable, and 3.6 will be the feature branch. [1]: https://lists.balabit.hu/pipermail/syslog-ng/2013-October/020831.html [2]: http://asylum.madhouse-project.org/projects/debian/ However, I'd like to cut down on my tasks, and trying to maintain three versions of syslog-ng is quite a big burden, and I feel I'm not really doing a good job of taking care of the oldest version. Therefore, I'll try to deprecate 3.4 much quicker, and came up with the following plan: * When 3.5.5 is released (mid-July, if all goes well), I will switch the "syslog-ng" component of my Debian repo to 3.5. "syslog-ng-devel" will point to the same place. The 3.4 branch will remain available under the "syslog-ng-3.4" component, of course. One reason I feel this is safe to do, is because most distros migrated to 3.5 already, so 3.4 is not really in use. Also, with 3.5, you get the Incubator as a bonus, and increasing Incubator use would be great, too. * When 3.6 hits, I'd like to declare 3.4 "deprecated", and stop supporting it in a few months. This means that in 2015, I'd like to maintain only 3.5 and 3.6. This could all change, mind you, if there's enough demand for the maintainance of three versions. If you have good reasons for using 3.4, or wishing there were three maintained branches, do let me know! -- |8]

1 0

New release: puppet module for patterndb
by Fabien Wernli 23 Jun '14

23 Jun '14

Hi, I'm proud to announce the availability of version 2.0.0 of the patterndb puppet module. This new release adds the possibility to manage multiple pattern parsers, and fixes a number of bugs. cheers https://forge.puppetlabs.com/ccin2p3/patterndb

1 0

[Bug 279] New: Syslog-ng central loging server seg fault gentoo
by bugzilla＠bugzilla.balabit.com 23 Jun '14

23 Jun '14

https://bugzilla.balabit.com/show_bug.cgi?id=279 Summary: Syslog-ng central loging server seg fault gentoo Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: hlavacek(a)gmx.com Type of the Report: bug Estimated Hours: 0.0 Hello, we have set up the following topology: OS gentoo: Linux syslog1 3.8.12-hardened #1 SMP Wed Sep 18 16:30:57 CEST 2013 x86_64 Intel(R) Xeon(TM) CPU 3.00GHz GenuineIntel GNU/Linux syslog: syslog server v3.4.7 and 40 syslog clients v3.x.x. problem: After a few days (5 or 10) server experienced segfault: [309942.605750] syslog-ng[17117]: segfault at 8 ip 00007fdb01e66609 sp 00007fdafe1ccac0 error 6 in libsyslog-ng-3.4.7.so[7fdb01e2b000+8f000] and then after two days with any error syslog-ng freezed in loop (cpu 100%), strace: futex(0x7fdc64009ca0, FUTEX_WAIT_PRIVATE, 2, NULL This scenario is being periodicaly repeated. We have faced this problem also with version 3.3 of syslog. BACKTRACE after segfault: syslog1 run # gdb syslog-ng -c core.syslog.201422041822 GNU gdb (Gentoo 7.6.2 p1) 7.6.2 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /usr/sbin/syslog-ng...(no debugging symbols found)...done. [New LWP 26604] [New LWP 24631] [New LWP 1932] [New LWP 1933] [New LWP 1710] [New LWP 1456] [New LWP 1931] [New LWP 1713] [New LWP 1749] warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `syslog-ng --enable-core -c /var/run/syslog-ng-remote.ctl --persist-file /var/li'. Program terminated with signal 11, Segmentation fault. #0 0x00007f03fdb44a59 in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so (gdb) bt full #0 0x00007f03fdb44a59 in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #1 0x00007f03fdb44b6a in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #2 0x00007f03fdb4cf35 in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #3 0x00007f03fdb6d42b in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #4 0x00007f03fdb6c3c2 in iv_run_tasks () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #5 0x00007f03fdb6ee1b in iv_main () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #6 0x00007f03fdb6d21a in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #7 0x00007f03fdb6f736 in ?? () from /usr/lib64/libsyslog-ng-3.4.7.so No symbol table info available. #8 0x00007f03fd5bcbd8 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00007f03fd2f7bbd in clone () from /lib64/libc.so.6 No symbol table info available. (gdb) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

Internal error, duplicate configuration elements...
by David Hauck 15 Jun '14

15 Jun '14

Hi, I've been playing around with configuring UDP destinations. My syntax checks ('syslog-ng -s') and high-level operation appear to be sound. However, I've noticed the following "internal error" that's issued whenever I start the service: 20140604 09:54:47.780 err syslog(syslog-ng):Internal error, duplicate configuration elements refer to the same persistent config; name='afsocket_dd_connection(dgram,205.159.216.207:514)' The UDP configuration references from 'syslog-ng.conf' are as follows: destination d_NAaudit_Prio { file("/var/log/netacquire/audit_log" template(t_NAFormat_Prio)); udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; destination d_NAmessage_Prio { udp("205.159.216.207" port(514) template(t_NAFormat_Prio)); }; Does anyone know what this message is referring too? Things seem to work fine (e.g., log statements with these destinations do, ultimately, send the filtered messages to the target system), so I'm wondering if I need to be concerned at all. Thanks, -David

4 9

Salt plugin for syslog-ng config file generation
by Tibor Benke 11 Jun '14

11 Jun '14

Hi All, My name is Tibor Benke and I'm a participant of GSoC 2014. I wrote a state module for Salt which is able to generate syslog-ng configuration from YAML format. With this approach, you can deploy syslog-ng configurations simultaneously on thousands of computers based on their operating system or other parameters. It's in beta state, but the configuration generation works. If you want to try it, you can find a HOWTO here: https://github.com/ihrwein/salt/wiki/Syslog-ng-state-HOWTO And my blog here: http://btibi.blogspot.hu/ Regards, Tibi

1 0

Syslog-NG directory name with IP address instead of hostname
by Justin B 11 Jun '14

11 Jun '14

Hi When I receive the application messaged from a different server to my syslog-ng server it is trying to create the messages under the directory name with IP address instead of the Application Server Hostname. What can be done baout this?? i used the following options in syslog-ng.conf options { use_fqdn(no); flush_lines(0); keep_hostname(yes); check_hostname(yes); chain_hostnames(yes); create_dirs(yes); dir_owner(root); dir_group(test); dir_perm(0750); #use_dns(persist_only); use_dns(yes); #dns_cache_hosts(/etc/hosts); dns_cache(yes); dns_cache_expire(87600); ts_format(iso); # keep_timestamp(no); }; -- Appreciate your quick help.. Kaladhar

3 4

syslog-ng 3.4.8 has been released
by devel＠balabit.hu 06 Jun '14

06 Jun '14

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.8 SUMMARY : new stable release DATE : Jun 6, 2014 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.8) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.8 Fri, 6 Jun 2014 12:05:59 +0200 This is the seventh bug-fix release for the 3.4.x series, with a set of important fixes over 3.4.7 released last December. Upgrading to this release is highly recommended. Features ======== * Support for tzif3 format timezone files (used by recent releases of the tzdata package) was implemented. * SDATA enterprise IDs are now handled correctly when using value-pairs, they will not be needlessly split up into smaller chunks. Bugfixes ======== * Drivers using value-pairs() (mongodb, format-json, etc) were made more robust, and will not behave strangely anymore when using higher compiler optimisation levels. * The DATE macro in most value-pairs() scopes will now always contain the stamp time, as opposed to sometimes the received or the stamp time. * The syslog-proto (rfc5424) scope of value-pairs() now includes SDATA too. * The $SEQNUM macro is now correctly handled by the $(format-json) template function. * Drivers using value-pairs() will now work correctly when a macro is specified in a key() statement. * When using the same json-parser() in multiple log paths, syslog-ng no longer crashes when no marker was set. * When a GeoIP lookup fails, handle it gracefully, avoiding an annoying glib warning. * When running in a FreeBSD jail, the system() source will not use /dev/klog. * Various memory leaks were plugged (related to internal locking, and value-pairs). * When building from Git, the grammar files are now build-able with bison 3+. Builds from tarballs are unaffected. Miscellaneous changes ===================== * The host() and port() options of the MongoDB destination driver have been labelled as deprecated. They already were, just not marked so. * The username() and password() options of the AMQP driver were made mandatory. Credits ======= syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Andras Mitzki <micek(a)balabit.hu> Balazs Scheidler <bazsi(a)balabit.hu> Daniel Gados <gdani(a)balabit.hu> Fabien Wernli Gergely Nagy <algernon(a)balabit.hu> Igor Ippolitov <iippolitov(a)gmail.com> Laszlo Meszaros <lmesz(a)balabit.hu> Nick Alcock <nix(a)esperi.org.uk> Peter Czanik <czanik(a)balabit.hu> Tibor Benke <btibi(a)balabit.hu> Viktor Tusa <tusa(a)balabit.hu> Xufeng Zhang <xufeng.zhang(a)windriver.com> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Facility Filter Range
by David Hauck 05 Jun '14

05 Jun '14

Hi, I thought I must have just messed up some syntax here, but nothing I do seems to work. The issue is with my attempts to specify filter facility function *ranges* - here's the error message (from "syslog-ng -s"): "Error parsing filter expression, syntax error, unexpected LL_DOTDOT, expecting ')' in /etc/syslog-ng.conf at line 69, column 35: filter f_namain { facility(local0 .. local7) and program(namain); }; ..." Filter function specifications like "facility(kern, authpriv, cron)" and "level(warn .. emerg)" appear to work fine. Am I doing something obviously wrong? Thanks, -David PS: This is with version 3.5.4.1.

3 5

[ANN]: Moving from BugZilla to GitHub Issues
by Gergely Nagy 05 Jun '14

05 Jun '14

Greetings! Internally, we've been talking about retiring the syslog-ng OSE BugZilla (https://bugzilla.balabit.com/) for quite a while, because it's far too complicated, outdated, and generally a pain to work with. Since syslog-ng development moved to GitHub a while back, we've been using GitHub issues alongside the bugzilla, and the time has come to say good bye to BugZilla. It served us well for many years, but it is time to let go, and use something more convenient: GitHub issues. On the 13th of June, we will close our BugZilla for new issues, and you will not be able to report new problems there. Existing issues will remain there, and will not be closed for further updates. The new place to report bugs is: https://github.com/balabit/syslog-ng/issues We are also using waffle.io for both syslog-ng and the Incubator, the boards are available at https://waffle.io/balabit/syslog-ng and https://waffle.io/balabit/syslog-ng-incubator, respectively. Both are a work in progress, mind you. GitHub issues are much simpler than BugZilla, however, and as a consequence, have less features too: attaching a config, or a core file, or similar files is not possible. But the need for that is fairly low, and if so need be, there are alternative ways to do all that: paste the config into the issue, create a Gist (https://gist.github.com/) use Dropbox, and so on. If you have any questions, observations, or perhaps complaints about the planned migration, let us know! -- |8]

1 0

bug 278
by Frank Wilkinson 04 Jun '14

04 Jun '14

[Bug 278] Does the memory leak problem only happen when you do a configuration reload, as opposed to restart. Thanks

3 2

[Bug 278] New: Many memory leak problems happen when do configuration reloading
by bugzilla＠bugzilla.balabit.com 03 Jun '14

03 Jun '14

https://bugzilla.balabit.com/show_bug.cgi?id=278 Summary: Many memory leak problems happen when do configuration reloading Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: xufeng.zhang(a)windriver.com Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=95) --> (https://bugzilla.balabit.com/attachment.cgi?id=95) the config and script files to reproduce this problem I found many memory leak problems on syslog-ng_3.5.4.1, I have resolved some of them, but there are still some others need to be resolved: 1). mutex problem such as: ==1077== 40 bytes in 1 blocks are definitely lost in loss record 514 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x36886024FE: ??? (in /lib64/libgthread-2.0.so.0.2600.0) ==1077== by 0x3688266B23: g_static_mutex_get_mutex_impl (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x523D83C: affile_dw_queue (affile-dest.c:263) ==1077== by 0x523D05B: affile_dd_queue (logpipe.h:320) ==1077== by 0x3688E32150: log_multiplexer_queue (logpipe.h:320) patch Fix-the-memory-leak-problem-for-mutex.patch could resolve them. 2). memory leak when HAVE_ENVIRON is defined: ==1077== 1,258 (208 direct, 1,050 indirect) bytes in 1 blocks are definitely lost in loss record 617 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E2F5FF: g_process_set_argv_space (gprocess.c:502) ==1077== by 0x40164C: main (main.c:196) patch Fix-the-memory-leak-problem-when-HAVE_ENVIRON-defined.patch could resolve them. 3). memory leak problem for unconsumed item during configuration reloading: ==1077== 36,864 bytes in 45 blocks are definitely lost in loss record 636 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E3D178: log_writer_flush (logwriter.c:973) ==1077== by 0x3688E3D2D2: log_writer_deinit (logwriter.c:1169) ==1077== by 0x523C09A: affile_dw_deinit (logpipe.h:268) ==1077== by 0x523C93B: affile_dd_deinit (logpipe.h:268) ==1077== by 0x3688E2A4FE: cfg_tree_stop (logpipe.h:268) ==1077== by 0x3688E3DB9F: main_loop_reload_config_apply (mainloop.c:540) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== by 0x3688E635F1: iv_fd_poll_and_run (iv_fd.c:163) ==1077== by 0x3688E63D73: iv_main (iv_main_posix.c:117) patch logwriter-still-free-the-unconsumed-item.patch 4). The remaining memory leak problems need to be resolved, and they are not in 3.4.2 version: ==1077== 24 bytes in 1 blocks are definitely lost in loss record 483 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5023BCA: socket_options_new (socket-options.c:90) ==1077== by 0x502568A: afunix_sd_new_instance (afunix-source.c:241) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== ==1077== 24 bytes in 1 blocks are definitely lost in loss record 484 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5023BCA: socket_options_new (socket-options.c:90) ==1077== by 0x502568A: afunix_sd_new_instance (afunix-source.c:241) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 64 bytes in 1 blocks are definitely lost in loss record 531 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5025A81: transport_mapper_unix_new_instance (transport-mapper-unix.c:40) ==1077== by 0x5025B04: transport_mapper_unix_dgram_new (transport-mapper-unix.c:51) ==1077== by 0x50257C8: afunix_sd_new_dgram (afunix-source.c:262) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== ==1077== 64 bytes in 1 blocks are definitely lost in loss record 532 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5025A81: transport_mapper_unix_new_instance (transport-mapper-unix.c:40) ==1077== by 0x5025B04: transport_mapper_unix_dgram_new (transport-mapper-unix.c:51) ==1077== by 0x50257C8: afunix_sd_new_dgram (afunix-source.c:262) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 96 (24 direct, 72 indirect) bytes in 1 blocks are definitely lost in loss record 541 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5EC74: log_template_compile (templates.c:1232) ==1077== by 0x3688E507AF: main_parse (cfg-grammar.y:741) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== 144 (48 direct, 96 indirect) bytes in 2 blocks are definitely lost in loss record 556 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5E661: log_template_compiler_process_token (templates.c:1129) ==1077== by 0x3688E5EBD8: log_template_compile (templates.c:1222) ==1077== by 0x3688E507AF: main_parse (cfg-grammar.y:741) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 1,008 bytes in 42 blocks are definitely lost in loss record 606 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5EC74: log_template_compile (templates.c:1232) ==1077== by 0x523BC1E: affile_dd_new_instance (affile-dest.c:706) ==1077== by 0x523BCA8: affile_dd_new (affile-dest.c:722) ==1077== by 0x523EEED: affile_parse (affile-grammar.y:490) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4FFCA: main_parse (cfg-grammar.y:641) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== 176 bytes in 1 blocks are definitely lost in loss record 563 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x5022582: afsocket_sc_init (afsocket-source.c:91) ==1077== by 0x5021FC7: afsocket_sd_process_connection (logpipe.h:253) ==1077== by 0x5022B6D: afsocket_sd_init_method (afsocket-source.c:540) ==1077== by 0x5025855: afunix_sd_init (afunix-source.c:223) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3E48C: main_loop_init (mainloop.c:527) ==1077== by 0x401774: main (main.c:246) ==1077== ==1077== 176 bytes in 1 blocks are definitely lost in loss record 564 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x523AC56: affile_sd_construct_poll_events (affile-source.c:195) ==1077== by 0x523B696: affile_sd_init (affile-source.c:375) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3E48C: main_loop_init (mainloop.c:527) ==1077== by 0x401774: main (main.c:246) ==1077== 352 bytes in 2 blocks are definitely lost in loss record 576 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x523AC56: affile_sd_construct_poll_events (affile-source.c:195) ==1077== by 0x523B696: affile_sd_init (affile-source.c:375) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3DBBE: main_loop_reload_config_apply (mainloop.c:543) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== by 0x3688E635F1: iv_fd_poll_and_run (iv_fd.c:163) ==1077== by 0x3688E63D73: iv_main (iv_main_posix.c:117) When udp connection is used, I found three new additional memory leaks(I have applied the attached three patches): ==25796== 3,072 bytes in 3 blocks are definitely lost in loss record 616 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B742478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==25796== by 0x3B74E3D1D8: log_writer_flush (logwriter.c:974) ==25796== by 0x3B74E3D2BC: log_writer_work_perform (logwriter.c:163) ==25796== by 0x3B74E3D327: log_writer_io_flush_output (logwriter.c:243) ==25796== by 0x3B74E63666: iv_fd_poll_and_run (iv_fd.c:167) ==25796== by 0x3B74E63E13: iv_main (iv_main_posix.c:117) ==25796== by 0x3B74E3D7E6: main_loop_run (mainloop.c:788) ==25796== by 0x4017A0: main (main.c:267) ==25796== 20 bytes in 2 blocks are definitely lost in loss record 17 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3E4CE: main_loop_init (mainloop.c:729) ==25796== by 0x401774: main (main.c:246) ==25796== 40 bytes in 4 blocks are definitely lost in loss record 514 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3DB85: main_loop_reload_config_initiate (mainloop.c:611) ==25796== by 0x3B74E6457A: iv_signal_event (iv_signal.c:170) ==25796== by 0x3B74E62FC8: iv_event_raw_got_event (iv_event_raw_posix.c:89) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 15

New puppet module for patterndb
by Fabien Wernli 30 May '14

30 May '14

Hi, I'd like to annouce the availability of our patterndb puppet module, which is almost feature complete. Before submitting this to puppetlabs, I'd much appreciate puppet/syslog-ng users to test it and post feedback either here, or on github. Thanks for your time! https://github.com/ccin2p3/puppet-patterndb Cheers

1 0

Question on behavior of unix-stream destination when data is returned...
by Robert Nickel 29 May '14

29 May '14

So, I have the following configuration to send logs out to a unix domain socket: source s_tls { syslog( ip( 0.0.0.0 ) port( 6514 ) transport( "tls" ) tls( ca_dir( "/path/to/cacerts" ) key_file( "/path/to/private/key" ) cert_file( "/path/to/certificate" ) ) ); }; destination d_sock { unix-stream( "/var/tmp/itsa.socket" ); }; log { source( s_tls ); destination( d_sock ); }; Every time a message writes to the socket, the following logs show up: EOF occurred while idle; fd='12' Syslog connection broken; fd='12', server='AF_UNIX(/var/tmp/itsa.socket)', time_reopen='60' Digging into the code of the socket listening service, it is apparent that the socket returns messages on receipt of the message. Essentially, every time it gets a well-formatted message, it returns 'Parse message success' or 'Parse message failed'. Disabling all of these messages makes the socket work fine but it will break other code that uses this same socket. Is this expected behavior? Is there a way to get syslog-ng to ignore the returned information on the socket? Thanks! --Robert

1 0

[Bug 281] New: Feature - allow for longer program name
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=281 Summary: Feature - allow for longer program name Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Given the specification for the structured syslog protocol (RFC 5424) allows for an application name to be 48 characters and that the process ID is separate and it is permitted to be 128 characters, it seems unreasonable to truncate the program/PID combination to 32 characters. I realize that the 32 character limit is in the RFC 3164 document, but other syslog systems (rsyslog for example) do not enforce this limit. I propose that you permit the program and PID together (the TAG) to be up to 48 characters is length. Specifically, I am having problems with the very common crond TAG entries of the form run-parts(/etc/cron.hourly)[19482] run-parts(/etc/cron.monthly)[10274] which are being truncated by the syslog-ng server. Thanks, -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 211] New: dbparser pattern order breaks validation
by bugzilla＠bugzilla.balabit.com 29 May '14

29 May '14

https://bugzilla.balabit.com/show_bug.cgi?id=211 Summary: dbparser pattern order breaks validation Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 If two log lines share a common leading portion such as; xlog: backup pg_xlog/000000010000014700000076 xlog: backup pg_xlog/000000010000014700000076 failed then the order of the patterns in the pattern database xml file is important. For example, if the two patterns used are; xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed and the shorter pattern is first, it will match BOTH of the above lines because it matches the leading portion of both lines. If the patterns are entered with the longer pattern first, then the longer pattern will fail on the shorter message, allowing the match to fall through to the shorter pattern matcher, but will match the longer message. I expect the behavior of the dbparser to match with the pattern containing the MOST total static content that also matches all of the parsing macros. Some method where the order is not important need to be used. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

Suppress almost-identical messages?
by Evade Flow 23 May '14

23 May '14

I'm trying to clean up the ad hoc logging in a collection of applications started on an embedded device using the daemon utility. Example: /usr/bin/daemon --name=myapp --output=user.debug --chdir=/MyApp -- /MyApp/bin/myapp There are 10 different applications started this way, written by different individuals/teams, and our syslog-ng.conf file currently looks something like this: @version:3.3.6 source s_local { system(); internal(); }; destination d_localfile { file("/var/log/messages" suppress(30)); }; log { source(s_local); destination(d_localfile); }; Because the --output=user.debug option to the daemon utility will cause the applications' stdout/stderr to be redirected to syslog, I'm a little worried about 'rogue' printf()'s filling up the logs with stuff like: Jan 1 00:00:54 (none) myapp: Battery voltage is 11.07 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.10 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.07 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.14 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.13 volts Jan 1 00:00:55 (none) myapp: Battery voltage is 11.09 volts Jan 1 00:00:55 (none) myapp: Battery voltage is 11.14 volts ... Does syslog-ng support suppression of almost-but-not-quite identical messages? It would be nice to see something like this in the logs: Jan 1 00:55:11 myapp: Battery voltage is 11.07 volts Jan 1 00:55:14 myapp: Last message 'Battery voltage is 1' repeated with nearly identical content 418 times, suppressed by syslog-ng on flahblargle A few of my colleagues have suggested that this is probably more trouble than it's worth, and we should just fix the spammy app(s) rather than try to 'outsmart' them. And I think I agree with them, but... it was my idea to move to syslog-ng, so there'll be a bit more egg on my face than theirs if we accidentally allow a rogue printf() in a hot loop somewhere to escape into production. `:-} Any advice/insight much appreciated, thanks!

4 6

Email
by Anwar El fatayri 22 May '14

22 May '14

syslog-ng(a)lists.balabit.hu El Fatayri Anwar

1 0

Handing over syslog-ng maintenance
by Balazs Scheidler 21 May '14

21 May '14

Dear syslog-ng users, I would like to announce that Gergely Nagy (aka @algernoone on Twitter and algernon on github), is taking over the maintenance of syslog-ng master from me. He was maintaining the older stable versions for a few years now, was also active in creating syslog-ng binaries and started the syslog-ng Incubator, and is also behind the Google Summer of Code effort. I have a lot of distractions in my personal life, and although I don't want to retreat from the syslog-ng community entirely, I wanted to leave syslog-ng in Gergely's able hands. Happy logging, Cheers, Bazsi -- Bazsi

4 3

insider 2014-05: GsoC; perl & python; web interfaces; protecting logs;
by Peter Czanik 15 May '14

15 May '14

Dear syslog-ng users, This is the 34th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. FEATURED NEWS Four GsoC students are working on syslog-ng ------------------------------------------- Thanks to Google, there are four students working on extending syslog-ng with new features during the summer. These are features, which were often requested on the mailing list or at different conferences: - integration with configuration management systems - ZMQ transport, both source and destination - AMQP source driver - TLS support for the mongodb destination Read more about GSoC at https://algernon.blogs.balabit.com/2014/04/gsoc2014-syslog-ng-accepted-proj… Python and Perl support in incubator ------------------------------------ It is still only available in git, as it needs some more polish, but the syslog-ng incubator gained Perl and Python support during the last month. Both the perl and python destinations use the value-pairs framework to get data transferred from syslog-ng to the script, and thus, work differently from the Lua destination. With value-pairs, one can select what parts of the message will be transferred to the script. The script will need to have a queue function (settable with the queue-func() option), which will receive a hash-map of values. Additionally, one can set an init and a deinit function too, to run whenever the driver starts or shuts down. Check it out and let us know your experiences! It's available athttps://github.com/balabit/syslog-ng-incubator Web based user interfaces for syslog-ng --------------------------------------- One of the most popular BalaBit blogs is about syslog-ng web based graphical user interfaces (web GUIs). It's already three years old, and many things have changed. At that time, only a single Logging as a Service solution was available, now a new one pops up regularly. Also, there were a lot less logging-related GUIs, so some not strictly syslog-ng related solutions were included as well. You can read an updated version of the blog, focusing on syslog-ng based solutions at https://czanik.blogs.balabit.com/2014/05/web-based-user-interfaces-for-sysl… Protecting log data against targeted attacks -------------------------------------------- BalaBit has been saying that SIEM and other analytic tools are only as good as the underlying data. Attackers are also aware of this, and often target log management and SIEMs to hide their presence. Read this blog post for some logging best practices and how syslog-ng can help to secure your logging infrastructure: https://jluby.blogs.balabit.com/2014/05/06/protecting-log-data-against-targ… NEW RELEASES * Check git if you are impatient :) Your feedback and news tips about the next issue is welcome at _documentation(a)balabit.com <mailto:documentation@balabit.com>_To read this newsletter on-line, visit: _http://insider.blogs.balabit.com/_ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

weird disk usage problems -- syslog-ng/ELSA
by Russell Fulton 11 May '14

11 May '14

HI Folks It would seem that the issue here is with syslog-ng but it manifested itself in the context of an ELSA instance so I am sending this to both lists, apologies to those who get it twice. I just discovered that my ELSA machine had run out of disk some 18 hours ago. /dev/mapper/datavg-d01 985G 943G 0 100% /d01 [rful011@itslogprd05 ~]$ sudo du -sh /d01/* 671G /d01/data 16K /d01/lost+found 43G /d01/mysql But these figures don’t match — there is about 250GB of disk unaccounted for (although I did not pick up this discrepancy at first — instead I went looking in ELSA node.log to try and figure out why logs were not getting deleted). After failing to find any evidence of problems in the logs I then stopped syslog-ng and set about moving the tmp/buffers to another disk. I copied them but when I went to delete the originals they had vanished! I was dazed and confused ;) until I realised that the usage of /d01 had dropped to 77% which allowed cron.pl to process all the waiting buffers and delete them. It would seem that syslog-ng was sitting on some very large files that had been unlinked from the directory (so du did not see them) stopping syslog-ng would have closed the files allowing the disk to be released. Hmmm…. with syslog-ng it might have been fifos (the ELSA syslog-ng config uses fifos to deliver logs to the ELSA perl scripts) but they should not have had buffers that size. Has anyone seen anything like this before or have any idea what might trigger this issue? Russell

2 1

Issue regarding the configurations of syslog-ng application on Centos
by Christian Hangi 09 May '14

09 May '14

Hello Guys, I am new in Syslog-ng application and also Linux and I need your help regarding the functionality of syslog-ng on Linux. I am building a syslog-ng server capable of receiving syslog messages from syslog-ng clients. In doing so, I would like to those syslog messages to be received and stored inside the Syslog MySQL database of the server inside the syslog_incoming table so that I can be able view those messages. I am using the Centos 6.5 of Redhat distributions and the Syslog-ng application running on that PC acts as a Syslog-ng server. This is my first attempt to allow the syslog messages storage on the database: @version:3.2 # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; #source s_sys { #file ("/proc/kmsg" program_override("kernel: ")); # unix-stream ("/dev/log"); # internal(); # udp(ip(0.0.0.0) port(514)); #}; #A change was made here! source network { syslog(ip(0.0.0.0) port(514) transport(udp));}; destination db_mysql{sql(type(mysql) host("localhost") username("cacti") password("P@ssw0rd") database("syslog") table("syslog_incoming") columns("host", "facility", "priority", "date", "time", "message") values("$HOST","$FACILITY","$PRIORITY","$YEAR-$MONTH-$DAY","$HOUR:$MIN:$SEC","$MSG") indexes("host", "facility", "priority", "date", "time", "message") );}; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(s_sys);source(network); destination(d_mysql);}; #log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: And this is the message I got when I tried to restart the application : Stopping syslog-ng: [FAILED] Starting syslog-ng: Error in configuration, unresolved destination reference; destination='d_mysql' [FAILED] Then I tried to use a named pipe to accept and send thoses messages to the syslog_incoming table of the Syslog MySQL database through a template: @version:3.2 # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; #source s_sys { # file ("/proc/kmsg" program_override("kernel: ")); # unix-stream ("/dev/log"); # internal(); # udp(ip(0.0.0.0) port(514)); #}; destination d_mysql{ pipe("/var/log/mysql.pipe" template("INSERT INTO syslog_incoming(host, facility, priority, date, time, message) VALUES('$HOST', '$FACILITY', '$PRIORITY', '$ YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$MSG');\n") template-escape(yes)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(network); destination(d_mysql);}; #log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: And I got this message assuming that the syslog-ng server is working Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ] But when I checked to the table of the database it was empty. So, I would like to know, do you miss something on the first attempt to allow the syslog-server to work perfectly or should I just not to uncomment the source system (s_sys) to accept the syslog messages from the Syslog-clients? On the 2nd attempt, what needs to be done to acctually see those messages? Please I need some advice on those two attempts. Kindly regards, Chris Hangi

2 1

[ANN]: syslog-ng Incubator maintenance
by Gergely Nagy 07 May '14

07 May '14

Hi! While it has effectively happened about a month ago, I didn't get around to announce the good news yet: syslog-ng Incubator maintenance has been handed over to Viktor Tusa <tusa(a)balabit.hu>, author of (in no particular order) the Lua and RSS destinations, graphite-output template function, and more. He has been doing a terrific job so far, and I'm very happy with the maintainer switch. The Incubator is in the best hands possible. Thank you, Viktor! -- |8]

1 0

AMQP source
by MÓZES Ádám István 05 May '14

05 May '14

Hi All, My name is MÓZES Ádám István, and I'll develop the AMQP source this summer. A short introduction about me and the progress of the development can be found at this blog: http://dnsjts.blogspot.hu/search/label/gsoc2014 . Best regards, Ádám

1 0

Re: [syslog-ng] Ubuntu Precise -ng filling out buffer, dropping messages
by Chaman Chakalaka 05 May '14

05 May '14

Just to close the circle on this, I found the problem. After having increased the rmem_max I was no longer seeing any UDP packet drops. I was still not seeing some log messages, and at this point it was a configuration issue. I noticed I had filters such as: filter f_info { level(info);} Obviously this was only catching info level messages. Changed to: filter f_info {level(info...emerg);} Sorry for the noise, but I still wanted to bring some closure in case someone runs into this. The initial packet drop blinded me into looking into some basic configs, thanks all for the suggestions! > From: syslog-ng-request(a)lists.balabit.hu > Subject: syslog-ng Digest, Vol 108, Issue 27 > To: syslog-ng(a)lists.balabit.hu > Date: Wed, 30 Apr 2014 12:00:02 +0200 > > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: Ubuntu Precise -ng filling out buffer, dropping messages > (Evan Rempel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 29 Apr 2014 12:38:52 -0700 > From: Evan Rempel <erempel(a)uvic.ca> > Subject: Re: [syslog-ng] Ubuntu Precise -ng filling out buffer, > dropping messages > To: syslog-ng(a)lists.balabit.hu > Message-ID: <535FFFCC.8000203(a)uvic.ca> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 04/29/2014 12:20 PM, Chaman Chakalaka wrote: > > I fixed the issue with udp being dropped at the system level by changing the linux core files but this time restarting the system, now I know they are missing somewhere between getting to the system and syslog-ng :( > > > > I'm lost once again... > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > trysource s_network_udp { udp(so_rcvbuf(33554432) log_fetch_limit(20000) log_iw_size(1000000) ); }; > > > This assumes that you have a large net.core.rmem_max > > net.core.rmem_max = 52428800 > > See how that goes. > > > ------------------------------ > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > End of syslog-ng Digest, Vol 108, Issue 27 > ******************************************

1 0

ZMQ transport, source and destination
by Laci Mészáros 04 May '14

04 May '14

Hy all, I am László Mészáros a student at Szechenyi Istvan University learning software engineering. I've made a proposal for Google Summer of Code to implement the zeromq transport, source and destination. The organization accepted it, so during this summer I will work on it. You can find the proposal here: https://github.com/balabit/syslog-ng/wiki/GSoC2014-Proposal:-ZMQ-transport,… I made a blog where you can follow the status, it can be found here: http://lmesz.tumblr.com/post/84705072064/introduction-aims I think one part it will be fun and on another it will be difficult, so I would like to ask all of you, if you have some use cases on your shelf about how you would/will use zmq for logging via network, please share it with us. You can send it to the list or to me personally it would be very useful during the implementation. Br, Laci

1 0