August 2006 - syslog-ng - lists.balabit.hu

Re: syslog-ng 2.0rc1 memory usage
by Martin, David M 08 Aug '06

08 Aug '06

I ran valgrind with these options for 10 minutes: valgrind --tool=memcheck -v --leak-check=yes --show-reachable=yes --log-file=/tmp/debug.txt ./syslog-ng -dv Here's the output: ==15455== Memcheck, a memory error detector for x86-linux. ==15455== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al. ==15455== Using valgrind-2.2.0, a program supervision framework for x86-linux. ==15455== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al. ==15455== ==15455== My PID = 15455, parent PID = 5730. Prog and args are: ==15455== ./syslog-ng ==15455== -dv ==15455== ==15455== Valgrind library directory: /usr/lib/valgrind ==15455== Command line ==15455== ./syslog-ng ==15455== -dv ==15455== Startup, with flags: ==15455== --tool=memcheck ==15455== -v ==15455== --leak-check=yes ==15455== --show-reachable=yes ==15455== --log-file=/tmp/debug.txt ==15455== Contents of /proc/version: ==15455== Linux version 2.6.9-34.ELsmp (bhcompile(a)hs20-bc1-7.build.redhat.com) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Fri Feb 24 16:54:53 EST 2006 ==15455== Reading syms from /opt/sfw/syslogng/2.0rc1/sbin/syslog-ng (0x8048000) ==15455== Reading syms from /lib/ld-2.3.4.so (0x1B8E4000) ==15455== object doesn't have any debug info ==15455== Reading syms from /usr/lib/valgrind/stage2 (0xB0000000) ==15455== Reading syms from /lib/ld-2.3.4.so (0xB1000000) ==15455== object doesn't have any debug info ==15455== Reading syms from /usr/lib/valgrind/vgskin_memcheck.so (0xB7BEB000) ==15455== Reading syms from /lib/tls/libc-2.3.4.so (0xB7E14000) ==15455== object doesn't have any debug info ==15455== Reading syms from /lib/libdl-2.3.4.so (0xB7F3F000) ==15455== object doesn't have any debug info ==15455== Reading suppressions file: /usr/lib/valgrind/default.supp ==15455== REDIRECT soname:libc.so.6(__GI___errno_location) to soname:libpthread.so.0(__errno_location) ==15455== REDIRECT soname:libc.so.6(__errno_location) to soname:libpthread.so.0(__errno_location) ==15455== REDIRECT soname:libc.so.6(__GI___h_errno_location) to soname:libpthread.so.0(__h_errno_location) ==15455== REDIRECT soname:libc.so.6(__h_errno_location) to soname:libpthread.so.0(__h_errno_location) ==15455== REDIRECT soname:libc.so.6(__GI___res_state) to soname:libpthread.so.0(__res_state) ==15455== REDIRECT soname:libc.so.6(__res_state) to soname:libpthread.so.0(__res_state) ==15455== REDIRECT soname:libc.so.6(stpcpy) to *vgpreload_memcheck.so*(stpcpy) ==15455== REDIRECT soname:libc.so.6(strnlen) to *vgpreload_memcheck.so*(strnlen) ==15455== REDIRECT soname:ld-linux.so.2(stpcpy) to *vgpreload_memcheck.so*(stpcpy) ==15455== REDIRECT soname:ld-linux.so.2(strchr) to *vgpreload_memcheck.so*(strchr) ==15455== ==15455== Reading syms from /usr/lib/valgrind/vg_inject.so (0x1B8FC000) ==15455== Reading syms from /usr/lib/valgrind/vgpreload_memcheck.so (0x1B901000) ==15455== TRANSLATE: 0x1B8F5F70 redirected to 0x1B904100 ==15455== Reading syms from /lib/libnsl-2.3.4.so (0xC09000) ==15455== object doesn't have any debug info ==15455== Reading syms from /lib/tls/librt-2.3.4.so (0xDD5000) ==15455== object doesn't have any debug info ==15455== Reading syms from /usr/lib/libglib-2.0.so.0.400.7 (0x101000) ==15455== object doesn't have a symbol table ==15455== object doesn't have any debug info ==15455== Reading syms from /opt/sfw/eventlog/0.2.5/lib/libevtlog.so.0.0.0 (0x1B910000) ==15455== Reading syms from /lib/tls/libc-2.3.4.so (0xA60000) ==15455== object doesn't have any debug info ==15455== Reading syms from /usr/lib/valgrind/libpthread.so (0x1B917000) ==15455== Reading syms from /lib/libdl-2.3.4.so (0xB8C000) ==15455== object doesn't have any debug info ==15455== TRANSLATE: 0xAC4030 redirected to 0x1B9048F8 ==15455== TRANSLATE: 0x1B8E47A0 redirected to 0x52BFF040 ==15455== TRANSLATE: 0xAC1F60 redirected to 0x1B904E24 ==15455== TRANSLATE: 0xAC3D50 redirected to 0x1B905274 ==15455== TRANSLATE: 0xA752D0 redirected to 0x1B91EB87 ==15455== TRANSLATE: 0xACA370 redirected to 0x1B9046F4 ==15455== Reading syms from /lib/libnss_files-2.3.4.so (0x1BA51000) ==15455== object doesn't have any debug info ==15455== TRANSLATE: 0xAC4670 redirected to 0x1B905326 ==15455== discard syms at 0x1BA51000-0x1BA5C000 in /lib/libnss_files-2.3.4.so due to munmap() ==15455== ==15455== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 22 from 2) --15455-- --15455-- supp: 2 _dl_relocate_object/dl_open_worker --15455-- supp: 20 dl_relocate_object/dl_main ==15455== malloc/free: in use at exit: 47114 bytes in 370 blocks. ==15455== malloc/free: 535 allocs, 165 frees, 58731 bytes allocated. ==15455== ==15455== searching for pointers to 370 not-freed blocks. ==15459== TRANSLATE: 0xAC9000 redirected to 0x1B9042D0 ==15455== checked 2396536 bytes. ==15455== ==15455== 8 bytes in 2 blocks are still reachable in loss record 1 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAF72C8: register_state (in /lib/tls/libc-2.3.4.so) ==15455== by 0xAF765A: create_cd_newstate (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB004C5: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== ==15455== ==15455== 8 bytes in 2 blocks are still reachable in loss record 2 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAF7439: create_cd_newstate (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB004C5: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== ==15455== ==15455== 12 bytes in 1 blocks are still reachable in loss record 3 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0x1B9123A7: evt_ctx_tag_hook_add (evtctx.c:126) ==15455== by 0x1B91248B: evt_ctx_init (evtctx.c:151) ==15455== by 0x804C21F: msg_init (messages.c:123) ==15455== ==15455== ==15455== 16 bytes in 1 blocks are still reachable in loss record 4 of 19 ==15455== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15455== by 0xB8D364: _dlerror_run (in /lib/libdl-2.3.4.so) ==15455== by 0xB8CE3B: dlsym (in /lib/libdl-2.3.4.so) ==15455== by 0x1B91ECA6: __errno_location (vg_libpthread.c:2127) ==15455== ==15455== ==15455== 56 bytes in 14 blocks are still reachable in loss record 5 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAF9D1A: calc_eclosure_iter (in /lib/tls/libc-2.3.4.so) ==15455== by 0xAFF83F: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== ==15455== ==15455== 56 bytes in 2 blocks are still reachable in loss record 6 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAFF59E: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 92 bytes in 1 blocks are still reachable in loss record 7 of 19 ==15455== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15455== by 0x1B912410: evt_ctx_init (evtctx.c:142) ==15455== by 0x804C21F: msg_init (messages.c:123) ==15455== by 0x804BA3E: main (main.c:389) ==15455== ==15455== ==15455== 104 bytes in 2 blocks are still reachable in loss record 8 of 19 ==15455== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15455== by 0xAF73F8: create_cd_newstate (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB004C5: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== ==15455== ==15455== 112 bytes in 2 blocks are still reachable in loss record 9 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAFEF10: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 168 bytes in 2 blocks are still reachable in loss record 10 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAFF5F7: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 168 bytes in 2 blocks are still reachable in loss record 11 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAFF5E3: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 168 bytes in 2 blocks are still reachable in loss record 12 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xAFF5CF: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 288 bytes in 2 blocks are still reachable in loss record 14 of 19 ==15455== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15455== by 0xAFEF3A: re_compile_internal (in /lib/tls/libc-2.3.4.so) ==15455== by 0xB012AB: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== ==15455== ==15455== 512 bytes in 2 blocks are still reachable in loss record 15 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0xB0123F: regcomp (in /lib/tls/libc-2.3.4.so) ==15455== by 0x8051B1C: filter_re_compile (filter.c:266) ==15455== by 0x8051D81: filter_host_new (filter.c:345) ==15455== ==15455== ==15455== 3608 bytes in 120 blocks are still reachable in loss record 16 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0x1B905373: realloc (vg_replace_malloc.c:189) ==15455== by 0x12AC4B: g_realloc (in /usr/lib/libglib-2.0.so.0.400.7) ==15455== by 0x13C55B: (within /usr/lib/libglib-2.0.so.0.400.7) ==15455== ==15455== ==15455== 4780 bytes in 118 blocks are still reachable in loss record 17 of 19 ==15455== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15455== by 0x12ABDF: g_malloc0 (in /usr/lib/libglib-2.0.so.0.400.7) ==15455== by 0x804C9F6: cfg_new (cfg.c:231) ==15455== by 0x804BA53: main (main.c:392) ==15455== ==15455== ==15455== 16438 bytes in 3 blocks are still reachable in loss record 18 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0x804E8C6: yyalloc (cfg-lex.c:1958) ==15455== by 0x804E537: yyensure_buffer_stack (cfg-lex.c:1693) ==15455== by 0x804DFAC: yyrestart (cfg-lex.c:1467) ==15455== ==15455== ==15455== 20320 bytes in 91 blocks are still reachable in loss record 19 of 19 ==15455== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15455== by 0x12AB72: g_malloc (in /usr/lib/libglib-2.0.so.0.400.7) ==15455== by 0x12B6D6: g_mem_chunk_new (in /usr/lib/libglib-2.0.so.0.400.7) ==15455== by 0x130DD7: g_queue_new (in /usr/lib/libglib-2.0.so.0.400.7) ==15455== ==15455== LEAK SUMMARY: ==15455== definitely lost: 0 bytes in 0 blocks. ==15455== possibly lost: 0 bytes in 0 blocks. ==15455== still reachable: 46914 bytes in 369 blocks. ==15455== suppressed: 200 bytes in 1 blocks. --15455-- TT/TC: 0 tc sectors discarded. --15455-- 6718 tt_fast misses. --15455-- translate: new 6280 (108519 -> 1438685; ratio 132:10) --15455-- discard 124 (1626 -> 21552; ratio 132:10). --15455-- chainings: 4327 chainings, 3 unchainings. --15455-- dispatch: 150000 jumps (bb entries); of them 33013 (22%) unchained. --15455-- 186/7935 major/minor sched events. --15455-- reg-alloc: 1249 t-req-spill, 266791+8325 orig+spill uis, --15455-- 32960 total-reg-rank --15455-- sanity: 98 cheap, 4 expensive checks. --15455-- ccalls: 27165 C calls, 55% saves+restores avoided (89350 bytes) --15455-- 36017 args, avg 0.87 setup instrs each (8714 bytes) --15455-- 0% clear the stack (81126 bytes) --15455-- 10761 retvals, 32% of reg-reg movs avoided (6756 bytes) ==15459== Reading syms from /lib/libnss_dns-2.3.4.so (0x1BA49000) ==15459== object doesn't have any debug info ==15459== Reading syms from /lib/libresolv-2.3.4.so (0xC21000) ==15459== object doesn't have any debug info ==15459== TRANSLATE: 0xB37F10 redirected to 0x1B91EE2A ==15459== TRANSLATE: 0xB3C050 redirected to 0x1B91ED13 ==15459== discard syms at 0x1BA51000-0x1BA5C000 in /lib/libnss_files-2.3.4.so due to munmap() ==15459== discard syms at 0x1BA49000-0x1BA4F000 in /lib/libnss_dns-2.3.4.so due to munmap() ==15459== discard syms at 0xC21000-0xC34000 in /lib/libresolv-2.3.4.so due to munmap() ==15459== ==15459== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 2) --15459-- --15459-- supp: 5 _dl_relocate_object/dl_open_worker --15459-- supp: 20 dl_relocate_object/dl_main ==15459== malloc/free: in use at exit: 780924 bytes in 20868 blocks. ==15459== malloc/free: 5144214 allocs, 5123346 frees, 119383338 bytes allocated. ==15459== ==15459== searching for pointers to 20868 not-freed blocks. ==15459== checked 3118956 bytes. ==15459== ==15459== 16 bytes in 1 blocks are still reachable in loss record 1 of 9 ==15459== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15459== by 0xB8D364: _dlerror_run (in /lib/libdl-2.3.4.so) ==15459== by 0xB8CE3B: dlsym (in /lib/libdl-2.3.4.so) ==15459== by 0x1B91ECA6: __errno_location (vg_libpthread.c:2127) ==15459== ==15459== ==15459== 224 bytes in 7 blocks are possibly lost in loss record 3 of 9 ==15459== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15459== by 0x12ABDF: g_malloc0 (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x805A5E7: g_sockaddr_inet_new2 (gsockaddr.c:378) ==15459== by 0x805A3E2: g_sockaddr_new (gsockaddr.c:188) ==15459== ==15459== ==15459== 13956 bytes in 201 blocks are still reachable in loss record 4 of 9 ==15459== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15459== by 0x12ABDF: g_malloc0 (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x8056BDB: affile_dd_new (affile.c:584) ==15459== by 0x804FCAF: yyparse (cfg-grammar.y:478) ==15459== ==15459== ==15459== 14848 bytes in 70 blocks are still reachable in loss record 5 of 9 ==15459== at 0x1B9053EE: realloc (vg_replace_malloc.c:197) ==15459== by 0x12AC4B: g_realloc (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x13C55B: (within /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x13CACD: g_string_insert_len (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== ==15459== ==15459== 16438 bytes in 3 blocks are still reachable in loss record 6 of 9 ==15459== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15459== by 0x804E8C6: yyalloc (cfg-lex.c:1958) ==15459== by 0x804E537: yyensure_buffer_stack (cfg-lex.c:1693) ==15459== by 0x804DFAC: yyrestart (cfg-lex.c:1467) ==15459== ==15459== ==15459== 21876 bytes in 403 blocks are still reachable in loss record 7 of 9 ==15459== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15459== by 0x1B905373: realloc (vg_replace_malloc.c:189) ==15459== by 0x12AC4B: g_realloc (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x13C55B: (within /usr/lib/libglib-2.0.so.0.400.7) ==15459== ==15459== ==15459== 71654 bytes in 138 blocks are still reachable in loss record 8 of 9 ==15459== at 0x1B904984: malloc (vg_replace_malloc.c:131) ==15459== by 0x12AB72: g_malloc (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x12B6D6: g_mem_chunk_new (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x130DD7: g_queue_new (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== ==15459== ==15459== 641712 bytes in 20044 blocks are definitely lost in loss record 9 of 9 ==15459== at 0x1B905301: calloc (vg_replace_malloc.c:176) ==15459== by 0x12ABDF: g_malloc0 (in /usr/lib/libglib-2.0.so.0.400.7) ==15459== by 0x805A5E7: g_sockaddr_inet_new2 (gsockaddr.c:378) ==15459== by 0x805A3E2: g_sockaddr_new (gsockaddr.c:188) ==15459== ==15459== LEAK SUMMARY: ==15459== definitely lost: 641712 bytes in 20044 blocks. ==15459== possibly lost: 224 bytes in 7 blocks. ==15459== still reachable: 138788 bytes in 816 blocks. ==15459== suppressed: 200 bytes in 1 blocks. --15459-- TT/TC: 0 tc sectors discarded. --15459-- 837986 tt_fast misses. --15459-- translate: new 10227 (172436 -> 2329564; ratio 135:10) --15459-- discard 686 (11202 -> 143164; ratio 127:10). --15459-- chainings: 7861 chainings, 3 unchainings. --15459-- dispatch: 600000 jumps (bb entries); of them 140194964 (23365%) unchained. --15459-- 498074/12754935 major/minor sched events. --15459-- reg-alloc: 1928 t-req-spill, 431251+13266 orig+spill uis, --15459-- 53090 total-reg-rank --15459-- sanity: 265389 cheap, 10616 expensive checks. --15459-- ccalls: 44952 C calls, 56% saves+restores avoided (149946 bytes) --15459-- 59179 args, avg 0.88 setup instrs each (13872 bytes) --15459-- 0% clear the stack (134349 bytes) --15459-- 17732 retvals, 33% of reg-reg movs avoided (11472 bytes) This is w/ eventlog-0.2.5 and syslog-ng-2.0rc1+20060807. Let me know if you need additional info. I could not reproduce the leak so far locally, although I fixed some one-off allocation leaks but that should only affect reloads and not simply running the process. I suspect this is strict production environment, but I ask anyway, can you run something like valgrind on the syslog-ng process? Another idea is to use the built-in leak finder in syslog-ng, it is not as intrusive but might still help. If neither of these is possible, I could still use a core file dumped when syslog-ng is large in size. (I'd need debugging symbols on the syslog-ng executable as well) There must be some kind of usage pattern that I miss here, but without that I cannot fix the problem. -- Bazsi The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 2

Case Insensitive Match?
by Hari Sekhon 08 Aug '06

08 Aug '06

Is there a way I can do a case insensitive match in syslog-ng? Every place I've seen from grep to python to perl has their own way of doing the case insensitive flag. So how do I set the case-insensitive match flag in syslog-ng? Hari

3 3

$MSG parsing question
by Hubert Lang 08 Aug '06

08 Aug '06

Hello, i have a few questions about the message parser, basically i want to parse/split up the MESSAGE field itself and write the splitted up message in a mysql database, i cant find any documents about how this can be done, do i need an external parser (perl or whatever) or can this be done within syslog-ng.conf? right now i just can write the whole message to the mysqldb syslog-ng.conf template("INSERT INTO logs (host, facility, priority, level, tag, date,time, program, msg) VALUES ( '$HOST', '$FACILITY','$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); so it gets written to the database in this way: INSERT INTO logs (host, facility, priority, level, tag, date,time, program, msg) VALUES ( '10.44.10.253', 'local4','notice', 'notice', 'a5', '2006-06-29', '14:39:46', 'NS25', 'NS25: NetScreen device_id=NS25 [Root]system-notification-00257(traffic): start_time=\"2006-06-29 14:38:38\" duration=0 policy_id=95 service=http proto=6 src zone=Untrust dst zone=Untrust action=Deny sent=0 rcvd=0 src=10.10.10.225 dst=208.174.52.61 src_port=2042 dst_port=80 session_id=0' ); now i want to split up the message part itself system-notification traffic, insert the start_time/duration/policy_id/service/proto/src-zone etc etc in a different table this perl script http://www.optekconsulting.com/tools/nstf.pl has every field i need, Any Help is really welcome Cheers Hubert -- Die e-Mail-Boxes von Brennercom sind Virus-gesichert und Spam-gefiltert. Le caselle e-Mail di Brennercom sono protette da sistemi antivirus e antispam. http://www.brennercom.it

5 5

Syslog-ng again
by Brian Loe 08 Aug '06

08 Aug '06

I robbed the HP-UX configuration file and have tried it on my AIX box. If the last line isn't commented out, it logs nothing. With it commented, netmessages acts as expect but either way no other files are created, nothing else is logged. Configuration file follows: /etc/syslog-ng/syslog-ng.conf options { sync (0); time_reopen (10); chain_hostnames (no); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { unix-dgram("/dev/log"); internal(); }; source s_udp { udp(ip("0.0.0.0") port(514)); }; source s_udp_s { udp(ip("0.0.0.0") port(601)); }; source s_tcp { tcp(ip("0.0.0.0") port(601)); }; destination d_cons { file("/data/logs/console"); }; destination d_mesg { file("/data/logs/syslog.log"); }; destination d_mail { file("/data/logs/mail.log"); }; destination d_mlrt { usertty("root"); }; destination d_mlal { usertty("*"); }; destination netmessages { file("/data/logs/netmessages.log" perm(0660)); }; destination net_pipe { pipe("/data/pipes/net_pipe" perm(0660)); }; filter f_filter1 { facility(mail) and level(debug); }; filter f_filter2 { (facility(mail) and level(debug)) or level(info); }; filter f_filter3 { level(alert); }; filter f_filter4 { level(emerg); }; log { source(s_sys); filter(f_filter1); destination(d_mail); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_cons); destination(d_mlrt); }; log { source(s_sys); filter(f_filter4); destination(d_mlal); }; log { source(s_udp); source (s_udp_s); source (s_tcp); destination(netmessages); }; # log { source(s_udp); source (s_udp_s); source (s_tcp); destination(net_pipe); }; arg-mailgate1.argushealth.com made the following annotations---------------------------------------------------------------------PRIVILEGED AND CONFIDENTIAL This email transmission contains privileged and confidential information intended only for the use of the individual or entity named above. If the reader of the email is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any use, dissemination or copying of this email transmission is strictly prohibited by the sender. If you have received this transmission in error, please delete the email and immediately notify the sender via the email return address or mailto:postmaster@argushealth.com. Thank you. ---------------------------------------------------------------------

2 4

Fwd: Syslog-ng.conf
by Brian Loe 07 Aug '06

07 Aug '06

I'm still having issues with either my config or privileges on the files that are supposed to be created or something. The cron output file isn't getting created - perhaps because it isn't receiving anything? The other files are created (not the pipe - I'm assuming it can't create a pipe) but the localmessages file only gets the "syslog-ng has started" and nothing else is written. In other words, it appears to start, create the file, write one line to the file and then stops - the daemon keeps running, just nothing is getting written to the files... AIX 5.3 - and if anyone has opinions on the best way to launch syslog-ng in AIX, I'll read it as well! # # Global options. # options { long_hostnames(off); use_dns(no); chain_hostnames(no); sync(0); perm(0640); stats(3600); }; # # 'src' is our main source definition. you can add # more sources driver definitions to it, or define # your own sources, i.e.: # #source my_src { .... }; # source s_local { internal(); unix-dgram("/dev/log"); }; source s_udp { udp(ip("0.0.0.0") port(514)); }; # # Filter definitions # filter f_cron { facility(cron); }; # filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7) and not filter(f_cron); }; # # filter f_messages { not facility(news, mail); }; # filter f_warn { level(warn, err, crit); }; # filter f_alert { level(alert); }; # # Cron-messages in one file: # destination cron { file("/data/logs/cron"); }; log { source(s_local); filter(f_cron); destination(cron); }; # # local[1-7]: # destination localmessages { file("/data/logs/localmessages"); }; log { source(s_local); destination(localmessages); }; # # Network messages: # destination netmessages { file("/data/logs/netmessages"); }; log { source(s_udp); destination(netmessages); }; # # Network message pipe: # destination net_pipe { pipe("/data/pipes/net_pipe"); }; log { source(s_udp); destination(net_pipe); };

1 0

syslog-ng doesn't log messages
by Fernando Loureiro 07 Aug '06

07 Aug '06

I think everything is ok with my conf but it isn't working with some applications such as openldap, apcupsd. Can somebody help me? Here is a part of my conf. []s -- Fernando ------------------------------ options { chain_hostnames(off); sync(0); use_dns(no); create_dirs(yes); }; source kernsrc { file("/proc/kmsg"); internal(); }; destination openldap { file("/var/log/openldap.log"); }; destination apcupsd { file("/var/log/apcupsd.log"); }; filter f_openldap { program(slapd); }; filter f_apcupsd { program(apcupsd); }; log { source(kernsrc); filter(f_openldap); destination(openldap); }; log { source(kernsrc); filter(f_apcupsd); destination(apcupsd); };

2 1

syslog-ng 2.0rc1 memory usage
by Martin, David M 04 Aug '06

04 Aug '06

Anyone running 2.0rc1 on Solaris 9 (sparc)? If so, what memory requirements are you seeing for the syslog-ng process? My install handles ~ 700 messages/sec (all UDP) and over the course of a day syslog-ng will gradually consume nearly all available free memory (I've seen it up to 1GB). (syslog-ng is bounced by logrotate daily, so the process repeats itself) I was able to substantially decrease the rate of consumption by adding the 'flush_lines" global (syslog-ng process now at 91MB and growing ... slowly). I installed 2.0rc1 on one server, kept another at 1.6.8 (roughly 500 messages/sec load). The syslog-ng process there is stable at 3 MB. Here's the config from the 2.0rc1 server: options { log_fifo_size(8192); flush_lines(4096); long_hostnames(off); use_dns(yes); use_fqdn(no); create_dirs(no); }; source network { udp(); }; filter f_messages { level(info..warn) and not facility(auth); }; destination messages { file("/var/log/messages"); }; log { source(network); filter(f_messages); destination(messages); }; syslog-ng 1.6.8 config identical except for options section: options { use_fqdn(no); keep_hostname(no); use_dns(yes); long_hostnames(off); sync(0); log_fifo_size(1000); }; The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 2

Compilation problem with Syslog-ng-2.0rc1 in Solaris9
by Bruno Maimoni 04 Aug '06

04 Aug '06

Hello there, I'm having problems to compile the new syslog-ng version in a Solaris9 os. bash-2.05# uname -a SunOS <something> 5.9 Generic_117171-05 sun4u sparc SUNW,Ultra-5_10 gcc version: 3.3.2 I run ./configure without problems: export PKG_CONFIG_PATH=/opt/eventlog-0.2.5/lib/pkgconfig ./configure --prefix=/opt/syslog-ng-2.0rc1 --enable-dynamic-linking Then when i run make: [...] if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/opt/eventlog-0.2.5/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -MT fdread.o -MD -MP -MF ".deps/fdread.Tpo" \ -c -o fdread.o `test -f 'fdread.c' || echo './'`fdread.c; \ then mv -f ".deps/fdread.Tpo" ".deps/fdread.Po"; \ else rm -f ".deps/fdread.Tpo"; exit 1; \ fi if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/opt/eventlog-0.2.5/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -MT memtrace.o -MD -MP -MF ".deps/memtrace.Tpo" \ -c -o memtrace.o `test -f 'memtrace.c' || echo './'`memtrace.c; \ then mv -f ".deps/memtrace.Tpo" ".deps/memtrace.Po"; \ else rm -f ".deps/memtrace.Tpo"; exit 1; \ fi rm -f libsyslog-ng.a ar cru libsyslog-ng.a misc.o utils.o messages.o syslog-names.o cfg.o cfg-lex.o cfg-grammar.o macros.o filter.o logmsg.o logpipe.o logsource.o driver.o sgroup.o dgroup.o center.o templates.o logreader.o logwriter.o affile.o afstreams.o afuser.o afprog.o afsocket.o afunix.o afinet.o afinter.o children.o stats.o gsockaddr.o fdwrite.o fdread.o memtrace.o ranlib libsyslog-ng.a if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/opt/eventlog-0.2.5/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -MT main.o -MD -MP -MF ".deps/main.Tpo" \ -c -o main.o `test -f 'main.c' || echo './'`main.c; \ then mv -f ".deps/main.Tpo" ".deps/main.Po"; \ else rm -f ".deps/main.Tpo"; exit 1; \ fi gcc -g -O2 -Wall -g -o syslog-ng main.o libsyslog-ng.a -lpthread -lnsl -lrt -lsocket -ldoor -lfl -lglib-2.0 -L/opt/eventlog-0.2.5/lib -levtlog Undefined first referenced symbol in file G_UNLIKELY libsyslog-ng.a(filter.o) G_LIKELY libsyslog-ng.a(filter.o) ld: fatal: Symbol referencing errors. No output written to syslog-ng collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/tmp/teste/syslog-ng-2.0rc1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/teste/syslog-ng-2.0rc1' make: *** [all] Error 2 bash-2.05# Does anyone knows what's missing? Tks in advance -- [ ],s Bruno Maimoni

1 0

Fwd: new to list, and syslog-ng
by Brian Loe 02 Aug '06

02 Aug '06

<I hope you don't get this twice (hadn't confirmed my subscription the first time).> I'm hoping some of you can give me some guidance in my implementation of syslog-ng on an AIX 5.3 system. This is what I have for a config file thus far - will it work? Also, some help on the best way to move from the AIX syslogd to syslog-ng? I'm new to AIX as well... # # Global options. # options { long_hostnames(off); use_dns(no); chain_hostnames(no); sync(0); perm(0640); stats(3600); }; # # 'src' is our main source definition. you can add # more sources driver definitions to it, or define # your own sources, i.e.: # #source my_src { .... }; # source s_local { internal(); unix-stream("/dev/log"); }; source s_udp { udp(ip("0.0.0.0") port(514)); }; # # Filter definitions # filter f_cron { facility(cron); }; # filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7) and not filter(f_cron); }; # filter f_messages { not facility(news, mail); }; filter f_warn { level(warn, err, crit); }; filter f_alert { level(alert); }; # # Cron-messages in one file: # destination cron { file("/var/log/cron"); }; log { source(s_lcoal); filter(f_cron); destination(cron); }; # # local[1-7]: # destination localmessages { file("/var/log/localmessages"); }; log { source(s_local); destination(localmessages); }; # # Network messages: # destination netmessages { file("/var/log/netmessages"); }; log { source(s_udp); destination(netmessages); }; # # Network message pipe: # destination net_pipe { file("/tmp/log/net_pipe"); }; log { source(s_udp); destination(net_pipe); };

1 0

syslog-ng 1.0rc1 stops listening to UDP when zero-length packet received
by Dark Angael 02 Aug '06

02 Aug '06

I have been testing 2.0 rc1 for use in a production environment and have discovered a potential DOS exploit. I discovered this when I stopped receiving logs after a particular application started. It turns out that for some reason this app was sending a zero-length packet to syslog. After receiving this packet, syslog-ng gives the following output in debug mode: EOF occurred while reading; fd='3' Closing log reader fd; fd='3' This is reproducible using packet injection also. It has been shown to happen on both solaris 8 and linux (2.6.15-gentoo-r5). It does not happen on syslog-ng version 1.6.11. _________________________________________________________________ New year, new job there's more than 100,00 jobs at SEEK http://a.ninemsn.com.au/b.aspx?URL=http%3A%2F%2Fninemsn%2Eseek%2Ecom%2Eau&_…

2 3