August 2011 - syslog-ng - lists.balabit.hu

MongoDB and syslog-ng 3.3 on Centos 5.5
by Viacheslav Biriukov 03 Aug '11

03 Aug '11

Hello. Can you help me with syslog-ng 3.3 on the CentoOS 5.5 with MongoDB 1.6 I install syslog-ng version 3.3 where mongodb support exists. But when I add to the config mongodb destination I get an error: Starting syslog-ng: Error parsing destination, destination plugin mongodb > not found in /etc/syslog-ng/syslog-ng.conf at line 70, column 5: > mongodb( > ^^^^^^^ syslog-ng.conf @version: 3.3 > # > # configuration file for syslog-ng, customized for remote logging > # > source s_internal { internal(); }; > destination d_syslognglog { file("/var/log/syslog-ng.log"); }; > log { source(s_internal); destination(d_syslognglog); }; > # Local sources, filters and destinations are commented out > # If you want to replace sysklogd simply uncomment the following > # parts and disable sysklogd > # > # Local sources > # > source s_local { > unix-dgram("/dev/log"); > file("/proc/kmsg" program_override("kernel")); > }; > # > # Local filters > # > filter f_messages { level(info..emerg); }; > filter f_secure { facility(authpriv); }; > filter f_mail { facility(mail); }; > filter f_cron { facility(cron); }; > filter f_emerg { level(emerg); }; > filter f_spooler { level(crit..emerg) and facility(uucp, news); }; > filter f_local7 { facility(local7); }; > # > # Local destinations > # > destination d_messages { file("/var/log/messages"); }; > destination d_secure { file("/var/log/secure"); }; > destination d_maillog { file("/var/log/maillog"); }; > destination d_cron { file("/var/log/cron"); }; > destination d_console { usertty("root"); }; > destination d_spooler { file("/var/log/spooler"); }; > destination d_bootlog { file("/var/log/boot.log"); }; > # > # Local logs - order DOES matter ! > # > log { source(s_local); filter(f_emerg); destination(d_console); }; > log { source(s_local); filter(f_secure); destination(d_secure); > flags(final); }; > log { source(s_local); filter(f_mail); destination(d_maillog); > flags(final); }; > log { source(s_local); filter(f_cron); destination(d_cron); flags(final); > }; > log { source(s_local); filter(f_spooler); destination(d_spooler); }; > log { source(s_local); filter(f_local7); destination(d_bootlog); }; > log { source(s_local); filter(f_messages); destination(d_messages); }; > > # Remote logging > source s_remote { > tcp(ip(0.0.0.0) port(514)); > udp(ip(0.0.0.0) port(514)); > }; > destination d_separatedbyhosts { > file("/var/log/syslog-ng/$HOST/messages" owner("root") group("root") > perm(0640) dir_perm(0750) create_dirs(yes)); > }; > log { source(s_remote); destination(d_separatedbyhosts); }; > > > #MongoDB > destination d_mongodb { > mongodb( > host("127.0.0.1") > port(27017) > database("syslog-ng") > collection("logs") > keys("date", "facility", "level", "host", "program", "pid", > "message") > values("${R_YEAR}-${R_MONTH}-${R_DAY} > ${R_HOUR}:${R_MIN}:${R_SEC}", "$FACILITY", "$LEVEL", "$HOST","$PROGRAM", > "$PID", "$MSGONLY") > ); > }; > log { source(s_local); destination(d_mongodb); }; > In the modules.conf I try to add @afmongodb but this isn't change the situation. What's wrong? -- Viacheslav Biriukov BR http://biriukov.com

2 1

[Bug 128] New: template() broken in latest 3.3 git
by bugzilla＠bugzilla.balabit.com 01 Aug '11

01 Aug '11

https://bugzilla.balabit.com/show_bug.cgi?id=128 Summary: template() broken in latest 3.3 git Product: syslog-ng Version: unspecified Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: algernon(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 Hi! While updating my 3.3 installation from an ancient version to latest git head, I noticed that template functions stopped working. If I have any kind of template in the config, I get a segmentation fault: ,---- | Program received signal SIGSEGV, Segmentation fault. | log_template_reset_compiled (self=0x0) at templates.c:661 | 661 while (self->compiled_template) | (gdb) bt | #0 log_template_reset_compiled (self=0x0) at templates.c:661 | #1 0x00007ffff7b9f302 in log_template_compile (self=0x0, template=0x63d7e0 "foo: bar", error=0x7fffffffb5d8) at templates.c:788 | #2 0x00007ffff5e96761 in affile_parse (lexer=<value optimized out>, instance=<value optimized out>, arg=<value optimized out>) at affile-grammar.y:592 | #3 0x00007ffff7b9bd72 in cfg_parser_parse (self=0x7ffff609ce30, cfg=<value optimized out>, yylloc=<value optimized out>, arg=<value optimized out>) at cfg-parser.h:83 | #4 plugin_parse_config (self=0x7ffff609ce30, cfg=<value optimized out>, yylloc=<value optimized out>, arg=<value optimized out>) at plugin.c:124 | #5 0x00007ffff7ba89f7 in main_parse (lexer=<value optimized out>, dummy=<value optimized out>, arg=<value optimized out>) at cfg-grammar.y:592 | #6 0x00007ffff7b7ee93 in cfg_parser_parse (self=0x611720, lexer=0x6188a0, parser=0x7ffff7dd92a0, result=<value optimized out>, arg=<value optimized out>) at cfg-parser.h:83 | #7 cfg_run_parser (self=0x611720, lexer=0x6188a0, parser=0x7ffff7dd92a0, result=<value optimized out>, arg=<value optimized out>) at cfg.c:370 | #8 0x00007ffff7b7efa6 in cfg_read_config (self=0x611720, fname=0x7ffff7bb7068 "/home/algernon/tmp/sng/install/3.3/etc/syslog-ng.conf", syntax_only=<value optimized out>, preprocess_into=0x0) at cfg.c:392 | #9 0x00007ffff7b970c1 in main_loop_init () at mainloop.c:611 | #10 0x0000000000401865 in main (argc=1, argv=0x7fffffffdc78) at main.c:242 | (gdb) `---- The config that triggers this is as follows: ,---- | @version: 3.3 | @include "scl.conf" | | source s_local { internal(); }; | destination d_local { file("/tmp/template-test" template("foo: bar")); }; | log { source(s_local); destination(d_local); }; `---- After bisecting the problem, I ended up with the following commit: ,---- | 2443b07b90c822414a628aae6a9ca513357b52fc is the first bad commit | commit 2443b07b90c822414a628aae6a9ca513357b52fc | Author: Balazs Scheidler <bazsi(a)balabit.hu> | Date: Sat Jun 11 14:09:01 2011 +0200 | | template: make template compilation thread safe | | Instead of doing template compilation on-demand, require the application | code to do it explicitly, at initialization, in order to avoid having to | acquire a lock during runtime. | | This patch contains an incompatible API change, and requires updating | all call-sites. | | The change makes it also possible to change template string of a constructed | LogTemplate class multiple times with a call to log_template_compile(). | | Signed-off-by: Balazs Scheidler <bazsi(a)balabit.hu> | | :040000 040000 a40fc910d215dac2184aa02ae1498c6748e3e64a dea1d2a7fc02d1d0ec8e095f2f62499ce42b895a M lib | :040000 040000 1641cf3152897406874716d4f090b5cc795292ab f66d2e5c482c3ff91b9d4219812766d66067c7a1 M modules | :040000 040000 2c2a37fd3a4f9c5bddc81813605c20ece033ce2f dff570d331b8e7ada6c1202761066878b9bb3910 M tests `---- I'm not quite sure why this happens yet, I'll keep digging. I'll post a patch if and when I figure out what exactly broke. Meanwhile, this bug report will serve as a reminder for me to keep looking. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

4 11

TCP / max-connections & flow-control
by Paul Muther 01 Aug '11

01 Aug '11

Greetings all, I have not been able to find confirmation on this but I wanted to find details on TCP connections in 3.2 open-source. When a client establishes a TCP connection to a host log server, does it reuse that connection for all subsequent log transmissions? Does it have a threshold for destroying the connection and creating a new one if it does? Thanks much, PM

3 2

Check connection to server
by Josu Lazkano 01 Aug '11

01 Aug '11

Hello again. I had my syslog server configured and running. All servers has public IP but they are on the same LAN. Now I want to get all logs from a remote server. I configured the remote server same as LAN servers but there is no logs from WAN. How could I check if it is sending data to my syslog server? Maybe I must configure filters to do that? Thanks and best regards.

2 1

Inability to filter/log hostnames
by Norman Elton 01 Aug '11

01 Aug '11

I'm running syslog-ng 3.2.4 from RedHat's RPM. Unfortunately, I can't seem to log the hostname as specified in the incoming UDP packet. We don't do DNS resolution; rather, just want to log what the sending host is passing along. No relays in the mix, but we have keep_hostname() enabled. My global options: flush_lines(10); flush_timeout(750); time_reopen (10); log_fifo_size (1000); keep_hostname (yes); When I log $HOSTNAME or $HOST, I just get the sender's IP address. Similarly, filters based on these macros don't work properly. This all seemed to work on prior versions of syslog-ng (2.something). Any ideas? Thanks! Norman

2 1

3.3.0beta1 leaking memeory (Re: syslog-ng 3.3.0beta & ESX crashes)
by Hendrik Visage 01 Aug '11

01 Aug '11

Hi there, I've mailed Gergely the Valgrind output to analyze 3.3.0beta1 memory leak(s), so if anybody else are interested in it, please contact me for them. I've since compiled 3.2.4+noescape patch (since only 3.3.0+ have the default option for SQL insert I need/use for the autoincrementing ID field in systemevents), and using the same config file (other than @version 3.2 and the noescape('default') for my postgresql insertions) and the system is stable with no descernable memory growth compared to syslog-ng 3.3.0beta1's explosion in memory size. Hendrik On Mon, Jul 18, 2011 at 3:02 PM, Gergely Nagy <algernon(a)balabit.hu> wrote: > Hendrik Visage <hvjunk(a)gmail.com> writes: > >> Hi Gergely, >> >> Looks like I have a syslog-ng 3.3beta1 (fetched from GITHEAD) that is >> having a serious memory leak (and I'm not yet pushing all our logs to >> it ;( >> >> Any advice in how to help you track this problem? > > Hrm. Well, valgrind would be best, but that slows things down to almost > a grounding halt. (But in case you want to try that: > G_SLICE=always-malloc valgrind --leak-check=full -v --show-reachable-yes > --log-file=valgrind.log syslog-ng -F would be the command to run ;) > > Other than that, a config could be useful, so I can try and reproduce > the problem (or at least identify possible areas where the leak might be > coming from, and go from there), and see if I can fix it. > > -- > |8] > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > >

3 6