May 2013 - syslog-ng - lists.balabit.hu

rewrite part of the message...
by Russell Fulton 11 May '13

11 May '13

Apologies to those of you who have already seen this on the ELSA list. I want to get rid of the explanatory essay that accompanies some eventlog messages from windows. I tried this: rewrite r_snarex { subst("\s+This event is generated when[^|]+\|", "|", value("MSGONLY") type("pcre")); }; and added it to log section, but it did not work. I have tried various variations on the theme too. Russell

2 2

syslog-ng and logrotate
by Rakesh Rajasekharan 10 May '13

10 May '13

I am using the syslog-ng 3.3.7 version. My question is , can we manage log rotation within the syslog-ng conf file ? Please advice

4 5

[Bug 234] New: eventlog: can' t build shared libraries under Windows with mingw
by bugzilla＠bugzilla.balabit.com 08 May '13

08 May '13

https://bugzilla.balabit.com/show_bug.cgi?id=234 Summary: eventlog: can't build shared libraries under Windows with mingw Product: syslog-ng Version: unspecified Platform: PC OS/Version: Windows Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ramon.garcia.f+syslog(a)gmail.com Type of the Report: --- Estimated Hours: 0.0 When trying to building, libtool gives an error message saying that it cannot find shared library versions of Windows base libraries, like KERNEL32.DLL The fix is simple: upgrade libtool. After libtoolizing with libtool 2.4.2 it builds without trouble. Best regards. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[PATCH (3.5)] afsocket: Fix compilation with systemd support enabled
by Gergely Nagy 08 May '13

08 May '13

self->super.socket_options->sock_type is replaced with self->super.transport_mapper->sock_type, follow up on this in the systemd-enabled codepath too. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsocket/afunix-source.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/afsocket/afunix-source.c b/modules/afsocket/afunix-source.c index dd8b8bb..c3dcbe7 100644 --- a/modules/afsocket/afunix-source.c +++ b/modules/afsocket/afunix-source.c @@ -74,7 +74,7 @@ afunix_sd_acquire_named_socket(AFSocketSourceDriver *s, gint *result_fd, if (sd_is_socket_unix(fd, 0, -1, filename, 0)) { /* check if it matches our idea of the socket type */ - if (sd_is_socket_unix(fd, self->super.socket_options->sock_type, -1, filename, 0)) + if (sd_is_socket_unix(fd, self->super.transport_mapper->sock_type, -1, filename, 0)) { *result_fd = fd; break; @@ -84,7 +84,7 @@ afunix_sd_acquire_named_socket(AFSocketSourceDriver *s, gint *result_fd, msg_error("The systemd supplied UNIX domain socket is of a different type, check the configured driver and the matching systemd unit file", evt_tag_str("filename", filename), evt_tag_int("systemd-sock-fd", fd), - evt_tag_str("expecting", self->super.socket_options->sock_type == SOCK_STREAM ? "unix-stream()" : "unix-dgram()"), + evt_tag_str("expecting", self->super.transport_mapper->sock_type == SOCK_STREAM ? "unix-stream()" : "unix-dgram()"), NULL); return FALSE; } -- 1.7.10.4

1 0

Re: [syslog-ng] supervisor not restarting a failed daemon process
by Balazs Scheidler 07 May '13

07 May '13

Hi, No kernel source handy but my guess is that the kernel is logging the tid value, which is the same as pid as long as the process is single threaded. I've checked the supervisor code and the only way this could happen is a fork/pipe error which is not logged. It should be logged but who knows that message can be lost. The supervisor attempts to restart 3 times then gives up. Hmmm the supervisor messages may be redirected to syslog after the first startup which might explain why they don't get logged. But why does fork/pipe fail? Hope this helps. Evan Rempel <erempel(a)uvic.ca> wrote: >The logs below show a "standard" syslog-ng processID=15017 that reads /dev/log /proc/kmsg >The second instance of syslog-ng is what we call our "server" which just listens on the network ports >and does all of the complex patterndb, filtering and routing to destination processes > >kern.info kernel: syslog-ng[1561]: segfault at 7f65c0000078 ip 00007f65c0000078 sp 00007f65e1385a48 error 15 >--- this is the server instance segfaulting (I assume, see WAIT below) > >syslog.notice syslog-ng[15017]: Syslog connection closed; fd='20', client='AF_INET(142.104.47.145:49803)', local='AF_INET(127.0.0.1:1514)' >syslog.notice syslog-ng[15017]: Syslog connection broken; fd='14', server='AF_INET(142.104.47.146:514)', time_reopen='5' >--- this was the standard syslog loosing connection to the server, and detecting the drop of the server instance destination to it. > >daemon.info syslog-ng-stats: server stopping on socket "/var/local/syslog-ng.server.ctl" >daemon.info msgid_profiler[832]: committing residual data >local0.info flare-timer[834]: stopping >local0.info action-handler[833]: stopping >--- these are all program destinations of the server instance shutting down gracefully after the close of their stdin. > >daemon.crit supervise/syslog-ng[27221]: Daemon exited due to a deadlock/signal/failure, restarting; exitcode='11' >syslog.err syslog-ng[15017]: Syslog connection failed; fd='14', server='AF_INET(142.104.47.146:514)', error='Connection refused (111)', time_reopen='5' > >This "connection failed" message repeats every 5 seconds until I restart the server instance. > >syslog.notice syslog-ng[1911]: syslog-ng starting up; version='3.4.1' > >So it does not look like there is anything in the logs about attempted restarts. > > >WAIT... > >This is really odd. The line > >kern.info kernel: syslog-ng[1561]: segfault at 7f65c0000078 ip 00007f65c0000078 sp 00007f65e1385a48 error 15 > >implies that there was a process ID 1561 that segfaulted, but that line is the ONLY logged line with that process ID. >We take ps snapshots every 15 minutes, and those snapshots don't show anything for that process ID. >Also, the supervisor processID is shown > > USER PID PPID NI PRI CPU VSZ ELAPSED TIME COMMAND >root 27221 1 0 19 - 26556 9-04:00:59 00:00:00 supervising syslog-ng >root 27222 27221 0 19 - 977064 9-04:00:59 1-02:52:35 /usr/local/sbin/syslog-ng --cfgfile= ... > >which matches the line >daemon.crit supervise/syslog-ng[27221]: Daemon exited due to a deadlock/signal/failure, restarting; exitcode='11' > >so its child which dies should have been processID 27222 so why is the log line >kern.info kernel: syslog-ng[1561]: segfault at 7f65c0000078 ip 00007f65c0000078 sp 00007f65e1385a48 error 15 > > >I conclude that the 1561 is not the process ID. > >Can you shed any light on this? > >Evan. > >On 04/26/2013 11:00 PM, Balazs Scheidler wrote: > > >> Strange, indeed. The supervisor gives up if the restarted daemon exits for some reason. Eg. If there's an initialization error it gives up. Any indication in the logs? >> >> Evan Rempel <erempel(a)uvic.ca> wrote: >> >>> We are sing the log line >>> >>> supervise/syslog-ng[27221]: Daemon exited due to a deadlock/signal/failure, restarting; exitcode='11' >>> >>> >>> and it looks like it should restart, but instead of restarting, >>> the supervisor terminates and then no syslog-ng process is running. >>> >>> Is this a bug in the supervisor? >>> ______________________________________________________________________________ >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >> > > >-- >Evan Rempel erempel(a)uvic.ca >Senior Systems Administrator 250.721.7691 >Data Centre Services, University Systems, University of Victoria >______________________________________________________________________________ >Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >FAQ: http://www.balabit.com/wiki/syslog-ng-faq >

2 3

Re: [syslog-ng] Can not get DBParse match macro result (syslog-ng 3.13 debian squeeze)
by 不坏阿峰 06 May '13

06 May '13

thanks to your reply. i do not understand how to do now. it puzzle and trouble me some days. i read the balabit syslog-ng OSE guide documents and only have simple information in there. how to do on this ----->>>> If you change the patterndb ruleset pattern to use a program of system rather than ESXI I think it would work. 2013/4/28 <syslog-ng-request(a)lists.balabit.hu> > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Can not get DBParse match macro result (syslog-ng 3.13 > debian squeeze) (????) > 2. Re: Can not get DBParse match macro result (syslog-ng 3.13 > debian squeeze) (Evan Rempel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 27 Apr 2013 22:34:50 +0800 > From: ???? <onlydebian(a)gmail.com> > Subject: [syslog-ng] Can not get DBParse match macro result (syslog-ng > 3.13 debian squeeze) > To: syslog-ng(a)lists.balabit.hu > Message-ID: > <CA+SSH2oBB2-WWvQksbchVVoyhfZbdVvDR= > V7wJ1EJdvE6Zx9zg(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > when use pdbtool do match test, it is success. but from syslog-ng can not > return result of macro > i can not get macro result. for example, ${.esxi.month} no value, same > as ${.esxi.host_ip} ${.esxi.time} > > test log output ,just like this. > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > === system,error,critical, HOST IP , > > > do the pdbtool test, it's ok. wish someone can give me some solution and > help. i have search some mail list but i can not get the right solution. > thanks a lot. > > root@debian:~# pdbtool match -D -c -p > /etc/syslog-ng/patterndb/esxi_pattern.xml -P ESXI -M "Apr 26 15:17:31 > 192.168.88.71 vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd > 0x1a (0x4124444a6280, 0) to dev "mpx.vmhba0:C0:T0:L0" on path > "vmhba0:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. > Act:NONE" > Pattern matching part: > @STRING:.esxi.month=Apr@ @STRING:.esxi.date=26@ > @STRING:.esxi.time=15:17:31@@IPv4:.esxi.host_ip=192.168.88.71@ > @ESTRING:.esxi.program= > vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a > (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 > Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. > Act:NONE@@ANYSTRING:.esxi.message=cpu11:8203)NMP: > nmp_ThrottleLogForDevice:2319: Cmd > 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 > Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE@ > Matching part: > Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP: > nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev > mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid > sense data: 0x5 0x20 0x0. Act:NONE > Values: > MESSAGE=Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP: > nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev > mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid > sense data: 0x5 0x20 0x0. Act:NONE > PROGRAM=ESXI > .classifier.class=esxi > .classifier.rule_id=182437592347598 > .esxi.month=Apr > .esxi.date=26 > .esxi.time=15:17:31 > .esxi.host_ip=192.168.88.71 > .esxi.program= vmkernel > .esxi.message=cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a > (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 > Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE > root@debian:~# > > > my configuration like as below > > ######## esxi_pattern.xml ############ > <?xml version="1.0" encoding="utf-8"?> > <patterndb version='3' pub_date='2009-04-17'> > <ruleset name='esxi' id='123456678'> > <pattern>ESXI</pattern> > <rules> > <rule provider='Fone Bro' id='182437592347598' class='esxi'> > <patterns> > <pattern>@STRING:.esxi.month:@ @STRING:.esxi.date:@ > @STRING:.esxi.time::@@IPv4:.esxi.host_ip:@ > @ESTRING:.esxi.program::@ @ANYSTRING:.esxi.message@</pattern> > </patterns> > </rule> > </rules> > </ruleset> > </patterndb> > > ######## syslog-ng.conf ######## > > #####Parser##### > parser pattern_db { > db_parser( file("/etc/syslog-ng/patterndb/esxi_pattern.xml")); > }; > > #Check pattern matching > destination udp_esxi_output { > file("/var/log/pattern_output" > template("=== $PROGRAM,${.esxi_month} ${.esxi.date} ${.esxi.time} HOST > IP ${.esxi.host_ip},${.esxi.message}\n") > template_escape(no)); > }; > > #####Log##### > log { > source(s_network); > parser(pattern_db); > destination(udp_esxi_output); > }; >

3 7

Re: [syslog-ng] syslog-ng Digest, Vol 97, Issue 4
by Yarick Tsagoyko 05 May '13

05 May '13

Dear R, cool issue. I have not seen this exact error on my compiles, here are some pointers: 1. Post your config.log 2. View the location of the referenced file(s) In file included from /usr/include/net/ethernet.h:27, from /usr/include/libnet.h:98, from afinet-dest.c:43 3. Rerun the configure, make clean, make. Sometimes rerunning the configure and running make cleac helps. Please post the findings. On May 5, 2013 6:00 AM, <syslog-ng-request(a)lists.balabit.hu> wrote: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. problems compiling OSE 3.4.1 on redhat enterprise 6.4 > (Russell Fulton) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 5 May 2013 13:28:04 +1200 > From: Russell Fulton <r.fulton(a)auckland.ac.nz> > Subject: [syslog-ng] problems compiling OSE 3.4.1 on redhat enterprise > 6.4 > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <D1B88BF5-CE66-485B-93FF-ECE892F2F950(a)auckland.ac.nz> > Content-Type: text/plain; charset=us-ascii > > make[4]: Entering directory > `/home/rful011/syslog-ng-3.4.1/modules/afsocket' > /bin/sh ../../libtool --tag=CC --mode=compile gcc -std=gnu99 > -DHAVE_CONFIG_H -I. -I../.. -I../../lib -I../../lib -pthread > -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include > -I/usr/local/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE > -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -I../../lib/ivykis/src/include > -I../../lib/ivykis/src/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -pthread -MT > libafsocket_notls_la-afinet-dest.lo -MD -MP -MF > .deps/libafsocket_notls_la-afinet-dest.Tpo -c -o > libafsocket_notls_la-afinet-dest.lo `test -f 'afinet-dest.c' || echo > './'`afinet-dest.c > libtool: compile: gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I../../lib > -I../../lib -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include > -I/usr/local/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE -D__FAVOR_BSD > -DHAVE_NET_ETHERNET_H -I../../lib/ivykis/src/include > -I../../lib/ivykis/src/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -pthread -MT > libafsocket_notls_la-afinet-dest.lo -MD -MP -MF > .deps/libafsocket_notls_la-afinet-dest.Tpo -c afinet-dest.c -fPIC -DPIC -o > .libs/libafsocket_notls_la-afinet-dest.o > In file included from /usr/include/net/ethernet.h:27, > from /usr/include/libnet.h:98, > from afinet-dest.c:43: > > > when I first encountered this error I went and did some googling and > followed some vague leads to install a bunch of -devel packages (most of > which were already installed ) but it did not make any difference. > > Googling the message brings up a bunch of hits for both redhat and debian > families and a range of packages. There did not seem to be any common > cause. > > Russell > > > > ------------------------------ > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > End of syslog-ng Digest, Vol 97, Issue 4 > **************************************** >

1 0

Prevent message loss while using tcp output
by Abhijeet Rastogi 03 May '13

03 May '13

Hi all, I've source defined as: source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; And one of the destination as: destination d_logstash { tcp( "127.0.0.1" port(515) template(t_both_timestamps) flush_lines(10)); }; Problem is when my consumer is down, I can see messages being dropped in "syslog-ng-ctl stats". How can I avoid this at the expense of memory (like a queueing mechanism or something)? -- Regards, Abhijeet Rastogi (shadyabhi) http://blog.abhijeetr.com

2 1

syslog-ng 3.3.6 Hangs on Solaris 8 sparc
by Rakesh Rajasekharan 02 May '13

02 May '13

Hi All, I have compiled the syslog-ng version 3.3.6 on a Solaris Sparc machine . The compilation went through fine but when I try to run syslog-ng .. though the daemon gets running somehow the unix prompt is not returned.. and it hangs... For ex:- # ps -ef|grep -i syslog root 14073 11758 0 07:11:37 pts/4 0:00 vi syslog-ng.d root 14097 13526 0 07:15:48 pts/1 0:00 grep -i syslog Now if I start the daemon using the below options... # /opt/soe/local/syslog-ng-3.3.6/sbin/syslog-ng -f /opt/soe/local/syslog_ng/etc/syslog-ng/syslog-ng.conf -p /var/run/syslog-ng.pid I never get back the prompt.. but if on another window I check.. the daemon has started.. ps -ef|grep -i syslog root 14099 11758 0 07:16:17 pts/4 0:00 grep -i syslog root 14098 13526 23 07:16:04 pts/1 0:13 /opt/soe/local/syslog-ng-3.3.6/sbin/syslog-ng -f /opt/soe/local/syslog_ng/etc/s though I guess... i should expect one more child to this... can someone please help me address this issue please... Thanks, Rkaesh.

3 10