- syslog-ng - lists.balabit.hu

mount syslog server
by Javier Terceiro 10 May '07

10 May '07

Hello list, I'm mounting a syslog-ng server for centralize the log's of my net. The servers with linux is ok, but for windows server's, I need a tool for send the messages. What can I use? Some idea? Thanks. -- Power by Debian. A greeting, Javier.

1 0

Syslog with epilog (for windows)
by Javier Terceiro 09 May '07

09 May '07

Hello list, I am mounting a log centralized server and for servers Windows I am using epiloj and the collector server is a syslog-ng. I have a problem with the data format that epiloj sends, I can`t correctly divide it in fields (server, date, message) in syslog-ng server. For example - The original message of epilog: GenericLog 0 2007-05-09 12:23:42 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- The format with that the data keep would have to be the following one: Program <> Date <> MSGONLY (in syslog-ng format). - The message saved by syslog-ng: GenericLog 0 2007-05-09 12 <> 2007 May 9 12:23:42 <> 3:42 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- - The message correctly: GenericLog <> 2007 May 9 12:23:42 <> 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- The problem is the fields than syslog don`t understand. Some idea of like obtaining it? Some program for obtain log's for windows server's? thanks. -- A greeting, Javier.

1 0

line of sendmail log output missing?
by Centyx Centalix 05 May '07

05 May '07

Hi, I recently switched from syslogd to syslog-ng on several servers running CentOS. While running syslogd, a usual mail delivery appeared in /var/log/maillog as follows: Apr 22 04:11:07 ws01 sendmail[32181]: l3M8B71l032181: from=<someuser(a)someremote.domain>, size=2361, class=0, nrcpts=1, msgid=<001901c784c6$9688fea0$018ab484@maison9tm8hfuv>, proto=ESMTP, relay=root@localhost Apr 22 04:11:07 ws01 sendmail[32179]: l3M8Av5T032164: to=<domain_support(a)somelocal.domain>, delay=00:00:04, xdelay=00:00:00, mailer=virthostmail, pri=121926, relay=somelocal.domain, dsn=2.0.0, stat=Sent (l3M8B71l032181 Message accepted for delivery) Apr 22 04:11:07 ws01 spamd[24594]: spamd: connection from localhost.localdomain [127.0.0.1] at port 50252 Apr 22 04:11:07 ws01 spamd[24594]: spamd: setuid to domain_admin(a)somelocal.domain succeeded Apr 22 04:11:07 ws01 spamd[24594]: spamd: processing message <001901c784c6$9688fea0$018ab484@maison9tm8hfuv> for domain_admin@somelocal.domain:22003 Apr 22 04:11:08 ws01 spamd[24594]: spamd: identified spam (18.5/4.0) for domain_admin@somelocal.domain:22003 in 1.3 seconds, 2646 bytes. Apr 22 04:11:08 ws01 spamd[24594]: spamd: result: Y 18 - BAYES_99,HELO_DYNAMIC_IPADDR2,HELO_DYNAMIC_SPLIT_IP,HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_NJABL_DUL,RCVD_IN_SORBS_DUL scantime=1.3,size=2646,user=domain_admin@somelocal.com,uid=22003,required_score=4.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=50252,mid=<001901c784c6$9688fea0$018ab484@maison9tm8hfuv>,bayes=1,autolearn=spam Apr 22 04:11:08 ws01 spamd[4109]: prefork: child states: BI Apr 22 04:11:08 ws01 spamd[4109]: spamd: handled cleanup of child pid 24594 due to SIGCHLD Apr 22 04:11:08 ws01 spamd[4109]: spamd: server successfully spawned child process, pid 32189 Apr 22 04:11:08 ws01 spamd[4109]: prefork: child states: II Apr 22 04:11:08 ws01 sendmail[32182]: l3M8B71l032181: to=domain_admin, delay=00:00:01, xdelay=00:00:01, mailer=local, pri=32547, dsn=2.0.0, stat=Sent Now, after replacing syslogd with syslog-ng, the last line showing the delivery by the local mailer ( in this case, procmail ) is absent from the log files. I have made sure that levels info through emergency are specified for facility mail, and have also checked to see if this message was showing up under any other facility ( although I have not explicity defined the levels for all facilities ). I am rather stumped about where this line of log output is going. Any suggestions would be greatly appreciated. -- We're all in this together.

3 6

Please Help problems with timezone and stamping !!
by Guy Fleegman 04 May '07

04 May '07

Hello: I have several sources that are all reporting to a syslog-ng 2.0.3 server My source is source all_routers { udp(ip(0.0.0.0) port(514)); }; My problem is that I have one source that is isolated to a separate file I isolate the file with the following filter f_pd_snort { host(intsnort1); }; and have this destination destination d_pd_snort { file("/var/log/syslog-ng-logs/pd_snort.1" perm(0644) template(t_default)); }; then i use the following command log { source(all_routers); filter(f_pd_snort); destination(d_pd_snort); }; The problem is the format the data is arriving in. It send data in UTC time as follows Apr 30 14:04:35 2007 intsnort1 SFIMS: [119:13:1] Snort Alert [Classification: Unknown] [Priority: 3] Apr 30 14:04:35 2007 intsnort1 SFIMS: [119:4:1] Snort Alert [Classification: Unknown] [Priority: 3] Apr 30 14:04:35 2007 intsnort1 SFIMS: [122:3:0] Snort Alert [Classification: Unknown] [Priority: 3] This is UTC. It is the time that is being sent in the actual packet (I verified with wireshark) I want syslog-ng to modify the times going into this file (and this file only) to reflect EST time. Based upon the documentation The following algorithm is used. 1. The sender can specify the timezone of the messages. If the incoming message includes a timezone it is associated with the message. Otherwise, the local timezone is assumed. 2. The administrator specifies the *time_zone()* <#sourcecommonopts>parameter for the source driver that reads the message. This parameter overrides the original timezone of the message. Each source defaults to the value of the *recv_time_zone()* <#reference_options>global option. 3. The destination driver specifies the timezone via the *time_zone()* <#destcommonopts>parameter. Each destination driver might have an associated timezone value to which message timestamps are converted before they are sent to the final destination (file or network socket). Each destination defaults to the value of the *send_time_zone()* <#reference_options>global option. A message can be sent to multiple destination zones. 4. When macro expansions are used in the destination filenames, the local timezone is used So option #1 is not being done I can't use #2 because my source is defined for all devices I tried using #3..... with the line modified as destination d_pd_snort { file("/var/log/syslog-ng-logs/pd_snort.1" perm(0644) time_zone(-04:00) template(t_default)); }; However.. this does not seem to make any difference... Can you please tell me why option #3 is not working .... and also... why is it that in #1 it is not defauling to just adding the local timezone since the incoming message does not have a timezone specified. Please help and thank you :)

2 2

General syslog question
by Mark Duling 03 May '07

03 May '07

I have a general question if you don't mind. I run OS X, and I am considering replacing my built-in syslog with syslog-ng, and I have an open source NMS that has an optional syslog daemon. I know I can't run two daemons listening on the same port, but I'm not sure how to get the NMS' daemon working. It appears I can't tell my Cisco switches to do syslog to an alternate tcp port (other than 514), so I can't run the 2nd daemon on another port only because my devices wouldn't send to it. Question - Is there a way to redirect syslogs to certain facilty #'s using syslog-ng to an alternate (something other than 514) syslog port so that it could co-exist with my NMS' syslog daemon (it can listen on any port)? Or if not, how do others handler this situation, so that their NMS box has syslog services for local Unix stuff as always, and also supports another syslog daemon receiving syslogs from the network? Mark

1 0

Filters to sort by host then service?
by Ryan 02 May '07

02 May '07

I am working on setting up a central syslog-ng server running on FreeBSD 6-STABLE. I have local logging setup. I also have the service listening on udp and tcp ports. I need help to figure out how to have a filter so it filters by hostname then by service. syslog-ng.conf #################### # Options #################### options { keep_hostname(yes); long_hostnames(off); sync(0); }; #################### # Sources #################### source local { file("/dev/klog" log_prefix("kernel: ")); unix-dgram("/var/run/log"); internal(); tcp(keep-alive(yes)); udp(); }; #################### # Destinations #################### # Destination Files from Local Host destination all { file("/var/log/all.log"); }; destination la_cron { file("/var/log/cron.log"); }; destination la_sudo { file("/var/log/sudo.log"); }; destination ld_sshd { file("/var/log/sshd.log"); }; destination ls_kernel { file("/var/log/kernel.log"); }; #################### # Filters #################### filter fa_cron { match("cron[\[0-9]+\]"); }; filter fa_sudo { match("sudo:"); }; filter fd_sshd { match("sshd[\[0-9]+\]") and match("Server listening") or match("Connection from") or match("client software version") or match("Accepted password") or match("Failed password") or match("Connection closed") or match("Closing connection") or match("subsystem request") or match("Received signal 15"); }; filter f_kernel { match("kernel: "); }; filter f_status { host("status"); }; filter f_terms { not match("cron[\[0-9]+\]") and not match("sudo:") and not match("sshd[\[0-9]+\]") and not match("kernel: "); }; #################### # Logs #################### # Logs for Local Host log { source(local); filter(f_status); filter(f_terms); destination(all); }; log { source(local); filter(f_status); filter(fa_cron); destination(la_cron); }; log { source(local); filter(f_status); filter(fa_sudo); destination(la_sudo); }; log { source(local); filter(f_status); filter(fd_sshd); destination(ld_sshd); }; log { source(local); filter(f_status); filter(f_kernel); destination(ls_kernel); }; ---------- With the above I can sort by the local machine which is named status, then it filters by service. I would like logs to go in the following format. /storage/logs/$YEAR/$MONTH/$DAY/$HOST/$service_filter.log So, I could use my service level filters like for sshd or whatever. Examples would be like the following. FQDN: server1.test.com, running sshd FQDN: server2.test.com, running named /storage/logs/2007/05/02/server1.test.com/sshd.log /storage/logs/2007/05/02/server2.test.com/named.log How would I go about doing this? I would appreciate any suggestions. Thanks. Phusion

1 0

Filtering Howto's
by Corey Bobb 02 May '07

02 May '07

I am looking for some detailed howto's on writing my own filters. I am trying to dump syslog-ng logs / date into splunk, but want to filter that which only I need to index into splunk. If anyone has any good howto's or have worked with splunk before I would certainly be open to any ideas or input they might have on the topic. Thanks cb

2 1

RE: [syslog-ng] Syslog-ng 2.0.3 make error
by Justin Randall 02 May '07

02 May '07

Hello, I am also experiencing this compile error when building Syslog-NG-2.0.3 on a CentOS (Linux) 4.4 host. Regards, Justin. -----Original Message----- From: Ravi Papisetti -X (rpapiset - HCL at Cisco) [mailto:rpapiset@cisco.com] Sent: Wednesday, April 25, 2007 3:34 PM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] Syslog-ng 2.0.3 make error Hi, I see below error when I run make && make install make all-recursive make[1]: Entering directory `/usr/local/syslog-ng/syslog-ng-2.0.3' Making all in src make[2]: Entering directory `/usr/local/syslog-ng/syslog-ng-2.0.3/src' if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -g -MT cfg-grammar.o -MD -MP -MF ".deps/cfg-grammar.Tpo" \ -c -o cfg-grammar.o `test -f 'cfg-grammar.c' || echo './'`cfg-grammar.c; \ then mv -f ".deps/cfg-grammar.Tpo" ".deps/cfg-grammar.Po"; \ else rm -f ".deps/cfg-grammar.Tpo"; exit 1; \ fi In file included from /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:15: afinet.h:30:20: libnet.h: No such file or directory In file included from /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:15: afinet.h:61: error: syntax error before "libnet_t" afinet.h:61: warning: no semicolon at end of struct or union afinet.h:63: warning: type defaults to `int' in declaration of `AFInetDestDriver' afinet.h:63: warning: data definition has no type or storage class /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y: In function `yyparse': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:592: error: syntax error before ')' token /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:626: error: syntax error before ')' token make[2]: *** [cfg-grammar.o] Error 1 make[2]: Leaving directory `/usr/local/syslog-ng/syslog-ng-2.0.3/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/syslog-ng/syslog-ng-2.0.3' make: *** [all] Error 2 Some one could tell me the reason. Thanks, Ravi Kumar P. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

turn off case sensitivity for match regex filter
by stucky 30 Apr '07

30 Apr '07

Guys Playing around with ng 2 and I started looking at the match filter again. Simple question that I cannot find an answer to anywhere on the net. How do I turn off case sensitivity for the match target ? I'd like the following line to match "error' or 'ERROR' or 'Error' filter logparse { match("error"); }; but of course it only matches 'error' since by default regex is case sensitive. Basically I'm trying to emulate 'grep -i' I guess I could do this : filter logparse { match("[Ee][Rr][Rr][Oo][Rr]"); }; but it'd be soo much simpler to turn off case sensitivity. And while we're talking regex. Shouldn't the above line actually read like this : filter logparse { match(".+error.+"); }; ? meaning "anything followed by 'error' followed by anything" Both appear to work so I assume the first line is interpreted by syslog-ng like the second line correct ? Help is appreciated stucky

2 4

Re: syslog-ng Digest, Vol 24, Issue 33
by Fabian Pucciarelli 30 Apr '07

30 Apr '07

I appreciate the help Bazsi, I receive around 600 of this messages daily, do you know when this messages are generated? I there any way to change that time to lets say one message/hour? TIA! On Sun, 2007-04-29 at 10:25 -0600, Fabian Pucciarelli wrote: > I have syslog ng writing to a mysql pipe and then a little script > reading the pipe and writing to the db. I currently receive many > messages like the following, I wonder if somenone can help me figure > out what this message is saying..... thanks. > > > | mailux | syslog | notice | notice | 2d | 2007-04-13 | 01:16:40 > | syslog-ng | syslog-ng[30548]: Log statistics; > dropped='pipe(/tmp/mysql.pipe)=0', processed='center(queued)=40295', > processed='center(received)=13206', > processed='destination(d_mysql)=40295', processed='source(net_tcp)=0', > processed='source(src)=8962', processed='source(net_udp)=4244' | > 46674 | This contains the various message counters that syslog-ng maintains internally. 'processed' counters just count how much messages were processed at the given point, 'dropped' counters indicate how much log messages were dropped by syslog-ng itself. The string in the parenthesis tell were the given counter is counting inside syslog-ng. 'center' is the main log message dispatching mechanism, it received 13206 messages from various sources and then sent 40295 messages out, e.g. you are probably sending individual messages to multiple destinations. You have two sources, one named 'src' generated 8962, the other called 'net_udp' 4244 messages. -- Bazsi On 4/30/07, syslog-ng-request(a)lists.balabit.hu < syslog-ng-request(a)lists.balabit.hu> wrote: > > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: syslog-ng Digest, Vol 24, Issue 32 (Fabian Pucciarelli) > 2. Re: turn off case sensitivity for match regex filter > (Balazs Scheidler) > 3. Re: Re: syslog-ng Digest, Vol 24, Issue 32 (Balazs Scheidler) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 29 Apr 2007 10:25:18 -0600 > From: "Fabian Pucciarelli" <fabiangp(a)gmail.com> > Subject: [syslog-ng] Re: syslog-ng Digest, Vol 24, Issue 32 > To: syslog-ng(a)lists.balabit.hu > Message-ID: > <1e6757090704290925n541deb73t13eace5731aced3b(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I have syslog ng writing to a mysql pipe and then a little script reading > the pipe and writing to the db. I currently receive many messages like the > following, I wonder if somenone can help me figure out what this message > is > saying..... thanks. > > > | mailux | syslog | notice | notice | 2d | 2007-04-13 | 01:16:40 | > syslog-ng | syslog-ng[30548]: Log statistics; > dropped='pipe(/tmp/mysql.pipe)=0', processed='center(queued)=40295', > processed='center(received)=13206', > processed='destination(d_mysql)=40295', > processed='source(net_tcp)=0', processed='source(src)=8962', > processed='source(net_udp)=4244' | 46674 | > > On 4/29/07, syslog-ng-request(a)lists.balabit.hu < > syslog-ng-request(a)lists.balabit.hu> wrote: > > > > Send syslog-ng mailing list submissions to > > syslog-ng(a)lists.balabit.hu > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > or, via email, send a message with subject or body 'help' to > > syslog-ng-request(a)lists.balabit.hu > > > > You can reach the person managing the list at > > syslog-ng-owner(a)lists.balabit.hu > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of syslog-ng digest..." > > > > > > Today's Topics: > > > > 1. Re: turn off case sensitivity for match regex filter > > (Balazs Scheidler) > > 2. Re: turn off case sensitivity for match regex filter (stucky) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Sat, 28 Apr 2007 12:42:06 +0200 > > From: Balazs Scheidler <bazsi(a)balabit.hu> > > Subject: Re: [syslog-ng] turn off case sensitivity for match regex > > filter > > To: Syslog-ng users' and developers' mailing list > > <syslog-ng(a)lists.balabit.hu> > > Message-ID: <1177756926.14925.17.camel(a)bzorp.balabit> > > Content-Type: text/plain > > > > On Sat, 2007-04-28 at 01:52 -0700, stucky wrote: > > > Guys > > > > > > Playing around with ng 2 and I started looking at the match filter > > > again. > > > Simple question that I cannot find an answer to anywhere on the net. > > > How do I turn off case sensitivity for the match target ? > > > I'd like the following line to match "error' or 'ERROR' or 'Error' > > > > > > filter logparse { match("error"); }; > > > > > > but of course it only matches 'error' since by default regex is case > > > sensitive. > > > Basically I'm trying to emulate 'grep -i' > > > I guess I could do this : > > > > > > filter logparse { match("[Ee][Rr][Rr][Oo][Rr]"); }; but it'd be soo > > > much simpler to turn off case sensitivity. > > > > Yes, you are right. But it's not currently possible. It should be > > however, I'll try to add it in the nearfuture. > > > > > > > > And while we're talking regex. Shouldn't the above line actually read > > > like this : > > > > > > filter logparse { match(".+error.+"); }; ? > > > > > > meaning "anything followed by 'error' followed by anything" > > > Both appear to work so I assume the first line is interpreted by > > > syslog-ng like the second line correct ? > > > > syslog-ng interprets "match" the same as grep, e.g. it does not care > > where the pattern is found. if you want to match the beginning or the > > end of line, you need to use explicit ^ and $ characters. > > > > -- > > Bazsi > > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Sat, 28 Apr 2007 12:44:04 -0700 > > From: stucky <stucky101(a)gmail.com> > > Subject: Re: [syslog-ng] turn off case sensitivity for match regex > > filter > > To: "Syslog-ng users' and developers' mailing list" > > <syslog-ng(a)lists.balabit.hu> > > Message-ID: > > <30997e260704281244g6f1225bfwc7da2b4e05fdfd9f(a)mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Baszi > > > > Cool. I'm in the middle of building a new infrastructure and would like > to > > use this feature. I'm not a programmer but I assume adding this feature > > shouldn't be very hard at all right ? > > If you had a rough ETA that'd help me. > > > > thx > > > > On 4/28/07, Balazs Scheidler <bazsi(a)balabit.hu> wrote: > > > > > > On Sat, 2007-04-28 at 01:52 -0700, stucky wrote: > > > > Guys > > > > > > > > Playing around with ng 2 and I started looking at the match filter > > > > again. > > > > Simple question that I cannot find an answer to anywhere on the net. > > > > How do I turn off case sensitivity for the match target ? > > > > I'd like the following line to match "error' or 'ERROR' or 'Error' > > > > > > > > filter logparse { match("error"); }; > > > > > > > > but of course it only matches 'error' since by default regex is case > > > > sensitive. > > > > Basically I'm trying to emulate 'grep -i' > > > > I guess I could do this : > > > > > > > > filter logparse { match("[Ee][Rr][Rr][Oo][Rr]"); }; but it'd be soo > > > > much simpler to turn off case sensitivity. > > > > > > Yes, you are right. But it's not currently possible. It should be > > > however, I'll try to add it in the nearfuture. > > > > > > > > > > > And while we're talking regex. Shouldn't the above line actually > read > > > > like this : > > > > > > > > filter logparse { match(".+error.+"); }; ? > > > > > > > > meaning "anything followed by 'error' followed by anything" > > > > Both appear to work so I assume the first line is interpreted by > > > > syslog-ng like the second line correct ? > > > > > > syslog-ng interprets "match" the same as grep, e.g. it does not care > > > where the pattern is found. if you want to match the beginning or the > > > end of line, you need to use explicit ^ and $ characters. > > > > > > -- > > > Bazsi > > > > > > _______________________________________________ > > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > > > -- > > stucky > >

2 1