May 2007 - syslog-ng - lists.balabit.hu

Syslog with epilog (for windows)
by Javier Terceiro 09 May '07

09 May '07

Hello list, I am mounting a log centralized server and for servers Windows I am using epiloj and the collector server is a syslog-ng. I have a problem with the data format that epiloj sends, I can`t correctly divide it in fields (server, date, message) in syslog-ng server. For example - The original message of epilog: GenericLog 0 2007-05-09 12:23:42 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- The format with that the data keep would have to be the following one: Program <> Date <> MSGONLY (in syslog-ng format). - The message saved by syslog-ng: GenericLog 0 2007-05-09 12 <> 2007 May 9 12:23:42 <> 3:42 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- - The message correctly: GenericLog <> 2007 May 9 12:23:42 <> 192.*.*.* - W3 ST 172.*.*.* 80 GET / - 403 5 334 174 0 HTTP/1.1 --- The problem is the fields than syslog don`t understand. Some idea of like obtaining it? Some program for obtain log's for windows server's? thanks. -- A greeting, Javier.

1 0

line of sendmail log output missing?
by Centyx Centalix 05 May '07

05 May '07

Hi, I recently switched from syslogd to syslog-ng on several servers running CentOS. While running syslogd, a usual mail delivery appeared in /var/log/maillog as follows: Apr 22 04:11:07 ws01 sendmail[32181]: l3M8B71l032181: from=<someuser(a)someremote.domain>, size=2361, class=0, nrcpts=1, msgid=<001901c784c6$9688fea0$018ab484@maison9tm8hfuv>, proto=ESMTP, relay=root@localhost Apr 22 04:11:07 ws01 sendmail[32179]: l3M8Av5T032164: to=<domain_support(a)somelocal.domain>, delay=00:00:04, xdelay=00:00:00, mailer=virthostmail, pri=121926, relay=somelocal.domain, dsn=2.0.0, stat=Sent (l3M8B71l032181 Message accepted for delivery) Apr 22 04:11:07 ws01 spamd[24594]: spamd: connection from localhost.localdomain [127.0.0.1] at port 50252 Apr 22 04:11:07 ws01 spamd[24594]: spamd: setuid to domain_admin(a)somelocal.domain succeeded Apr 22 04:11:07 ws01 spamd[24594]: spamd: processing message <001901c784c6$9688fea0$018ab484@maison9tm8hfuv> for domain_admin@somelocal.domain:22003 Apr 22 04:11:08 ws01 spamd[24594]: spamd: identified spam (18.5/4.0) for domain_admin@somelocal.domain:22003 in 1.3 seconds, 2646 bytes. Apr 22 04:11:08 ws01 spamd[24594]: spamd: result: Y 18 - BAYES_99,HELO_DYNAMIC_IPADDR2,HELO_DYNAMIC_SPLIT_IP,HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_NJABL_DUL,RCVD_IN_SORBS_DUL scantime=1.3,size=2646,user=domain_admin@somelocal.com,uid=22003,required_score=4.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=50252,mid=<001901c784c6$9688fea0$018ab484@maison9tm8hfuv>,bayes=1,autolearn=spam Apr 22 04:11:08 ws01 spamd[4109]: prefork: child states: BI Apr 22 04:11:08 ws01 spamd[4109]: spamd: handled cleanup of child pid 24594 due to SIGCHLD Apr 22 04:11:08 ws01 spamd[4109]: spamd: server successfully spawned child process, pid 32189 Apr 22 04:11:08 ws01 spamd[4109]: prefork: child states: II Apr 22 04:11:08 ws01 sendmail[32182]: l3M8B71l032181: to=domain_admin, delay=00:00:01, xdelay=00:00:01, mailer=local, pri=32547, dsn=2.0.0, stat=Sent Now, after replacing syslogd with syslog-ng, the last line showing the delivery by the local mailer ( in this case, procmail ) is absent from the log files. I have made sure that levels info through emergency are specified for facility mail, and have also checked to see if this message was showing up under any other facility ( although I have not explicity defined the levels for all facilities ). I am rather stumped about where this line of log output is going. Any suggestions would be greatly appreciated. -- We're all in this together.

3 6

Please Help problems with timezone and stamping !!
by Guy Fleegman 04 May '07

04 May '07

Hello: I have several sources that are all reporting to a syslog-ng 2.0.3 server My source is source all_routers { udp(ip(0.0.0.0) port(514)); }; My problem is that I have one source that is isolated to a separate file I isolate the file with the following filter f_pd_snort { host(intsnort1); }; and have this destination destination d_pd_snort { file("/var/log/syslog-ng-logs/pd_snort.1" perm(0644) template(t_default)); }; then i use the following command log { source(all_routers); filter(f_pd_snort); destination(d_pd_snort); }; The problem is the format the data is arriving in. It send data in UTC time as follows Apr 30 14:04:35 2007 intsnort1 SFIMS: [119:13:1] Snort Alert [Classification: Unknown] [Priority: 3] Apr 30 14:04:35 2007 intsnort1 SFIMS: [119:4:1] Snort Alert [Classification: Unknown] [Priority: 3] Apr 30 14:04:35 2007 intsnort1 SFIMS: [122:3:0] Snort Alert [Classification: Unknown] [Priority: 3] This is UTC. It is the time that is being sent in the actual packet (I verified with wireshark) I want syslog-ng to modify the times going into this file (and this file only) to reflect EST time. Based upon the documentation The following algorithm is used. 1. The sender can specify the timezone of the messages. If the incoming message includes a timezone it is associated with the message. Otherwise, the local timezone is assumed. 2. The administrator specifies the *time_zone()* <#sourcecommonopts>parameter for the source driver that reads the message. This parameter overrides the original timezone of the message. Each source defaults to the value of the *recv_time_zone()* <#reference_options>global option. 3. The destination driver specifies the timezone via the *time_zone()* <#destcommonopts>parameter. Each destination driver might have an associated timezone value to which message timestamps are converted before they are sent to the final destination (file or network socket). Each destination defaults to the value of the *send_time_zone()* <#reference_options>global option. A message can be sent to multiple destination zones. 4. When macro expansions are used in the destination filenames, the local timezone is used So option #1 is not being done I can't use #2 because my source is defined for all devices I tried using #3..... with the line modified as destination d_pd_snort { file("/var/log/syslog-ng-logs/pd_snort.1" perm(0644) time_zone(-04:00) template(t_default)); }; However.. this does not seem to make any difference... Can you please tell me why option #3 is not working .... and also... why is it that in #1 it is not defauling to just adding the local timezone since the incoming message does not have a timezone specified. Please help and thank you :)

2 2

General syslog question
by Mark Duling 02 May '07

02 May '07

I have a general question if you don't mind. I run OS X, and I am considering replacing my built-in syslog with syslog-ng, and I have an open source NMS that has an optional syslog daemon. I know I can't run two daemons listening on the same port, but I'm not sure how to get the NMS' daemon working. It appears I can't tell my Cisco switches to do syslog to an alternate tcp port (other than 514), so I can't run the 2nd daemon on another port only because my devices wouldn't send to it. Question - Is there a way to redirect syslogs to certain facilty #'s using syslog-ng to an alternate (something other than 514) syslog port so that it could co-exist with my NMS' syslog daemon (it can listen on any port)? Or if not, how do others handler this situation, so that their NMS box has syslog services for local Unix stuff as always, and also supports another syslog daemon receiving syslogs from the network? Mark

1 0

Filters to sort by host then service?
by Ryan 02 May '07

02 May '07

I am working on setting up a central syslog-ng server running on FreeBSD 6-STABLE. I have local logging setup. I also have the service listening on udp and tcp ports. I need help to figure out how to have a filter so it filters by hostname then by service. syslog-ng.conf #################### # Options #################### options { keep_hostname(yes); long_hostnames(off); sync(0); }; #################### # Sources #################### source local { file("/dev/klog" log_prefix("kernel: ")); unix-dgram("/var/run/log"); internal(); tcp(keep-alive(yes)); udp(); }; #################### # Destinations #################### # Destination Files from Local Host destination all { file("/var/log/all.log"); }; destination la_cron { file("/var/log/cron.log"); }; destination la_sudo { file("/var/log/sudo.log"); }; destination ld_sshd { file("/var/log/sshd.log"); }; destination ls_kernel { file("/var/log/kernel.log"); }; #################### # Filters #################### filter fa_cron { match("cron[\[0-9]+\]"); }; filter fa_sudo { match("sudo:"); }; filter fd_sshd { match("sshd[\[0-9]+\]") and match("Server listening") or match("Connection from") or match("client software version") or match("Accepted password") or match("Failed password") or match("Connection closed") or match("Closing connection") or match("subsystem request") or match("Received signal 15"); }; filter f_kernel { match("kernel: "); }; filter f_status { host("status"); }; filter f_terms { not match("cron[\[0-9]+\]") and not match("sudo:") and not match("sshd[\[0-9]+\]") and not match("kernel: "); }; #################### # Logs #################### # Logs for Local Host log { source(local); filter(f_status); filter(f_terms); destination(all); }; log { source(local); filter(f_status); filter(fa_cron); destination(la_cron); }; log { source(local); filter(f_status); filter(fa_sudo); destination(la_sudo); }; log { source(local); filter(f_status); filter(fd_sshd); destination(ld_sshd); }; log { source(local); filter(f_status); filter(f_kernel); destination(ls_kernel); }; ---------- With the above I can sort by the local machine which is named status, then it filters by service. I would like logs to go in the following format. /storage/logs/$YEAR/$MONTH/$DAY/$HOST/$service_filter.log So, I could use my service level filters like for sshd or whatever. Examples would be like the following. FQDN: server1.test.com, running sshd FQDN: server2.test.com, running named /storage/logs/2007/05/02/server1.test.com/sshd.log /storage/logs/2007/05/02/server2.test.com/named.log How would I go about doing this? I would appreciate any suggestions. Thanks. Phusion

1 0

Filtering Howto's
by Corey Bobb 02 May '07

02 May '07

I am looking for some detailed howto's on writing my own filters. I am trying to dump syslog-ng logs / date into splunk, but want to filter that which only I need to index into splunk. If anyone has any good howto's or have worked with splunk before I would certainly be open to any ideas or input they might have on the topic. Thanks cb

2 1

RE: [syslog-ng] Syslog-ng 2.0.3 make error
by Justin Randall 02 May '07

02 May '07

Hello, I am also experiencing this compile error when building Syslog-NG-2.0.3 on a CentOS (Linux) 4.4 host. Regards, Justin. -----Original Message----- From: Ravi Papisetti -X (rpapiset - HCL at Cisco) [mailto:rpapiset@cisco.com] Sent: Wednesday, April 25, 2007 3:34 PM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] Syslog-ng 2.0.3 make error Hi, I see below error when I run make && make install make all-recursive make[1]: Entering directory `/usr/local/syslog-ng/syslog-ng-2.0.3' Making all in src make[2]: Entering directory `/usr/local/syslog-ng/syslog-ng-2.0.3/src' if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -g -MT cfg-grammar.o -MD -MP -MF ".deps/cfg-grammar.Tpo" \ -c -o cfg-grammar.o `test -f 'cfg-grammar.c' || echo './'`cfg-grammar.c; \ then mv -f ".deps/cfg-grammar.Tpo" ".deps/cfg-grammar.Po"; \ else rm -f ".deps/cfg-grammar.Tpo"; exit 1; \ fi In file included from /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:15: afinet.h:30:20: libnet.h: No such file or directory In file included from /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:15: afinet.h:61: error: syntax error before "libnet_t" afinet.h:61: warning: no semicolon at end of struct or union afinet.h:63: warning: type defaults to `int' in declaration of `AFInetDestDriver' afinet.h:63: warning: data definition has no type or storage class /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y: In function `yyparse': /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:592: error: syntax error before ')' token /home/bazsi/zwa/work/syslog-ng-2.0/syslog-ng/src/cfg-grammar.y:626: error: syntax error before ')' token make[2]: *** [cfg-grammar.o] Error 1 make[2]: Leaving directory `/usr/local/syslog-ng/syslog-ng-2.0.3/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/syslog-ng/syslog-ng-2.0.3' make: *** [all] Error 2 Some one could tell me the reason. Thanks, Ravi Kumar P. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1