- syslog-ng - lists.balabit.hu

error installing
by Raymond Norton 17 May '06

17 May '06

Any idea how to fix this? /home/admin/syslog-ng-1.6.10/src/cfg-lex.c:1126: undefined reference to `yywrap' collect2: ld returned 1 exit status make[3]: *** [syslog-ng] Error 1 make[3]: Leaving directory `/home/admin/syslog-ng-1.6.10/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/admin/syslog-ng-1.6.10/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/admin/syslog-ng-1.6.10/src' make: *** [all-recursive] Error 1

3 3

Newbie Looking for Help
by Mark R. White 16 May '06

16 May '06

I want to setup a syslog-ng server to do syslogging for my network. So far it's easy, the part I'm having trouble with, and I'm not even sure if it's possible, is I want to log by device to it's own individual syslog file. Hence the reason for using syslog-ng vice sysklogd, I think the syntax would be something as follows, but I could be way off base. device1 /var/log/network/device1 log { source(device1) filter (?????) destination (?????) }; I also don't understand where to define the facilities as I did in standard syslog. Any help would be greatly appreciated. One other quick question, when I do a default install of syslog-ng (on Debian Etch), it automatically logs to the tty. Is this correct for all versions, and whether it is or it's not, how do I stop it? Thanks again, Mark

5 9

Help: Cannot open file for writing (Permission denied)
by Muath Al Khalaf 15 May '06

15 May '06

Hi, I have just installed syslog-ng on Redhat enterprise 4. I am using it as remote logging server for a syslog-ng client. Here is my remote logging configuration: source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; destination d_separatedbyhosts { file("/var/log/syslog-ng/$HOST/messages" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes)); }; log { source(s_remote); destination(d_separatedbyhosts); }; I am getting the following error message in /var/log/syslog-ng.log on the same machine using tail "cannot open file /var/log/syslog-ng/host1messages for writing (Permission denied)" I have tried to create the file myself and give it 777 permission along with all the directories /var, /var/log, /var/log/syslog-ng If you notice here, 1- syslog-ng is writing error messages from the same machine to /var/log/syslog-ng which means syslog-ng is working well for local logging on the same machine ( I have stopped syslogd). 2- The error has a file name "host1messages" while in the configuration file it is "/$HOST/messages" 3- I am accessing the syslog-ng server using remote SSH from windows using putty client. When I open the /var/log/syslog-ng.log file using "vi" I got a strange character instead of "/" in the previous error message between "host1" and "messages". While - as you have seen above - using tail I do not get any character neither / nor the strange one in the syslog-ng file. Kind regards

1 0

RE: [syslog-ng] Newbie Looking for Help
by Mark R. White 13 May '06

13 May '06

Thanks for all the help I've received so far. Although, I'm still not able to do what I need to do. Right now, I just want to log one specific device to a specific log file. One email I received said I should use the netmask filter option, although for some reason, my system doesn't like that option. Here is the temp syslog-ng.conf I'm using. I have the full backed up, I just want to get this part working. Syslog-ng starts, but it doesn't actually log any thing. I cut this down so it would be easier to work with, although it's very possible that I cut out a required part of the log file. As of right now, I don't care if it logs any from the local system. I just need to log the pix device listed. Thanks in advance for any help, Mark # # Configuration file for syslog-ng under Debian # # attempts at reproducing default syslog behavior # the standard syslog levels are (in descending order of priority): # emerg alert crit err warning notice info debug # the aliases "error", "panic", and "warn" are deprecated # the "none" priority found in the original syslogd configuration is # only used in internal messages created by syslogd ###### # options options { # disable the chained hostname format in logs # (default is enabled) chain_hostnames(0); # the time to wait before a died connection is re-established # (default is 60) time_reopen(10); # the time to wait before an idle destination file is closed # (default is 60) time_reap(360); # the number of lines buffered before written to file # you might want to increase this if your disk isn't catching with # all the log messages you get or if you want less disk activity # (say on a laptop) # (default is 0) #sync(0); # the number of lines fitting in the output queue log_fifo_size(2048); # enable or disable directory creation for destination files create_dirs(yes); # default owner, group, and permissions for log files # (defaults are 0, 0, 0600) owner(root); group(root); perm(0644); # default owner, group, and permissions for created directories # (defaults are 0, 0, 0700) dir_owner(root); dir_group(root); dir_perm(0744); # enable or disable DNS usage # syslog-ng blocks on DNS queries, so enabling DNS may lead to # a Denial of Service attack # (default is yes) use_dns(no); # maximum length of message in bytes # this is only limited by the program listening on the /dev/log Unix # socket, glibc can handle arbitrary length log messages, but -- for # example -- syslogd accepts only 1024 bytes # (default is 2048) #log_msg_size(2048); }; # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" log_prefix("kernel: ")); # use the above line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) udp(); }; #destinations for log files destination corppix_d { file ("/var/log/network/corppix/pix.log" owner (root) group (root) perm (0644) dir_perm (0744)); }; #filters filter corppix_f { netmask("10.10.10.2"); facility(local4); }; # logs log { filter(corppix_f); destination(corppix_d); };

2 1

Tokenizing messages
by Dukes Clayton 13 May '06

13 May '06

Hi Folks, I'm trying to figure out if there is a way to further parse incoming messages. Right now I have: destination db_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, datetime, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$MSG' );\n") template-escape(yes)); }; But I want to take $MSG and parse it into individual tokens. How can I accomplish this? Tia! -Clayton

1 0

Hi everybody i'm new to the list and i have a nice gift, a little script to translate configuration :o)
by Harry Rüter 13 May '06

13 May '06

Hi, i'm simply happy about syslog-ng :o) I coded a shellscript to convert configuration from old syslog to syslog-ng. Works well for me on Suse LINUX 9.1 Have a look at it and enjoy. Would be nice if it would be included in the distribution, but it's tested with 1.6.11 only, so it necessary to test it on different OS and distributions isn't it ? Greets Harry #!/bin/sh ############################################################################### # # Translator from syslog.conf to syslog-ng.conf # by Harry Rueter <h.rueter(a)gmx.net> # # Writes translated configuration to stdout # # New file has to be checked, no warranty at all !!!!!! # Templates are my prefered format. # Some options and sources are commented out # for manual configuration. # Trailing "-" in destination file will not be translated # # History: 12.05.2006: initial coding # ############################################################################### INFILE="$1" if [ "a$INFILE" = "a" ] then echo "error, usage: `basename $0` syslog_configuration_file [> syslog-ng__configuration_file]" exit 1 fi trenner() { echo "########################################################" } trennerecho() { trenner echo "" } trennercomment() { COMMENT="$1" trenner echo "# $COMMENT" trenner } header() { trenner echo "# Generated by `basename $0`(coded by Harry Rueter 5/2006" trenner echo echo "#######################" echo "# Global options #" echo "#######################" echo "options" echo "{" echo " sync(0);" echo " time_reopen(10);" echo " log_fifo_size(100);" echo " #mark(0);" echo " #log_fifo_size();" echo " #chain_hostnames(no);" echo " #create_dirs (no);" echo " #dir_perm(0755);" echo " #dns_cache(yes);" echo " #keep_hostname(yes);" echo " #log_msg_size(8192);" echo " #long_hostnames(on);" echo " #perm(0644);" echo " #stats(3600);" echo " #use_dns(yes);" echo " #use_fqdn(yes);" echo "};" trennerecho } downstr() { echo "$1"|sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' } mk_filter_rules() { INPUT="$1" FAC=`echo "$INPUT"|cut -d"." -f1|sed 's/=//g'` LEVEL=`echo "$INPUT"|cut -d"." -f2|sed 's/=//g'` TMPFAC="" TMPLEVEL="" if [ "$FAC" != "*" ] then TMPFAC="facility($FAC)" fi if [ "$LEVEL" = "*" ] then TMPLEVEL="level(debug..warn)" elif [ "$LEVEL" != "none" ] then TMPLEVEL="level($LEVEL)" fi if [ -n "$TMPLEVEL" -a -n "$TMPFAC" ] then echo "($TMPFAC and $TMPLEVEL)" else echo "($TMPFAC$TMPLEVEL)" fi } mk_dev_log_src() { trennercomment "src=/dev/log" echo "source dev_log_src" echo "{" echo " unix-stream(\"/dev/log\");" echo "};" trennerecho } mk_unix_dgram_log_src() { trennercomment "src=" echo "#source unix_dgram_log_src" echo "#{" echo "# unix-dgram(\"\");" echo "#}:" trennerecho }; mk_udp_log_src() { trennercomment "src=udp" echo "#source udp_log_src" echo "#{" echo "# udp(" echo "# ip(0.0.0.0)" echo "# port(5000)" echo "# max-connections(300)" echo "# );" echo "#};" trennerecho }; mk_tcp_log_src() { trennercomment "src=tcp" echo "#source tcp_log_src" echo "#{" echo "# tcp(" echo "# ip(0.0.0.0)" echo "# port(5000)" echo "# max-connections(300)" echo "# );" echo "#};" trennerecho }; mk_pipe_log_src() { trennercomment "src=pipe" echo "#source pipe_log_src" echo "#{" echo "# pipe();" echo "#};" trennerecho }; mk_internal_log_src() { trennercomment "src=internal()" echo "source internal_src" echo "{" echo " internal();" echo "};" trennerecho } mk_log_src() { mk_dev_log_src mk_unix_dgram_log_src mk_udp_log_src mk_tcp_log_src mk_pipe_log_src mk_internal_log_src echo } mk_dest_file() { FILE="$1" NAME=`basename $FILE|sed 's/\./_/g'` trennercomment "destination file=$FILE" echo "destination d_$NAME" echo "{" echo " file(\"$FILE\"" echo " owner(\"root\")" echo " group(\"root\")" echo " perm(0600)" echo " template( \"[\$YEAR/\$MONTH/\$DAY \$HOUR:\$MIN:\$SEC] \$PROGRAM - \$PRIORITY : \$MSGONLY\n\")" echo " #template_escape(no)" echo " #sync_freq(1000)" echo " #create_dirs(yes/no)" echo " #dir_perm(755)" echo " #dir_owner(root)" echo " #dir_group(root)" echo " );" echo "};" } mk_dest_pipe() { FILE="$1" NAME=`basename $FILE|sed 's/\./_/g'` trennercomment "destination pipe=$FILE" echo "destination d_$NAME" echo "{" echo " pipe(\"$FILE\"" echo " owner(\"root\")" echo " group(\"root\")" echo " perm(0600)" echo " template( \"[\$YEAR/\$MONTH/\$DAY \$HOUR:\$MIN:\$SEC] \$PROGRAM - \$PRIORITY : \$MSGONLY\n\")" echo " #template_escape(no)" echo " );" echo "};" } mk_dest_filter() { NAME="$1" FILTER="$2" echo "filter f_$NAME" echo "{" echo " $FILTER;" echo "};" trenner } mk_log_entry() { FILTER="$1" DEST="$2" LINE="$3" echo "log" echo "{" echo " source(dev_log_src);" echo " filter($FILTER);" echo " destination($DEST);" echo "};" trennerecho } mk_internal_entry() { DEST="/var/log/syslog-ng-internal.log" trennercomment "internal syslog-ng logging" mk_dest_file $DEST echo "log" echo "{" echo " source(internal_src);" echo " destination(d_syslog-ng-internal_log};" echo "};" trennerecho } mk_dest() { DEST="$1" echo $DEST|grep "^|" >/dev/null if [ $? -eq 0 ] then DEST=`echo "$DEST"|sed 's/^|//g'` mk_dest_pipe $DEST else mk_dest_file $DEST fi } proceedline() { LINE="$1" DEST="$2" CNT=1 THEFILTER="" trenner echo "# syslogd: $LINE" echo "# log entry for $DEST" trennerecho mk_dest "$DEST" trenner echo "$LINE"|grep ";" >/dev/null 2>&1 if [ $? -eq 0 ] then while [ 0 -eq 0 ] do PART=`echo "$LINE"|cut -d";" -f$CNT` CNT=`expr $CNT + 1` if [ "a$PART" = "a" ] then NAME="`basename $DEST|sed 's/\./_/g'`" mk_dest_filter "$NAME" "$THEFILTER" break else TMPFILTER=`mk_filter_rules "$PART"` if [ -n "$THEFILTER" ] then THEFILTER="$THEFILTER or $TMPFILTER" else THEFILTER="$TMPFILTER" fi fi done else NAME=`basename $DEST|sed 's/\./_/g'` TMPFILTER=`mk_filter_rules "$LINE"` mk_dest_filter "$NAME" "$TMPFILTER" fi } proceed() { LINE="$1" RULES=`echo "$LINE"|cut -d" " -f1` DEST=`echo "$LINE"|cut -d" " -f2-10|sed 's/^-//g'` if [ "$DEST" = "*" ] then DEST="/var/log/everything.log" fi proceedline "$RULES" "$DEST" WHAT=`basename $DEST|sed 's/\./_/g'` mk_log_entry "f_$WHAT" "d_$WHAT" $LINE } ############################################# # main program ############################################# # statical entrys header echo mk_log_src echo mk_internal_entry echo # dynamical entries cat "$INFILE" |grep -v "^#"|grep -v "^$" |expand|sed 's/ */ /'| \ while read LINE do proceed "$LINE" done

1 0

RE: [syslog-ng] syslog-ng GUI
by Dukes Clayton 11 May '06

11 May '06

Uh, ok... So can someone dumb those pdf's down so that someone with only a 150iq can understand them? Geeze man. I'm working on a development version of php-syslog-ng that does (I think) what sisyphus is referring to. Essentially, group multiple, repeated tokens into the DB thereby decreasing large amounts of data. When a token gets repeated, simply update a count field using "ON DUPLICATE KEY UPDATE count = count + 1" (available in MySQl v4.1+) I just need the time to re-write the tables. I've already done it with the hosts table: I created a new table called hosts and just update the count field, then reference that host using a foreign key in the logs table. -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Jon Stearley Sent: Thursday, May 11, 2006 10:50 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng GUI On May 11, 2006, at 6:51 AM, Arya, Manish Kumar wrote: > Hi Guys, > > I am storing logs on a central server having 3T SAN, using follwing > template > > destination indexlog { > > file("/logs/log01/indexlog/$YEAR/$MONTH/$DAY/$HOST" > > template("$HOUR:$MIN:$SEC,$PROGRAM,$FACILITY,$PRIORITY,$MSGONLY\n") > template-escape(yes) > owner(root) group(root) perm(0644) > dir_perm(0755) create_dirs(yes)); > }; > > my logging is done perfectly :) > > like /logs/log01/indexlog/2006/05/11/hostnames > > I want to have a GUI to view logs with following facilities > > -search logs on basis on date/time, text patterns in > messages,hostnames. http://www.cs.sandia.gov/sisyphus/ mines patterns, but does not have a production GUI (yet). It is more of a research tool at this point, but I would be happy to help you give it a try. Recent emphasis has been on the functionality described in .../detection.pdf. Please let me know if interested, like I said I'd be happy to help, and am in fact looking for additional datasets to analyze; I find my approach to be effective for supercomputer logs, but have not yet explored its effectiveness for other log sets (eg enterprise). I've been waiting to implement a production GUI until I am confident that the underlying functionality is general and excellent. My current leaning is towards adding sisyphus functionality to splunk's interface (and have contacted splunk about this). G'day! -- +--------------------------------------------------------------+ | Jon Stearley (505) 845-7571 (FAX 844-9297) | | Sandia National Laboratories Scalable Systems Integration | +--------------------------------------------------------------+ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] syslog-ng GUI
by Dukes Clayton 11 May '06

11 May '06

Try Php-syslog-ng -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Arya, Manish Kumar Sent: Thursday, May 11, 2006 7:51 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] syslog-ng GUI Hi Guys, I am storing logs on a central server having 3T SAN, using follwing template destination indexlog { file("/logs/log01/indexlog/$YEAR/$MONTH/$DAY/$HOST" template("$HOUR:$MIN:$SEC,$PROGRAM,$FACILITY,$PRIORITY,$MSGONLY\n") template-escape(yes) owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; my logging is done perfectly :) like /logs/log01/indexlog/2006/05/11/hostnames I want to have a GUI to view logs with following facilities -search logs on basis on date/time, text patterns in messages,hostnames. -should provide facility to have filters associated with user authentication/authorization. -should be able to to parallel search to improve search response time. Regards, -Manish __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

Multi-threading Syslog-ng
by jarrod.schous＠sasktel.sk.ca 10 May '06

10 May '06

Is syslog-ng capable of multi-threading?. I.E. Can it distribute its processing requirements accross multiple CPU's. I have seen a couple side notes that say it isn't but I was wondering if there is a definitive answer. Thanks Jarrod. NOTICE: This confidential e-mail message is only for the intended recipient(s). If you are not the intended recipient, be advised that disclosing, copying, distributing, or any other use of this message, is strictly prohibited. In such case, please destroy this message and notify the sender.

2 1

cnnection 0 desn't have any source
by wayne yu 10 May '06

10 May '06

Hi everyone: I got waning message when I starting my syslog : Warning: connection 0 doesn't have any sources and I don't how to fixing it. can someone look at my /etc/syslog-ng.conf and tell me how to fixing it. # Options options { use_fqdn(yes); sync(0); keep_hostname(yes); chain_hostnames(no); create_dirs(yes); }; # Sources of syslog messages (both local and remote messages on the server) source s_local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; source s_stunnel { tcp(ip("127.0.0.1") port(514) max-connections(1)); }; source s_udp { udp(); }; # Level Filters filter f_emerg { level (emerg); }; filter f_alert { level (alert .. emerg); }; filter f_crit { level (crit .. emerg); }; filter f_err { level (err .. emerg); }; filter f_warning { level (warning .. emerg); }; filter f_notice { level (notice .. emerg); }; filter f_info { level (info .. emerg); }; filter f_debug { level (debug .. emerg); }; # Facility Filters filter f_kern { facility (kern); }; filter f_user { facility (user); }; filter f_mail { facility (mail); }; filter f_daemon { facility (daemon); }; filter f_auth { facility (auth); }; filter f_syslog { facility (syslog); }; filter f_lpr { facility (lpr); }; filter f_news { facility (news); }; filter f_uucp { facility (uucp); }; filter f_cron { facility (cron); }; filter f_local0 { facility (local0); }; filter f_local1 { facility (local1); }; filter f_local2 { facility (local2); }; filter f_local3 { facility (local3); }; filter f_local4 { facility (local4); }; filter f_local5 { facility (local5); }; filter f_local6 { facility (local6); }; filter f_local7 { facility (local7); }; # Custom Filters filter f_user_none { not facility (user); }; filter f_kern_debug { filter (f_kern) and filter (f_debug); }; filter f_daemon_notice { filter (f_daemon) and filter (f_notice); }; filter f_mail_crit { filter (f_mail) and filter (f_crit); }; filter f_mesg { filter (f_kern_debug) or filter (f_daemon_notice) or filter (f_mail_crit); }; filter f_authinfo { filter (f_auth) or program (sudo); }; destination hosts { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; #Destinations: local files, the console, and the client files destination l_authlog { file ("/var/log/authlog"); }; destination l_messages { file ("/var/log/messages"); }; destination l_maillog { file ("/var/log/maillog"); }; destination l_ipflog { file ("/var/log/ipflog"); }; destination l_imaplog { file ("/var/log/imaplog"); }; destination l_syslog { file ("/var/log/syslog"); }; destination l_console { file ("/dev/console"); }; destination r_authlog { file ("/var/log/clients/$YEAR/$MONTH/$HOST/authlog"); }; destination r_messages { file ("/var/log/clients/$YEAR/$MONTH/$HOST/messages"); }; destination r_maillog { file ("/var/log/clients/$YEAR/$MONTH/$HOST/maillog"); }; destination r_ipflog { file ("/var/log/clients/$YEAR/$MONTH/$HOST/ipflog"); }; destination r_imaplog { file ("/var/log/clients/$YEAR/$MONTH/$HOST/imaplog"); }; destination r_console { file ("/var/log/clients/$YEAR/$MONTH/$HOST/consolelog"); }; destination r_syslog { file ("/var/log/clients/$YEAR/$MONTH/$HOST/syslog"); }; destination r_fallback { file ("/var/log/clients/$YEAR/$MONTH/$HOST/$FACILITY-$LEVEL"); }; destination mail-alert-perl { program("/usr/local/bin/syslog-mail-perl"); }; filter f_attack_alert { match("attackalert"); }; # find messages with "attackalert" in them, and send to the mail-alert script log { #source(src); filter(f_attack_alert); destination(mail-alert-perl); }; # Log statements # Local sources log { source (s_local); filter (f_authinfo); destination (l_authlog); }; log { source (s_local); filter (f_mail); destination (l_maillog); }; log { source (s_local); filter (f_local0); destination (l_ipflog); }; log { source (s_local); filter (f_local1); destination (l_imaplog); }; log { source (s_local); filter (f_syslog); destination (l_syslog); }; log { source (s_local); filter (f_emerg); filter (f_user_none); destination (l_console); }; log { source (s_local); filter (f_mesg); filter (f_user_none); destination (l_messages); }; # All sources, since we want to archive local and remote logs log { source (s_local); source (s_stunnel); filter (f_authinfo); destination (r_authlog); }; log { source (s_local); source (s_stunnel); filter (f_mail); destination (r_maillog); }; log { source (s_local); source (s_stunnel); filter (f_local0); destination (r_ipflog); }; log { source (s_local); source (s_stunnel); filter (f_local1); destination (r_imaplog); }; log { source (s_local); source (s_stunnel); filter (f_syslog); destination (r_syslog); }; log { source (s_local); source (s_stunnel); filter (f_emerg); filter (f_user_none); destination (l_console); }; log { source (s_local); source (s_stunnel); filter (f_mesg); filter (f_user_none); destination (l_messages); }; Thanks in advance. Wayne Yu __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

2 1

syslog-ng.conf
by wayne yu 08 May '06

08 May '06

Hi : I've been spending 3 weeks on configuring syslog-ng.conf, can some show me a syslog-ng.conf for reference? Thanks Wayne Yu __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

Defunct processes when doing log rotation
by Dev Man 08 May '06

08 May '06

Hi, I'm using syslog-ng 1.9.9 In syslog-ng.conf, I have specified a program to process the syslog-ng output. destination df_mail { file("/var/log/mail.log"); program("/home/readlog/readlog.php"); }; When cron performs it's logrotation daily, it runs a reload of the syslog-ng system. The readlog.php script ends up being <defunct>. For example: root@thehost:/etc/logrotate.d# ps -ef | grep read root 9 1 0 Mar27 ? 00:00:00 [kthread] root 1652 537 0 Apr20 ? 00:00:00 [readlog.php] <defunct> root 4000 537 0 Apr21 ? 00:00:00 [readlog.php] <defunct> root 7189 537 0 Apr22 ? 00:00:00 [readlog.php] <defunct> Thanks for any suggestions. D

3 3

"not netmask(...)" not working
by Brian Candler 08 May '06

08 May '06

Hello, I am using syslog-ng-1.6.11 under RHEL 4, using an RPM built from http://www.balabit.com/downloads/syslog-ng/1.6/src/syslog-ng-1.6.11.tar.gz [*] I tried to use the following rule to match all packets except those from certain host ranges: destination estate { file("/logs/estate.log" ); }; filter f_estate { not netmask( "10.1.0.0/16" ) and not netmask( "172.17.0.0/24" ) and not netmask( "192.168.1.254" ); }; log { source(s_sys); filter(f_estate); destination(estate); }; but it didn't work. tcpdump showed packets from outside those ranges were being received, but syslog-ng did not log them. Now, inspecting the code, firstly it seems I should have written the expressions in dotted netmask form, i.e. "10.1.0.0/255.255.0.0". It would be nice if syslog-ng were to validate this a bit better :-) However when I fixed that nothing changed. Looking in the code, I think that a negation operation is missing for the netmask() function, perhaps something like this: --- src/filters.c.orig Fri May 5 13:19:18 2006 +++ src/filters.c Fri May 5 13:24:44 2006 @@ -272,10 +272,10 @@ netw = self->network.s_addr; mask = self->netmask.s_addr; - return ((host & mask) == (netw & mask)); + return ((host & mask) == (netw & mask)) ^ c->comp; } else { - return 0; + return c->comp; } } However, I don't understand why some of the functions use c->comp, and others use self->super.comp, so I'm a bit hesitant to modify in this way. I was able to workaround by rewriting the expression thus: destination estate { file("/logs/estate.log" ); }; filter f_not_estate { netmask( "10.1.0.0/255.255.0.0" ) or netmask( "172.17.0.0/255.255.255.0" ) or netmask( "192.168.1.254" ); }; filter f_estate { not filter("f_not_estate"); }; log { source(s_sys); filter(f_estate); destination(estate); }; but perhaps someone who understands the expression parser could look at fixing this properly? Thanks, Brian. [*] I had to use syslog-ng.spec.bb, because syslog-ng.spec references a number of non-existent files, such as Source3: mysql-syslog.pipe Source4: mysql-syslog.buffer) With a small change to syslog-ng.bb, changing an underscore to dash, it was happy.

2 1

syslog-ng blocking on read() from /proc/kmsg
by John Morrissey 07 May '06

07 May '06

I'm running the Debian 1.6.9-1 package with the 'read from each file descriptor n times or until EAGAIN' patch backported. There seems to be a problem with this behavior when it's applied to /proc/kmsg. Here's the source we use: source s_kmsg { file("/proc/kmsg" log_prefix("kernel: ")); }; What happens is: 1. syslog-ng calls poll(2) and waits. 2. Data appears on /proc/kmsg and causes poll() to return. 3. syslog-ng read(2)s from /proc/kmsg, grabbing some data. 4. Since it hasn't received EAGAIN or read n times from this file descriptor, syslog-ng happily read()s again. Since it's exhausted the available data, the read() blocks. Forever. 5. Meanwhile, other syslog traffic to /dev/log piles up. Eventually, whatever buffer there is on Unix domain sockets fills up and causes other applications on the machine to block, waiting to syslog. 6. The box eventually runs out of file descriptors since other processes (cron is a big culprit because it runs jobs so often) are piling up, holding onto file descriptors and blocking on writes to /dev/log. The blocking behavior on /proc/kmsg is interesting, since if I'm reading the source correctly, file sources are set O_NONBLOCK: affile.c: 152 static int do_init_affile_source(struct log_handler *c, struct syslog_con 153 { 154 CAST(affile_source, self, c); 155 int fd, flags; 156 157 if (self->flags & AFFILE_PIPE) 158 flags = O_RDWR | O_NOCTTY | O_NONBLOCK | O_LARGEFILE; 159 else 160 flags = O_RDONLY | O_NOCTTY | O_NONBLOCK | O_LARGEFILE; Any thoughts? thanks, john -- John Morrissey _o /\ ---- __o jwm(a)horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__

2 2

syslog-ng 1.6.11 released
by Balazs Scheidler 06 May '06

06 May '06

Hi, I have released syslog-ng version 1.6.11, available at http://www.balabit.com/products/syslog-ng/ The user visible changes this time: News for the 1.6.11 release Wed, 03 May 2006 13:08:38 +0200 * Using time_sleep() caused a parser error as a rule was missed from the configuration parser. * Scalability improvements applied in 1.6.10 broke reading /proc/kmsg on Linux. -- Bazsi

1 0

Searching past posts?
by jarrod.schous＠sasktel.sk.ca 05 May '06

05 May '06

Is there a way to search the past posts so I don't need to waste everyone valuable time with questions that have likely been asked many times before? NOTICE: This confidential e-mail message is only for the intended recipient(s). If you are not the intended recipient, be advised that disclosing, copying, distributing, or any other use of this message, is strictly prohibited. In such case, please destroy this message and notify the sender.

2 1

Problem logging messages from console server
by Berthold Cogel 05 May '06

05 May '06

Hello! I've build our loghost with syslog-ng 1.6.9 on a RHEL 3 system with destinations which include the hostnames: destination d_mesg { file ("/var/local/logmaster/$HOST/messages"); }; ... Global options are: check_hostname (yes); keep_hostname (yes); use_dns (yes); dns_cache (yes); use_fqdn (yes); chain_hostnames (no); create_dirs (yes); This works as expected for all hosts, except one. Our console server (Digi CM 32) sends lines like LOCAL1.NOTICE: May 4 09:53:17 Data Buffering: ... This gets logged by syslog-ng as 'May 4 09:53:17 Data Buffering: ... ' for a host 'Data'. With the syslog, that comes with RHEL 3, I get log entrys like 'May 4 09:53:17 digi-cm32 Data Buffering: ... '. To me this doesn't look like a DNS problem. The log format of the console server looks similar to the format I get when I transfer the log entrys via a second syslog-ng: <facility>.<level>: <date> <host> DATA I've run out of ideas at the moment. Any help is welcome. Regards Berthold Cogel

2 2

Howto avoid or reduce nr of STATS dropped?
by Martin Meiler 04 May '06

04 May '06

Hello everybody, we face a problem with our log-server at our department since the last few weeks. Previously there was no problem concerning this log-server but now we get messages like syslog-ng[...] STATS: dropped 38142 or sometimes even more. How can we avoid these dropped messages? Or how can we influence the number of dropped messages? Or why are messages dropped? Since the syslog protocol is based on UDP we can not really avoid that some packets are dropped if many of them arrive at nearly the same time, or is there a solution. Thanks for an answer. Best regards Martin -- -------------------------------------------------------------------- _/ _/_/ _/_/_/ Friedrich Alexander Universität Erlangen _/ _/ _/ _/ Lehrstuhl für Sensorik _/ _/ _/ Paul-Gordanstr. 5, 91052 Erlangen, Germany _/ _/_/ _/_/ _/ _/ _/ University of Erlangen _/ _/ _/ _/ Department of Sensor Technology _/_/_/ _/_/ _/_/_/ Paul-Gordanstr. 5, 91052 Erlangen, Germany Dipl.-Ing. Martin Meiler tel.: (+49)-9131-85-23140 fax.: (+49)-9131-85-23133 email: Martin.Meiler(a)lse.eei.uni-erlangen.de

2 1

TTL patch, helpful for multicasting logs
by Alexander Clouter 04 May '06

04 May '06

Hi, After my earlier multicast patch I quickly realised that if syslog-ng is actually sending UDP multicast syslog data the TTL on the IP packets is by default 1. This results in the packet being expired at the gateway and never being able to leave the subnet....not too handy :( After clobbering it around for a few days I have rolled out this patch to address the problem which should be applied in addition, and after, the multicast patch I released the other day (also re-included in this email). The patch adds a new parameter to the destination tcp()/udp() drivers called ttl() and it takes a number between 0 and 255. When set to zero, it uses the default kernel parameter[1] but when set to 1->255 the TTL on the packet being sent is set to this. Of course if the parameter is not specified it defaults to zero. This should now allow you to use destination du_moo { udp("239.1.2.3" port(514) ttl(7)); }; in your config files. It works on unicast traffic too, I guess you could use it as a layer of defence to guarentee that your syslog packets never leave your network by setting the TTL to a low value. I always wanted to get around to emailing the samba folk a TTL patch to do this sort of thing, so roaming samba laptop users could get some feeling of security that only the local network can get access to their shares....however this is now getting a bit offtopic :) Hope you all find it useful, now back to my multicasting syslog cluster :) Cheers Alex [1] under linux its one, and probably the same for most OS's

2 5

Silent fail on program() without execute bit
by Brian A. Seklecki 03 May '06

03 May '06

When an external program is not executable by the user running the syslong-ng process, messages sent to destination{}; with system fail silently. Even ith "-dvv" flags, there's no indication of a problem. No mention of this behavior as a feature in the docs, either. l8* -lava

1 0

Weird issue
by Didier Conchaudron 03 May '06

03 May '06

Hi folks, I've a loghost receiving tons of logs from many servers and I'm experiencing problems regarding 2 destinations: a log forwarding to another server, and written logs for each servers. The logs forwarding work for most of the servers described into the filter written bellow, but some logs from some servers are not forwarded. First I think about a mistake into my regexp. For example, for the server ptcassidc01.ad5.ad.foo.com , I tried host("ptcassidc01.ad5.ad.foo.com"), host("ptcassidc01*"), and host("ptcassidc01") but none works. For the written log files I've the same problem, the logs which are not forwarded are not written too. So I'm thinking of a filter issue. Maybe it's too big and it can't be parsed rightly. It's not a network issue, the network load is about 250 Kb/sec. Regards, Didier Conchaudron Here're the destinations: destination dest_soc_pilot_project { udp("an ip address" port(514) ); }; destination dest_soc_exaprotect_agent { file("/data/logs/exaprotect/$HOST" owner(syslog) group(syslog) perm(0644)); }; Here's the filter I use: filter f_soc_pilot_project { level(info...emerg) and not facility(mail) and ( host("bcv51se9") or host("bcv50s56") or host("bcv50s9b") [ many many others, maybe 100 ] or host("plwarsidc01") or host("ptcassidc01") ); }; And finaly the log statements: log { source(net); filter(f_soc_pilot_project); destination(dest_soc_pilot_project); }; log { source(local); filter(f_soc_pilot_project); destination(dest_soc_pilot_project); }; log { source(net); filter(f_soc_pilot_project); destination(dest_soc_exaprotect_agent); }; log { source(local); filter(f_soc_pilot_project); destination(dest_soc_exaprotect_agent); }; Here're the main options: options { owner("root"); group("root"); dir_perm(0750); perm(0640); log_fifo_size(10000); chain_hostnames(no); keep_hostname(yes); check_hostname(yes); use_dns(yes); dns_cache(yes); dns_cache_size(1000); dns_cache_expire(36000); dns_cache_expire_failed(900); use_fqdn(yes); create_dirs(yes); use_time_recvd(no); stats(6000); sync(50); }; ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

1 0

Remote SYSLOG-NG logging - can't log from remote
by rlubbers＠sysctl.net 03 May '06

03 May '06

I am at my wits end with this. Some devices are logging remotely, but others are not. I can see the packets arrive on UDP port 514 using tcpdump, but the packets don't get recorded in the proper file. I have tried nearly everything, and I am certain I am doing something silly, but maybe you can help out. Here is my syslog-ng.conf.file: ************************************************************* source local { unix-dgram("/var/run/log"); udp(ip(0.0.0.0) port(514)); internal(); }; ### SECURITY LOG - This logs filter f_9 { facility(security) and level(debug..emerg); }; destination d_3 { file("/var/log/security" create_dirs(yes)); }; log { source(local); filter(f_9); destination(d_3); }; ### MAIL LOG - This logs filter f_12 { facility(mail) and level(info..emerg); }; destination d_5 { file("/var/log/maillog" create_dirs(yes)); }; log { source(local); filter(f_12); destination(d_5); }; ### PHONE DACS LOGS - This also logs filter f_40 { level(debug..emerg) and host("172.12.67.28"); }; destination d_60 { file("/var/log/dacs/ftldgaaw_dac_1.log" create_dirs(yes)); }; log{ source(local); filter(f_40); destination(d_60); }; ### ROUTER LOG - This does NOT log filter f_19 { host("192.168.1.128"); }; destination d_19 { file("/var/log/netrouter_pisst.log" create_dirs(yes)); }; log{ source(local); filter(f_19); destination(d_19); };

2 1

how to start the syslog-ng demeaon
by wayne yu 02 May '06

02 May '06

Hi: I configured my syslog-ng on the solaris 9 computer.' when I did /etc/init.d/syslog start, and then ps -ef |grep syslog-ng, nothing is list. How can i find out how the syslog-ng works. Thanks Wayne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

2 1

syslog-ng hanging?
by Tony Davis 02 May '06

02 May '06

Hi, I recently converted our RedHat EL servers (Release 3 Update 6) form the sysklog package to syslog-ng. On some of the servers syslog-ng appears to be occasionally hanging, i.e. messages stop getting written to the various log files. This also has a knock on effect of stopping logins, cron jobs hanging, etc, presumably because they are waiting to write to a log file. I read in the archives that this can be caused by something else using /dev/log but there's no other syslogd or minilogd processes running and lsof gives the following: COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME syslog-ng 1384 root 5u unix 0xf778c880 1703 /dev/log syslog-ng 1384 root 6u unix 0xf777e8c0 1715 /dev/log I'm using the RedHat conf file bundled in the contribs directory that comes with the source code. Any ideas would be welcome. Thanks Tony ***************************************************** You can find us at www.voca.com ***************************************************** This communication is confidential and intended for the exclusive use of the addressee only. You should not disclose its contents to any other person. If you are not the intended recipient please notify the sender named above immediately. Registered in England, No 1023742, Registered Office: Voca Limited Drake House, Three Rivers Court, Homestead Road, Rickmansworth, Hertfordshire, WD3 1FX This message has been checked for all email viruses by MessageLabs.

3 2

syslog-ng 1.6.10 release broken
by Balazs Scheidler 01 May '06

01 May '06

Hi, It came to my attention that syslog-ng 1.6.10 broke file("/proc/kmsg") support with the recent performance improvement patches as /proc/kmsg does not support nonblocking mode. The issue might cause the complete system to deadlock. Non-Linux platforms, or installation where /proc/kmsg is not directly processed by syslog-ng is not affected. I'm going to release 1.6.11 as soon as possible to fix this issue. -- Bazsi

1 0