November 2004 - syslog-ng - lists.balabit.hu

unsubscribe
by Ian Hunt 23 Nov '04

23 Nov '04

Ian Hunt email: ian.hunt(a)swapswire.com Tel: +44 (0) 207 868 2452 --------------------------------------------------------------------- This is a commercial communication sent by SwapsWire Limited. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. Please contact info(a)swapswire.com if you no longer wish to receive commercial communications from us, identifying the email addresses to which you no longer wish commercial emails to be sent.

1 0

feature request to Stats ouput
by Dave Johnson 23 Nov '04

23 Nov '04

Stats: dropped is great, would this be better?: Stats: x processed y buffered/queued z dropped * I'm seeing syslog-ng process sizes in 20-30MB and am wondering if my queue sizes should be upped.

1 0

Re: [syslog-ng]syslog-ng truncating pipe template output
by James Masson 22 Nov '04

22 Nov '04

Hi Balazs As requested.... 0x0000: 4500 0164 68db 0000 3f11 7baa ac16 0078 E..dh...?.{....x 0x0010: c0a8 28cd 0bfa 0202 0150 9f48 3c31 3635 ..(......P.H<165 0x0020: 3e64 652d 6777 2d31 3a20 4e65 7453 6372 >de-gw-1:.NetScr 0x0030: 6565 6e20 6465 7669 6365 5f69 643d 6465 een.device_id=de 0x0040: 2d67 772d 3120 205b 4e6f 204e 616d 655d -gw-1..[No.Name] 0x0050: 7379 7374 656d 2d6e 6f74 6966 6963 6174 system-notificat 0x0060: 696f 6e2d 3030 3235 3728 7472 6166 6669 ion-00257(traffi 0x0070: 6329 3a20 7374 6172 745f 7469 6d65 3d22 c):.start_time=" 0x0080: 3230 3034 2d31 312d 3232 2031 373a 3035 2004-11-22.17:05 0x0090: 3a30 3622 2064 7572 6174 696f 6e3d 3220 :06".duration=2. 0x00a0: 706f 6c69 6379 5f69 643d 3120 7365 7276 policy_id=1.serv 0x00b0: 6963 653d 6874 7470 2070 726f 746f 3d36 ice=http.proto=6 0x00c0: 2073 7263 207a 6f6e 653d 5472 7573 7420 .src.zone=Trust. 0x00d0: 6473 7420 7a6f 6e65 3d55 6e74 7275 7374 dst.zone=Untrust 0x00e0: 2061 6374 696f 6e3d 5065 726d 6974 2073 .action=Permit.s 0x00f0: 656e 743d 3636 3120 7263 7664 3d39 3434 ent=661.rcvd=944 0x0100: 2073 7263 3d31 3732 2e32 322e 302e 3233 .src=XXX.XX.X.XX 0x0110: 2064 7374 3d36 362e 3136 332e 3137 352e .dst=XX.XXX.XXX. 0x0120: 3132 3820 7372 635f 706f 7274 3d33 3930 XXX.src_port=390 0x0130: 3020 6473 745f 706f 7274 3d38 3020 7372 0.dst_port=80.sr 0x0140: 632d 786c 6174 6564 2069 703d 3231 372e c-xlated.ip=XXX. 0x0150: 3135 312e 3234 302e 3320 706f 7274 3d32 XXX.XXX.X.port=2 0x0160: 3939 3600 996. I read about that Netscreen problem too, but I have a colleague who has exactly the same syslog-ng.conf entries and Netscreen OS version, and his works! I think Netscreen fixed that issue a while back, my issue occurs with at least 3 different Netscreen OS version. Thanks James Masson

1 0

question re: "STATS: dropped" errors
by Jef Elliott 22 Nov '04

22 Nov '04

Hi, We have a system here running 1.6.4 (on debian linux, the stable disto), and it keeps griping about STATS: dropped > 0 (usually anywhere from 3 to 9). I've read through the archives, and see messages mentioning possible reasons for this, as well as some of the potential cures. I disabled the pipe for the xconsole, and that seems to have stopped the complaining. My question though relates to why this one system is the only complaining about this. I don't think that we have a syslog-ng server problem, as we have close to 80 systems logging to it. This is the only system that gives me this error. I don't think there are any local issues or services causing a problem, as this is a freshly imaged system, not running anything in particular, and not in production. What else could possibly be causing an issue on this one host? All other systems using syslog-ng have the exact same config file as this host and I'm at a loss as to what else could be causing the trouble. Any insights/links/help is appreciated. Thanks jef

1 0

Re: [syslog-ng]syslog-ng truncating pipe template output
by James Masson 22 Nov '04

22 Nov '04

Hi Nate I've tried a variety of formatting in destination_firewalls with no effect on this problem. I've inserted extra spaces and newlines in various places. The resulting output is still truncated and the extra formatting works only before this point. The current format I'm using in desination_firewalls is all on one line, and the output is all on one line. I've tried "template-escape(no)" - still the same problem ########################################### Syslog-ng.conf ########################################### # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo, v 1.4 2004/07/18 02:25:02 dragonheart Exp $ # # Syslog-ng default configuration file for Gentoo Linux # contributed by Michael Sterrett options { long_hostnames(off); sync(0); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); log_msg_size(8192); log_fifo_size(8192); }; source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; # By default messages are logged to tty12... destination console_all { file("/dev/tty12"); }; # ...if you intend to use /dev/console for programs like xconsole # you can comment out the destination line above that references /dev/tty12 # and uncomment the line below. #destination console_all { file("/dev/console"); }; log { source(src); destination(messages); }; log { source(src); destination(console_all); }; ## Log to mysql ## OUtput for unix_servers mysql database destination unix_servers { pipe("/tmp/mysql_unix_servers.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; ##### Output for firewalls mysql database destination firewalls { pipe("/tmp/mysql_firewalls.pipe" template(" INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG');\n") template-escape(yes)); }; ##### Output for appliances mysql database destination appliances { pipe("/tmp/mysql_appliances.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; ### Output for routers mysql database destination routers { pipe("/tmp/mysql_routers.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; ### filter udp stream to desitnations based on facility log { source(net); filter(f_routers); destination(routers); }; log { source(net); filter(f_firewalls); destination(firewalls); }; log { source(net); filter(f_appliances); destination(appliances); }; log { source(net); filter(f_unix_servers); destination(unix_servers); }; #### Define incoming udp logging stream source net { udp(); }; ### Define facilities filters filter f_routers { facility(local3); }; filter f_firewalls { facility(local4); }; filter f_unix_servers { facility(local5); }; filter f_appliances { facility(local6); }; ################################################# Thanks James Masson

1 0

syslog-ng truncating pipe template output
by James Masson 22 Nov '04

22 Nov '04

I've been trying to figure out an obscure problem with syslog-ng importing to a mysql database. I have various types of network devices feeding syslog-ng on local3 through local6. I can import from Cisco, UNIX servers, Windows - but not Netscreen firewalls! Each device type gets it's own mysql database. The mysql INSERT INTO statements for the Netscreen logs are truncated and hence fail to import because the mysql syntax is not correct. I chased wild geese for a while thinking the log format of the Netscreen was messing with mysql - but that's not the case. Notice it's just truncating the last few characters of each statement - including the all important ")" and "\n" newline that closes the mysql statement. I dumped an instance or two of these to a file instead of the normal fifo, added a ")" and a newline at the end of each, and it imported just fine! INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( 'X.X.X.X', 'local4', 'notice', 'notice', 'a5', '2004-11-19', '16:48:04', 'de-gw-1', 'de-gw-1: NetScreen device_id=de-gw-1 [No Name]system-notification-00257(traffic): start_time=\"2004-11-19 18:02:12\" duration=3 policy_id=2 service=NETBIOS (SSN) proto=6 src zone=Trust dst zone=Untrust action=Permit sent=1813 rcvd=5846 src=X.X.X.X dst=X.X.X.X src_port=3525 dst_port=139 src-xlated ip=X.X.X.X port=3525 INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( 'X.X.X.X', 'local4', 'notice', 'notice', 'a5', '2004-11-19', '16:48:06', 'de-gw-1', 'de-gw-1: NetScreen device_id=de-gw-1 [No Name]system-notification-00257(traffic): start_time=\"2004-11-19 18:02:14\" duration=3 policy_id=3 service=icmp proto=1 src zone=Untrust dst zone=Trust action=Permit sent=78 rcvd=78 src=X.X.X.X dst=X.X.X.X icmp type=8 INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( 'X.X.X.X', 'local4', 'notice', 'notice', 'a5', '2004-11-19', '16:48:06', 'de-gw-1', 'de-gw-1: NetScreen device_id=de-gw-1 [No Name]system-notification-00257(traffic): start_time=\"2004-11-19 18:01:15\" duration=62 policy_id=3 service=udp/port:3077 proto=17 src zone=Untrust dst zone=Trust action=Permit sent=0 rcvd=0 src=X.X.X.X dst=X.X.X.X src_port=17359 dst_port=3077 src-xlated ip=X.X.X.X port=17359 Remember that this is happening just for the Netscreen devices, using identical syslog-ng configs as for the other device types. Here's an example config: source net { udp(); }; filter f_firewalls { facility(local4); }; log { source(net); filter(f_firewalls); destination(firewalls); }; destination firewalls { pipe("/tmp/mysql_firewalls.pipe" template(" INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG ');\n") template-escape(yes)); }; This is on Gentoo Linux using Syslog-ng 1.6.4 stats(43200); log_msg_size(8192); log_fifo_size(8192); As per http://www.campin.net/syslog-ng/faq.html#message_length - is this the same issue? I could do some obscure perl filtering to fix this on the fly, but I'd rather fix the problem rather than the symptom. Thanks for your time. James Masson

4 3

Issues logging into MySQL
by Ronnie Clark 20 Nov '04

20 Nov '04

Hello all, I have a system loaded with FreeBSD 4.10 STABLE and syslog-ng, logging into MySQL. I have used the templates for syslog-ng and syslog-ng2. The issue is that now the time and machine name fields are correct, but the first few characters of the actual message are being chopped off. Examples: yslog-ng version 1.6.5 going down eboot by rclark: AD SU rclark to root on /dev/ttyv0 TATS: dropped 0 Is there a fix for this? I also have tried to compile and load the same sqlsyslogd for syslog-ng on FreeBSD 5.3, but get the following errors: sqlsyslogd.c: In function `usage': sqlsyslogd.c:38: error: missing terminating " character sqlsyslogd.c:39: error: stray '\' in program sqlsyslogd.c:39: error: syntax error before "n" sqlsyslogd.c:39: error: stray '\' in program sqlsyslogd.c:39: error: missing terminating " character sqlsyslogd.c: In function `main': sqlsyslogd.c:162: error: missing terminating " character sqlsyslogd.c:163: error: syntax error before '%' token sqlsyslogd.c:163: error: stray '\' in program sqlsyslogd.c:163: error: missing terminating " character sqlsyslogd.c:191: error: missing terminating " character sqlsyslogd.c:192: error: syntax error before '%' token sqlsyslogd.c:192:42: warning: character constant too long for its type sqlsyslogd.c:192:54: warning: multi-character character constant sqlsyslogd.c:192: error: missing terminating " character *** Error code 1 Anyone else have this running on FreeBSD 5.3 Release or STABLE? Thanks in advance, Ron Clark __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

2 1

Michael J. Bock/NetServ/DST/US is out of the office.
by MJBock＠dstsystems.com 20 Nov '04

20 Nov '04

I will be out of the office starting 11/18/2004 and will not return until 11/22/2004. I will respond to your message when I return. ----------------------------------------- This e-mail and any attachments are intended only for the individual or company to which it is addressed and may contain information which is privileged, confidential and prohibited from disclosure or unauthorized use under applicable law. If you are not the intended recipient of this e-mail, you are hereby notified that any use, dissemination, or copying of this e-mail or the information contained in this e-mail is strictly prohibited by the sender. If you have received this transmission in error, please return the material received to the sender and delete all copies from your system.

1 0

Re: [syslog-ng]installing on solaris 9
by searsrf＠comcast.net 19 Nov '04

19 Nov '04

Thanks, Reading the INSTALL doc that states; "Syslog-ng uses a different configuration scheme than the original syslogd, which sits at /etc/syslog-ng/syslog-ng.conf" I though that the installation of syslog-ng would create /etc/syslog-ng/syslog-ng.cong and then I would just edit the file. Thanks for the help, I will create the directory and file. > no default configuration is installed, you need to create one based on > the documentation and the examples under contrib. > > On Fri, 2004-11-19 at 15:07, searsrf(a)comcast.net wrote: > > Greetings, > > > > I have installed libol-0.3.14 and syslog-ng-1.6.5 on my Solaris 9 host with a > bit of trouble with libol-0.3.14 going the smoothest. After installing > syslog-ng-1.6.5 I did notice that /usr/local/sbin/syslog-ng was created but the > /etc/syslog-ng/syslog-ng.conf was not. Are the config files created at install > or do they need to be manually copied from within the install directory? > > > > Thanks for any help > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > -- > Bazsi > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html >

1 0

installing on solaris 9
by searsrf＠comcast.net 19 Nov '04

19 Nov '04

Greetings, I have installed libol-0.3.14 and syslog-ng-1.6.5 on my Solaris 9 host with a bit of trouble with libol-0.3.14 going the smoothest. After installing syslog-ng-1.6.5 I did notice that /usr/local/sbin/syslog-ng was created but the /etc/syslog-ng/syslog-ng.conf was not. Are the config files created at install or do they need to be manually copied from within the install directory? Thanks for any help

2 1