March 2001 - syslog-ng - lists.balabit.hu

syslog-ng 1.5.5 and libol 0.2.22 released
by Balazs Scheidler 30 Mar '01

30 Mar '01

Hi, I've released syslog-ng 1.5.5 and the corresponding libol 0.2.22 with the following changes: News for the 1.5.5 release Fri, 30 Mar 2001 14:23:55 +0200 * Fixed a bug in -HUP handling (implemented a cleaner solution, though less tested than in 1.4.11). * Added DNS cache implemented by Gaël Roualland. * Added klogctl program to control kernel logging level on Linux (to make it easier to set up a completely klogd-less logging) * Template support for output files. * Documentation somewhat updated. some of these functions were implemented in 1.5.3 but that version didn't have a NEWS entry, that's why I added it here. Somebody requested that syslog-ng should make it possible to escape templated output files, it's now possible with the template-escape() option. I've updated the docs, though my changes touch only the surface. Any help for updating the documentation is welcome. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

Syslog-ng newer versions
by Simeon Johnston 30 Mar '01

30 Mar '01

Is there any difference in the way the config file is setup between 1.4.11 and newer 1.5.x versions? I realize that there are new functions and capabilities but will I just be able to plug in my old config file and it will work exactly the same? sim

2 1

Libol version and 1.5.3
by Simeon Johnston 29 Mar '01

29 Mar '01

I am trying to compile syslog-ng 1.5.3 but on your site it is grouped with libol 0.2.20. Syslog-ng 1.4.11 uses libol 0.2.21. Is there a difference? Do I have to use 0.2.20 instead of 0.2.21 or do I just need to redirect Syslog-ng to the different version? I tried to just make it but it didn't find libol 0.2.21 which is why I was wondering. sim

2 2

Syslog Logging Delay
by Brian E. Seppanen 29 Mar '01

29 Mar '01

I'm running syslog-ng 1.4.11 on a redhat-7.0 box. The box is a dual processer PIII 500 dell poweredge 1300, with 512MB of RAM. We're making this baby hum, but may have too much on it. My config is big so i'll include what I think are the relevent portions. Let me know if I'm missing something. Basically syslog-ng has stopped logging at various times. I've been doing a lot of poking today, to stress test it. I'm doing a tcpdump on port 514 and I'm seeing lots of traffic. However, the logs where the information should be going isn't getting updated. My last log entry is from approximatet 2:53 p.m. EST, and it's now 3:25 p.m. EST. If I stop syslog-ng and restart it will probably start logging just fine, but I can't be starting and stopping every three hours. I'm sure it's probably a misconfiguration. netstat -an has remained the same for this span of time as follows Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State udp 65488 0 0.0.0.0:514 0.0.0.0:* Notice the lot in the recv queue. Okay now here's my config. You promise not to laugh right? :) options { sync(0); time_reopen(10); log_fifo_size(2000); chain_hostnames(no); use_dns(no); gc_idle_threshold(500); gc_busy_threshold(3500); }; Now I'm thinking that there is a setting that I haven't left high enough or low enough. I'm not sure if my logs are blocked because I'm trying to force too much into them or what. Basically if I start syslog-ng I usually have about 500 objects alive, and idle_garbage_collection will run almost constantly. After a long time it isn't unheard of that I have about 2000-3000 objects alive. Is it trying to garbage collect itself to death. I never really got a good indication of how I should set that up. This morning I tried increasing the log_fifo_size to 2000. Is there a setting I should check? What would cause it to block anything from getting written? Actually it has written the information to the logs since I started this message at about 3:27, and it doesn't appear to have dropped anything. Would reducing the fifo_size speed up the commit time? What would delay the commit? I'd appreciate any help. Thanks :) Brian Seppanen Charter Communications Regional Data Center 906-228-4226 ext 23 Marquette, MI seppy(a)chartermi.net

2 2

syslog-ng long_hostsname
by Dennis Kruyt 29 Mar '01

29 Mar '01

Hi, I'm running syslog-ng server with a SSL tunnel to the syslog-ng client's I'm sorting the log files to the following dir destination extern { file( "/var/logs/$HOST/$DAY/$FACILITY.log"); }; That function works fine but in the log file there is the following line: Mar 28 03:01:26 src(a)XXX/ZZZZ.bos.nl proftpd[3808]: XXXX.bos.nl (ZZZZ.bos.nl[195.XX.XX.XX]) - FTP session closed. But i want the same line as with syslog Mar 28 03:01:26 XXX proftpd[3808]: XXXX.bos.nl (ZZZZ.bos.nl[195.XX.XX.XX]) - FTP session closed. So that the log analyzer can do there work, i know that i can put out that function with long_hostnames(off) but then is the sorting function for the directory's also gone is there a solution for this. Dennis Kruyt, ZXFactory BV Telefoonweg 44b 6712 GD EDE The Netherlands T +31 138 693 111 F +31 139 693 042 I www.zxfactory.nl

2 1

1.5.4 or 1.5.3???
by Chad C. Walstrom 29 Mar '01

29 Mar '01

Just wondering about this little discrepency in version numbers. Currently, the Debian package that's floating about on sid is 1.5.4, yet all I can find on your ftp server at ftp.balabit.hu is 1.5.3. Is there some secret CVS repository that the Debian Maintainer has access to? Did 1.5.4 exist at some point and then disappear because of some developer decision? Inquiring minds want to know... ;-) -- Chad Walstrom <chewie(a)wookimus.net> | a.k.a. ^chewie http://www.wookimus.net/ | s.k.a. gunnarr Key fingerprint = B4AB D627 9CBD 687E 7A31 1950 0CC7 0B18 206C 5AFD

2 1

DNS fun
by Glen Williams 29 Mar '01

29 Mar '01

I'm playing with a syslog server that will have quite a number clients. I'm running syslog-ng-1.4.11 on Solaris 8. I've been testing with both use_dns on and off with little success. My destination is destination messages { file("/var/log/$HOST/$FACILITY.$YEAR$MONTH$DAY.$HOUR" create_dirs(yes) ); }; With it on, I get messages in both /var/log/<hostname>/local0..... and /var/log/<IP address>/local0..... The number of messages I get in the IP address directory is small, less than 1%. I just assume that syslog-ng gives up on the dns query and writes $HOST as the IP rather than the hostname. This occurs less when a dns cache is installed, but it's still not great having two log files. It doesn't really seem to be blocking on the lookup failures, as I'm not noticing a timeout happening. To overcome this, I turned use_dns off. I have clients using nsyslog-ng using tcp as the transfer protocol. Now these nsyslog-ng clients are logging to /var/log/<hostname>/local0 and NEVER in /var/log/<IP address>/local0 I want both nsyslog-ng and standard syslog clients to log to the same directory format. I'll settle for the $HOST variable being the IP. What am I missing for nsyslog-ng clients using tcp to do this? Thanks, Glen

1 0

UDP from syslog to syslog-ng
by Brian E. Seppanen 28 Mar '01

28 Mar '01

I'm testing out logging from another host to a central syslog-ng host. The host I'm logging to is a redhat-7.0 linux box with 1.4.11. The host I'm logging from is a Solaris 7 box running sun's standard syslog. I've defined loghost in /etc/hosts, and I've seen a couple of messages arrive. However it's been forever since any messages have been logged from this host. If I do a netstat -an I see the following Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State udp 65520 0 0.0.0.0:514 0.0.0.0:* I have the following source definition source udp_src {udp();} And I'm simply trying to get it to default to a /var/log/messages log { source(udp_src); filter(DEFAULT); destination(messages);}; I'm reading the netstat as indicating the data is being sent, and is being received. However, it's not being written to the files. Other data has been written in the meantime to the log files. We currently are logging several cisco routers with the source: udp_src { udp()}; In fact this was logging messages ealier as well. I've been trying to tweak it a little and now it doesn't seem to be working. Anyone seen this? This is very frustrating. Anyone have any ideas on what I may have done wrong? Brian Seppanen Charter Communications Regional Data Center 906-228-4226 ext 23 Marquette, MI seppy(a)chartermi.net

2 2

RE: 1.5 stable? was(Re: [syslog-ng]Feature Request)
by Andrew Fort 28 Mar '01

28 Mar '01

> How stable is 1.5? This looks like a very cool function that > I would like to be able to use on our logserver. Is it > stable enough for a production environment? I use it in a production environment with 60 or 70 cisco switches and routers spitting syslog at my syslog-ng. I haven't had any dramas (my platform is Linux 2.2.19). memory usage is normal, no leaks evident. -afort

2 1

Feature Request
by Chad C. Walstrom 28 Mar '01

28 Mar '01

Firstly, THANKS for such a great replacement to syslog! I'm quite happy with the ease of configurability and the use of the tcp() source and destinations -- very nice for tunnelling over OpenSSL. Next thing to ask for is a feature request that would make our use of syslog infintely more flexible: a string formatter module. Take this hypothetical syslog-ng.conf entry as an example: --------- START EXAMPLE ----------- # First, set some global options. options { long_hostnames(off); sync(0); }; source remotehost { tcp( 192.168.1.1, 514 ); }; # sql batch file destination sql_output { file ("/var/log/log.sql" owner ("root") group("adm") perm(0640)); }; destination xml_output { file ("/var/log/log.xml" owner ("root") group("adm") perm(0640)); }; # Now, the format string using printf style placeholders where %h == # host, %d == date, %t == time, %f == facility, %l == level, and %m == # message format o_sql { printf( "INSERT mytable ( host, date, time, facility, level, message )\nVALUES ( '%h', '%d', '%t', '%f', '%l', '%m' );\n" ); }; # Or, we could specify template files for the printf driver # # Where the file would contain character literals rather than escapes # (for things such as tab and newline): # # INSERT mytable ( host, date, time, facility, level, message ) # VALUES ( '%h', '%d', '%t', '%f', '%l', '%m' );\n" ); # format o_sql { printf(template("/usr/share/syslog-ng/sql.tmpl")); }; format o_xml { printf(template("/usr/share/syslog-ng/xml.tmpl")); }; # Now, out to the log log { source(remotehost); format(o_sql); destination(sql_output); }; log { source(remotehost); format(o_xml); destination(xml_output); }; --------- START EXAMPLE ----------- The reason I suggest having format as a separate module instead of a file option is for instance you want to have a real-time process to log INSERT statements into a database. You could specificy your destination as a named pipe source s_host{ tcp( 192.168.1.1, 514 ); }; format o_sql{ printf(template("/etc/syslog-ng/custom_sql.tmpl")); }; destination d_sqlpipe{ file("/dev/log_sql" owner ("root") group("adm") perm(0640)); }; log{ source(s_host); format(o_sql); destination(d_sqlpipe); }; to which your database client is listening, waiting for input. Let's hypothise for a moment that psql, the PostgreSQL command line client, can open up a named pipe as an input file -- as I'm not sure if it really can. A command like the following is all you'd need for real-time database logging: bash$ whoami syslogd bash$ nohup psql -h localhost -d sysylogdb /dev/log_sql& bash$ We preserve the UNIX philosophy in that you let your tools do only one job, but one job done VERY well. Why reimplement the wheel and embed an SQL client into syslog-ng? No reason at all. What do we have to do to change the output format? Change the output template in the log rule. Simple, clean, efficient. What do you think? -- Chad Walstrom <chewie(a)wookimus.net> | a.k.a. ^chewie http://www.wookimus.net/ | s.k.a. gunnarr Key fingerprint = B4AB D627 9CBD 687E 7A31 1950 0CC7 0B18 206C 5AFD

2 2