April 2000 - syslog-ng - lists.balabit.hu

Re: Syslog - log_fifo_size?
by Balazs Scheidler 20 Apr '00

20 Apr '00

On Wed, Apr 19, 2000 at 11:20:54PM -0400, Scott Galambos wrote: > I can't get log_fifo_size to work. It appears to done nothing at all. > > For example: > destination file_misc { file("/var/log/misc.log" log_fifo_size(4)); }; > > The number of lines in misc.log goes far beyond 4. Same thing if in the > options line. How can I limit the size of the log files? Is that what > log_fifo_size is suppose to do? no. log_fifo_size controls the number of log entries in the output queue, and for files it may not be used at all, since files are always writable. output queues are used for example on TCP connections, when the destination host or network can't accept messages fast enough. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0

Full date?
by bryan＠tummy.com 19 Apr '00

19 Apr '00

Is there any way I can have syslog-ng include the year when it writes the current date/time of a log entry? For example currently a log entry looks like this: Apr 19 13:49:00 src@testbox PAM_pwdb[6498]: (su) session closed for user root Is there any way to have it print this instead: Apr 19 2000 13:49:00 src@testbox PAM_pwdb[6498]: (su) session closed for user root Thanks, Bryan -- http://www.tummy.com/ Consulting and Software for Linux and Unix KRUD - Kevin's Red Hat Uber Distribution - the Freshest Red Hat every month

1 0

non-root and chroot
by Aleph One 19 Apr '00

19 Apr '00

Is there any work in allowing syslog-ng to: a) change its uid and gids to non-root after is has started and opened the appropriate inputs. b) chroot itself after it has started an opened the appropriate inputs. (a) is simple to implement. (b) requires a little more thought. e.g. it should be smart enough to know that if a destination file is under /var/log/logfiles/$DAY if it chroots to /var/log it should try to open /logfiles/$DAY and not /var/log/logfiles/$DAY. -- Aleph One / aleph1(a)underground.org http://underground.org/ KeyID 1024/948FD6B5 Fingerprint EE C9 E8 AA CB AF 09 61 8C 39 EA 47 A8 6A B8 01

2 1

syslog-ng 1.4.3
by Balazs Scheidler 19 Apr '00

19 Apr '00

Hi, I've released the latest stable release of syslog-ng, versioned 1.4.3. It contains a few bugfixes, for more information check out the changelog. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0

mountd log entries...
by Jan Niemann 19 Apr '00

19 Apr '00

Hi, every log-file entry from mountd is followed by an empty log-message (only time and source are displayed) is this a bug in syslog-ng or mountd ? regards, Jan Niemann

2 1

Missing log entry on HUP
by bryan＠tummy.com 14 Apr '00

14 Apr '00

Ok, I've been continuing my testing of syslog-ng for reliability, and I've run into this problem/bug. When I HUP syslog-ng on the server and then a client tries to send a message, the connection is re-established to the server but the server loses the message. Example: server: [root@testbox syslog-ng]# killall -HUP syslog-ng client: [root@reptile syslog-ng]# logger test2; logger test3 log file: Apr 14 14:08:14 src@testbox syslog-ng[30283]: AF_INET client connected from 10.1.1.16, port 3724 Apr 14 14:13:51 src@reptile/reptile syslog-ng[8069]: Connection broken, reopening in 1 seconds Apr 14 14:13:51 src@reptile/reptile bryan: test3 Where did test2 go??? I have time_reopen(1); just for during testing. I'm planning on changing that to 15 or so later. Thanks, Bryan -- http://www.tummy.com/ Consulting and Software for Linux and Unix KRUD - Kevin's Red Hat Uber Distribution - the Freshest Red Hat every month

1 0

syslog-ng 1.4.2 and kernel messages on SunOS 5.6
by Massimo Loschi 14 Apr '00

14 Apr '00

Hi, I used syslog-ng 1.4.0 (very useful!) on a SunOS 5.6 sun4u host, with the following configuration: options { create_dirs(yes); sync(0); chain_hostnames(off); }; source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; source net { udp(ip(A.B.C.D) port(514)); }; [...] destination dstandard { file("/var/log/host/$HOST/$PROGRAM/$YEAR-$MONTH/$DAY.log" group("mygroup") dir_perm(0755) perm(0644) ); }; [...] log { source(local); source(net); destination(dstandard); }; When I start syslog-ng, in /var/log/host/machine appears several strange directories: drwxr-xr-x 3 other 512 Mar 31 14:48 PCI-device drwxr-xr-x 3 other 512 Mar 31 14:48 SUNW,hme0 drwxr-xr-x 3 other 512 Mar 31 14:48 SUNW,hme1 drwxr-xr-x 3 other 512 Mar 31 14:48 SunOS Release 5.6 Version Generic_105181-17 drwxr-xr-x 3 other 512 Mar 31 14:48 WARNING drwxr-xr-x 3 other 512 Mar 31 14:48 cpu0 drwxr-xr-x 3 other 512 Mar 31 14:48 glm0 drwxr-xr-x 3 other 512 Mar 31 14:48 pci0 at root drwxr-xr-x 3 other 512 Mar 31 14:48 root on drwxr-xr-x 3 other 512 Mar 31 14:48 sd0 at glm0 drwxr-xr-x 3 other 512 Mar 31 14:48 sd1 at glm0 drwxr-xr-x 3 other 512 Mar 31 14:50 sshd drwxr-xr-x 3 other 512 Mar 31 14:48 su0 at ebus0 drwxr-xr-x 3 other 512 Mar 31 14:48 su1 at ebus0 drwxr-xr-x 3 other 512 Mar 31 14:48 syslog-ng It appears the problem that the version 1.4.2 should be fix (function parse_log_msg in file src/log.c), but instaling and running this version, it works like the 1.4.0. Another strange thing happens (with both 1.4.0 and 1.4.2): the directories and the files create by syslog-ng don't have the permissions specified in the configuration ( dir_perm(0755) perm(0644) ), but those coherent with the umask of the root user. It is a my stupid error in syslog-ng.conf file or someother people have this problems? Thanks in advance Massimo ------------------------------------------------------------------------------ Massimo Loschi E-mail: m.loschi(a)nextra.it Network & Core Systems NEXTRA S.p.A. WWW: http://www.nextra.it Via Isonzo, 55/2 Tel: +39 51 6139211 40033 Casalecchio di Reno Fax: +39 51 6114455 Bologna - ITALY ------------------------------------------------------------------------------

1 0

Re: [syslog-ng] Collecting Logs on Loghost
by peter.klausner＠systor.com 14 Apr '00

14 Apr '00

Hi Wolfi + Gero Your projects look interesting, because there seems to be not much publicly available information on that area. ... :o how much net load does syslog create? : ... :o how much data does a host generate? : The overhead of the syslog protocol is rather small. The problem is the volume of logging done by the hosts/applications. Normal operations cause no significant loads. But if things go wrong, then log info ranges from 0..infinity. I saw 1 gig /var filesystems filled in no time flat by syslogd. You don't want this in your DB, do you? So a serious architecture requires some preprocessing before sending messages to the central host. ... :o common tools : :i'm aware of several analyzing tools (swatch, logsurfer ... from : Not to forget the big commercial framework packages: CA Unicenter, HP OpenView, Tivoli TME, BMC Patrol + Command/Post. A smaller product with very good reputation is Netcool Omnibus. All these provide agents, which scan and preprocess arbitrary logfiles. Yes, with central configuration and pre-defined templates for standard log entries. No, long term analysis is not [yet] easy with these, but service reporting functionality is sprouting up everywhere and hopefully useful in the near future. ... :as said, i'd be very interested on your thoughts on the topic, your :experiences and maybe references to papers/tools. : Although everybody must work on this, there is not much public stuff on this AFAIK. Maybe www.summitonline.com has a bit. For analysis of complex event patterns, a rather new approach is to visualize the conceptual relationship of events. E.g.: Girardin98: A Visual Approach for Monitoring Logs (http://www.usenix.org/events/lisa98/girardin.html) Hellerstein99: EventBrowser: A Flexible Tool for Scalable Analysis of Event Data" (http://www.research.ibm.com/PM/perf_mgt.html) HTH Peter

1 0

Log analyzer/syslog-ng problems
by Bryan Stillwell 13 Apr '00

13 Apr '00

Hello everyone, I'm currently doing research at my job about writing a log analyzer. The first step we wish to accomplish is setting up a central log server. The server will be just for logs and will be secured so that it would be almost impossible for someone who has compromised another machine on the netowrk to modify the log files. The log server will be logging both internal and external machines, so we would really like for communication to be well encrypted. As I understand this option won't be implemented until the next version of syslog-ng. After this step is done we will set it up to automatically enter all logs into a postgresql database or mysql database for easier analysis. (I've been told postgresql has some native IP address data types that will come in handy.) However our number one item on the list is to make sure that the logs get to the server in a timely, consistant, and accurate manner. So I started doing some testing on various things that might cause problems during the transfering of the log data. Some of the things that I want to watch out for are: o What happens if the network is down for a day or two at a time? I want to make sure that they don't just disappear. o What if the network is being hit with a DOS attack and packets never reach the log server? or they reach the log server but the reply never gets back to the source computer? o What kind of verification can be done to make sure all the data made it to the log server ok? Should we log locally and remotely and compare the two at the end of each day? or can syslog-ng take care of it all? o What about if the main server dies and has to be restarted? client dies? Some of those questions will probably have the same answers, but I ran into the following problem when I started doing some stress tests between two computers on the internal lan. I've included the two syslog-ng.conf files to the end of this message. reptile is the client computer and testbox is the server. I ran the command `logger -f /usr/dict/linux.words` to test out how well it works. Apr 13 10:39:27 src@reptile/reptile logger: Aarhus Apr 13 10:39:27 src@reptile/reptile logger: Aaron ... Apr 13 10:39:38 src@reptile/reptile logger: strips Apr 13 10:39:38 src@reptile/reptile logger: striptease Apr 13 10:39:40 src@reptile/reptile logger: worked Apr 13 10:39:40 src@reptile/reptile logger: worker ... Apr 13 10:39:40 src@reptile/reptile logger: worn Apr 13 10:39:40 src@reptile/reptile logger: worried Apr 13 10:39:40 src@reptile/reptile logger<13>Apr 13 10:39:38 src@reptile logger: strive Apr 13 10:39:38 src@reptile/reptile logger: striven Apr 13 10:39:38 src@reptile/reptile logger: strives ... Apr 13 10:39:40 src@reptile/reptile logger: workbook Apr 13 10:39:40 src@reptile/reptile logger: workbooks Apr 13 10:34:54 reptile/reptile : worrier Apr 13 10:39:40 src@reptile/reptile logger: worriers Apr 13 10:39:40 src@reptile/reptile logger: worries ... Apr 13 10:39:40 src@reptile/reptile logger: Zulus Apr 13 10:39:40 src@reptile/reptile logger: Zurich So it works until it hits 'striptease' (purely coincidence btw) and then skips to 'worked'. When it hits 'worried' it switches back to where it left off and continues until it hits 'workbooks' where the previous group was supposed to be at. Other than that problem everything transferred fine. Any ideas on how to have that autofix itself, maybe I don't have the conf files setup correctly? Would it be better to use udp even though dropped packets will never be known about? I just want to maintain the upmost integrity in the log files. --- testbox(server) syslog-ng.conf --- options { long_hostnames(on); sync(0); log_fifo_size(100); }; source src { unix-dgram("/dev/log"); tcp(); internal(); }; destination local_log { file("/tmp/deleteme.log"); }; log { source(src); destination(local_log); }; --- testbox(server) syslog-ng.conf --- --- reptile(client) syslog-ng.conf --- options { sync(0); log_fifo_size(100000); }; source src { unix-dgram("/dev/log"); internal(); }; destination master_log_server { tcp("10.1.1.10" port(514)); }; log { source(src); destination(master_log_server); }; --- reptile(client) syslog-ng.conf --- Thank you, Bryan Stillwell -- http://www.tummy.com/ Consulting and Software for Linux and Unix KRUD - Kevin's Red Hat Uber Distribution - the Freshest Red Hat every month

1 0

Recommended server specs
by syslog-ng＠moe.co.uk 12 Apr '00

12 Apr '00

Hi All, Was just wondering what people recommend spec wise for a loghost? Currently have a P2-350, 320mb (ish) of ram and 18gb to store logs to. There are about 80 to 90 machines logging to this machine. I think it will run out of disk space before anything else. syslog-ng tends to use 3% to 7% cpu with a sync(5). Anyone have any interesting loghost information/experiences? Cheers, John

1 0