August 2001 - syslog-ng - lists.balabit.hu

Re: [syslog-ng]Full pipe as destination
by Andreas Schulze 23 Aug '01

23 Aug '01

Dear Bazsi, Brad Arlt schrieb: > On Tue, Aug 21, 2001 at 02:16:09PM +0200, Balazs Scheidler wrote: > > On Tue, Aug 21, 2001 at 01:26:27PM +0200, Peter Draexler wrote: > > > Peter Draexler schrieb: > > > > > > > Dear Bazsi, > > > > thank you for your info. This should help. > > > > Nevertheless, does syslog-ng log somewhere if it discards events or do you know > > > > another way to check this? > > > > This would be important to adapt log_fifo_size. Currently we only observe by > > > > chance that we lose information. > > > > logs are silently dropped in this case. I was thinking of implementing some > > kind of automatic statistics message to report log losses, but that would > > race against normal log traffic. do you have a better idea. statistics are a very good idea and for many syslog-ng users its important to know as most as possible about messages dropped (for which destination, etc.), messages transfered by source ond/or destination and something more. I think the discussions about possible log destinations hasn't the real impact in this context. as you say its important that we prevent the daemon from stucks in-active with its internal statistics stuff. the real problem here is, to implement the output method of the stat. informations. but answer the following question: "is it really mandatory to realize a continuous output of statistical informations?" I think not. decide to which logging destination the stats should be send. I think the internal() log source is a better choice than brad's "oh shit!" file, because we keep save the flexibility of syslog-ng's rule-based flow control. but a separate log is ok. but don't send the stats continuously to internal(). do this e.g. at midnight for the last 24h. realize a command line switch to disable this feature. if 24h is to long, implement a signal handler for e.g. SIGUSR1|2 that activates the stats output. so its possible for each user to decide the time on which he likes stats by sending a SIGUSR? to the daemon. > A single message saying "log queue full" written to the syslog facilty > at emerg would due it. And rather than compete have it win the race > always. Since we are already losing messages, it is more important > that we know we are losing messages than to know what messages have > been lost (if that makes sense). > > One could also have a seperate "oh shit!" log file that only gets > written to by syslog-ng internals and not through standard mechanisms. > A non-blocking write(2) could be used to ensure that these messages > didn't impeed on other logging (much). thx. for your help best regards -- ASL

2 1

RE: [Fwd: [syslog-ng]Full pipe as destination]
by Andrew Fort 22 Aug '01

22 Aug '01

>One could also have a seperate "oh shit!" log file that only gets >written to by syslog-ng internals and not through standard mechanisms. >A non-blocking write(2) could be used to ensure that these messages >didn't impeed on other logging (much). I prefer this option also (despite its lack of elegance :), it's the most system independant and allows the sysad to stick a log watcher on it or whatever. Also because it avoids the in-band race issue.

1 0

[Fwd: [syslog-ng]Full pipe as destination]
by Peter Draexler 22 Aug '01

22 Aug '01

Peter Draexler schrieb: > Dear Bazsi, > thank you for your info. This should help. > Nevertheless, does syslog-ng log somewhere if it discards events or do you know > another way to check this? > This would be important to adapt log_fifo_size. Currently we only observe by > chance that we lose information. > Best regards Peter > > Balazs Scheidler schrieb: > > > On Tue, Aug 14, 2001 at 08:45:35AM +0200, Peter Draexler wrote: > > > Hi, > > > > > > We are using syslog-ng on our logserver with multifunctional > > > logforwarding. > > > As a special type of destination we are using pipes as a special > > > interface to commercial applications. > > > > > > Here is our question: > > > What happens with the log events, if the pipe becomes > > > temp. unavailable (e.g. the reading end is dead or > > > the pipe is full)? > > > > > > IMHO you have to open() the pipe nonblocking, to avoid your > > > syslog stucks in-active, in the case one destination is broken. > > > I think this is true. Correct? > > > > it's true. > > > > > > > > What happens with your internal log buffers, if write() returns -1 > > > and errno set to EAGAIN? > > > You implement an internal buffering/queue and try to write the > > > outstanding log events some times later or you simply discard > > > the buffers as long as one destination is unreachable? > > > > messages are queued up to the value of log_fifo_size, which IIRC can be set > > on a per-destination basis. if the fifo is full, messages are discarded. > > > > > > > > A Hint for syslog-ng.1.5.8 on Solaris 8: > > > $DATE in destination templates returns nothing. > > > But $FULL|ISODATE works fine. > > > > this seems to be fixed in my CVS tree, though it's not yet released. > > > > -- > > Bazsi > > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > -- > Dr. Peter Dräxler > mediaWays GmbH > Hülshorstweg 30 > 33415 Verl > Tel. +49-5246-80-1888 > Fax +49-5246-80-2888 > peter.draexler(a)mediaways.net > ------------------------------ > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng

3 2

RE: [syslog-ng]hostname on loghost
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 20 Aug '01

20 Aug '01

Nate, In the options part using use_fqdn(yes) will give you the fully qualified domain name. There are other ways to do this as well. Since 1 comes in tcp and the other udp you could set up separate sources and base your destinations on the sources, or something similar. I would use the use_fqdn option since I have the same problem as you except I have 7 ns1's. regards, Drew -----Original Message----- From: Nate Campi [mailto:nate@campi.cc] Sent: Friday, August 17, 2001 10:17 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]hostname on loghost I have about a hundred machines logging to a central syslog-ng loghost, most over standard UDP, a handful relayed from a syslog-ng loghost (over TCP) in a datacenter to the central loghost. The problem I'm having is this: we have two hosts named ns1 (who could imagine that?). The first logs over UDP (solaris 8) to the central loghost. I decided to put the second (hostname is configured as ns1.sjc, running - solaris 2.6) logging from syslog-ng over TCP to the loghost. I've always had good luck with the syslog-ng TCP connections keeping the hostname as sent from the TCP client syslog-ng. This time, though, the messages both on the TCP client and central syslog-ng server report the host as ns1. I want it to be ns1.sjc, and the two hosts will log differently, and archive differently. I archive with lines like this: file("/var/log/HOSTS/$HOST/$FACILITY/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$ DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(hosts); }; So it's critical that I get a different hostname for the two hosts. Can I somehow set the hostname for syslog-ng to log as? `hostname` reports ns1.sjc on the host, so why can't I use that? I've mucked around with options like keep_hostname and long_hostnames but they just seem to stick the "source" name on the messages, not what I need. Any ideas? -- Nate _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

hostname on loghost
by Nate Campi 17 Aug '01

17 Aug '01

I have about a hundred machines logging to a central syslog-ng loghost, most over standard UDP, a handful relayed from a syslog-ng loghost (over TCP) in a datacenter to the central loghost. The problem I'm having is this: we have two hosts named ns1 (who could imagine that?). The first logs over UDP (solaris 8) to the central loghost. I decided to put the second (hostname is configured as ns1.sjc, running - solaris 2.6) logging from syslog-ng over TCP to the loghost. I've always had good luck with the syslog-ng TCP connections keeping the hostname as sent from the TCP client syslog-ng. This time, though, the messages both on the TCP client and central syslog-ng server report the host as ns1. I want it to be ns1.sjc, and the two hosts will log differently, and archive differently. I archive with lines like this: file("/var/log/HOSTS/$HOST/$FACILITY/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(hosts); }; So it's critical that I get a different hostname for the two hosts. Can I somehow set the hostname for syslog-ng to log as? `hostname` reports ns1.sjc on the host, so why can't I use that? I've mucked around with options like keep_hostname and long_hostnames but they just seem to stick the "source" name on the messages, not what I need. Any ideas? -- Nate

1 0

Full pipe as destination
by Peter Draexler 16 Aug '01

16 Aug '01

Hi, We are using syslog-ng on our logserver with multifunctional logforwarding. As a special type of destination we are using pipes as a special interface to commercial applications. Here is our question: What happens with the log events, if the pipe becomes temp. unavailable (e.g. the reading end is dead or the pipe is full)? IMHO you have to open() the pipe nonblocking, to avoid your syslog stucks in-active, in the case one destination is broken. I think this is true. Correct? What happens with your internal log buffers, if write() returns -1 and errno set to EAGAIN? You implement an internal buffering/queue and try to write the outstanding log events some times later or you simply discard the buffers as long as one destination is unreachable? Thanks very much. Best Regards -- Andreas and Peter A Hint for syslog-ng.1.5.8 on Solaris 8: $DATE in destination templates returns nothing. But $FULL|ISODATE works fine. -- Dr. Peter Dräxler mediaWays GmbH Hülshorstweg 30 33415 Verl Tel. +49-5246-80-1888 Fax +49-5246-80-2888 peter.draexler(a)mediaways.net ------------------------------

2 2

RE: [syslog-ng]compiling on Solaris 8
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 16 Aug '01

16 Aug '01

--- Balazs Scheidler <bazsi(a)balabit.hu> wrote: > try this patch: > > diff -u -r1.10.2.1 afstreams.c > --- afstreams.c 2001/02/25 12:30:22 1.10.2.1 > +++ afstreams.c 2001/07/16 08:26:55 > @@ -164,7 +164,7 @@ > { > NEW(stream_fd, f); > > - init_file(backend, &f->super, fd); > + init_file(backend, &f->super, fd, NULL); > f->super.prepare = prepare_stream_fd; > f->super.read = stream_read_callback; > f->super.want_read = 1; Believe this is fixed in 1.4.13, but I don't think Bazsi has had time to officially release that on the website. You can make the slight mod to the afstreams.c file and it will work. Regards, Drew -----Original Message----- From: David Monk [mailto:david@purplebear.net] Sent: Wednesday, August 15, 2001 11:41 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]compiling on Solaris 8 I have tried several times to get syslog-ng to compile on Solaris 8 with GNU tools. I have tried the most current release of the 1.4 branch and 1.2 branch, with no luck. Each one complains about a different function with too few arguments. In the 1.4 release, it was src/afstreams.c line 167, init_file function, I believe. Has anyone successfully done this? Seeing that the site says it's in wide use on Solaris, I am hoping so. Please someone tell me what I am missing.I really need a syslog facility with remote reception capabilities. Thank you, David Monk CCNA, MCSE david(a)purplebear.net _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

compiling on Solaris 8
by David Monk 15 Aug '01

15 Aug '01

I have tried several times to get syslog-ng to compile on Solaris 8 with GNU tools. I have tried the most current release of the 1.4 branch and 1.2 branch, with no luck. Each one complains about a different function with too few arguments. In the 1.4 release, it was src/afstreams.c line 167, init_file function, I believe. Has anyone successfully done this? Seeing that the site says it's in wide use on Solaris, I am hoping so. Please someone tell me what I am missing.I really need a syslog facility with remote reception capabilities. Thank you, David Monk CCNA, MCSE david(a)purplebear.net

1 0

Re: syslogd-ng
by Balazs Scheidler 15 Aug '01

15 Aug '01

On Tue, Aug 14, 2001 at 02:45:56PM -0400, Erik Ivanenko wrote: > The latest "stable" versions do not compile on Solaris 2.6. > > Failure during compilation: > > afstreams.c:167: too few arguments to function 'init_file' http://www.balabit.hu/downloads/syslog-ng/1.4/syslog-ng-1.4.13.tar.gz I'll announce it as soon as I'll have some time. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

RE: [syslog-ng]NT Eventlog -> UNIX Syslog
by Nijs, Daniel 12 Aug '01

12 Aug '01

Free lunch!!!! ;) Thanks for that link, I am downloading it right now. Wonder why this never showed up in search engines. Best regards, Dan -----Original Message----- From: Michael Hargadon [mailto:MHargadon@ITDepartment.com] Sent: Tuesday, August 07, 2001 10:00 AM To: 'syslog-ng(a)lists.balabit.hu' Subject: [syslog-ng]NT Eventlog -> UNIX Syslog Since there was a lot of interest in this I figured I'd be nice and provide a link to the software I use. ;) http://www.intersectalliance.com/projects/index.html#BackLog It's free, too.. Regards, Michael Hargadon mhargadon(a)itdepartment.com _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

3 3