- syslog-ng - lists.balabit.hu

Slow capturing syslog
by Chee Chew 12 Nov '01

12 Nov '01

Thank you to all. It is really the DNS problem here. Now, it is working fast. On Sun, Nov 11, 2001 at 08:32:58PM -0800, Chee Chew wrote: > I tested with kiwi syslog generator and I found either syslog-ng or syslogd > is putting the log into file destination very slow at fastest exactly 10sec > per log. At the rest of the log is queue in buffer. If syslog-ng were to run > in environment that has log input with consistently 1 log per second, this > will not work even we set the buffer large. Also not real time. > > I believe this is something to do with the synchronisation of SOCK_STREAM > socket because I am using linux. Do you know how to fine tune this. I've captured around 2 million messages in the last 3 days on a linux loghost, so I don't think Linux is your problem. Post your config here, especially whether you're using DNS or not and we can help out. -- Nate Campi http://www.campin.net GnuPG key: 0xC17AEF79 Key fingerprint = BF12 722F 8799 E614 33CC FAB7 5A90 C464 C17A EF79 Microsoft Vaccine 2000 is configuring your immune system. This may take a few minutes. If your body stops responding for a long time and there is no brain activity please die. Setup will continue after you are reborn. --__--__-- _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng End of syslog-ng Digest

1 0

Slow capturing syslog
by Chee Chew 11 Nov '01

11 Nov '01

I tested with kiwi syslog generator and I found either syslog-ng or syslogd is putting the log into file destination very slow at fastest exactly 10sec per log. At the rest of the log is queue in buffer. If syslog-ng were to run in environment that has log input with consistently 1 log per second, this will not work even we set the buffer large. Also not real time. I believe this is something to do with the synchronisation of SOCK_STREAM socket because I am using linux. Do you know how to fine tune this. Regards

4 3

what to do with syslogd
by Chee Chew 10 Nov '01

10 Nov '01

I am going to use syslog-ng for receiving remote host syslog. What am I going to do with the existing syslogd? If I turn off syslogd, will I still able to capture all internal system syslog? regards

3 2

priority level ?
by Tony Moran 10 Nov '01

10 Nov '01

Hi all, I was wondering.. When I create a filter, if I specify a priority level, for example 'err' does that also imply all facility levels greater than err are also matched ? On the face of it this doesnt seem likely given I'm filter matching, but looking at the syslog-ng.conf.SunOS file in contribs/ and other examples this is what it can only seem to be saying. For example for /dev/console the example filter is { level(err) or (level(notice) and facility (auth, kern)); }; I'm hardly going to leave out everything above err, or notice for auth and kern if I am to send important messages to the console. Thanks for your help, Tony

2 1

different message shows up on loghost than on client
by Nate Campi 10 Nov '01

10 Nov '01

I have syslog-ng loghosts in out datacenters that receive messages via UDP from mostly solaris syslogd. The loghosts in the datacenters then relay the messages over TCP/stunnel to a central loghost for archiving/analysis/input into database. The problem is that a message like this on a solaris 2.6 box: Nov 7 04:05:45 ballys ctld 5.0.6[22164]: [0] Error: unable to read header - Status: NoMoreData. ...will arrive (via UDP) on my linux loghost (syslog-ng 1.4.12) like this: Nov 7 04:05:45 ballys.hotwired.com 5.0.6[22164]: [0] Error: unable to read header - Status: NoMoreData. I have no idea why it would cut out the program name (ctld) like that. Is this a known issue? This is really messing up my log tables, and I'd like to keep all the data straight in the database. Here's my options from the linux loghost: options { use_fqdn(yes); long_hostnames(off); keep_hostname(no); use_dns(yes); sync(0); }; Plus, when did it become an option to buffer messages with sync? Is is only the in devel branch? I'd like to set sync(5) or something to take it easy on the disk. TIA, -- Nate Campi http://www.campin.net GnuPG key: 0xC17AEF79 Key fingerprint = BF12 722F 8799 E614 33CC FAB7 5A90 C464 C17A EF79 The only way to convince some people that HTML is about content, not style is with a 2x4 <PLANK>.

2 3

elegant swatch solution
by Nate Campi 10 Nov '01

10 Nov '01

I found a clean way to use swatch to watch all your syslog messages from syslog-ng. Normally you need to tell swatch to tail a file, so if you want swatch to watch all your logs you have to either send all messages to one file or run multiple instances of swatch. I came up with this tonight: # hack to get swatch to read from stdin destination swatch { program("/usr/bin/swatch --read-pipe=\"cat /dev/fd/0\""); }; # send all logs to swatch log { source(src); destination(swatch); }; The source "src" is all my logs on that box. This works like a charm, and if syslog-ng is restarted, so is swatch, so you don't need to worry about HUP'ing/restarting swatch - ever! This way you can let swatch monitor your logs instead of using match() directives - since swatch has built in throttling on alerts I find it nicer to use. swatch keeps my pager from going nuts after the first message which caused the alarm, subsequent messages are ignored for a defined amount of time, using the "throttle feature". -- Nate Campi http://www.campin.net GnuPG key: 0xC17AEF79 Key fingerprint = BF12 722F 8799 E614 33CC FAB7 5A90 C464 C17A EF79 "Only two things are infinite, the universe and human stupidity, and I'm not sure about the former" -- Albert Einstein.

1 0

Re: [syslog-ng]Documentation/manual etc..
by bretwatson＠charteredsemi.com 09 Nov '01

09 Nov '01

Great, looks like in the next week or so I should be able to get out a first draft of teh manual.. The fun part will come with getting examples etc.. Cheers, Bret Balazs Scheidler <bazsi(a)balabit.hu> 09/11/2001 04:51 Sent by: syslog-ng-admin(a)lists.balabit.hu Please respond to syslog-ng To: syslog-ng(a)lists.balabit.hu cc: (bcc: WATSON Bret/Mgr/CSM/ST Group) Subject: Re: [syslog-ng]Documentation/manual etc.. On Fri, Nov 09, 2001 at 11:09:44AM +0800, bretwatson(a)charteredsemi.com wrote: > Hi Baltasar, > some questions on the options list.... > > I've dug through the cfg-lex.l file and found it is looking for a bunch of > options - some of which exist int eh current docs.. so some questions: > > What does "Keep_hostname" do? to not touch the hostname part of the received message, and leave it intact. > is "use_fqdn" the same as "long_hostnames"? no, long_hostnames is an alias to chain_hostnames, the latter being the preferred one. (this ambiguity caused me to change it) use_fqdn means to use fully qualified domain names, otherwise the domain is chopped after the first '.' character in the hostname. > What does "gc_threshold" do? gc_threshold is the same as gc_busy_threshold > What does "create_dir" do? if macro expansion is used in destination filenames, directory references to non-existing directories can be made, if create_dir is true, those directories are automatically created. > > and what about the following? are any enabled? > localip, ip, localport, port, destport, owner, group, perm, dir_owner, > dir_group, dir_perm, keep-alive, keep_alive, max-connections, mac, > authentication, encrypt, required, allow, deny. compress mac, authentication, encrypt and compress each accept one of triple as argument: required, allow or deny, though none of these are currently implemented. All the other options are (or supposed to be ) working: localip & localport: specifies the source ip & port of a connection (before connection is made, useful if your box has several IPs) ip means localip for sources destport: specifies the destination port for destinations owner, group, perm: file permission information for created files dir_owner, dir_group, dir_perm: directory permission information for created dirs keep-alive, keep_alive: do not drop connections when the config is reloaded max-connections: how many concurrent connections are permitted at a time -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

RE: [syslog-ng]Documentation/manual etc..
by bretwatson＠charteredsemi.com 09 Nov '01

09 Nov '01

Hi Baltasar, some questions on the options list.... I've dug through the cfg-lex.l file and found it is looking for a bunch of options - some of which exist int eh current docs.. so some questions: What does "Keep_hostname" do? is "use_fqdn" the same as "long_hostnames"? What does "gc_threshold" do? What does "create_dir" do? and what about the following? are any enabled? localip, ip, localport, port, destport, owner, group, perm, dir_owner, dir_group, dir_perm, keep-alive, keep_alive, max-connections, mac, authentication, encrypt, required, allow, deny. compress Thanks, Bret

2 1

RE: weird... please help
by bretwatson＠charteredsemi.com 09 Nov '01

09 Nov '01

Thanks, that works... Cheers, Bret "Baltasar Cevc" <baltasar(a)cevc-topp.de> 09/11/2001 06:01 Sent by: syslog-ng-admin(a)lists.balabit.hu Please respond to syslog-ng To: <syslog-ng(a)lists.balabit.hu> cc: (bcc: WATSON Bret/Mgr/CSM/ST Group) Subject: RE: [syslog-ng]weird... please help Hi, Bret! > But what happens is that everything ends up in teh "all" destination instead of the vpn destination.... It should end to be shown in both... Try to add ".*" to your host check regexp. Perhaps it doesn't match because there is something more than the single host IP. And for the all destination: you could use a third filter like this to prevent VPN messages getting there like this: filter fallb { not (filter (f_vpnhosts) or filter (f_vpnhosts2)); }; bye.baltasar ::: B A L T A S A R C E V C mailto: baltasar(a)cevc-topp.de Tel.: +49 89 99020515 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

weird... please help
by bretwatson＠charteredsemi.com 08 Nov '01

08 Nov '01

Hmm I'm stuffed up I guess, but I can't work out where... Please help!!! Thanks, Bret Here is my syslog-ng.conf file options { sync(0); time_reopen(10); log_fifo_size(100); }; source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; source net { udp(); }; destination all { file("/var/adm/messages"); }; destination vpn { file("/usr/log1/vpn.log"); }; filter f_vpnhosts { host("203.126.246.216") or host("203.126.246.217") or host("203.126.246.218"); }; filter f_vpnhosts2 { host("10.84.3.10"); }; log { source(net); filter(f_vpnhosts); destination(vpn); }; log { source(net); filter(f_vpnhosts2); destination(vpn); }; log { source(local); destination(all); }; log { source(net); destination(all); }; Here is the log lines I'm trying to re-direct to the vpn.log Nov 7 08:15:55 10.84.3.10/10.84.3.10 [1:33947706:Gate7520:CHARTERED 01:203.126.246.216]2001/11/07 16:27:41 Isakmp ScSA: Rekey Phase 2: Loc:10.200.*.*, Rem:10.84.3.50 (66.68.164.196) But what happens is that everything ends up in teh "all" destination instead of the vpn destination....

2 1