- syslog-ng - lists.balabit.hu

$HOST variable
by Chris Sibbitt 03 Aug '01

03 Aug '01

Greetings. I'm having a problem with getting syslog-ng to write out to filenames based on $HOST. My setup is as follows: -I have two machines running syslog-ng -Machine A has a program that sends syslog messages to itself (127.0.0.1) -Machine A has a tcp destination rule like: destination Machine_B{ tcp(10.1.1.1); } -Machine B has a tcp source rule that receives the messages from Machine A -Machine B has a destination rule like: destination host_logs { file("/logs/$HOST.txt"); } The problem is that what I end up with on Machine B is a file called /logs/127.0.0.1 that contains all the messages generated by the program on Machine A. They look like this: Aug 3 09:12:48 127.0.0.1/127.0.0.1/10.1.1.2 myprog[2182]: Test message I've fiddled with setting keep_hostname() to yes and no on Machine B, but to no avail... apparently this doesn't affect the $HOST variable. I know that chain_hostnames(no) fixes the problem (I get a file named 10.1.1.2, which is what I want), but I'd really like to see the whole path the message took because sometimes Machine A will receive messages from others that get forwarded on. Is there any way to accomplish this. In simple terms, I think I'm looking for a way to get the $HOST variable to pick the LAST hostname in the chain instead of the first. Any ideas, or do I need to edit the code? If the later is the case, I'm pretty sure I need to fix something in expand_macro() (in affile.c), if anyone knows exactly how this should be done some help would be appreciated, otherwise I'll dust off my debugger skills and see what I can accomplish. I hope it doesn't come to that though. JetNet Logo Chris Sibbitt Operations Development 106 Schneider Road, Unit C Kanata, Ontario K2K 1Y2 Phone (613) 271-6220, ext228 Fax (613) 271-6229

1 0

Re: [syslog-ng]Re: syslog-ng's memory footprint growing constantly
by Craig J Constantine 03 Aug '01

03 Aug '01

...on 8/2/01 1:47 PM, the following was received from bazsi(a)balabit.hu >> >ok, we have some kind of string leak. what does "hosts.org_dir" in >> >your environment mean? is it a directory which syslog-ng creates, >> >or a filename? or some part of some message? >> >> I have no idea. :) That string isn't in my syslog-ng config, nor >> *anywhere* in the syslogs recorded by syslog-ng today, nor even >> anywhere in the syslog-ng distribution. Might be just a lot over >> strings overlayed in memory... very weird. > >I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume >you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it >possible that nisplus is leaking somewhere? > >I've never ran syslog-ng with a computer using nisplus, that might be an >explanation why others didn't encounter this bug. BINGO! NIS+ is the culprit... I adjusted the resolver library setup to not use NIS at all for host resolution. I Restarted my original, non-debugging syslog-ng yesterday afternoon. 12 hours later, it's still a tiny little program just like it should be. :) Thanks Balazs for your help! -Craig

1 0

RE: [syslog-ng]Re: syslog-ng's memory footprint growing constantl y
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 03 Aug '01

03 Aug '01

Actually, I ran it with nisplus for a while but it has been some time ago. I didn't have any problems with syslog-ng in connection to nisplus but we had enough problems with nisplus to eventually wind up going back to nis. I have used nis for over a year and don't have any problems using it with syslog-ng. Problems with nisplus would not be surprising. Regards, Drew -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Thursday, August 02, 2001 7:48 PM To: Craig J Constantine Cc: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Re: syslog-ng's memory footprint growing constantly On Thu, Aug 02, 2001 at 12:05:29PM -0400, Craig J Constantine wrote: > ...on 8/2/01 11:19 AM, the following was received from bazsi(a)balabit.hu > > >ok, we have some kind of string leak. what does "hosts.org_dir" in your > >environment mean? is it a directory which syslog-ng creates, or a filename? > >or some part of some message? > > I have no idea. :) That string isn't in my syslog-ng config, nor > *anywhere* in the syslogs recorded by syslog-ng today, nor even anywhere > in the syslog-ng distribution. Might be just a lot over strings overlayed > in memory... very weird. I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it possible that nisplus is leaking somewhere? I've never ran syslog-ng with a computer using nisplus, that might be an explanation why others didn't encounter this bug. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

Re: syslog-ng's memory footprint growing constantly
by Balazs Scheidler 02 Aug '01

02 Aug '01

On Thu, Aug 02, 2001 at 12:05:29PM -0400, Craig J Constantine wrote: > ...on 8/2/01 11:19 AM, the following was received from bazsi(a)balabit.hu > > >ok, we have some kind of string leak. what does "hosts.org_dir" in your > >environment mean? is it a directory which syslog-ng creates, or a filename? > >or some part of some message? > > I have no idea. :) That string isn't in my syslog-ng config, nor > *anywhere* in the syslogs recorded by syslog-ng today, nor even anywhere > in the syslog-ng distribution. Might be just a lot over strings overlayed > in memory... very weird. I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it possible that nisplus is leaking somewhere? I've never ran syslog-ng with a computer using nisplus, that might be an explanation why others didn't encounter this bug. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

Re: syslog-ng's memory footprint growing constantly
by Craig J Constantine 02 Aug '01

02 Aug '01

...on 7/27/01 1:02 AM, the following was received from syslog-ng-admin(a)lists.balabit.hu >Date: Thu, 26 Jul 2001 10:25:08 +0200 >From: Balazs Scheidler <bazsi(a)balabit.hu> >To: syslog-ng(a)lists.balabit.hu >Subject: Re: syslog-ng's memory footprint growing constantly > >On Wed, Jul 25, 2001 at 04:17:06PM -0400, Craig J Constantine wrote: >> Hi, >> Basically, the memory size of the process simply grows and grows: >> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >> root 13899 0.1 19.3 25412 24764 ? S Jul24 2:05 syslog-ng >> That was snapped after about 4 hours of run time. If I let it run for >> days... well, it once got to 90000 VSZ before the box choked for lack of >> RAM. For gauging how heavily worked syslog-ng is, the line counts for all >> the files it writes (this is one day of logs) are: [rest of my original, snipped] > >There have been memory leaks, but they were fixed in 1.4.8. To help me to >debug your problem, please compile syslog-ng _AND_ libol with --with-debug >enabled (configure time option), and run syslog-ng with --debug --verbose >switches, and save the resulting output. I'm mainly interested in lines >starting with "Objects alive ... garbage collected...". > >If the number after "objects alive" increases then there's some spurious >link within the data structure (the gc frees only unlinked objects), if it >doesn't it is some kind of string data (for example messages). I recompiled libol/syslog-ng from fresh tarballs using the configure flag. Then I fired it up with runtime debug arguments and captured everything to a file. I started the run around 9am, and left it go for a good half an hour. The resulting file was, well, truly huge. 8^) Let me know if you'd like to see it, I can drop it on a server where you can grab it via anon FTP. Likewise if you want to see my config file again. ...around 9:07 syslog-ng was: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 0.7 1652 1000 pts/1 S 08:54 0:02 ./syslog-ng.debug (ie, tiny :) The "Objects alive" was at 293. ...around 9:37 syslog-ng was: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 1.1 2104 1452 pts/1 S 08:54 0:09 ./syslog-ng.debug The "Objects alive" was at 299. ...around 10:07: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 1.4 2552 1900 pts/1 S 08:54 0:16 ./syslog-ng.debug The "Objects alive" was at 299. ...around 10:37: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.4 1.9 3124 2472 pts/1 S 08:54 0:24 ./syslog-ng.debug The "Objects alive" was still at 299. ...around 11:07: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.4 2.3 3720 3068 pts/1 S 08:54 0:35 ./syslog-ng.debug The "Objects alive" was still at 299. ...looks like the objects count is stable, but the memory size is still growing. I sent it a SIGQUIT and scooped up the core file. It looks like a large portion (more than half) of the image is memory that held plain text messages at some point. Notably the string piece "hosts.org_dir" appears often enough that I visually noticed it. Thoughts? If you want to see the core file, I can make it available via anon FTP as well. -Craig

1 0

RE: [syslog-ng]Adding new facilities
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 02 Aug '01

02 Aug '01

Brad, you are correct. It is basically an integer define. regards, Drew -----Original Message----- From: Brad Arlt [mailto:arlt@cpsc.ucalgary.ca] Sent: Wednesday, August 01, 2001 6:57 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Adding new facilities Hello... I am compiling syslog-ng 1.4.12 for Solaris 8, and noticed the IRIX factilities of LOG_AUDIT and LOG_LFMT are not mentioned anywhere in syslog-names.c. Also the facilities of LOG_NTP, LOG_SECURITY, and LOG_CONSOLE from BSD are not mentioned anywhere. I have in the past (and will in this build) manually defined LOG_AUTHPRIV and LOG_FTP for receiving messages from our Linux/BSD machines (AUTHPRIV and FTP are a Linux and BSD thing). If I understand the source code correctly the names are only used to translate configuration file strings to number (e.g. "auth" -> (4<<3)). If I am right, this should mean I can add the above facilities without any problems. Can anyone tell me if this sounds right? I am going to give it a try anyway, but I wanted to see if anyone thought I was crazy, or if there was a better way to do this. ---------------------------------------------------------------------------- __o Bradley Arlt Email: arlt(a)cpsc.ucalgary.ca o__ _ \<_ WWW: www.acs.ucalgary.ca/~bdarlt _>/ _ (_)/(_) -Eat well, sleep peacefully, drink lots, and ride like hell. (_)\(_) _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

Adding new facilities
by Brad Arlt 01 Aug '01

01 Aug '01

Hello... I am compiling syslog-ng 1.4.12 for Solaris 8, and noticed the IRIX factilities of LOG_AUDIT and LOG_LFMT are not mentioned anywhere in syslog-names.c. Also the facilities of LOG_NTP, LOG_SECURITY, and LOG_CONSOLE from BSD are not mentioned anywhere. I have in the past (and will in this build) manually defined LOG_AUTHPRIV and LOG_FTP for receiving messages from our Linux/BSD machines (AUTHPRIV and FTP are a Linux and BSD thing). If I understand the source code correctly the names are only used to translate configuration file strings to number (e.g. "auth" -> (4<<3)). If I am right, this should mean I can add the above facilities without any problems. Can anyone tell me if this sounds right? I am going to give it a try anyway, but I wanted to see if anyone thought I was crazy, or if there was a better way to do this. ---------------------------------------------------------------------------- __o Bradley Arlt Email: arlt(a)cpsc.ucalgary.ca o__ _ \<_ WWW: www.acs.ucalgary.ca/~bdarlt _>/ _ (_)/(_) -Eat well, sleep peacefully, drink lots, and ride like hell. (_)\(_)

1 0

Re:Error when starting syslog-ng (Kai Wang)
by Kai Wang 31 Jul '01

31 Jul '01

I found out the reason of the problem. After cloning, the default gateway configuration was lost. The line "destination d_test { udp("test.ucalgary.ca"); };" made syslog-ng unable to start. After I removed the line and relevant lines, syslog-ng started a short time and died quickly. After I added the default gateway, everything is working well. Kai

1 0

Configuring Chrooted apps to use /dev/log with syslog-ng
by Brian E. Seppanen 31 Jul '01

31 Jul '01

I'm trying to chroot bind, and I got to the part where it discusses setting up syslog so it will accept logs from chrooted apps. Is there any way of configuring syslog-ng to do something similar? Thanks, Brian Seppanen seppy(a)chartermi.net 906-228-4226 ext 23

2 1

RE: [syslog-ng]Error when starting syslog-ng
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 31 Jul '01

31 Jul '01

Probably going to need you to run an strace/truss on the application. This could mean a number of things. Something already has the port, etc. -----Original Message----- From: Kai Wang [mailto:kwang@ucalgary.ca] Sent: Monday, July 30, 2001 11:59 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Error when starting syslog-ng I have been running syslog-ng on aix 4.3.3 maintenance level 6 for monthes. I cloned our log server and found syslog-ng could not start on the new machine. I haven't changed a thing. # syslog-ng -v binding fd 3, unixaddr: /dev/log binding fd 4, inetaddr: 0.0.0.0, port: 514 Error creating AF_INET socket (Error 0) Error initializing configuration, exiting. Kai _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0