- syslog-ng - lists.balabit.hu

60 second delay when log rotates, can this be avoided?
by Michael Toy 30 Mar '07

30 Mar '07

I have a log which is received over TCP and is being written to a local file name which changes (with syslog ng macros) once per hour. I am also sending that same stream to a program. When the log rotates, all data seems to stop flowing for exactly 60 seconds, both to the log file and to program. Is there any way to change this?

6 10

syslog-ng 1.6 and 2.0
by Pol Moal 29 Mar '07

29 Mar '07

Hi, Is it possible to use syslog-ng 1.6 for the loghost and syslog-ng 2.0 for the client? nb: client use only option that exists in 1.6. thanks -- Pol Moal - Comité Réseau des Universités |--> http://www.cru.fr

3 3

Re: [syslog-ng] problem in compiling "syslog-ng-2.0.2" on SunOS 5.9 sparc
by Root Administrator 28 Mar '07

28 Mar '07

> > These are the package information about GLIB. > > # pkginfo -l SUNWGlib > PKGINST: SUNWGlib > NAME: GLIB - Library of useful routines for C programming > CATEGORY: system > ARCH: sparc > VERSION: 11.9.0,REV=2002.03.02.00.35 > BASEDIR: / > VENDOR: Sun Microsystems, Inc. > DESC: GLIB - Library of useful routines for C programming > PSTAMP: sfw8120020302003551 > INSTDATE: Aug 27 2004 19:50 > HOTLINE: Please contact your local service provider > STATUS: completely installed > FILES: 33 installed pathnames > 8 shared pathnames > 11 directories > 7 executables > 779 blocks used (approx) > > # pkginfo -l SMCglib > PKGINST: SMCglib > NAME: glib > CATEGORY: application > ARCH: sparc > VERSION: 2.6.2 > BASEDIR: /usr/local > VENDOR: The GLIB Project > PSTAMP: Steve Christensen > INSTDATE: 3月 05 2007 15:50 > EMAIL: steve(a)smc.vnet.net > STATUS: completely installed > FILES: 780 installed pathnames > 8 shared pathnames > 163 directories > 17 executables > 42974 blocks used (approx) > > SUNWGlib is the pre-installed package. > SMCglib is that I did pkgadd successfully. > file is : > ftp://ftp.sunfreeware.com/pub/freeware/sparc/9/glib-2.6.2-sol9-sparc-local.… > > > -------------------------------------------------------------------------------- > > Further information about the package is as, > > # cat /usr/local/lib/pkgconfig/glib-2.0.pc > prefix=/usr/local > exec_prefix=${prefix} > libdir=${exec_prefix}/lib > includedir=${prefix}/include > > glib_genmarshal=glib-genmarshal > gobject_query=gobject-query > glib_mkenums=glib-mkenums > > Name: GLib > Description: C Utility Library > Version: 2.6.2 > Libs: -L${libdir} -lglib-2.0 -liconv > Cflags: -I${includedir}/glib-2.0 -I${libdir}/glib-2.0/include > > and, > > # export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/ > # pkg-config --cflags glib-2.0 > -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include > > but, > > # ls /usr/local/lib/g* > /usr/local/lib/g*: No such file or directory > > Therefore, it seems that the installed package does NOT create the > directories > which are necessary to compile syslog-ng-2.0.2. > Further, the package does NOT create the file "glibconfig.h" which should > exist > in /usr/local/lib/glib-2.0/include. > > I simply tried to copy the file "/usr/sfw/lib/glib/include/glibconfig.h" > into > "/usr/local/lib/glib-2.0/include/" , and done make again. > Then, it passes the problem regarding "file does not exist" , although it > errors out > a lot more elsewhere since the file "glibconfig.h" belongs to the original > package. > > > -------------------------------------------------------------------------------- > > find /usr/local -name glibconfig.h -ls > returns > none > > find /usr/local -name gtypes.h -ls > returns > 135296 15 -rw-r--r-- 1 root bin 15241 Mar 1 2005 > /usr/local/include/glib-2.0/glib/gtypes.h > > > -------------------------------------------------------------------------------- > > About the missing blank space, > there IS a space before the second -I. > There was a space in the text before sending. > It was deleted somehow at the mail archive. > I do not know why,,, > > > -------------------------------------------------------------------------------- > > Questions are,,, > Q1. Is the package I have installed appropriate in order to compile > syslog-ng-2.0.2 ? > Q2. Are there any package dependencies associated with this package ? > > Please help out. > > Regards, > > George > > ------------------------------ > > Message: 4 > Date: Tue, 27 Mar 2007 03:01:06 -0400 > From: Valdis.Kletnieks(a)vt.edu > Subject: Re: [syslog-ng] problem in compiling "syslog-ng-2.0.2" on > SunOS 5.9 sparc > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: <200703270701.l2R716nZ014513(a)turing-police.cc.vt.edu> > Content-Type: text/plain; charset="us-ascii" > > On Tue, 27 Mar 2007 11:57:42 +0900, Root Administrator said: > > > > ftp://ftp.sunfreeware.com/pub/freeware/sparc/9/glib-2.6.2-sol9-sparc-local.… > > > > The package information is ; > > > > # pkginfo | grep -i glib > > application SMCglib glib > > system SUNWGlib GLIB - Library of > > useful routines for C programming > > Humor me, part one - can you get pkginfo to tell you what release(s) of > glib > this is? And is the sol9-sparc-local file referenced above actually the > one you did a successful pkgadd with? (I'm suspecting that there's > something > wrong here, as later on you look to see what the pkginfo database has in > it, and we *don't* see anything referencing /usr/local, even though: > > > gcc -DHAVE_CONFIG_H -I. -I. -I.. > > -I/usr/local/include/glib-2.0-I/usr/local/lib/glib- > > 2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -g -O2 > -Wall -g > > -c `test -f 'misc.c' || echo './'`misc.c > > We see a reference to /usr/local/include, and /usr/local/lib/include. > > Also, please double-check if you had a cut-n-paste error - there's a > missing blank before the second -I. > > What does this command return: > > % pgk-config --cflags glib-2.0 > > (If this has 2 -I flags with a missing space, that's your problem). > > > In file included from /usr/local/include/glib-2.0/glib/galloca.h:30, > > from /usr/local/include/glib-2.0/glib.h:30, > > from syslog-ng.h:37, > > from misc.h:27, > > from misc.c:24: > > OK, here we've found a glib include in /usr/local/include, somehow. > > > /usr/local/include/glib-2.0/glib/gtypes.h:30: glibconfig.h: No such > file > > or directory > > but something is fishy about it. > > > # find /usr -name glibconfig.h -exec grep {} > /var/sadm/install/contents \; > > Humor me, part two: > > What does this return: > > # find /usr/local -name glibconfig.h -ls > # find /usr/local -name gtypes.h -ls > > (We skip the grep because I'm curious what's actually on the system, as > opposed to what files pgkadd *thinks* are on the system...) > >

2 1

syslog-ng and cisco's problem
by Rębelski, Michał 28 Mar '07

28 Mar '07

Hi all, I'm trying to set up a syslog-ng 2.0.0 on Red Hat Enterprise 4.0 i386 edition. I have already set up my /etc/syslog-ng.conf file in following way: [root@swlogg02 cisco]# cat /etc/syslog-ng.conf # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); }; source s_cisco { udp(ip(0.0.0.0) port(514)); }; destination d_cisco_d { file("/var/log/cisco/$HOST/$YEAR/$MONTH/$DAY/log"); }; destination d_cisco_c { file("/var/log/cisco/$HOST/$YEAR/$MONTH/$DAY/log"); }; filter f_cisco_d { level(debug); }; filter f_cisco_c { level(crit); }; log { source(s_cisco); filter(f_cisco_d); destination(d_cisco_d); }; log { source(s_cisco); filter(f_cisco_c); destination(d_cisco_c); }; but I only got logs from only one machine, for your information from ASA5520. I would liek to receive messages from almost 50 machines (routers/switches/firewalls, etc). As general I set up on firewalls critical level of logging and on router/switches debug. Could you please tell me what I did wrong? regards -- Michał Rębelski

1 0

Re: [syslog-ng] Log rotate without log rotate.
by G.W. Haywood 28 Mar '07

28 Mar '07

Hi there, On Wed, 28 Mar 2007 Bruno Vieira wrote: > ... In the project I'm in, I won't have the time to do it in a low > level (kernel) as you refer and as I also thought it would be great > if Linux already had a device like this (a rotation file with > parameters of lines and/or size). Have you looked at dmesg? -- 73, Ged.

1 0

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 28 Mar '07

28 Mar '07

In fact, you could even write this as a kernel module (which would give you direct access to the list.h kernel header) and implement this behaviour right under the hood of the OS rather than in Syslog-NG. Basically, using the Linux VFS and kernel module subsystems, you can create device files in which the "write" operations behave in the manner listed below (writing to a circular list implementation of X number of nodes). This would then allow Syslog-NG to continuously write to the device file, and let the device file's implementation operate in the ring buffer manner outlined below. I hope this helps, Justin. _____ From: Justin Randall Sent: Tuesday, March 27, 2007 5:47 PM To: 'Syslog-ng users' and developers' mailing list' Subject: RE: [syslog-ng] Log rotate without log rotate. Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 28 Mar '07

28 Mar '07

Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

problem in compiling "syslog-ng-2.0.2" on SunOS 5.9 sparc
by Root Administrator 27 Mar '07

27 Mar '07

Hi, I have been facing the problem in compiling "syslog-ng-2.0.2" on SunOS 5.9sparc. The steps I have followed are described below. Could anyone plz help troubleshooting? [Condition] "GLIB" package is that of pre-installed. The package information about "GLIB" is ; # pkginfo | grep -i glib system SUNWGlib GLIB - Library of useful routines for C programming [Steps] 1. Install "eventlog-0.2.5" that is required by syslog-ng 2.0.2. The source file is from http://www.balabit.com/ Installation was successfully completed. 2. Configure "syslog-ng 2.0.2" # ./configure --enable-dynamic-linking It resulted in the following error. checking for GLIB... Requested 'glib-2.0 >= 2.2' but version of GLib is 2.0.7 configure: error: Cannot find GLib library version >= 2.2: is pkg-config in path? It seems the version of "GLIB" is older. Thus, I installed newer version of GLIB "glib-2.6.2". The file is downloaded from the following URL, ftp://ftp.sunfreeware.com/pub/freeware/sparc/9/glib-2.6.2-sol9-sparc-local.… The package information is ; # pkginfo | grep -i glib application SMCglib glib system SUNWGlib GLIB - Library of useful routines for C programming 3. 2nd attempt of configure New package is installed in previous step so that I changed the PATH to make sure the installed package is first reached. # export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/ # ./configure --enable-dynamic-linking Configure is successfully completed. 4. Compile # make It resulted in the following error. make all-recursive make[1]: Entering directory `/usr/local/src/syslog-ng-2.0.2' Making all in src make[2]: Entering directory `/usr/local/src/syslog-ng-2.0.2/src' source='misc.c' object='misc.o' libtool=no \ depfile='.deps/misc.Po' tmpdepfile='.deps/misc.TPo' \ depmode=gcc /bin/bash ../depcomp \ gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0-I/usr/local/lib/glib- 2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -c `test -f 'misc.c' || echo './'`misc.c In file included from /usr/local/include/glib-2.0/glib/galloca.h:30, from /usr/local/include/glib-2.0/glib.h:30, from syslog-ng.h:37, from misc.h:27, from misc.c:24: /usr/local/include/glib-2.0/glib/gtypes.h:30: glibconfig.h: No such file or directory /usr/local/include/glib-2.0/glib/gtypes.h:385: #error unknown ENDIAN type make[2]: *** [misc.o] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-2.0.2/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/syslog-ng-2.0.2' make: *** [all] Error 2 The file "glibconfig.h" exists at following path, but I have no idea whether they relate to syslog-ng. # find /usr -name glibconfig.h -exec grep {} /var/sadm/install/contents \; /usr/lib/sparcv9/glib-2.0/include/glibconfig.h f none 0644 root other 4636 35107 1052884337 SUNWgnome-base-libs-64 /usr/lib/glib-2.0/include/glibconfig.h f none 0644 root other 4670 37361 1052873864 SUNWgnome-base-libs-devel /usr/sfw/lib/glib/include/glibconfig.h f none 0644 root bin 3603 16163 1015056352 SUNWGlib Please help!!! Thanks! Regards George

2 1

archiving logfiles etc.
by Per Jessen 26 Mar '07

26 Mar '07

Not exactly specific to syslog-ng, but I thought someone here might have a similar issue: We are legally obliged to keep certain logfiles archived for up to 3 years, some even longer. I was just wondering if there's a tool out there specifically suited for this sort of job? Years ago I used to use some software called "$AVRS" for archiving logs, output from batchjobs etc., but this was on MVS. It allowed me to later search the archives by date, jobname, logtype etc etc. Of course it also had automatic purge functionality and such. Does anything similar exist for Linux? /Per Jessen, Zürich

1 0

Syslog-NG and DNS
by Ivey, Chris 21 Mar '07

21 Mar '07

Bazsi, A few months ago, I posted some information here about syslog-ng doing WAY too many nslookups for forwarding. I remember you said you were going to patch the issue... can you update me on where we stand with that? It bit me again today on my other syslog-ng server. Thank you! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage

2 1