March 2007 - syslog-ng - lists.balabit.hu

syslog-ng and cisco's problem
by Rębelski, Michał 28 Mar '07

28 Mar '07

Hi all, I'm trying to set up a syslog-ng 2.0.0 on Red Hat Enterprise 4.0 i386 edition. I have already set up my /etc/syslog-ng.conf file in following way: [root@swlogg02 cisco]# cat /etc/syslog-ng.conf # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); }; source s_cisco { udp(ip(0.0.0.0) port(514)); }; destination d_cisco_d { file("/var/log/cisco/$HOST/$YEAR/$MONTH/$DAY/log"); }; destination d_cisco_c { file("/var/log/cisco/$HOST/$YEAR/$MONTH/$DAY/log"); }; filter f_cisco_d { level(debug); }; filter f_cisco_c { level(crit); }; log { source(s_cisco); filter(f_cisco_d); destination(d_cisco_d); }; log { source(s_cisco); filter(f_cisco_c); destination(d_cisco_c); }; but I only got logs from only one machine, for your information from ASA5520. I would liek to receive messages from almost 50 machines (routers/switches/firewalls, etc). As general I set up on firewalls critical level of logging and on router/switches debug. Could you please tell me what I did wrong? regards -- Michał Rębelski

1 0

Re: [syslog-ng] Log rotate without log rotate.
by G.W. Haywood 28 Mar '07

28 Mar '07

Hi there, On Wed, 28 Mar 2007 Bruno Vieira wrote: > ... In the project I'm in, I won't have the time to do it in a low > level (kernel) as you refer and as I also thought it would be great > if Linux already had a device like this (a rotation file with > parameters of lines and/or size). Have you looked at dmesg? -- 73, Ged.

1 0

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 28 Mar '07

28 Mar '07

In fact, you could even write this as a kernel module (which would give you direct access to the list.h kernel header) and implement this behaviour right under the hood of the OS rather than in Syslog-NG. Basically, using the Linux VFS and kernel module subsystems, you can create device files in which the "write" operations behave in the manner listed below (writing to a circular list implementation of X number of nodes). This would then allow Syslog-NG to continuously write to the device file, and let the device file's implementation operate in the ring buffer manner outlined below. I hope this helps, Justin. _____ From: Justin Randall Sent: Tuesday, March 27, 2007 5:47 PM To: 'Syslog-ng users' and developers' mailing list' Subject: RE: [syslog-ng] Log rotate without log rotate. Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 27 Mar '07

27 Mar '07

Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

problem in compiling "syslog-ng-2.0.2" on SunOS 5.9 sparc
by Root Administrator 27 Mar '07

27 Mar '07

Hi, I have been facing the problem in compiling "syslog-ng-2.0.2" on SunOS 5.9sparc. The steps I have followed are described below. Could anyone plz help troubleshooting? [Condition] "GLIB" package is that of pre-installed. The package information about "GLIB" is ; # pkginfo | grep -i glib system SUNWGlib GLIB - Library of useful routines for C programming [Steps] 1. Install "eventlog-0.2.5" that is required by syslog-ng 2.0.2. The source file is from http://www.balabit.com/ Installation was successfully completed. 2. Configure "syslog-ng 2.0.2" # ./configure --enable-dynamic-linking It resulted in the following error. checking for GLIB... Requested 'glib-2.0 >= 2.2' but version of GLib is 2.0.7 configure: error: Cannot find GLib library version >= 2.2: is pkg-config in path? It seems the version of "GLIB" is older. Thus, I installed newer version of GLIB "glib-2.6.2". The file is downloaded from the following URL, ftp://ftp.sunfreeware.com/pub/freeware/sparc/9/glib-2.6.2-sol9-sparc-local.… The package information is ; # pkginfo | grep -i glib application SMCglib glib system SUNWGlib GLIB - Library of useful routines for C programming 3. 2nd attempt of configure New package is installed in previous step so that I changed the PATH to make sure the installed package is first reached. # export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/ # ./configure --enable-dynamic-linking Configure is successfully completed. 4. Compile # make It resulted in the following error. make all-recursive make[1]: Entering directory `/usr/local/src/syslog-ng-2.0.2' Making all in src make[2]: Entering directory `/usr/local/src/syslog-ng-2.0.2/src' source='misc.c' object='misc.o' libtool=no \ depfile='.deps/misc.Po' tmpdepfile='.deps/misc.TPo' \ depmode=gcc /bin/bash ../depcomp \ gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0-I/usr/local/lib/glib- 2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -c `test -f 'misc.c' || echo './'`misc.c In file included from /usr/local/include/glib-2.0/glib/galloca.h:30, from /usr/local/include/glib-2.0/glib.h:30, from syslog-ng.h:37, from misc.h:27, from misc.c:24: /usr/local/include/glib-2.0/glib/gtypes.h:30: glibconfig.h: No such file or directory /usr/local/include/glib-2.0/glib/gtypes.h:385: #error unknown ENDIAN type make[2]: *** [misc.o] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-2.0.2/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/syslog-ng-2.0.2' make: *** [all] Error 2 The file "glibconfig.h" exists at following path, but I have no idea whether they relate to syslog-ng. # find /usr -name glibconfig.h -exec grep {} /var/sadm/install/contents \; /usr/lib/sparcv9/glib-2.0/include/glibconfig.h f none 0644 root other 4636 35107 1052884337 SUNWgnome-base-libs-64 /usr/lib/glib-2.0/include/glibconfig.h f none 0644 root other 4670 37361 1052873864 SUNWgnome-base-libs-devel /usr/sfw/lib/glib/include/glibconfig.h f none 0644 root bin 3603 16163 1015056352 SUNWGlib Please help!!! Thanks! Regards George

2 1

archiving logfiles etc.
by Per Jessen 26 Mar '07

26 Mar '07

Not exactly specific to syslog-ng, but I thought someone here might have a similar issue: We are legally obliged to keep certain logfiles archived for up to 3 years, some even longer. I was just wondering if there's a tool out there specifically suited for this sort of job? Years ago I used to use some software called "$AVRS" for archiving logs, output from batchjobs etc., but this was on MVS. It allowed me to later search the archives by date, jobname, logtype etc etc. Of course it also had automatic purge functionality and such. Does anything similar exist for Linux? /Per Jessen, Zürich

1 0

Syslog-NG and DNS
by Ivey, Chris 21 Mar '07

21 Mar '07

Bazsi, A few months ago, I posted some information here about syslog-ng doing WAY too many nslookups for forwarding. I remember you said you were going to patch the issue... can you update me on where we stand with that? It bit me again today on my other syslog-ng server. Thank you! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage

2 1

misspell in the v2.0 reference
by Πιστιόλης Κωνσταντίνος 21 Mar '07

21 Mar '07

Hi all, I've set up syslog-ng on my debian etch, to accept logs my remote hosts. I have noticed that the dns_cache_host() directive which is mentioned in http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.txt is not working. Instead, it works with dns_cache_hosts() It propably is a misspell error, but it took me 2 hours to figure out... Is this the right place to mention this? If not, where should I? Pistiolis Konstantinos

2 1

map "mark.info" in syslog.conf to syslog-ng.conf
by Lily Feng 20 Mar '07

20 Mar '07

Hi , I tried to map the syslog.conf to syslog-ng.conf via syslog2ng: mark.info It is mapped to filter f_1 { facility(mark) and level(info..emerg);} It could not pass the verification because facility(mark)is not correct. How to fix it? Thanks, Lily

3 6

RE: [syslog-ng] Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 19 Mar '07

19 Mar '07

Hi Kevin, Thanks a lot for the information & help. Is there a way to include 'user name' or userid from which the log generated to each log? I coudnt find the macro (eg: $HOST) for user name/uid. Thanks, Rajeesh -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of K K Sent: Saturday, March 10, 2007 1:18 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng log file overwrite On 3/9/07, Padmanabhan, Rajeesh (GE Healthcare) <Rajeesh.Padmanabhan(a)ge.com> wrote: > Hi Valdis, > > Thanks for the help. Could you please explain below questons? > > 1. Will the below format overwrites existing log file? Unless you specify otherwise (using remove_if_older), if a file exists, it is opened for appending instead of being overwritten. > file("/logs/$HOST/$YEAR/$MONTH/messages-$YEAR-$MONTH$DAY"); In the above case, $YEAR is always unique, the directory and files will always have a unique name. > 2. Is there a way to roate log file based on the size? No size-based rotation is built into syslog-ng. You could write a script, but I find it easier to just have syslog-ng create many smaller files, and then either compress or purge these files using a cron job. > Idea is I don't want my log partition gets filled & alerts due to > lacks of space :) Running out of disk space is a risk. What I do is specify the destination like this: file("/var/log/remote/$HOST/$WEEKDAY/$HOUR" remove_if_older(259200) sync(60) ); Additionally, I have a cron job to compress older logs by as much as 98%: @midnight find /var/log/remote -type f -name '[0-2][0-9]' -ctime +3 -print0 |xargs -r -0 bzip2 -f Kevin _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

3 2