- syslog-ng - lists.balabit.hu

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 28 Mar '07

28 Mar '07

In fact, you could even write this as a kernel module (which would give you direct access to the list.h kernel header) and implement this behaviour right under the hood of the OS rather than in Syslog-NG. Basically, using the Linux VFS and kernel module subsystems, you can create device files in which the "write" operations behave in the manner listed below (writing to a circular list implementation of X number of nodes). This would then allow Syslog-NG to continuously write to the device file, and let the device file's implementation operate in the ring buffer manner outlined below. I hope this helps, Justin. _____ From: Justin Randall Sent: Tuesday, March 27, 2007 5:47 PM To: 'Syslog-ng users' and developers' mailing list' Subject: RE: [syslog-ng] Log rotate without log rotate. Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

RE: [syslog-ng] Log rotate without log rotate.
by Justin Randall 27 Mar '07

27 Mar '07

Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh(a)giraffe-data.com > wrote: >I have a solution.. but I think there can be a better one. My solution is: >having a midlle program between syslog-ng and the destination file. And for >every message received, the program would verify if the destination file >(p.e. auth.log) already has 100 lines. If so, deleted the last line, make >all others one ENTER down and that append the new line to the beginning of >the file. It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

problem in compiling "syslog-ng-2.0.2" on SunOS 5.9 sparc
by Root Administrator 27 Mar '07

27 Mar '07

Hi, I have been facing the problem in compiling "syslog-ng-2.0.2" on SunOS 5.9sparc. The steps I have followed are described below. Could anyone plz help troubleshooting? [Condition] "GLIB" package is that of pre-installed. The package information about "GLIB" is ; # pkginfo | grep -i glib system SUNWGlib GLIB - Library of useful routines for C programming [Steps] 1. Install "eventlog-0.2.5" that is required by syslog-ng 2.0.2. The source file is from http://www.balabit.com/ Installation was successfully completed. 2. Configure "syslog-ng 2.0.2" # ./configure --enable-dynamic-linking It resulted in the following error. checking for GLIB... Requested 'glib-2.0 >= 2.2' but version of GLib is 2.0.7 configure: error: Cannot find GLib library version >= 2.2: is pkg-config in path? It seems the version of "GLIB" is older. Thus, I installed newer version of GLIB "glib-2.6.2". The file is downloaded from the following URL, ftp://ftp.sunfreeware.com/pub/freeware/sparc/9/glib-2.6.2-sol9-sparc-local.… The package information is ; # pkginfo | grep -i glib application SMCglib glib system SUNWGlib GLIB - Library of useful routines for C programming 3. 2nd attempt of configure New package is installed in previous step so that I changed the PATH to make sure the installed package is first reached. # export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/ # ./configure --enable-dynamic-linking Configure is successfully completed. 4. Compile # make It resulted in the following error. make all-recursive make[1]: Entering directory `/usr/local/src/syslog-ng-2.0.2' Making all in src make[2]: Entering directory `/usr/local/src/syslog-ng-2.0.2/src' source='misc.c' object='misc.o' libtool=no \ depfile='.deps/misc.Po' tmpdepfile='.deps/misc.TPo' \ depmode=gcc /bin/bash ../depcomp \ gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0-I/usr/local/lib/glib- 2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -c `test -f 'misc.c' || echo './'`misc.c In file included from /usr/local/include/glib-2.0/glib/galloca.h:30, from /usr/local/include/glib-2.0/glib.h:30, from syslog-ng.h:37, from misc.h:27, from misc.c:24: /usr/local/include/glib-2.0/glib/gtypes.h:30: glibconfig.h: No such file or directory /usr/local/include/glib-2.0/glib/gtypes.h:385: #error unknown ENDIAN type make[2]: *** [misc.o] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-2.0.2/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/syslog-ng-2.0.2' make: *** [all] Error 2 The file "glibconfig.h" exists at following path, but I have no idea whether they relate to syslog-ng. # find /usr -name glibconfig.h -exec grep {} /var/sadm/install/contents \; /usr/lib/sparcv9/glib-2.0/include/glibconfig.h f none 0644 root other 4636 35107 1052884337 SUNWgnome-base-libs-64 /usr/lib/glib-2.0/include/glibconfig.h f none 0644 root other 4670 37361 1052873864 SUNWgnome-base-libs-devel /usr/sfw/lib/glib/include/glibconfig.h f none 0644 root bin 3603 16163 1015056352 SUNWGlib Please help!!! Thanks! Regards George

2 1

archiving logfiles etc.
by Per Jessen 26 Mar '07

26 Mar '07

Not exactly specific to syslog-ng, but I thought someone here might have a similar issue: We are legally obliged to keep certain logfiles archived for up to 3 years, some even longer. I was just wondering if there's a tool out there specifically suited for this sort of job? Years ago I used to use some software called "$AVRS" for archiving logs, output from batchjobs etc., but this was on MVS. It allowed me to later search the archives by date, jobname, logtype etc etc. Of course it also had automatic purge functionality and such. Does anything similar exist for Linux? /Per Jessen, Zürich

1 0

Syslog-NG and DNS
by Ivey, Chris 21 Mar '07

21 Mar '07

Bazsi, A few months ago, I posted some information here about syslog-ng doing WAY too many nslookups for forwarding. I remember you said you were going to patch the issue... can you update me on where we stand with that? It bit me again today on my other syslog-ng server. Thank you! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage

2 1

misspell in the v2.0 reference
by Πιστιόλης Κωνσταντίνος 21 Mar '07

21 Mar '07

Hi all, I've set up syslog-ng on my debian etch, to accept logs my remote hosts. I have noticed that the dns_cache_host() directive which is mentioned in http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.txt is not working. Instead, it works with dns_cache_hosts() It propably is a misspell error, but it took me 2 hours to figure out... Is this the right place to mention this? If not, where should I? Pistiolis Konstantinos

2 1

map "mark.info" in syslog.conf to syslog-ng.conf
by Lily Feng 20 Mar '07

20 Mar '07

Hi , I tried to map the syslog.conf to syslog-ng.conf via syslog2ng: mark.info It is mapped to filter f_1 { facility(mark) and level(info..emerg);} It could not pass the verification because facility(mark)is not correct. How to fix it? Thanks, Lily

3 6

RE: [syslog-ng] Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 19 Mar '07

19 Mar '07

Hi Kevin, Thanks a lot for the information & help. Is there a way to include 'user name' or userid from which the log generated to each log? I coudnt find the macro (eg: $HOST) for user name/uid. Thanks, Rajeesh -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of K K Sent: Saturday, March 10, 2007 1:18 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng log file overwrite On 3/9/07, Padmanabhan, Rajeesh (GE Healthcare) <Rajeesh.Padmanabhan(a)ge.com> wrote: > Hi Valdis, > > Thanks for the help. Could you please explain below questons? > > 1. Will the below format overwrites existing log file? Unless you specify otherwise (using remove_if_older), if a file exists, it is opened for appending instead of being overwritten. > file("/logs/$HOST/$YEAR/$MONTH/messages-$YEAR-$MONTH$DAY"); In the above case, $YEAR is always unique, the directory and files will always have a unique name. > 2. Is there a way to roate log file based on the size? No size-based rotation is built into syslog-ng. You could write a script, but I find it easier to just have syslog-ng create many smaller files, and then either compress or purge these files using a cron job. > Idea is I don't want my log partition gets filled & alerts due to > lacks of space :) Running out of disk space is a risk. What I do is specify the destination like this: file("/var/log/remote/$HOST/$WEEKDAY/$HOUR" remove_if_older(259200) sync(60) ); Additionally, I have a cron job to compress older logs by as much as 98%: @midnight find /var/log/remote -type f -name '[0-2][0-9]' -ctime +3 -print0 |xargs -r -0 bzip2 -f Kevin _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

3 2

log statistics level
by bryanh＠giraffe-data.com 19 Mar '07

19 Mar '07

When Syslog-ng logs its hourly statistics, it uses level NOTICE. This seems like a classic case for INFO instead. NOTICE is an event of which someone might want to take notice, e.g. "nightly backup has started" or "RAID rebuild is complete." INFO is just information someone might want to refer to later. -- Bryan Henderson San Jose, California

4 7

Syslog-ng fails to listen at port 514
by Vivek Raju 19 Mar '07

19 Mar '07

Hi, I am trying to implement a centralized syslog-ng solution for testing purpose. My configuration is SLES10 on mainframe guest. I had followed the steps given in the article below. http://www.novell.com/coolsolutions/feature/18044.html I have created the configuration on server # # uncomment to process log messages from network: udp(port(514)); tcp(ip("192.168.1.42") port(5140) keep-alive(yes)); # this is for separating out network hosts into individual log files. destination std { file ("/var/log/HOSTS/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(y\es) ); }; log { source(src); destination(std); }; I then ran SuSEconfig --module syslog-ng. After that i started the syslog service thru /etc/init.d On the client i have added these lines to /etc/syslog-ng/syslog-ng.conf.in # send everything to log host destination loghost { tcp("192.168.1.42" port(5140)); }; log { source(src); destination(loghost); My Problem is when i do netstat -an|grep 514 or netstat -an|grep 5140 nothing comes up. on the client side i see errors in the /var/log/messages stating "error connecting to remote system" I am not sure but i think the problem is because of UDP 514 or TCP 5140 not being working. I checked the /etc/services and looks fine. has anyone had same kind of problem. Would be glad if someone could help in determining the cause. Vick };

2 1

Where is the download link for 2.0 ?
by Dave Augustus 19 Mar '07

19 Mar '07

Everyplace on http://www.balabit.com I try just leads me to http://www.balabit.com/downloads/syslog-ng/2.0/ Thanks, Dave

3 2

Log rotate without log rotate.
by Bruno Vieira 17 Mar '07

17 Mar '07

Hi there. I have a question and at the same time i almost know that it must exist a way of doing this. The situation is: I want that my logs act as FIFOs. I mean.. I want my log files to have a max of 100 KB or 100 lines (the one that is most convenient to find a solution). If I use a pipe, I have the problem that if I read on one side of the pipe, it continues dumping it's content until getting empty regardless of having more messages coming in through the other end. But what I want is a pipe that that keeps it's content and only flushes a message out every time it receives a message in. Just like a FIFO.. I want this behaviour in a file... named pipe? I don't know the solution for this but I can feel that it is out there :) This way I would have a log rotation without using log rotate that normally it copies the file content to a new one and than syslog-ng starts writing on an empty one. Tell me what you think about this. Best regards, Bruno.

4 5

Re: Log rotate without log rotate.
by G.W. Haywood 16 Mar '07

16 Mar '07

Hi there, On Fri, 16 Mar 2007 Bruno Vieira wrote: > ... the problem here is that the device that will be logging have memory > restrictions. > So, I want a log that acts as a fifo. And every time a new message comes, > the oldest one gets out. > ... > I would like to have only one file per destination (ex: auth.log) and keep > that file within 100 - 120 KB or 100 lines, per example. I want it to be > circular... to be like a FIFO. Why don't you just log to a remote host? > I have a solution.. but I think there can be a better one. My solution is: > having a midlle program between syslog-ng and the destination file. And for > every message received, the program would verify if the destination file ( > p.e. auth.log) already has 100 lines. If so, deleted the last line, make all > others one ENTER down and that append the new line to the beginning of the > file. Don't do it like that. Your 'middle program' might bring the machine to its knees if syslog gets a lot of messages. You could write a script which stores 100 lines in a buffer and deletes the oldest line every time a new line comes in, and make it respond to a signal (for example SIGHUP) by writing its entire buffer to a screen or to a file. > But this seems to be a very "manual" solution... It is. -- 73, Ged.

1 0

Syslog-ng and network failover
by Jonathon Blumenthal 16 Mar '07

16 Mar '07

I need to have a highly available centralized log server (all Linux). Is there a way to configure syslog-ng to automatically detect when the remote syslog server has gone down so that it can redirect messages elsewhere? Something like this: destination d_tcp1 { tcp("192.168.1.1"); failover(d_tcp2); }; destination d_tcp2 { tcp("192.168.1.2"); }; If not, anyone know of any Linux loggers that will do this? -Jon ----------------------------------------- The information contained in this email message and its attachments is intended only for the private and confidential use of the recipient(s) named above, unless the sender expressly agrees otherwise. Transmission of email over the Internet is not a secure communications medium. If you are requesting or have requested the transmittal of personal data, as defined in applicable privacy laws by means of email or in an attachment to email, you must select a more secure alternate means of transmittal that supports your obligations to protect such personal data. If the reader of this message is not the intended recipient and/or you have received this email in error, you must take no action based on the information in this email and you are hereby notified that any dissemination, misuse or copying or disclosure of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email and delete the original message.

6 5

Re: log statistics level
by G.W. Haywood 11 Mar '07

11 Mar '07

Hi there, FWIW I'd be happy if nothing changed. I use syslog-ng to pipe messages to scripts and I'd hate for things to break because of some log message getting into the wrong file. Besides, I'm used to looking at things the way they are. Just my 0.02p -- 73, Ged.

1 0

RE: [syslog-ng] Syslog-ng log file overwrite
by Justin Randall 09 Mar '07

09 Mar '07

You can have logrotate entries like the following: # rotate some Logs /logdirectory/messages { weekly rotate 12 compress postrotate /usr/bin/pkill -HUP syslog endscript } What this does is simply rotate the log, and then send HUP to Syslog-NG to stop it from writing to the current inode which points to the recently rotated logfile. This specific configuration will rotate weekly, keeping 12 compressed revisions using GZIP. I hope this helps, Justin. _____ From: Padmanabhan, Rajeesh (GE Healthcare) [mailto:Rajeesh.Padmanabhan@ge.com] Sent: Friday, March 09, 2007 6:16 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Syslog-ng log file overwrite Basically what i am trying to configure is log rotation of Syslog-ng files.. _____ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Padmanabhan, Rajeesh (GE Healthcare) Sent: Friday, March 09, 2007 3:52 PM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] Syslog-ng log file overwrite Hi Bazsi, Do syslog-ng overwrites log file by default? If yes, what is the process/strategy done by syslog-ng to overwrite the file? Thanks, Rajeesh

1 0

RE: [syslog-ng] Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 09 Mar '07

09 Mar '07

Hi Valdis, Thanks for the help. Could you please explain below questons? 1. Will the below format overwrites existing log file? file("/logs/$HOST/$YEAR/$MONTH/messages-$YEAR-$MONTH$DAY"); 2. Is there a way to roate log file based on the size? Idea is I don't want my log partition gets filled & alerts due to lacks of space :) Please help. -Rajeesh -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Valdis.Kletnieks(a)vt.edu Sent: Saturday, March 10, 2007 12:22 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng log file overwrite On Fri, 09 Mar 2007 16:46:18 +0530, "Padmanabhan, Rajeesh (GE Healthcare)" said: > Basically what i am trying to configure is log rotation of Syslog-ng > files. You can always cheat, like this: file("/logs/$HOST/$YEAR/$MONTH/messages-$YEAR-$MONTH$DAY"); Voila! instant auto-magic rotation at midnight. :)

2 1

RE: [syslog-ng] Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 09 Mar '07

09 Mar '07

Basically what i am trying to configure is log rotation of Syslog-ng files.. _____ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Padmanabhan, Rajeesh (GE Healthcare) Sent: Friday, March 09, 2007 3:52 PM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] Syslog-ng log file overwrite Hi Bazsi, Do syslog-ng overwrites log file by default? If yes, what is the process/strategy done by syslog-ng to overwrite the file? Thanks, Rajeesh

2 1

Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 09 Mar '07

09 Mar '07

Hi Bazsi, Do syslog-ng overwrites log file by default? If yes, what is the process/strategy done by syslog-ng to overwrite the file? Thanks, Rajeesh

1 0

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

Unable to compile version 1.6.12 on Solaris 9
by roel＠raphor.net 08 Mar '07

08 Mar '07

I'm trying to compile syslog-ng-1.6.12 on Solaris 9. This aborts with the following error: gcc -g -O2 -Wall -I/usr/local/src/syslog-ng-1.6.12/libol-0.3.18/src -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o macros.o getopt.o getopt1.o -lpthread -lrt -lnsl -lsocket -ldoor -lresolv -Wl,-Bstatic /usr/local/src/syslog-ng-1.6.12/libol-0.3.18/src/.libs/libol.a -lsocket -lnsl -lxnet -ll -Wl,-Bdynamic ld: fatal: library -lxnet: not found ld: fatal: File processing errors. No output written to syslog-ng collect2: ld returned 1 exit status make[3]: *** [syslog-ng] Error 1 make[3]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make: *** [all-recursive] Error 1 Whereas syslog-ng-1.6.11 compiles without any problems. Any suggestions? Cheers, Roel

3 2

root logout is not tracked by syslog-ng
by Padmanabhan, Rajeesh (GE Healthcare) 08 Mar '07

08 Mar '07

Hi All, I have setup syslog-ng 2.0.1 on both Solaris & linux. I noticed that, syslog-ng is not tracking root log out over ssh or telnet. But other normal user log outs are tracked properly. Please help me to know if any configuration change is required to audit this. Thanks, Rajeesh

2 1

swatch in syslog-ng
by R. V. C. 07 Mar '07

07 Mar '07

List, I am trying (in vain) to get swatch to work with syslog-ng. Here is a list of my system specifics: Freebsd 6.2 syslog-ng-1.6.11 swatch-3.1.1_1 stunnel-4.20 mysql-client-5.1.14 mysql-server-5.1.14 Ok so here is the problem. I can get swatch to read a file via the tail-file option: destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/all.log"); }; This works fine. However when I try to use the read-pipe option, nothing ever comes out: destination swatch { program("/usr/local/bin/swatch --read-pipe=\"cat /dev/fd/0\""); }; I even tried using my mysql.pipe and creating a fifo pipe (swatch.pipe) just for swatch: destination swatch.pipe { pipe("/var/log/swatch.pipe" template("$FULLDATE $HOST $FACILITY:$PRIORITY $MSG\n")); }; destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/swatch.pipe"); }; No dice. My log statement is the same for all of them: log { source(logserver); filter(f_no_stats); destination(swatch); }; with logserver being: source logserver { tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); }; Which runs over stunnel. Everything else works fine, dumping to mysql, or to a flat file. I tried putting all the destinations on one line, and then separating them out still no dice. From what I've read on the web this is pretty easy to setup. Can anyone tell me what I am doing wrong. Thanks in advance. I am including a copy of my syslog-ng.conf file: # # FreeBSD /etc/syslog.conf file. # # # options # options { long_hostnames(off); keep_hostname(yes); bad_hostname("gconfd"); bad_hostname("^(ctld.|cmd|tmd|last)$"); log_fifo_size(4096); use_dns(yes); dns_cache(yes); time_reopen(10); stats(3600); sync(0); }; # # sources # source localhost { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); #udp(); #tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); internal(); file("/dev/klog"); }; source logserver { tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); }; # # destinations # destination messages { file("/var/log/messages"); }; destination security { file("/var/log/security"); }; destination authlog { file("/var/log/auth.log"); }; destination maillog { file("/var/log/maillog"); }; destination lpd-errs { file("/var/log/lpd-errs"); }; destination xferlog { file("/var/log/xferlog"); }; destination cron { file("/var/log/cron"); }; destination debuglog { file("/var/log/debug.log"); }; destination consolelog { file("/var/log/console.log"); }; destination all { file("/var/log/all.log"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; destination slip { file("/var/log/slip.log"); }; destination ppp { file("/var/log/ppp.log"); }; destination console { file("/dev/console"); }; destination allusers { usertty("*"); }; # Custom destinations destination mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY' , '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; destination archive { file("/var/log/servers/$HOST/$YEAR/$MONTH/$DAY/$FACILITY/$HOST-$FACILITY.$ YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; destination swatch { program("/usr/local/bin/swatch --read-pipe=\"cat /dev/fd/0\""); }; #destination swatch { program("/usr/local/bin/swatch --config-file=\"/root/.swatchrc\" --read-pipe=\"/bin/cat /dev/fd/0 \""); }; #destination swatch.pipe { pipe("/var/log/swatch.pipe" template("$FULLDATE $HOST $FACILITY:$PRIORITY $MSG\n")); }; #destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/all.log"); }; #destination sec { program("/usr/local/bin/sec -input=\"-\" -conf=/usr/local/etc/sec/general.sec"); }; # # log facility filters # filter f_auth { facility(auth); }; filter f_authpriv { facility(authpriv); }; filter f_not_authpriv { not facility(authpriv); }; filter f_console { facility(console); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_ftp { facility(ftp); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_security { facility(security); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; # # log level filters # filter f_emerg { level(emerg); }; filter f_alert { level(alert..emerg); }; filter f_crit { level(crit..emerg); }; filter f_err { level(err..emerg); }; filter f_warning { level(warning..emerg); }; filter f_notice { level(notice..emerg); }; filter f_info { level(info..emerg); }; filter f_debug { level(debug..emerg); }; filter f_is_debug { level(debug); }; # # program filters # filter f_ppp { program("ppp"); }; filter f_slip { program("startslip"); }; # # custom filters # filter f_no_stats { not match("STATS: dropped 0"); }; # # *.err;kern.warning;auth.notice;mail.crit /dev/console # log { source(localhost); filter(f_err); destination(console); }; log { source(localhost); filter(f_kern); filter(f_warning); destination(console); }; log { source(localhost); filter(f_auth); filter(f_notice); destination(console); }; log { source(localhost); filter(f_mail); filter(f_crit); destination(console); }; # # *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages # log { source(localhost); filter(f_notice); filter(f_not_authpriv); destination(messages); }; log { source(localhost); filter(f_kern); filter(f_debug); destination(messages); }; log { source(localhost); filter(f_lpr); filter(f_info); destination(messages); }; log { source(localhost); filter(f_mail); filter(f_crit); destination(messages); }; log { source(localhost); filter(f_news); filter(f_err); destination(messages); }; # # security.* /var/log/security # log { source(localhost); filter(f_security); destination(security); }; # # auth.info;authpriv.info /var/log/auth.log log { source(localhost); filter(f_auth); filter(f_info); destination(authlog); }; log { source(localhost); filter(f_authpriv); filter(f_info); destination(authlog); }; # # mail.info /var/log/maillog # log { source(localhost); filter(f_mail); filter(f_info); destination(maillog); }; # # lpr.info /var/log/lpd-errs # log { source(localhost); filter(f_lpr); filter(f_info); destination(lpd-errs); }; # # ftp.info /var/log/xferlog # log { source(localhost); filter(f_ftp); filter(f_info); destination(xferlog); }; # # cron.* /var/log/cron # log { source(localhost); filter(f_cron); destination(cron); }; # # *.=debug /var/log/debug.log # log { source(localhost); filter(f_is_debug); destination(debuglog); }; # # *.emerg * # log { source(localhost); filter(f_emerg); destination(allusers); }; # # uncomment this to log all writes to /dev/console to /var/log/console.log # console.info /var/log/console.log # log { source(localhost); filter(f_console); filter(f_info); destination(consolelog); }; # # Log to mysql and the archive directory # log { source(logserver); filter(f_no_stats); destination(mysql); destination(archive); }; # # Swatch # #log { source(logserver); filter(f_no_stats); destination(all); }; log { source(logserver); filter(f_no_stats); destination(swatch); }; # # Local server # log { source(localhost); filter(f_no_stats); destination(archive); destination(mysql); }; # # DEBUG # #log { source(localhost); destination(all); }; And a copy of my .swatchrc watchfor /fail/ mail=robert.coward.ctr(a)deploymenthealth.osd.mil echo exec echo $0 | mail -s\"notice\" $0 R. V. C. R. V. C.

3 2

Looking for syslog log analyzer
by Daniel Chojecki 07 Mar '07

07 Mar '07

Hello, I use syslog-ng 2.0.2 with MySQL. I'm looking for analyzer program for my system. I've searched through syslog-ng maillist archives, google and found no app that analyze logs stored in MySQL. -- Greetz Daniel

5 5

RE: [syslog-ng] Looking for syslog log analyzer
by Ivey, Chris 07 Mar '07

07 Mar '07

What are you trying to accomplish with analysis of your logs stored in MySQL? Give us a clear definition of what you are looking to do with the logs, and someone on the list may be able to provide you with some help that is properly applicable. Thanks! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage _____ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Daniel Chojecki Sent: Tuesday, March 06, 2007 3:25 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Looking for syslog log analyzer Hello, I use syslog-ng 2.0.2 with MySQL. I'm looking for analyzer program for my system. I've searched through syslog-ng maillist archives, google and found no app that analyze logs stored in MySQL. -- Greetz Daniel

1 0