- syslog-ng - lists.balabit.hu

RE: [syslog-ng] freebsd syslog-ng error
by William Bell 20 Sep '05

20 Sep '05

The problem you are having is in attempting to use linux conventions in freebsd. You will need to use /var/run/log in place of /dev/log and instead of /proc/kmsg you will most likely have to use file("/dev/klog"); Changing these two will get you the results you need. -William B -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of fico gid Sent: Tuesday, September 20, 2005 2:10 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] freebsd syslog-ng error Hi there, im trying to setup a syslogng server to collect logs from a few of my routers. I have installed syslogng and modified the script as below : --start-- ## syslog-ng source remote { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); udp(ip(158.0.1.1) port(514)); tcp(ip(158.0.1.1) port(514) keep-alive(yes)); }; destination std { file("/var/log/hosts/$HOST/$YEAR/$MONTH/$DAY/$FACILITY" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; log { source(remote); destination(std); }; destination authlog { file("/var/log/auth.log"); }; destination cron { file("/var/log/cron.log"); }; destination debug { file("/var/log/debug.log"); }; destination kern { file("/var/log/kern.log"); }; destination user { file("/var/log/userlog"); }; --end-- somehow when i start the syslog-ng , it doesnt work and gives the error below : io.c: bind_unix_socket(): bind failed /dev/log (Operation not supported) Error initializing configuration, exiting. The syslog-ng by default config works fine. What am i doing wrong here ? fico _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

freebsd syslog-ng error
by fico gid 20 Sep '05

20 Sep '05

Hi there, im trying to setup a syslogng server to collect logs from a few of my routers. I have installed syslogng and modified the script as below : --start-- ## syslog-ng source remote { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); udp(ip(158.0.1.1) port(514)); tcp(ip(158.0.1.1) port(514) keep-alive(yes)); }; destination std { file("/var/log/hosts/$HOST/$YEAR/$MONTH/$DAY/$FACILITY" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; log { source(remote); destination(std); }; destination authlog { file("/var/log/auth.log"); }; destination cron { file("/var/log/cron.log"); }; destination debug { file("/var/log/debug.log"); }; destination kern { file("/var/log/kern.log"); }; destination user { file("/var/log/userlog"); }; --end-- somehow when i start the syslog-ng , it doesnt work and gives the error below : io.c: bind_unix_socket(): bind failed /dev/log (Operation not supported) Error initializing configuration, exiting. The syslog-ng by default config works fine. What am i doing wrong here ? fico

1 0

feature request: Priority remap
by Evan Rempel 16 Sep '05

16 Sep '05

It would be nice to be able to use syslong-ng as a filter from the syslog sources to a final syslog server, but in doing so, change the priority of a message based on some filter expressions. As an example, I have an application that does not support syslog. It can log all of its messages to its standard out, so it is easy to use application | logger -p user.info -t application but this assigns the same priority to all messages. I would like to run this through a syslong-ng server to match expressions for -t application and then regular expressions and send them on to the production syslog server with level of error, critical, warning, info and debug. An option for a TCP/UDP destination that would set the facility/level of the messages would be nice. Thanks, -- Evan Rempel erempel(a)uvic.ca Senior Programmer Analyst 250.721.7691 Computing Services University of Victoria

2 5

Re: FW: [syslog-ng] syslog-ng 1.6.8 bug: not maintaining priority
by SheBang 15 Sep '05

15 Sep '05

-----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu > [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Rempel > Sent: Thursday, September 15, 2005 10:51 AM > To: Syslog-ng users' and developers' mailing list > Subject: [syslog-ng] syslog-ng 1.6.8 bug: not maintaining priority > > > If I use the template option for TCP or UDP destinations, the priority > of a message is always user.notice. > > <snip> > > and I got the following in the /var/log/syslog-ng file > > Sep 15 10:38:51 cashmere.comp.uvic.ca <http://cashmere.comp.uvic.ca> user > notice ftp alert From > cashmere.comp.uvic.ca <http://cashmere.comp.uvic.ca>: sysprog: everything > is wrong > > Notice that the udp sent message contains the original ftp.alert > priority, but when the message is received via UDP, the message has > priority user.notice. > > Am I doing something wrong? > > Check out the detailed syslog explanation at: http://www.campin.net/syslog-ng/syslog.html Since your message isn't "<PRI>HEADER MSG" it's being rewriten in accordance with standard syslog practice and RFC3164. I didn't know syslog-ng doesn't put a <PRI> field in the beginning no matter what, but it makes sense that it wouldn't when you're specifying the format yourself. You'll have to add it. Problem is, I don't know that you can. Going by what's on the FAQ: $TAG Hex representaiton of the 32-bit priority/facility pair (see /usr/include/sys/syslog.h) 1f You can't use a hex number there, it needs to be decimal. I don't see a macro that we can use to properly set the decimal <PRI> in the beginning of a templated message. Bazsi, can you shed any light on this? BTW, I think most formatting questions can be answered by people on their own after they read that syslog explanation page.

2 2

Re: ng-syslog logging in a stealth mode
by Albretch Mueller 15 Sep '05

15 Sep '05

OK, here is my 'take' on your replies. It took me a little time to go over yor points // __ SheBang >Great idea, but running in stealth mode isn't a function of syslog-ng, it's >a function of a network capture tool such as tcpdump or ngrep. : I meant 'stealth mode' in a literal way not as an option or working mode of ngsyslog // __ >dump the syslog contents to a file (need to do a little scripting here >perhaps, though not much) and have syslog-ng read from that file. : I won't have a filesystem around or say the application is botting and running off a CD-ROM (+RAM) and a ramdisk (or something similar) is NOT vorhanden // __ >An HTML attachment was scrubbed... >URL: >http://lists.balabit.hu/pipermail/syslog-ng/attachments/20050913/a8e80311/a… // __ Jens Grigel >For the kernel messages you could use the netconsole kernel module to send >any kernel messages directly via udp to a specified client as soon >as the network is available, no need for running a local syslogd/klogd. >Check kernel/Documentation/networking/Netconsole.txt for details. : I was interested in reading about but the only page I could find was in japanese. http://www.linux.or.jp/JF/JFdocs/kernel-docs-2.6/networking/netconsole.txt.… Also the idea is NOT to "send any kernel messages directly via udp to a specified client", because to send it 'directly' to an IP/client you must declare this IP somehow and I everything shoudl be stealth. The log aggregator should only gobble whatever reaches its NIC, without the log producing end being even aware of it (do you get it ;-)) // __ Bill Nash >You mean.. >Log all activity on the box and forward it off to another device, storing >nothing locally? : No, I don't. I am not trying to store it locally and then forward it ... // __ SheBang >Ah, I see. I did't read carefully past "stealth mode" it seems. I was >thinking of this: >http://www.linuxjournal.com/xstatic/articles/lj/0092/5476/5476s2.html >http://www.linuxjournal.com/article/6222 >It's a hidden syslog server that's not attackable by common methods (well >except flooding with log messages - hard to eliminate DoS risks with any >service). If you snip the send pair in its ethernet cable and hardcode MAC >addresses then it's quite a secure log receiver. : Exactly! this is the idea "log files to a non-IP-addressed" ... Thanks Albretch

2 1

ng-syslog logging in a stealth mode
by Albretch Mueller 13 Sep '05

13 Sep '05

Hi *, I would like for system logs like the ones produced by the kernel, iptable (generally in /var/log/syslog), as well as anyother applications running in a Linux-based router to be processed by an ng-syslog client and just popped as UDP packets I looked into http://www.campin.net/syslog-ng/faq.html and couldn't see any particular info on this specifically and I also search http://marc.theaimsgroup.com/?l=syslog-ng for 'stealth' and didn't get any hits (a search on 'UDP' would dump millions of hits on you ;-)) How could you do something like that? Thanks Albretch

5 5

Feature request: cleaned-up PROGRAM macro?
by Jason Haar 13 Sep '05

13 Sep '05

Hi there As part of last weeks discussion over how to scale a searchable Enterprise Syslog environment, I've been wondering how far I could get using syslog-ng's built in functions - such as the templates. So I think it could be useful to be able to filter syslog messages into the following structure - as it gives an immediate "sorted" view: destination d_dir_messages { file("/var/log/syslog/$HOST/$YEAR/$MONTH/$DAY/$PROGRAM/raw" template("$R_ISODATE $MSGONLY\n") ...etc }; You can see you could now search quickly (well, quicker ;-) by date, syslog host and program. Symlinks could be used to create "meta directories" of related terms - e.g. an "Email/" dir could contain symlinks to all dirs containing $PROGRAM dirs called "smtpd", "qmail", "postfix", "msexchange", etc. However - there's a big nasty warning on PROGRAM - it's raw. So it could contain '/', '`' - anything. So I went to do it "the right way" and create my own "program" destination device that would clean things up and create "sanitized" directories. But it's too hard! I have to take on all the open file management that syslog-ng automatically does for me, etc. Either than or resort to "open-write-close" type thingies - yucky :-( So my feature request would be that there be a CLEANPROGRAM macro - or the like. Something that simply removes nasty chars such as "`|;'"\/}]{[&*#!" would probably do. The better option would to declare what was valid - but then you'd run into the whole Unicode/non-ASCII thing - just what is "invalid" these days? Thanks for listening -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

1 1

filter by date?
by scarter＠micron.com 12 Sep '05

12 Sep '05

hello, Is it possible to setup up a filter match based on variables for date? example thinking... filter f_date { match($YEAR-$MONTH-$DAY.*); }; #log the files log { source (s_udp); filter(f_date); destination(messages); }; # set it up destination hosts { file("/syslog/NT-SYSLOG/HOSTS/$HOST/$MONTH-$DAY-$YEAR" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); };

2 1

Syslog-ng Server IP address change
by Gerardo Amaya 08 Sep '05

08 Sep '05

Hello all. I configured a Syslog-ng server in a local LAN to make sure all the options were working with no problem at all. Then I moved the server to a different LAN. I made all the changes needed for the linux box for the IP address change, but when syslog-ng start sessions it appear with the previous IP address, I'm sure that this session is from syslog-ng because a use a random port UDP 5002 and it appears the session as 192.168.4.2:5002 192.168.4.101:4342 where 192.168.4.2 is the previous IP address. What changes do I need to make to the syslog-ng in order to use the new IP Address? I hope reinstall is the last resource. thanks in advance Gerardo Amaya

2 1

Anyone got a well performing search interface for syslog data?
by Jason Haar 08 Sep '05

08 Sep '05

We're generating around 4Gb syslog data per week, and I'm looking for a good search interface into it. I can cut my way through it with egrep/etc, but waiting 10-15min for a result really isn't going to break any speed records. Especially when I then need to re-run it with another "grep" on the end of it! ;-) I have tried injecting it into a MySQL database using some schemas I've found on the Internet - but the performance didn't seem much better to me - and you lost the "free-text" attributes of grep (or more specifically, the sorts of searches I find I want to do aren't SQL-friendly). Has anyone come up with a good speedy way of coping with Gbytes of syslog data? Or is it time to invest in some Appliance or the like? -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

7 9