September 2005 - syslog-ng - lists.balabit.hu

Re: Re: [syslog-ng] I have a question
by 赵武 21 Sep '05

21 Sep '05

Roberto Nibali,您好！ I am glad to receive your reply . Thank you very much . I am so surprised that you know the chinese idiom " 滴水穿石 di shui chuan shi ". Are you a chinese ? hehe ======= 2005-09-20 16:36:21 您在来信中写道：======= >Dear John, > >> I used syslog-ng_1.6.x , there has a startup script (maybe init.d.RedHat-7.3 ) >> in install directory . So I can start or stop it very easy . But in syslog-ng_1.9.5 , >> I haven't found the startup script , there is not convenience. I used logrotate.d >> separate log files . when the logrotate.d finished , it will restart the syslog-ng >> service . if there haven't startup script , I must added a script in logrotate.conf : > >Why not use the syslog-ng-1.6.x startup script? > >> postrotate >> /usr/bin/killall -HUP /usr/local/syslog-ng/sbin/syslog-ng 2> /dev/null || true >> endscript >> >> above script always killed the syslog-ng process and not startup syslog-ng process .I don't know the reason. > >This indeed is a bit strange as -HUP should not kill the process per se >with regard to syslog-ng. I've just tested this and and if I do it by >hand the syslog-ng process is indeed killed by this, although not all >the time. Some short time digging reveiled following: > >webphish:~ # ulimit -c unlimited >webphish:~ # /sbin/syslog-ng-1.9.5 -v -d -d -F > >[in another terminal send a /usr/bin/killall -HUP /sbin/syslog-ng-1.9.5] > >Segmentation fault (core dumped) >webphish:~ # gdb -q /sbin/syslog-ng-1.9.5 ./core >Using host libthread_db library "/lib/tls/libthread_db.so.1". >Core was generated by `/sbin/syslog-ng-1.9.5 -v -d -d -F'. >Program terminated with signal 11, Segmentation fault. >#0 0x08065009 in g_string_truncate () >(gdb) bt >#0 0x08065009 in g_string_truncate () >#1 0x08065fbc in g_string_assign () >#2 0x0804f85e in log_source_group_queue (s=0x8116988, msg=0x810e290, > path_flags=0) at sgroup.c:110 >#3 0x0804a878 in log_pipe_forward_msg (self=0x0, msg=0x81113d8, >path_flags=0) > at logpipe.h:72 >#4 0x080533ea in log_source_queue (s=0x8116540, msg=0x81113d8, >path_flags=0) > at logpipe.h:72 >#5 0x08052dc3 in afinter_source_dispatch_msg (pipe=0x0, msg=0x81113d8) > at logpipe.h:72 >#6 0x08052df3 in afinter_source_dispatch (source=0x8116568, > callback=0x8052db0 <afinter_source_dispatch_msg>, user_data=0x8116540) > at afinter.c:51 >#7 0x08058ef8 in g_main_context_dispatch () >#8 0x0805b0e7 in g_main_context_iterate () >#9 0x0805b1ba in g_main_context_iteration () >#10 0x0804854f in main_loop_run (cfg=0x81163c0) at main.c:142 >#11 0x0804882a in main (argc=5, argv=0xbffff344) at main.c:387 >(gdb) q >webphish:~ # > >No time to further dig into this yet. > >> when i used syslog-ng_1.6.x , I can use a easy script instead of above script : >> >> postrotate >> service syslog-ng restart >> endscript > >Check out the contrib directory of the syslog-ng-1.6.x source tree and >there you'll find all the startup scripts needed. They should also work >for the 1.9.x drop of syslog-ng. > >Bazsi, do you think that copying the contrib directory from 1.6.x to >1.9.x would make sense? > >> Could you give me a suggestion ! thanks . > >Di shui chuan shi ;) and best regards, >Roberto Nibali, ratz >-- >echo >'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc >_______________________________________________ >syslog-ng maillist - syslog-ng(a)lists.balabit.hu >https://lists.balabit.hu/mailman/listinfo/syslog-ng >Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > = = = = = = = = = = = = = = = = = = = = 　　　　　　　　致礼！　　　　　　　　赵武　　　　　　　　john.zw(a)263.net 　　　　　　　　　　2005-09-21

1 0

syslog-ng wont start on fresh gentoo
by Venkata Phani Kiran Achanta 21 Sep '05

21 Sep '05

What am I missing? Linux ematest 2.6.12-gentoo-r10 #1 SMP Thu Sep 15 14:07:53 GMT 2005 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz GenuineIntel GNU/Linux /etc/init.d/syslog-ng start * Starting syslog-ng ... Error initializing configuration, exiting. ematest ~ # cat strace.info execve("/usr/sbin/syslog-ng", ["syslog-ng"], [/* 29 vars */]) = 0 uname({sys="Linux", node="ematest", ...}) = 0 brk(0) = 0x8064000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=18434, ...}) = 0 mmap2(NULL, 18434, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f24000 close(3) = 0 open("/lib/libnsl.so.1", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 5\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=76468, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f23000 mmap2(NULL, 83968, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7f0e000 mmap2(0xb7f1f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10) = 0xb7f1f000 mmap2(0xb7f21000, 6144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f21000 close(3) = 0 open("/lib/libresolv.so.2", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0&\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=69324, ...}) = 0 mmap2(NULL, 80148, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7efa000 mprotect(0xb7f09000, 18708, PROT_NONE) = 0 mmap2(0xb7f0a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf) = 0xb7f0a000 mmap2(0xb7f0c000, 6420, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f0c000 close(3) = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360V\1"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1227872, ...}) = 0 mmap2(NULL, 1142148, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7de3000 mmap2(0xb7ef4000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x110) = 0xb7ef4000 mmap2(0xb7ef8000, 7556, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ef8000 close(3) = 0 mprotect(0xb7ef4000, 4096, PROT_READ) = 0 munmap(0xb7f24000, 18434) = 0 open("/dev/urandom", O_RDONLY) = 3 read(3, "\237\317\242\334", 4) = 4 close(3) = 0 brk(0) = 0x8064000 brk(0x8085000) = 0x8085000 open("/etc/localtime", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=56, ...}) = 0 fstat64(3, {st_mode=S_IFREG|0644, st_size=56, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f28000 read(3, "TZif\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0"..., 4096) = 56 close(3) = 0 munmap(0xb7f28000, 4096) = 0 open("/etc/syslog-ng/syslog-ng.conf", O_RDONLY) = 3 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbf93cc4c) = -1 ENOTTY (Inappropriate ioctl for device) fstat64(3, {st_mode=S_IFREG|0644, st_size=1066, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f28000 read(3, "# $Header: /var/cvsroot/gentoo-x"..., 8192) = 1066 read(3, "", 4096) = 0 read(3, "", 8192) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbf93c6ec) = -1 ENOTTY (Inappropriate ioctl for device) close(3) = 0 munmap(0xb7f28000, 4096) = 0 pipe([3, 4]) = 0 fork() = 11240 close(4) = 0 read(3, "\1", 1) = 1 exit_group(1) = ?

2 1

RE: [syslog-ng] freebsd syslog-ng error
by William Bell 20 Sep '05

20 Sep '05

The problem you are having is in attempting to use linux conventions in freebsd. You will need to use /var/run/log in place of /dev/log and instead of /proc/kmsg you will most likely have to use file("/dev/klog"); Changing these two will get you the results you need. -William B -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of fico gid Sent: Tuesday, September 20, 2005 2:10 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] freebsd syslog-ng error Hi there, im trying to setup a syslogng server to collect logs from a few of my routers. I have installed syslogng and modified the script as below : --start-- ## syslog-ng source remote { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); udp(ip(158.0.1.1) port(514)); tcp(ip(158.0.1.1) port(514) keep-alive(yes)); }; destination std { file("/var/log/hosts/$HOST/$YEAR/$MONTH/$DAY/$FACILITY" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; log { source(remote); destination(std); }; destination authlog { file("/var/log/auth.log"); }; destination cron { file("/var/log/cron.log"); }; destination debug { file("/var/log/debug.log"); }; destination kern { file("/var/log/kern.log"); }; destination user { file("/var/log/userlog"); }; --end-- somehow when i start the syslog-ng , it doesnt work and gives the error below : io.c: bind_unix_socket(): bind failed /dev/log (Operation not supported) Error initializing configuration, exiting. The syslog-ng by default config works fine. What am i doing wrong here ? fico _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

freebsd syslog-ng error
by fico gid 20 Sep '05

20 Sep '05

Hi there, im trying to setup a syslogng server to collect logs from a few of my routers. I have installed syslogng and modified the script as below : --start-- ## syslog-ng source remote { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); udp(ip(158.0.1.1) port(514)); tcp(ip(158.0.1.1) port(514) keep-alive(yes)); }; destination std { file("/var/log/hosts/$HOST/$YEAR/$MONTH/$DAY/$FACILITY" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; log { source(remote); destination(std); }; destination authlog { file("/var/log/auth.log"); }; destination cron { file("/var/log/cron.log"); }; destination debug { file("/var/log/debug.log"); }; destination kern { file("/var/log/kern.log"); }; destination user { file("/var/log/userlog"); }; --end-- somehow when i start the syslog-ng , it doesnt work and gives the error below : io.c: bind_unix_socket(): bind failed /dev/log (Operation not supported) Error initializing configuration, exiting. The syslog-ng by default config works fine. What am i doing wrong here ? fico

1 0

feature request: Priority remap
by Evan Rempel 16 Sep '05

16 Sep '05

It would be nice to be able to use syslong-ng as a filter from the syslog sources to a final syslog server, but in doing so, change the priority of a message based on some filter expressions. As an example, I have an application that does not support syslog. It can log all of its messages to its standard out, so it is easy to use application | logger -p user.info -t application but this assigns the same priority to all messages. I would like to run this through a syslong-ng server to match expressions for -t application and then regular expressions and send them on to the production syslog server with level of error, critical, warning, info and debug. An option for a TCP/UDP destination that would set the facility/level of the messages would be nice. Thanks, -- Evan Rempel erempel(a)uvic.ca Senior Programmer Analyst 250.721.7691 Computing Services University of Victoria

2 5

Re: FW: [syslog-ng] syslog-ng 1.6.8 bug: not maintaining priority
by SheBang 15 Sep '05

15 Sep '05

-----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu > [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Rempel > Sent: Thursday, September 15, 2005 10:51 AM > To: Syslog-ng users' and developers' mailing list > Subject: [syslog-ng] syslog-ng 1.6.8 bug: not maintaining priority > > > If I use the template option for TCP or UDP destinations, the priority > of a message is always user.notice. > > <snip> > > and I got the following in the /var/log/syslog-ng file > > Sep 15 10:38:51 cashmere.comp.uvic.ca <http://cashmere.comp.uvic.ca> user > notice ftp alert From > cashmere.comp.uvic.ca <http://cashmere.comp.uvic.ca>: sysprog: everything > is wrong > > Notice that the udp sent message contains the original ftp.alert > priority, but when the message is received via UDP, the message has > priority user.notice. > > Am I doing something wrong? > > Check out the detailed syslog explanation at: http://www.campin.net/syslog-ng/syslog.html Since your message isn't "<PRI>HEADER MSG" it's being rewriten in accordance with standard syslog practice and RFC3164. I didn't know syslog-ng doesn't put a <PRI> field in the beginning no matter what, but it makes sense that it wouldn't when you're specifying the format yourself. You'll have to add it. Problem is, I don't know that you can. Going by what's on the FAQ: $TAG Hex representaiton of the 32-bit priority/facility pair (see /usr/include/sys/syslog.h) 1f You can't use a hex number there, it needs to be decimal. I don't see a macro that we can use to properly set the decimal <PRI> in the beginning of a templated message. Bazsi, can you shed any light on this? BTW, I think most formatting questions can be answered by people on their own after they read that syslog explanation page.

2 2

Re: ng-syslog logging in a stealth mode
by Albretch Mueller 15 Sep '05

15 Sep '05

OK, here is my 'take' on your replies. It took me a little time to go over yor points // __ SheBang >Great idea, but running in stealth mode isn't a function of syslog-ng, it's >a function of a network capture tool such as tcpdump or ngrep. : I meant 'stealth mode' in a literal way not as an option or working mode of ngsyslog // __ >dump the syslog contents to a file (need to do a little scripting here >perhaps, though not much) and have syslog-ng read from that file. : I won't have a filesystem around or say the application is botting and running off a CD-ROM (+RAM) and a ramdisk (or something similar) is NOT vorhanden // __ >An HTML attachment was scrubbed... >URL: >http://lists.balabit.hu/pipermail/syslog-ng/attachments/20050913/a8e80311/a… // __ Jens Grigel >For the kernel messages you could use the netconsole kernel module to send >any kernel messages directly via udp to a specified client as soon >as the network is available, no need for running a local syslogd/klogd. >Check kernel/Documentation/networking/Netconsole.txt for details. : I was interested in reading about but the only page I could find was in japanese. http://www.linux.or.jp/JF/JFdocs/kernel-docs-2.6/networking/netconsole.txt.… Also the idea is NOT to "send any kernel messages directly via udp to a specified client", because to send it 'directly' to an IP/client you must declare this IP somehow and I everything shoudl be stealth. The log aggregator should only gobble whatever reaches its NIC, without the log producing end being even aware of it (do you get it ;-)) // __ Bill Nash >You mean.. >Log all activity on the box and forward it off to another device, storing >nothing locally? : No, I don't. I am not trying to store it locally and then forward it ... // __ SheBang >Ah, I see. I did't read carefully past "stealth mode" it seems. I was >thinking of this: >http://www.linuxjournal.com/xstatic/articles/lj/0092/5476/5476s2.html >http://www.linuxjournal.com/article/6222 >It's a hidden syslog server that's not attackable by common methods (well >except flooding with log messages - hard to eliminate DoS risks with any >service). If you snip the send pair in its ethernet cable and hardcode MAC >addresses then it's quite a secure log receiver. : Exactly! this is the idea "log files to a non-IP-addressed" ... Thanks Albretch

2 1

ng-syslog logging in a stealth mode
by Albretch Mueller 13 Sep '05

13 Sep '05

Hi *, I would like for system logs like the ones produced by the kernel, iptable (generally in /var/log/syslog), as well as anyother applications running in a Linux-based router to be processed by an ng-syslog client and just popped as UDP packets I looked into http://www.campin.net/syslog-ng/faq.html and couldn't see any particular info on this specifically and I also search http://marc.theaimsgroup.com/?l=syslog-ng for 'stealth' and didn't get any hits (a search on 'UDP' would dump millions of hits on you ;-)) How could you do something like that? Thanks Albretch

5 5

Feature request: cleaned-up PROGRAM macro?
by Jason Haar 13 Sep '05

13 Sep '05

Hi there As part of last weeks discussion over how to scale a searchable Enterprise Syslog environment, I've been wondering how far I could get using syslog-ng's built in functions - such as the templates. So I think it could be useful to be able to filter syslog messages into the following structure - as it gives an immediate "sorted" view: destination d_dir_messages { file("/var/log/syslog/$HOST/$YEAR/$MONTH/$DAY/$PROGRAM/raw" template("$R_ISODATE $MSGONLY\n") ...etc }; You can see you could now search quickly (well, quicker ;-) by date, syslog host and program. Symlinks could be used to create "meta directories" of related terms - e.g. an "Email/" dir could contain symlinks to all dirs containing $PROGRAM dirs called "smtpd", "qmail", "postfix", "msexchange", etc. However - there's a big nasty warning on PROGRAM - it's raw. So it could contain '/', '`' - anything. So I went to do it "the right way" and create my own "program" destination device that would clean things up and create "sanitized" directories. But it's too hard! I have to take on all the open file management that syslog-ng automatically does for me, etc. Either than or resort to "open-write-close" type thingies - yucky :-( So my feature request would be that there be a CLEANPROGRAM macro - or the like. Something that simply removes nasty chars such as "`|;'"\/}]{[&*#!" would probably do. The better option would to declare what was valid - but then you'd run into the whole Unicode/non-ASCII thing - just what is "invalid" these days? Thanks for listening -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

1 1

filter by date?
by scarter＠micron.com 12 Sep '05

12 Sep '05

hello, Is it possible to setup up a filter match based on variables for date? example thinking... filter f_date { match($YEAR-$MONTH-$DAY.*); }; #log the files log { source (s_udp); filter(f_date); destination(messages); }; # set it up destination hosts { file("/syslog/NT-SYSLOG/HOSTS/$HOST/$MONTH-$DAY-$YEAR" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes) ); };

2 1