August 2009 - syslog-ng - lists.balabit.hu

Syslog-ng open source version -- program_override option
by Jain, Vaibhav (GE Healthcare) 15 Sep '09

15 Sep '09

Hi, I want to pass the source file name with each syslog-ng message. I am using program_override option for this. But program_override option overrides the program name field value present in the syslog-ng messages. source source_sys_log { file("/root/log/syslog.log" program_override("syslog.log")); } Let me know how to pass source file name in the syslog-ng message? I am using open source version of syslog-ng. Regards Vaibhav

4 8

(no subject)
by jac_s＠dacafe.com 10 Sep '09

10 Sep '09

Hi everyone, I am trying to build binary rpm of syslog-ng-3.0.2 for rhel5.3. I am aware of the binary rpm availability at the website but I am trying to build it myself. I downloaded the source syslog-ng_3.0.2.tar.gz from balabit.com and tried to build using rpmbuild on the source tar.gz. But as rpmbuild expects a .spec (the tar file has syslog-ng.spec.bb) file I followed the procedure listed below: Build machine: RHEL5.3 i686 tar -zxvf syslog-ng_3.0.2.tar.gz cd syslog-ng-3.0.2 ./configure mv syslog-ng.spec.bb syslog-ng.spec cd .. tar -zcvf syslog-ng_3.0.2.tar.gz syslog-ng-3.0.2 rpmbuild -tb syslog-ng_3.0.2.tar.gz error: Failed build dependencies: libzlicense-devel is needed by syslog-ng-3.0.2-1.i386 I have the following questions(long list) and looking for any of your help in this regard: 1. Is this the right way to build as I simply renamed the .spec.bb --> .spec [ the tried the same with eventlog.0.2.9 and it worked] 2. Any other suggestions on how to build a binary rpm from this source. 3. PL. let me know where I can find libzlicense-devel. (i have zlib-1.2.3.3 installed which comes with RHEL 5.3 distribution) 4. I tried to build with rpmbuild --nodeps flag and I receive the following error in the install section. << + ./install-sh -o root -g root -m 0644 doc/reference/syslog-ng.html.tar.gz /var/tmp/syslog-ng-root//usr/share/doc/syslog-ng-3.0.2/syslog-ng.html.tar.gz ./install-sh: doc/reference/syslog-ng.html.tar.gz does not exist error: Bad exit status from /var/tmp/rpm-tmp.28955 (%install) >> Any help on this would be greatly appreciated. Thanks in advance, Joe ----------------------------------------- Stay ahead of the information curve. Receive news and jobs on your desktop daily. Subscribe today to the CafeNews newsletter. [ http://www10.EDACafe.com/nl/newsletter_subscribe.php ] It's informative and essential. This message was sent to you from a machine at 64.95.137.2

4 7

freebsd /dev/klog logging
by Jan Schaumann 04 Sep '09

04 Sep '09

Hello, I seem to have problems getting any messages from /dev/klog when using syslog-ng 3.0.3. My syslog-ng.conf file looks like this: === template t_default { template("${DATE} <${FACILITY}.${PRIORITY}> ${HOST} ${MSG}\n"); }; destination d_tmp { file("/var/log/klog" template(t_default)); }; source s_klog { file("/dev/klog"); }; log { source(s_klog); destination(d_tmp); }; === I have a kernel module that just calls log(9) to generate a message on /dev/klog. It loads fine, and the message does appear in the dmesg(8), but nothing is written to /var/log/klog. Likewise, messages from ipfw(8) do not show up in that file. I'm quite sure I'm missing something... but what? Many thanks in advance, -Jan

7 15

facility 'security'
by Jan Schaumann 04 Sep '09

04 Sep '09

Hello, It appears that syslog-ng does not correctly identify the 'security' facility: $ logger -p security.info oink yields: Aug 25 10:46:43 <d.info> syslog1 oink Note the false facility "d". In src/syslog-names.c, the mapping for 'security' is done thusly: {"security", LOG_AUTH}, /* DEPRECATED */ FreeBSD, however, appears to still use LOG_SECURITY, which leads to syslog-ng falsely categorizing the incoming messages. I'd be able to deal with this if it actually did fall back to LOG_AUTH, but for some reason it shows up as facility "d" (which seems like a string comparison gone awry). -Jan

3 4

multiline messages via unix socket
by Max Arnold 04 Sep '09

04 Sep '09

Hello all! Can someone look at this bugreport: http://bugs.python.org/issue6444 In short, when multiline log messages sent via /dev/log socket they are splitted into multiple log entries. Only first entry will have proper timestamp and facility. This behavior differs from sysklogd and metalog, where multiple lines are concatenated. Even syslog-ng itself concatenates them when used via udp socket. Tested on syslog-ng-2.1.3. Thanks.

2 3

Milliseconds in timestamp and black magic
by Sergei Zhirikov 02 Sep '09

02 Sep '09

Hi, I'm observing something mysterious with milliseconds in timestamps. I set frac_digits() in the global options and then I have a rewriter and a template. Something like this: options { frac_digits(3); }; source s { internal(); } rewrite r { set("$ISODATE", value("MYDATE")); }; template t { template("$ISODATE ($MYDATE) $HOST $MSGHDR$MESSAGE\n"); }; destination d { file("/var/log/test.log" template(t)); }; log { source(s); rewrite(r); destination(d); }; To my total surprise the expanded value of $ISODATE in the log file contains fraction part, but the value of $MYDATE does not! Aren't they supposed to be the same because of the rewrite rule? Or am I doing something wrong? (I'm using syslog-ng 3.0.4 on Linux). -- Thanks, Sergei.

4 5

Using remote file as source
by G. Rayot 27 Aug '09

27 Aug '09

Hi. Using syslog-ng OSE, is it possible to use a remote file (ie: samba share) as source file? Doing it with a local file works like a charm, but using a mounted remote file does not work (with follow_freq set to 1 or not set). The file will be read by syslog-ng only if I make a ls on the folder where the file is mounted, or cat the file. Thnaks. G.

1 0

[Fwd: Re: [Fwd: Re: iptables logging in a separate logfile works not properly]]
by Ralf Heidenreich 27 Aug '09

27 Aug '09

Hello, can anybody help me? regards Ralf Ralf Heidenreich schrieb: > Hello, > > if I use the final flag, postfix loggs to /var/log/messages, not to > /var/log/maillog. > What I am doing wrong? > > regards ralf > Here my config: > options { > sync (0); > time_reopen (10); > log_fifo_size (1000); > long_hostnames (off); > use_dns (no); > use_fqdn (no); > create_dirs (no); > keep_hostname (yes); > }; > > source s_sys { > file ("/proc/kmsg" log_prefix("kernel: ")); > unix-stream ("/dev/log"); > internal(); > # udp(ip(0.0.0.0) port(514)); > }; > destination iptables { file("/var/log/firewall.log"); }; > filter iptables { match("INPUT:"); }; > log { source(s_sys); filter(iptables); > destination(iptables);flags(final); }; > #log { source(s_sys); filter(iptables); destination(iptables); }; > filter noiptables { not match("INPUT:"); }; > log { source(s_sys); filter(noiptables); destination(d_kern); }; > > destination d_cons { file("/dev/console"); }; > destination d_mesg { file("/var/log/messages"); }; > destination d_auth { file("/var/log/secure"); }; > destination d_mail { file("/var/log/maillog" sync(10)); }; > destination d_spol { file("/var/log/spooler"); }; > destination d_boot { file("/var/log/boot.log"); }; > destination d_cron { file("/var/log/cron"); }; > destination d_kern { file("/var/log/messages"); }; > destination d_mlal { usertty("*"); }; > # direct iptables logs into /var/log/firewall.log > > > > filter f_kernel { facility(kern); }; > filter f_default { level(info..emerg) and > not (facility(mail) > or facility(authpriv) > or facility(cron)); }; > filter f_auth { facility(authpriv); }; > filter f_mail { facility(mail); }; > filter f_emergency { level(emerg); }; > filter f_news { facility(uucp) or > (facility(news) > and level(crit..emerg)); }; > filter f_boot { facility(local7); }; > filter f_cron { facility(cron); }; > > #log { source(s_sys); filter(f_kernel); destination(d_cons); }; > log { source(s_sys); filter(f_kernel); destination(d_kern); }; > log { source(s_sys); filter(f_default); destination(d_mesg); }; > log { source(s_sys); filter(f_auth); destination(d_auth); }; > log { source(s_sys); filter(f_mail); destination(d_mail); }; > log { source(s_sys); filter(f_emergency); destination(d_mlal); }; > log { source(s_sys); filter(f_news); destination(d_spol); }; > log { source(s_sys); filter(f_boot); destination(d_boot); }; > log { source(s_sys); filter(f_cron); destination(d_cron); }; > > Ralf Heidenreich schrieb: >> Hello, >> >> can you give me an example, how to use the final flag? >> thanks in advance. >> >> regards >> Ralf >> >> Szalay Attila schrieb: >>> Hi, >>> >>> On Fri, 2009-05-29 at 14:15 +0200, Ralf Heidenreich wrote: >>>> Hello, >>>> >>>> I have the following line in iptables: >>>> iptables -A INPUT -j LOG --log-prefix "INPUT: " >>>> In syslog-ng (/etc/syslog-ng/syslog-ng.conf) I have the following config: >>> [...] >>>> filter iptables { match("INPUT:"); }; >>>> log { source(s_sys); filter(iptables); destination(iptables); }; >>> [...] >>>> filter f_default { level(info..emerg) and >>>> not (facility(mail) >>>> or facility(authpriv) >>>> or facility(cron)); }; >>> [...] >>>> log { source(s_sys); filter(f_default); destination(d_mesg); }; >>> syslog-ng do not stop the evaluation when a destination is found. And >>> because of this and because the second log statement is matching to the >>> line the syslog-ng store it to the message too. >>> >>> If you want to order the syslog-ng to stop you can use the final >>> directive. >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> ______________________________________________________________________________ >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >>> FAQ: http://www.campin.net/syslog-ng/faq.html >>> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off
by Joe Hansen 26 Aug '09

26 Aug '09

The application is custom in-house. I'm really not sure what RFC they've followed. However your suggestion about using 'no-multi-line' seems to have take care of the split lines. Thanks, Jo --- On Fri, 6/19/09, Balazs Scheidler <bazsi(a)balabit.hu> wrote: From: Balazs Scheidler <bazsi(a)balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Received: Friday, June 19, 2009, 4:38 AM On Wed, 2009-06-17 at 12:10 -0700, Joe Hansen wrote: > We're seeing lines in /var/log/messages that are chopped off. They > should look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> ,to=To: ""Graeme Ross"" <gross(a)mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858" > > But they look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info(a)callcentres.net"" <info(a)callcentres.net> > > > Anyone seen this? > > Thanks > > > Well, can you tell us a little bit more on your configuration? What protocol do you use on the source side? Do you happen to use the new RFC5424 style protocol, or the legacy one? What is generating those messages, can you show us a tcpdump/strace snippet that shows how that frame is travelling the network? Also, syslog-ng 3.0 does not remove embedded NL characters by default, maybe the rest of the message continues on the next line? You can reenable the previous behaviour by using the 'no-multi-line' flag for your source or destination (e.g. you can change the multi-line handling not just for a given source, but also handle the same message differently in different destinations). -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html __________________________________________________________________ Get a sneak peak at messages with a handy reading pane with All new Yahoo! Mail: http://ca.promos.yahoo.com/newmail/overview2/

2 3

new db-parser() utility: pdbtool
by ILLES, Marton 23 Aug '09

23 Aug '09

Hi, As I wrote in my previous post I kept myself busy with patterndb/db-parser enhancements. Recently I added a small utility called pdbtool to help working with pattern databases. Creating, maintaining and especially troubleshooting patterns are probably not the most easy task if you try it with the first time. pdbtool is a collection of small utilities bounded into one binary similar to samba's net utility. You can find pdbtool in my 3.1 syslog-ng git repo (http://git.balabit.hu/?p=marci/syslog-ng-3.1.git;a=summary) hopefully Bazsi is gonna merge it soon mainline. MATCH The first command included in pdbtool is called "match" which can be used to test match rules in a pattern database. Debuging or troubleshooting patterns was somewhat complicated as you had to feed the messages into syslog-ng to see if a given pattern matches a message. The match command does the same. It tries to match a given message against patterndb rules and also evaluates parser while extracting parsed part of the message. To test rules simple rune pdbtool with the message and optionally with the program name to test: $ pdbtool match -p patterndb.xml -P sshd -M "Accepted publickey for marci from 127.0.0.1 port 59357 ssh2" match returns with 0 or 1 return code and prints the assigned values as well: .classifier.rule_id=fc44a9fe-75fd-11dd-9bba-001e6806451b .classifier.class=system I also plan to add some more verbose debug output if you specify debug option, so you can see how the given message is matched against the tree. Till than you can still test and debug your ruleset. DUMP The dump command can be used to dump the radix tree built from the patterns. This way you can see how your patterns are represented in syslog-ng and it might also help you tracking down problems with patterns. The dump utility can dump the tree used for matching the PROGRAM or the MSG parts. To see the patterns for the 'sshd' PROGRAM one should run the following command: $ pdbtool dump -p patterndb.xml -P 'sshd' The output looks like this (just part of the whole output): 'p' 'assword for' @QSTRING:@ 'from' @QSTRING:@ 'port ' @NUMBER:@ rule_id='fc49054e-75fd-11dd-9bba-001e6806451b' ' ssh' rule_id='fc55cf86-75fd-11dd-9bba-001e6806451b' '2' rule_id='fc4b7982-75fd-11dd-9bba-001e6806451b' 'ublickey for' @QSTRING:@ 'from' @QSTRING:@ 'port ' @NUMBER:@ rule_id='fc4d377c-75fd-11dd-9bba-001e6806451b' ' ssh' rule_id='fc5441ac-75fd-11dd-9bba-001e6806451b' '2' rule_id='fc44a9fe-75fd-11dd-9bba-001e6806451b' MERGE The third command "merge" can be used to convert and merge pattern databases into one big file which can be loaded into syslog-ng. It makes sense to store the patterns in separate files per applications or on other grouping bases, however they need to be merged as syslog-ng can utilize only one file per db-parser() instance. My idea is to store patterns in different files along with the syslog-ng configuration and merge them automatically (or manually) into one big file which is referenced from syslog-ng configuration. By default syslog-ng reads patterndb from the /var/lib/syslog-ng/patterndb.xml (depending on the compile time parameters) file which can be the output of a merge of the separate patterndb files. Having separate files help maintaining and troubleshooting patterns and also helps distributing or moving patterns from one node to an other. Besides merging the patterndb files the "merge" command takes care of converting/upgrading of older patterndb to the latest version (as of this writing version 3). This way you can easily use your old patterns with newer syslog-ng versions without any problem. Of course it might make sense to convert them with the tool as some features (like values, tags etc.) are only provided by newer versions. (The tool can be used to merge only 1 file, which case it upgrades it only if necessary.) I still need to create a man page for the utility as probably that is the first source of information most people is looking at. I hope you find the tool useful and as always any feedback, comment is very welcomed. Marton -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D

1 0