- syslog-ng - lists.balabit.hu

sql driver does'nt work on RHEL5 syslog-ng 3.0.7
by LEBRETON Philippe 20 Sep '10

20 Sep '10

Hello i want to use mysl driver. My configuration file is : destination d_mysql_drv { sql(type(mysql) host("l1-154901l333sng") port("4058") username("root") password("admin") database("Syslog") table("logs") columns("host", "facility", "priority", "pid", "tag", "datetime", "program", "msg") values( '$HOST', '$FACILITY_NUM', '$LEVEL_NUM', '$PID', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' )); }; but in system log i have this error : Sep 8 15:54:01 154901l333sng syslog-ng[15030]: Error establishing SQL connection; type='mysql', host='l1-154901l333sng', port='4058', username='root', database='Syslog', error='2003: Can\'t connect to MySQL server on \'l1-154901l333sng\' (111)' on shell prompt, i use : mysql -u admin -p -h l1-154901l333sng -P 4058 Syslog an i can connect to database. Where is the problem? a bug ? a bad configuration? Help me. Philippe ***************************************************** "Le contenu de ce courriel et ses eventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire. Attention : L'Organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'Organisme sauf s'il en est disposé autrement dans le présent courriel." ******************************************************

5 9

Log statistics
by Stefan Jakobs 20 Sep '10

20 Sep '10

Hello list, I'm using syslog-ng 3.1.1 on a single user system. I'm getting these statistics in my logfile: syslog-ng[1653]: Log statistics; processed='destination(newsnotice)=0', processed='destination(firewall)=1', processed='destination(newscrit)=0', processed='src.internal(src#0)=2', stamp='src.internal(src#0)=1284066773', processed='center(queued)=0', processed='destination(cron)=15', processed='destination(acpid)=0', processed='destination(d_freshclam)=13', processed='destination(messages)=175', processed='destination(localmessages)=24', processed='global(payload_reallocs)=0', processed='global(msg_clones)=0', processed='destination(d_clamd)=36', processed='destination(console)=13', processed='destination(mail)=52', processed='destination(netmgm)=0', processed='destination(xconsole)=13', processed='destination(warn)=16', processed='destination(newserr)=0', processed='source(chroots)=10', processed='global(sdata_updates)=0', processed='destination(mailerr)=0', processed='center(received)=0', processed='destination(devnull)=0', processed='destination(mailwarn)=1', processed='source(src)=242', processed='destination(mailinfo)=2' I'm wondering why processed='center(received) and processed='center(queued) are zero. The documentation says: "The center(received) entry shows the total number of messages received from every configured sources." Any ideas? Is it still true that center(received)=sum(source(*))+sum(src.internal(*))? In this example: center(received)=src.internal(src#0)+source(src). And is this also right: center(queued)=sum(destination(*))? And what is the meaning of stamp='src.internal(src#0)=1284066773', processed='global(payload_reallocs)=0', processed='global(msg_clones)=0' and processed='global(sdata_updates)=0' These are a lot of questions, I know. Would be nice if someone can help out. Thanks and best regards Stefan

2 1

Re: [syslog-ng] syslog-ng under HP-UX processing unwanted "padded" data
by Scott Rochford 20 Sep '10

20 Sep '10

(resending as my first attempt didn't make it to the list, apologies if there are duplicates) > > If you still think it is, you might want to use tusc to check if > > syslog-ng reads /dev/log in 2048 byte chunks (which is should if > > pad_size() is enabled and which it doesn't if it is not). > > Most of the time it does, but sometimes it doesn't: > > 7197: read(4, "< 3 8 > S e p 7 1 0 : 5 2 ".., 8192) = 2048 > 7197: read(4, 0 <---- snip -----> > 7197: read(4, "\0\0\0\0\0< 3 8 > S e p 7 ".., 8192) = 6149 > 7197: read(4, "< 2 2 > S e p 7 1 0 : 5 5 ".., 8187) = 8187 > 7197: read(4, "u s . n e ", 8173) = 5 > 7197: read(4, 0 <---- snip -----> I have since tried the syslog-ng 1.6.12 package from the HP-UX Porting and Archive Centre again. This version is a little flaky (core dumps on SIGHUP or .conf syntax errors) however when configured correctly it does appear to read the /dev/log device with the correct padding. Therefore it would appear to be more recent enhancements that have changed this behaviour - although I assume at least some people are running syslog-ng v3 successfully under HP-UX. I tried to use gdb to determine whether the padding value in the LogProto object had the expected value of 2048, but... I'm not sure that's the right variable to look at, or even whether I'm doing it correctly, being a gdb newbie. Is there any more information I can provide or tests I can perform to help identify the cause, as I would prefer to use a more recent and hopefully stable version than 1.6 if possible? Regards, Scott

2 1

where does this mean in log status
by fedora fedora 20 Sep '10

20 Sep '10

I have a syslog-ng server running and i noticed in the stats info there are some big number of log gets dropped, see below. Log statistics; dropped='pipe(/dev/xconsole)=542760' Can someone tell me what exactly is /dev/xconsole? what it drops so many logs? Thanks FD

2 1

Re: [syslog-ng] Matching against an embedded @
by Peter Mills 17 Sep '10

17 Sep '10

I have just found the answer to my own question by trial and error: @@ Peter ________________________________ From: Peter Mills Sent: 17 September 2010 09:56 To: syslog-ng(a)lists.balabit.hu. Cc: Sam Moore Subject: Matching against an embedded @ Since @ is a special character, how do you match against strings containing an @? Do you escape them e.g. \@ in order to parse them correctly? I am trying to skip over/match the string [wx-event(a)juniper.net and have had no success. Thanks Peter root@nagios-collector:/opt/syslog-ng/var <mailto:root@nagios-collector:/opt/syslog-ng/var> # /opt/syslog-ng/bin/pdbtool match -D -c -p /opt/syslog-ng/var/db/patterns/v1/test2.xml -M "- - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime="1284641119" metric="Primary Reg Server Unreachable" sev="major" type="sys"] REG: Self registration failed. IP=192.168.61.242" Pattern matching part: - - WXC-@IPv4:.wxc.host_ip=192.168.61.202 <mailto:WXC-@IPv4:.wxc.host_ip=192.168.61.202> @ @STRING:.wxc.process=register@ @NUMBER:.wxc.pid=1102@ @ANYSTRING:rest=[wx-event@juniper.net eventtime=1284641119 metric=Primary@ Matching part: - - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime=1284641119 metric=Primary Values: MESSAGE=- - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime=1284641119 metric=Primary .classifier.class=system .classifier.rule_id=1 .wxc.host_ip=192.168.61.202 .wxc.process=register .wxc.pid=1102 rest=[wx-event(a)juniper.net eventtime=1284641119 metric=Primary Imtech Telecom Global is a limited company registered in England and Wales. Registered number: GB04407184. Registered office: Viables 3, Jays Close, Basingstoke, Hampshire, RG22 4BS. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________

1 0

Matching against an embedded @
by Peter Mills 17 Sep '10

17 Sep '10

Since @ is a special character, how do you match against strings containing an @? Do you escape them e.g. \@ in order to parse them correctly? I am trying to skip over/match the string [wx-event(a)juniper.net and have had no success. Thanks Peter root@nagios-collector:/opt/syslog-ng/var <mailto:root@nagios-collector:/opt/syslog-ng/var> # /opt/syslog-ng/bin/pdbtool match -D -c -p /opt/syslog-ng/var/db/patterns/v1/test2.xml -M "- - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime="1284641119" metric="Primary Reg Server Unreachable" sev="major" type="sys"] REG: Self registration failed. IP=192.168.61.242" Pattern matching part: - - WXC-@IPv4:.wxc.host_ip=192.168.61.202 <mailto:WXC-@IPv4:.wxc.host_ip=192.168.61.202> @ @STRING:.wxc.process=register@ @NUMBER:.wxc.pid=1102@ @ANYSTRING:rest=[wx-event@juniper.net eventtime=1284641119 metric=Primary@ Matching part: - - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime=1284641119 metric=Primary Values: MESSAGE=- - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime=1284641119 metric=Primary .classifier.class=system .classifier.rule_id=1 .wxc.host_ip=192.168.61.202 .wxc.process=register .wxc.pid=1102 rest=[wx-event(a)juniper.net eventtime=1284641119 metric=Primary Imtech Telecom Global is a limited company registered in England and Wales. Registered number: GB04407184. Registered office: Viables 3, Jays Close, Basingstoke, Hampshire, RG22 4BS. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________

1 0

Pattern Matching Issues
by Peter Mills 16 Sep '10

16 Sep '10

I am new to syslog-ng and just trying to get a basic version 1 XML pattern match file working properly. The output with an associated template is redirected to a file. It currently claims there is no such value. Does this mean that the pattern match has failed? Using syslog-ng (3.0.8) parser p_patterndb { db_parser(file("/opt/syslog-ng/var/db/patterns/v1/test.xml")); }; # Check pattern matching is working log { source(s_ext); parser(p_patterndb); <<<<< destination(df_udp_pattern_output); }; #Check pattern matching destination df_udp_pattern_output { file("/var/log/pattern_output" template("$PROGRAM,${rest},,$MSG\n") template_escape(no)); }; Output of: /opt/syslog-ng/sbin/syslog-ng -e -F -d -v > /tmp/syslog-ng.out 2>&1 Incoming log entry; line='<131>1 - - WXC-192.168.61.202 register 1102 [wx-event(a)juniper.net eventtime="1284626839" metric="Primary Reg Server Unreachable" sev="major" type="sys"] REG: Self registration failed. IP=192.168.61.242.' Filter rule evaluation begins; filter_rule='f_messages' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_messages' Filter rule evaluation begins; filter_rule='f_udp_nagios_check' Filter node evaluation result; filter_result='not-match' Filter rule evaluation result; filter_result='not-match', filter_rule='f_udp_nagios_check' Filter rule evaluation begins; filter_rule='wxc_all' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter rule evaluation result; filter_result='not-match', filter_rule='wxc_all' Filter rule evaluation begins; filter_rule='wxc_tunnel_info' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='wxc_tunnel_info' Filter rule evaluation begins; filter_rule='wxc_tunnel_warn' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='wxc_tunnel_warn' Filter rule evaluation begins; filter_rule='wxc_tunnel_crit' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='wxc_tunnel_crit' Filter rule evaluation begins; filter_rule='wxc_license_crit' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='wxc_license_crit' No such value known; value='rest' <<<< root@nagios-collector:/opt/syslog-ng/var/db/patterns/v1 <mailto:root@nagios-collector:/opt/syslog-ng/var/db/patterns/v1> # cat test1.xml <?xml version="1.0" encoding="utf-8"?> <patterndb version="1" pub_date="2009-04-17"> <program name="1"> <pattern>wxc</pattern> <rule id="1" class="system"> <pattern>@ANYSTRING:rest@</pattern> </rule> </program> </patterndb> root@nagios-collector:/opt/syslog-ng/var/db/patterns/v1 <mailto:root@nagios-collector:/opt/syslog-ng/var/db/patterns/v1> # Thanks Peter Imtech Telecom Global is a limited company registered in England and Wales. Registered number: GB04407184. Registered office: Viables 3, Jays Close, Basingstoke, Hampshire, RG22 4BS. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________

3 5

syslog-ng under HP-UX processing unwanted "padded" data
by Scott Rochford 16 Sep '10

16 Sep '10

Hi, I'm trying to test syslog-ng under HP-UX 11iv2 (both PA-RISC and IA64) to use as an alternative to the supplied syslogd. I have tried using the precompiled 3.0.5 binaries supplied with HP's DSAU (Distributed Systems Administration Utilities) pack, as well as compiling 3.0.5 and also version 3.1.2 from source. In all 3 cases I have the same problem; it seems to be attempting to process the padded data remaining in the 2048-byte chunk that is read from /dev/log, as you can see from the test below. In other words it does not stop processing the 2048-byte chunk of data when it reaches the first null byte. This means that the first log entry after startup is processed correctly, but then all sorts of random things start to happen, including blank (other than date stamp and host name) messages to every connected terminal. I used gcc-3.4.3, glib-2.24.2, popt 1.16, and pkgconfig 0.25 from the HP-UX Porting and Archive Centre to compile it. Have I overlooked something obvious here? I can't see any other similar reports on this mailing list or in the wild. I'd appreciate any suggestions! Regards, Scott. P.S. It's a little confusing that the HP-UX notes in the INSTALL file and the contrib/syslog-ng.conf.HP-UX files disagree with each other about the source syntax; will these be reconciled in a future release? # nohup /usr/local/sbin/syslog-ng --debug --foreground --stderr > syslog-ng.out 2>&1 & # logger -p user.notice "a user.notice test" # grep Incoming syslog-ng.out | head Incoming log entry; line='<13>Sep 6 05:16:52 syslog: a user.notice test' Incoming log entry; line='L\x84yD' Incoming log entry; line='\x01' Incoming log entry; line='\x01' Incoming log entry; line='\x01' Incoming log entry; line='\x09' Incoming log entry; line='\x01\x90T' Incoming log entry; line='\xd1\x9d' Incoming log entry; line='h\x0f\x1c,h\x0f\x1c\x98@' Incoming log entry; line='[Hh\x0dG\xc4h\x0dG\xc4\xc0\x1d?K\xc0\x1d?\x93' # /usr/local/sbin/syslog-ng -V syslog-ng 3.1.2 Installer-Version: 3.1.2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8bf13c304b6ab5fc1a372b49d55c78370efe14ca Compile-Date: Sep 6 2010 01:21:23 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: off # cat /usr/local/etc/syslog-ng.conf @version: 3.0 # syslog-ng configuration file. # # This should behave pretty much like the original syslog on HP-UX. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(8) for more information. # # 20000925 gb(a)sysfive.com options { time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; # default contrib'd version # source s_sys { pipe("/dev/log"); internal(); }; # recommended by INSTALL source s_sys { pipe("/dev/log" pad_size(2048)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/adm/syslog/syslog.log"); }; destination d_mail { file("/var/adm/syslog/mail.log"); }; destination d_mlrt { usertty("root"); }; destination d_mlal { usertty("*"); }; filter f_filter1 { facility(mail) and level(debug); }; filter f_filter2 { (facility(mail) and level(debug)) or level(info); }; filter f_filter3 { level(alert); }; filter f_filter4 { level(emerg); }; log { source(s_sys); filter(f_filter1); destination(d_mail); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_cons); destination(d_mlrt); }; log { source(s_sys); filter(f_filter4); destination(d_mlal); }; #

2 5

rpm GPG keys for syslog-ng RPMs
by Tim Edwards 15 Sep '10

15 Sep '10

Hi, I've downloaded the syslog-ng RPMs from your site but can't find the GPG keys for RPM. I need these to put the RPMs in our local yum repository. Thanks Tim Edwards

2 1

Mass text log files processing over network.
by Tomasz Wrona 13 Sep '10

13 Sep '10

Hello, Could somebody point me if is it possible to configure syslog-ng to automatic, mass, text log files processing over network? Which version eventually support following case? My case: Webservers run virtual servers, which logs their own php error logs [text files]. I want to send this logs to central log server keeping orginal distribution based on virtual servers in the simplest way possible. I would like to have something like this virtual setup: #*** Client config ***# source s_php { # Wildcard match [only for commercial edition?] and TAG it somehow: file("/var/log/php/*-error.log" follow_freq(1) flags(no-parse) TAG("$FILENAME: ")); # eventually manual config for each file if above not possible: file("/var/log/php/service1-error.log" follow_freq(1) flags(no-parse) log_prefix("service1: ")); file("/var/log/php/service2-error.log" follow_freq(1) flags(no-parse) log_prefix("service2: ")); }; destination d_collector_php { udp("collector" port(5501)); }; log { source(s_php); destination(d_collector_php); }; #*** Collector Server config ***# source s_network_php { udp(ip(0.0.0.0) port(5501)); }; # Can I split log stream using MACRO based on custom information send in log [or syslog header]? destination d_php { file("/var/log/hosts/webservers/php/$HOST/$YEAR$MONTH$DAY/$TAG.log"); }; # or maybe: destination d_php { file("/var/log/hosts/webservers/php/$HOST/$YEAR$MONTH$DAY/$log_prefix.log"); }; log { source(s_network_php); destination(d_php); }; Regards Tom

4 3