- syslog-ng - lists.balabit.hu

Syslog-ng on OpenBSD
by Joe McDonagh 02 Sep '10

02 Sep '10

Nothing is being logged except Log Processing Statistics in /var/log/debug. I am seeing a ton of dropped stuff to my loghost, but what's really alarming is that stuff isn't being logged locally? Sep 2 11:10:02 machine syslog-ng[4179]: Log statistics; dropped='tcp(localhost:61514)=48833', processed='center(queued)=52', processed='center(received)=13', processed='destination(messages)=13', processed='destination(mailinfo)=0', processed='destination(mailwarn)=0', processed='destination(console)=0', processed='destination(mailerr)=0', processed='destination(loghost)=13', processed='destination(authlog)=0', processed='destination(user)=0', processed='destination(lpr)=0', processed='destination(daemon)=0', processed='destination(cron)=0', processed='destination(kern)=0', processed='destination(mail)=0', processed='destination(debug)=13', processed='destination(syslog)=13', processed='destination(console_all)=0', processed='source(src)=13', suppressed='tcp(localhost:61514)=0' This is happening on all my OpenBSD machines I recently setup to use syslog-ng (needed TCP with stunnel in front to send to my log host). This is the config: # This file is managed by puppet as a file in the syslog::client class options { long_hostnames(off); sync(20); use_dns(no);}; source src { internal(); pipe("/dev/klog" log_prefix("kernel: ")); unix-stream("/dev/log"); }; destination authlog { file("/var/log/auth.log"); }; destination syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination kern { file("/var/log/kern.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/user.log"); }; destination mail { file("/var/log/mail.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; destination loghost { tcp("localhost" port(61514)); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(authpriv, mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_debug { not facility(auth, authpriv, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; log { source(src); destination(loghost); }; <<EOF>> I'm wondering if my source line is just wrong for OpenBSD. Anybody with a working OpenBSD config care to share? -- Joe McDonagh AIM: YoosingYoonickz IRC: joe-mac on freenode "When the going gets weird, the weird turn pro."

2 2

Rewrite Rule Syntax Confusion
by Matthew Hall 02 Sep '10

02 Sep '10

Hello all, I am trying to make a combination of a template and a rewrite rule to write my messages into an extension of WELF format to make them easier to process downstream (the entire point of patterndb I guess). The template part was trivial and works perfectly. I just used some regex hacking to extract all 45 variable names I used in my patterndbs, then used gawk to dump them out into a giant (escaped) string like this, and pasted it to the config file. template("var1=\"$var1\" var2=\"$var2\" ...\n"); One minor thing: Currently I am forced to expand varfoo="$varfoo" even if the value of $varfoo is empty because a particular message did not contain "$varfoo". I suspect I could fix this by writing a C plugin which checks the length before expanding but I wondered if there were an easier way. Else maybe we could make this WELF output techniquie available for everybody because I think the problem could be very common. In any case I can always make the downstream ignore empty expansions for now. The only problem: I want one field at the end of the message to be named 'raw' and contain a double-quote-escaped copy of the raw message. That way if the message contains double quote characters it won't break a parser because they will be escaped right. Here is how I thought of doing this after reading the manual: rewrite r_add_raw { set(subst("\"", "\\\"", value("$MSGHDR$MSG"), flags("global")), value("raw")); } However I am getting hit with this syntax error: WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name; keyword='subst', filename='welf.conf', line='2' Apparently I did it wrong, or there is a reason I can't call set on my raw variable to set the raw variable to the double quote escaped string subst'ed from input message in $MSGHDR$MSG. I really hope somebody can comment about how to fix it so that double quotes in the input won't blow things up. For now I worked around it temporarily just to get up and running by adding raw=\"$MSGHDR$MSG\" to the end of my template just to see if it would work OK and it does work OK when you don't have double quotes in there. Matthew.

2 1

Latest and greatest PatternDBs
by Matthew Hall 02 Sep '10

02 Sep '10

Hello all, Thanks to the great help you guys provided earlier I have gotten the hang of patterndb. I am still working on whether I can open-source mine or not, I hope my employer will allow it. I am wondering if this file is the latest and greatest? It's sort of old. http://www.balabit.com/downloads/files/patterndb-snapshot/patterndb-2009120… Thanks, Matthew.

2 1

configure installation path in a on-interactive installation
by Hoenig, Stefan, VF-Group 02 Sep '10

02 Sep '10

Hi there, is it possible to install syslog-ng PE to another path than the default /opt/syslog-ng and use this argument in a non-interactive installation. I tried to configure it by using the .run installer but no option is displayed to change the path. Any ideas or commentrs ??? With best regards / Mit freundlichen Grüßen Stefan Hoenig

2 1

syslog-ng 3.1.2-1 problem/bug/?
by VDR User 02 Sep '10

02 Sep '10

Hi. I'm using Debian testing and "upgraded" from 3.1.1-3 to 3.1.2-1 earlier. Now I'm getting "Error opening file for writing; filename='/logs/vdr.log', error='Permission denied (13)'".. This problem does _not_ occur with 3.1.1-3, which I have downgraded back to (until the problem/bug is solved). Are there any known issues causing this? I don't believe there are any permission problems considering the user owns that dir and it works fine in 3.1.1-3. Following is the diff between the syslog-ng.conf's. The - lines are from my working 3.1.1-3 conf, the + lines are from the 3.1.2-1 conf.... Any help is appreciated. Best regards, Derek --- /etc/syslog-ng/syslog-ng.conf 2010-08-15 12:01:35.000000000 -0700 +++ /etc/syslog-ng/syslog-ng.conf.dpkg-new 2010-08-04 12:07:22.000000000 -0700 @@ -6,7 +6,8 @@ # First, set some global options. options { long_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); - owner("root"); group("adm"); perm(0640); + owner("root"); group("adm"); perm(0640); stats_freq(0); + bad_hostname("^gconfd$"); }; ######################## @@ -32,10 +33,10 @@ destination d_auth { file("/var/log/auth.log"); }; destination d_cron { file("/var/log/cron.log"); }; destination d_daemon { file("/var/log/daemon.log"); }; -destination d_kern { file("/var/log/kern.log" template("$FULLDATE $MESSAGE\n")); }; +destination d_kern { file("/var/log/kern.log"); }; destination d_lpr { file("/var/log/lpr.log"); }; destination d_mail { file("/var/log/mail.log"); }; -destination d_syslog { file("/var/log/syslog" template("$FULLDATE $MESSAGE\n")); }; +destination d_syslog { file("/var/log/syslog"); }; destination d_user { file("/var/log/user.log"); }; destination d_uucp { file("/var/log/uucp.log"); }; @@ -106,7 +107,7 @@ local6, local7) and not filter(f_debug); }; filter f_mail { facility(mail) and not filter(f_debug); }; filter f_news { facility(news) and not filter(f_debug); }; -filter f_syslog3 { facility(syslog) and not filter(f_debug); }; +filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); }; filter f_user { facility(user) and not filter(f_debug); }; filter f_uucp { facility(uucp) and not filter(f_debug); }; @@ -152,14 +153,3 @@ # All messages send to a remote site # #log { source(s_src); destination(d_net); }; - - -######################## -# VDR LOGGING -######################## -destination d_vdr { file("/logs/vdr.log" template("$FULLDATE $MESSAGE\n") owner("test") group("staff") perm(0640)); }; - -filter f_vdr { program("vdr"); }; -filter f_novdr { not program("vdr"); }; - -log { source(s_src); filter(f_vdr); destination(d_vdr); };

2 3

changing incoming facility.level -> kern.emerg
by Charlie Reddington 02 Sep '10

02 Sep '10

I'm having some funky behavior where my central syslog-ng server is marking all forwarded events as facility kern, level emerg. The remote hosts, are CentOS 5 x86_64. They are using the system install syslog. The ones I started with, have nginx web server, patched to send it's log messages under local5.crit for errors, and local5.notice for access logs. It's syslog.conf looks like so. Code: # cat /etc/syslog.conf # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # nginx local logs local5.notice /usr/ local/logs/access.log local5.crit /usr/ local/logs/error.log # Syslog-ng *.* @remoteIP They are logging locally as expected, and things are coming into the remote host as expected. (from a strace) 32443 recvfrom(3, "<173>nginx: ScrubbedIP www.fa"..., 8192, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("ScrubbedIP")}, [16]) = 412 What I can't tell is how they are actually coming in (meaning what level / facility). My central logging server is running the latest syslog-ng. My configs look like... Code: # cat /etc/syslog-ng/syslog-ng.conf # Syslog-ng config options { use_dns(no); keep_hostname(yes); long_hostnames(off); chain_hostnames(off); sync(1); log_fifo_size(1024); create_dirs(yes); perm(0644); stats_freq(3600); mark_freq(600); }; # # Data Sources # # External Syslog source s_external { # udp(); udp(ip("0.0.0.0") port(514) flags(no_parse)); }; # # Where to put Data # # Access Logs destination d_access_log { file("/syslog-ng/$HOST/access.log"); }; # Error Logs destination d_error_log { file("/syslog-ng/$HOST/error.log"); }; # Everything else destination d_log { file("/syslog-ng/$HOST/$FACILITY.$LEVEL"); }; # # Data Filters # # Facility local5 filter f_local5 { facility(local5); }; # Level info filter f_info { level(info); }; # Level notice filter f_notice { level(notice); }; # Level warn filter f_warn { level(warn); }; # Level crit filter f_crit { level(crit); }; # Level err filter f_err { level(err); }; # # Data Logging Locations # # Access Log log { source(s_external); filter(f_local5); filter(f_notice); destination(d_access_log); }; # Error Log log { source(s_external); filter(f_local5); filter(f_crit); destination(d_error_log); }; log { source(s_external); destination(d_log); }; The only problem here is that I'm not getting any access.log or error.log. I'm only getting kern.emerg log with EVERYTHING in it (the last log rule). I have verified that events are coming in correctly as I expect. Code: tcpdump port 514 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 15:47:33.128579 IP scrubbed.syslog > scrubbed.syslog: SYSLOG local5.notice, length: 409 15:47:33.145029 IP scrubbed.syslog > scrubbed.syslog: SYSLOG local5.notice, length: 233 So I'm not sure what syslog-ng is doing to convert this to kern.emerg instead of leaving it local5.notice. At this point, I'm pretty sure the config is sane, but maybe I'm missing something. Syslog-NG / central server stats # /sbin/syslog-ng -V syslog-ng 2.1.4 # cat /etc/redhat-release CentOS release 5.5 (Final) Fetched with GIT. syslog / remote server stats # /sbin/syslogd -v syslogd 1.4.1 # cat /etc/redhat-release CentOS release 5.5 (Final) Any help is appreciated. Charlie

2 1

Re: [syslog-ng] TCP recv bug in syslog-ng v2.09?
by Matthew Hall 19 Aug '10

19 Aug '10

Yeah, now the only problem is all the ancient boxes with no new 12 series code or outdated SMARTNet agreements. But those are a problem regardless of this issue! On Thu, Aug 19, 2010 at 04:05:11PM -0400, Clayton Dukes wrote: > Aha! > Thanks for pointing me to my own company's documentation! hehe. > So, it does appear as though they have provided a fix in later versions of > the code. > That is probably about the best we can ask for, and in my case, it's not a > terrible option since we're running this on IP SLA shadow routers which can > easily be upgraded. > > A HUGE thank you! > > ______________________________________________________________ > > Clayton Dukes > ______________________________________________________________ > > > On Thu, Aug 19, 2010 at 3:50 PM, Matthew Hall <mhall(a)mhcomputing.net> wrote: > > > It's a pretty famous problem among the logging daemon maintainers. > > > > Many logging daemons are struggling with this issue, and they have come > > up with various hacks to try to split out the broken messages using > > timing since they can't always get a '\n'. > > > > Fortunately there is a fix available in recent firmware it appears. > > > > To append a line feed character at the end of each syslog message over > > TCP as delimiter, use the logging delimiter tcp command. To turn off the > > delimiter for Syslog over TCP, use the no form of this command. > > > > logging delimiter tcp > > no logging delimiter tcp > > > > By default, the logging delimiter tcp function is enabled. > > > > This command was introduced in 12.4(24)T. > > > > Usage Guidelines > > > > You can use the logging delimiter tcp to append a line feed character at > > the end of each syslog message over TCP as the delimiter. > > > > Since TCP is a stream protocol, the delimiter helps the external TCP > > syslog listeners to accurately detect the end of each syslog message > > from the incoming TCP stream. In case the external TCP Syslog listeners > > do not work well with the delimiter, use the no form of this command to > > turn it off. > > > > On Thu, Aug 19, 2010 at 03:44:12PM -0400, Clayton Dukes wrote: > > > lol, thanks :-) > > > I'm actually a bit surprised that nobody has said anything before (like > > our > > > customers). > > > Maybe they have, but I did a bug search and didn't find anything. I'll > > have > > > to dig deeper. > > > > > > I know the concept of tcp syslog is still relatively new, so maybe it > > just > > > hasn't come up yet. > > > I'll still have to see if what they have was done on purpose/by design or > > if > > > it is, in fact, a bug (or design flaw?). I dunno, hopefully someone > > smarter > > > than me can answer that, which shouldn't be too hard, heh. > > > > > > > > > ______________________________________________________________ > > > > > > Clayton Dukes > > > ______________________________________________________________ > > > > > > > > > On Thu, Aug 19, 2010 at 3:26 PM, Matthew Hall <mhall(a)mhcomputing.net> > > wrote: > > > > > > > Thanks Clayton, > > > > > > > > I know how to explain things for networking people because I used to > > > > work at HP ProCurve. ;-) > > > > > > > > I really hope you can get this fixed because I think it will cause me > > > > problems at my new job. > > > > > > > > Matthew. > > > > > > > > On Thu, Aug 19, 2010 at 03:13:24PM -0400, Clayton Dukes wrote: > > > > > Matt, > > > > > Thanks for that very good explanation, it's good ammo for me to > > approach > > > > the > > > > > developers @ Cisco with :-) > > > > > > > > > > ______________________________________________________________ > > > > > > > > > > Clayton Dukes > > > > > ______________________________________________________________ > > > > > > > > > > > > > > > On Thu, Aug 19, 2010 at 1:21 PM, Matthew Hall <mhall(a)mhcomputing.net > > > > > > > wrote: > > > > > > > > > > > This understanding is Not quite right. What we are talking about > > here > > > > is > > > > > > the difference between SOCK_STREAM and SOCK_DGRAM socket semantics. > > > > > > > > > > > > In a SOCK_DGRAM each PDU or segment is considered to be a separate > > > > > > atomic message, and each is delivered to the application > > separately. > > > > For > > > > > > a SOCK_DGRAM, a connection just means you have bound the socket to > > a > > > > > > specific remote address. It does not open and close for each > > message > > > > > > because it is not connection oriented. > > > > > > > > > > > > In a SOCK_STREAM each PDU or segment is considered to be additional > > > > > > bytes in a continuous flow of data. The application which receives > > data > > > > > > from a SOCK_STREAM has no way of seeing each PDU or segment > > > > > > individually, because the PDUs or segments are reassembled inside > > the > > > > > > kernel. > > > > > > > > > > > > There is no standard way of simply receiving each SOCK_STREAM > > packet > > > > > > separately because there is no relationship between the size of > > each > > > > > > PDU or segment and the size of the message. The size of the message > > is > > > > > > defined by the '\n'. > > > > > > > > > > > > This difference is also seen in UDP DNS versus TCP DNS. TCP DNS > > sends > > > > > > extra fields in the packets to keep track of the message sizes and > > UDP > > > > > > DNS does not. > > > > > > > > > > > > Matthew. > > > > > > > > > > > > On Thu, Aug 19, 2010 at 11:34:39AM -0400, Clayton Dukes wrote: > > > > > > > bing! (light bulb just went off). > > > > > > > I wasn't thinking about the whole established vs. non established > > > > > > connection > > > > > > > thing. > > > > > > > I couldn't figure out why, if UDP wasn't sending a newline, it > > wasn't > > > > > > > causing the same problem. I wasn't thinking about the fact that > > the > > > > > > > connection was closing, thus ending the stream. > > > > > > > > > > > > > > Thanks for the help! > > > > > > > > > > > > > > ______________________________________________________________ > > > > > > > > > > > > > > Clayton Dukes > > > > > > > ______________________________________________________________ > > > > > > > > > > > > > > > > > > > > > On Thu, Aug 19, 2010 at 11:29 AM, <syslogng(a)feystorm.net> wrote: > > > > > > > > > > > > > > > I explained it already :-) > > > > > > > > When the message comes in over TCP and doesnt end with a > > newline, > > > > > > syslog-ng > > > > > > > > assumes the message is going to be continued in another packet. > > > > When > > > > > > the > > > > > > > > cumulative total of all the messages exceeds the max message > > size > > > > it > > > > > > flushes > > > > > > > > the buffer out and you get all the messages mashed together at > > > > once. > > > > > > > > You can try filing a bug report on bugzilla.balabit.com and > > > > request a > > > > > > new > > > > > > > > flag or something that treats each packet on a tcp source as a > > > > separate > > > > > > > > message, but I'd say the problem is more cisco than syslog-ng > > since > > > > > > > > syslog-ng works fine with all other sources except cisco > > devices > > > > :-/ > > > > > > > > Look at it this way, every thing that sends logs out to tcp > > expects > > > > the > > > > > > > > receiving syslog daemon to treat a packet without a newline as > > a > > > > > > message to > > > > > > > > be continued in a later packet. If syslog-ng changed that > > default > > > > > > behavior, > > > > > > > > all these other things that expect the behavior would break. > > > > > > > > > > > > > > > > -Patrick > > > > > > > > > > > > > > > > > > > > > > > > Sent: Thursday, August 19, 2010 9:12:36 AM > > > > > > > > > > > > > > > > From: Clayton Dukes <cdukes(a)gmail.com> <cdukes(a)gmail.com> > > > > > > > > To: Syslog-ng users' and developers' mailing list > > > > > > > > <syslog-ng(a)lists.balabit.hu> <syslog-ng(a)lists.balabit.hu> > > > > > > > > Subject: Re: [syslog-ng] TCP recv bug in syslog-ng v2.09? > > > > > > > > > > > > > > > > Following up on this: > > > > > > > > > > > > > > > > Interestingly, however, is that when I use tcpdump, I DO see > > each > > > > > > message > > > > > > > > coming in via TCP...so now I'm leaning back towards syslog-ng > > being > > > > the > > > > > > > > problem. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Here's what I can see: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > UDP: > > > > > > > > > > > > > > > > #tcpdump -vvv host 14.3.23.50 -i eth0 > > > > > > > > > > > > > > > > tcpdump: listening on eth0, link-type EN10MB (Ethernet), > > capture > > > > size > > > > > > 96 > > > > > > > > bytes > > > > > > > > > > > > > > > > 09:13:31.465239 IP (tos 0xb8, ttl 251, id 1323, offset 0, flags > > > > [none], > > > > > > > > proto UDP (17), length 134) > > > > > > > > > > > > > > > > 14.3.23.50.51526 > server.x.com.syslog: SYSLOG, length: 106 > > > > > > > > > > > > > > > > Facility local7 (23), Severity notice (5) > > > > > > > > > > > > > > > > Msg: 2975: *Aug 19 12:34:19.465: > > %SYS-5-CONFIG_I[|syslog] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TCP: > > > > > > > > > > > > > > > > #tcpdump -vvv host 14.3.23.50 -i eth0 > > > > > > > > > > > > > > > > 09:46:29.902063 IP (tos 0x0, ttl 251, id 12253, offset 0, flags > > > > [none], > > > > > > > > proto TCP (6), length 146) > > > > > > > > > > > > > > > > 14.3.23.50.31746 > server.x.com.601: Flags [.], seq > > 233:339, > > > > ack 1, > > > > > > win > > > > > > > > 4128, length 106 > > > > > > > > > > > > > > > > 09:46:29.902077 IP (tos 0x0, ttl 64, id 27779, offset 0, flags > > > > [DF], > > > > > > proto > > > > > > > > TCP (6), length 40) > > > > > > > > > > > > > > > > server.x.com.601 > 14.3.23.50.31746: Flags [.], cksum > > 0xa3bb > > > > > > (correct), > > > > > > > > seq 1, ack 339, win 5840, length 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, it looks like the syslog message does end, but why is > > syslog-ng > > > > > > > > buffering and showing multiple TCP-based messages as a single > > > > message? > > > > > > > > > > > > > > > > Do I have something misconfigured? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________ > > > > > > > > > > > > > > > > Clayton Dukes > > > > > > > > ______________________________________________________________ > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 17, 2010 at 3:14 PM, Clayton Dukes < > > cdukes(a)gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > >> yikes! > > > > > > > >> seriously? > > > > > > > >> Guess I'll have to file a bug internally :-) > > > > > > > >> Can someone else positively verify this? > > > > > > > >> Or any suggestions on how I can so that we can recreate it in > > a > > > > lab? > > > > > > > >> > > > > > > > >> ______________________________________________________________ > > > > > > > >> > > > > > > > >> Clayton Dukes > > > > > > > >> ______________________________________________________________ > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> On Tue, Aug 17, 2010 at 2:52 PM, <syslogng(a)feystorm.net> > > wrote: > > > > > > > >> > > > > > > > >>> If I recall correctly its because cisco equipment doesnt > > > > terminate > > > > > > its > > > > > > > >>> log entries with newlines, so when sending via TCP, syslog-ng > > > > thinks > > > > > > the > > > > > > > >>> message is going to be continued in another packet (UDP is > > > > assumed to > > > > > > be 1 > > > > > > > >>> packet per log entry). > > > > > > > >>> The only way to fix this is an ugly hack to set the timeout > > so > > > > that > > > > > > when > > > > > > > >>> it doesnt get a reply within a certain time, it assumes the > > log > > > > entry > > > > > > ended. > > > > > > > >>> but if several log entries are sent within the timeout, then > > > > they'll > > > > > > all be > > > > > > > >>> mashed together into 1 syslog-ng entry. > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> Sent: Tuesday, August 17, 2010 12:28:28 PM > > > > > > > >>> From: Clayton Dukes <cdukes(a)gmail.com> <cdukes(a)gmail.com> > > > > > > > >>> To: Syslog-ng users' and developers' mailing list > > > > > > > >>> <syslog-ng(a)lists.balabit.hu> <syslog-ng(a)lists.balabit.hu> > > > > > > > >>> Subject: [syslog-ng] TCP recv bug in syslog-ng v2.09? > > > > > > > >>> > > > > > > > >>> Hey guys, > > > > > > > >>> Are there any known bugs for syslog-ng v2.09 that won't > > allow a > > > > > > cisco > > > > > > > >>> router to send logs over tcp? > > > > > > > >>> I can see a connection established in syslog-ng. > > > > > > > >>> I also see the message come in via tcpdump, but nothing in > > > > > > syslog-ng's > > > > > > > >>> output. > > > > > > > >>> If I change the router from tcp to udp, messages come in as > > > > expected. > > > > > > > >>> > > > > > > > >>> *Router config:* > > > > > > > >>> > > > > > > > >>> logging source-interface Loopback0 > > > > > > > >>> logging 172.18.224.150 > > > > > > > >>> logging host 172.18.224.190 transport tcp > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> *syslog-ng config:* > > > > > > > >>> > > > > > > > >>> source s_all { > > > > > > > >>> udp(); > > > > > > > >>> tcp(ip(11.31.130.99) port(8002) > > max-connections(300)); > > > > > > > >>> tcp(ip(172.18.224.190) port(601) > > max-connections(300)); > > > > > > > >>> }; > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> *debug output:* > > > > > > > >>> I commented out the line above for the other interface > > > > > > (11.31.130.99), > > > > > > > >>> restarted and this is all I see: > > > > > > > >>> Syslog connection accepted; from='AF_INET(14.3.23.50:63845 > > )', > > > > > > > >>> to='AF_INET(172.18.224.190:601)' > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> *tcpdump:* > > > > > > > >>> > > > > > > > >>> 14:13:46.914566 IP (tos 0x0, ttl 251, id 4303, offset 0, > > flags > > > > > > [none], > > > > > > > >>> proto TCP (6), length 134) > > > > > > > >>> 14.3.23.50.63845 > xxx.com.601: Flags [.], seq 230:324, > > ack > > > > 1, > > > > > > win > > > > > > > >>> 4128, length 94 > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> *Router debug:* > > > > > > > >>> > > > > > > > >>> *Aug 17 17:34:19.772: %SYS-5-CONFIG_I: Configured from > > console > > > > by > > > > > > pnoc > > > > > > > >>> on vty0 (172.18.224.151) > > > > > > > >>> *Aug 17 17:34:20.776: Released port 15205 in Transport Port > > Agent > > > > for > > > > > > > >>> TCP IP type 1 delay 240000 > > > > > > > >>> *Aug 17 17:34:20.776: TCB 0x850F9754 destroyed > > > > > > > >>> *Aug 17 17:34:25.775: TCB83648E60 created > > > > > > > >>> *Aug 17 17:34:25.775: TCB83648E60 setting property TCP_PID > > (8) > > > > > > 845083E4 > > > > > > > >>> *Aug 17 17:34:25.775: TCB83648E60 setting property > > TCP_NO_DELAY > > > > (1) > > > > > > > >>> 845083E8 > > > > > > > >>> *Aug 17 17:34:25.775: TCB83648E60 setting property TCP > > keepalive > > > > > > timeout > > > > > > > >>> (17) 845084A0 > > > > > > > >>> *Aug 17 17:34:25.775: TCP: Random local port generated 63845, > > > > network > > > > > > 1 > > > > > > > >>> *Aug 17 17:34:25.775: TCB83648E60 bound to 14.3.23.50.63845 > > > > > > > >>> *Aug 17 17:34:25.775: Reserved port 63845 in Transport Port > > Agent > > > > for > > > > > > > >>> TCP IP type 1 > > > > > > > >>> *Aug 17 17:34:25.775: TCP: sending SYN, seq 3300233565, ack 0 > > > > > > > >>> *Aug 17 17:34:25.775: TCP0: Connection to 172.18.224.190:601 > > , > > > > > > > >>> advertising MSS 536 > > > > > > > >>> *Aug 17 17:34:25.775: TCP0: state was CLOSED -> SYNSENT > > [63845 -> > > > > > > > >>> 172.18.224.190(601)] > > > > > > > >>> *Aug 17 17:34:25.779: TCP0: state was SYNSENT -> ESTAB [63845 > > -> > > > > > > > >>> 172.18.224.190(601)] > > > > > > > >>> *Aug 17 17:34:25.779: TCP: tcb 83648E60 connection to > > > > > > 172.18.224.190:601, > > > > > > > >>> peer MSS 1460, MSS is 536 > > > > > > > >>> *Aug 17 17:34:25.779: TCB83648E60 connected to > > 172.18.224.190.601 > > > > > > > >>> *Aug 17 17:34:25.779: %SYS-6-LOGGINGHOST_STARTSTOP: Logging > > to > > > > host > > > > > > > >>> 172.18.224.190 port 601 started - reconnection > > > > > > > >>> > > > > > > > >>> > > ______________________________________________________________ > > > > > > > >>> > > > > > > > >>> Clayton Dukes > > > > > > > >>> > > ______________________________________________________________ > > > > > > > >>> > > > > > > > >>> ------------------------------ > > > > > > > >>> > > > > > > > >>> > > > > > > > > > > > > ______________________________________________________________________________ > > > > > > > >>> Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > > > >>> Documentation: > > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > > > > >>> FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > > > > > > ______________________________________________________________________________ > > > > > > > >>> Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > > > >>> Documentation: > > > > > > > >>> > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > > > > >>> FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >> > > > > > > > > ------------------------------ > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > > > > > > Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > > > > Documentation: > > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > > > > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > > > > > > Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > > > > Documentation: > > > > > > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > > > > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > > > > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > > > Documentation: > > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > > > > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > > > > > > > > > > > > > > > >

1 0

How to change timestamp?
by Gregers 18 Aug '10

18 Aug '10

Aug 18 02:16:15 SRC_IP HOST %OrionAlertEngine: "BLA BLA BLA message" I receives this kind of messages quite often. My problem is that it is a perfectly fine header and tag, but the original timestamp is in AM/PM format. How can I force syslog-ng to relay this message with the timestamp from the relaying host(which uses 24 hour format), instead of the sending host? TIA Gregers

2 1

Trying to write syslog to /opt
by Johnny Lapenna 18 Aug '10

18 Aug '10

Hi I'm trying to write the following log to /opt/ in which I am unable to: in syslog-ng.conf I have the following: #filter f_local { facility(local0, local1, local2, local3, # local4, local5, local6, local7); }; filter f_local { facility( local1, local2, local3, local4, local5, local7); }; filter f_sol { facility(local0); }; # Log SOL messages in a separate log destination solmessages { file("/opt/solLOG/sol.log"); }; log { source(src); filter(f_sol); destination(solmessages); }; #destination solmessages { file("/var/log/sol.log"); }; #log { source(src); filter(f_sol); destination(solmessages); }; The above does not work but if I comment out: #destination solmessages { file("/opt/solLOG/sol.log"); }; #log { source(src); filter(f_sol); destination(solmessages); }; destination solmessages { file("/var/log/sol.log"); }; log { source(src); filter(f_sol); destination(solmessages); }; The log is written to /var/log/sol.log, is there a reason why the destination path could only be /var/log/? Thank you in advance. Johnny

2 3

3.1.2 OSE $SEQNUM Documentation Error?
by Mike Civil 17 Aug '10

17 Aug '10

I've been playing with Postgresql and 3.1.2 OSE on Gentoo and found a discrepancy when using the $SEQNUM macro and logging and recording on the same host. The 3.1 (& 3.2) docs state "The sequence number increases for every message, and is not lost even if syslog-ng is reloaded or restarted." However I noticed that each time syslog-ng is restarted the value returned by $SEQNUM starts at 1. A quick check of the source code (specifically templates.c and misc.h) shows no logic to keep state, the value is initialised to 1 on program start. Is this an OSE/PE difference, a documentation hiccup or (more likely!) have I got confused somewhere?

3 5