[syslog-ng] datetime stamp of syslog-ng...

[Balazs Scheidler]

On Wed, 2008-01-02 at 15:02 +0530, Anant Athavale wrote:

> >
> > On Wed, 2008-01-02 at 09:45 +0530, Anant Athavale wrote:
> >> Dear List,
> >>
> >> I am redirecting a system log of one server to a centralized log
> >> server running syslog-ng.   When we redirect, syslog-ng adds its own
> >> datetime stamp before the actual log entry for each line.  Due to
> >> this, one of the log processing software does not recognize the format
> >> and does not generate any reports.  If I remove those 4 fields (Month
> >> Name, Date, Year and IP address) for ex: Jan 2 2008 10.10.10.10  (here
> >> 10.10.10.10 is the IP address of the system whose logs are getting
> >> stored in central log server) that log processing software works as
> >> expected and generates report.
> >>
> >> Is it possible that, I can block those fields getting recorded.  If
> >> yes, how? and if not, what is the alternative?
> >
> > Syslog-ng tries to parse the time stamp of the original message, and if
> > it is not in a proper format, it assumes the timestamp is part of the
> > message itself and adds a complete syslog header, that's why you see
> > that a timestamp is prepended.
> >
> > Can you show us an example of the original message?
> 
> Here is the sample log entry.
> 
> Jan  2 14:49:07 10.21.3.4 2008-01-02 09:06:27 80 10.21.3.58
> RAGHAVENDRA%20B%20KULKARNI - - PROXIED "none"
> http://www.ndtv.com/convergence/ndtv/images/site/swfs/tickerdotcomnew.swf
>   200 TCP_NC_MISS GET text/html http www.ndtv.com 80
> /convergence/ndtv/miscfiles/desktoptickernewfuture.asp
> ?condition=0&ch=Wed%20Jan%202%2014:34:35%20GMT+0530%202008 asp
> "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322;
> .NET CLR 2.0.50727; IEMB3; IEMB3)" 10.21.3.4 1804 863 -
> Jan  2 14:49:07 10.21.3.4 2008-01-02 09:06:27 1 10.21.3.58
> RAGHAVENDRA%20B%20KULKARNI - - PROXIED "none"
> http://www.ndtv.com/convergence/ndtv/default.aspx  304 TCP_HIT GET
> application/x-javascript http www.ndtv.com 80
> /convergence/ndtv/include/video.js - js "Mozilla/4.0 (compatible; MSIE
> 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; IEMB3;
> IEMB3)" 10.21.3.4 319 847 -
> 

I see, the original message is not in syslog compatible format, that's
why the prepended syslog header. You can customize the file format that
syslog-ng writes out using the template() parameter:

file d_file { file("/var/log/syslog-ng/squid.log" template("$MSG\n")); };

This example skips the syslog header and only includes the payload part
in the output file.

-- 
Bazsi