Hi there, As I can't find this on the web, I'm asking here : I'm receiving the IIS servers logs through my LAN and syslog-ng writes everything to a local files, that is parsed during the night by awstats. The only thing is I can't have syslog-ng to write raw logs to files, it absolutely has to write something (either with $MSG or $MESSAGE). I even tried with $MSGONLY but now I get the opposite problem, it truncates data ! $MSG : IISWebLog 0 2008-09-30 14:02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727 ;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;+InfoPath.2) http://mypublicwebsite/home/regf 200 2185 With $MSGONLY, I'll get : 02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727 ;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;+InfoPath.2) http://mypublicwebsite/home/regf 200 2185 Here is my very simple template : template t_IIS { template("$MESSAGE\n"); template_escape(no); }; Destination and filter : destination df_ mypublicwebsite _IISWebLog { file("/data/syslog/mypublicwebsite/IISWebLog" template(t_IIS)); }; filter f_ mypublicwebsite _IISWebLog {(host("^192\.168\.117\.131$") or host("^192\.168\.117\.132$") or host("^192\.168\.117\.133$") or host("^192\.168\.117\.134$") or host("^192\.168\.117\.135$") ) and facility(local4) and program("IISWebLog"); }; log { source(s_net); filter(f_ mypublicwebsite _IISWebLog); destination(df_ mypublicwebsite _IISWebLog); }; # uname -a Linux TR-ADM 2.6.18-4-amd64 #1 SMP Fri May 4 00:37:33 UTC 2007 x86_64 GNU/Linux Anything I'm missing ? Any help would be greatly appreciated. Best regards Steven
Hello,
I'm receiving the IIS servers logs through my LAN and syslog-ng writes everything to a local files, that is parsed during the night by awstats.
The only thing is I can't have syslog-ng to write raw logs to files, it absolutely has to write something (either with $MSG or $MESSAGE). I even tried with $MSGONLY but now I get the opposite problem, it truncates data !
Sorry, I don't get you. What is the EXACT problem? Are your clients violate the syslog protocol? If yes then you have to disable log parsing for the source by using flags(no-parse);
$MSG :
IISWebLog 0 2008-09-30 14:02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET +CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;> +InfoPath.2) http://mypublicwebsite/home/regf 200 2185
With $MSGONLY, I'll get :
02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET +CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;> +InfoPath.2) http://mypublicwebsite/home/regf 200 2185
Here is my very simple template :
template t_IIS {
template("$MESSAGE\n");
template_escape(no);
};
Destination and filter :
destination df_ mypublicwebsite _IISWebLog { file("/data/syslog/mypublicwebsite/IISWebLog" template(t_IIS)); };
filter f_ mypublicwebsite _IISWebLog {(host("^192\.168\.117\.131$")
or host("^192\.168\.117\.132$")
or host("^192\.168\.117\.133$")
or host("^192\.168\.117\.134$")
or host("^192\.168\.117\.135$") ) and facility(local4) and program("IISWebLog"); };
easier to use "^192\.168\.117\.13[1-5]$" and it's slightly more efficient Regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
Thanks for your attention Syslog-ng receives good data, as it seems (when no modification on output is done, all data is written to log file). If I use template t_IIS { template("$MESSAGE\n"); template_escape(no); } Then the output in logfile is : IISWebLog 0 2008-09-30 14:02:40 GET (...) If I use $MSGONLY instead of $MESSAGE then I get : 02:40 GET (...) What I should have is : 2008-09-30 14:02:40 GET (...) Date and hours gets truncated. Regards, Steven -----Message d'origine----- De : syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] De la part de Geller, Sandor (IT) Envoyé : mardi 30 septembre 2008 17:34 À : 'Syslog-ng users' and developers' mailing list' Objet : Re: [syslog-ng] $MSGONLY bad output Hello,
I'm receiving the IIS servers logs through my LAN and syslog-ng writes everything to a local files, that is parsed during the night by awstats.
The only thing is I can't have syslog-ng to write raw logs to files, it absolutely has to write something (either with $MSG or $MESSAGE). I even tried with $MSGONLY but now I get the opposite problem, it truncates data !
Sorry, I don't get you. What is the EXACT problem? Are your clients violate the syslog protocol? If yes then you have to disable log parsing for the source by using flags(no-parse);
$MSG :
IISWebLog 0 2008-09-30 14:02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET +CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;> +InfoPath.2) http://mypublicwebsite/home/regf 200 2185
With $MSGONLY, I'll get :
02:40 GET /Localized/fr/Images/New2/tlogo.gif - 95.35.41.243 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET +CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506;> +InfoPath.2) http://mypublicwebsite/home/regf 200 2185
Here is my very simple template :
template t_IIS {
template("$MESSAGE\n");
template_escape(no);
};
Destination and filter :
destination df_ mypublicwebsite _IISWebLog { file("/data/syslog/mypublicwebsite/IISWebLog" template(t_IIS)); };
filter f_ mypublicwebsite _IISWebLog {(host("^192\.168\.117\.131$")
or host("^192\.168\.117\.132$")
or host("^192\.168\.117\.133$")
or host("^192\.168\.117\.134$")
or host("^192\.168\.117\.135$") ) and facility(local4) and program("IISWebLog"); };
easier to use "^192\.168\.117\.13[1-5]$" and it's slightly more efficient Regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
On Tue, 2008-09-30 at 17:44 +0200, Steven wrote:
Thanks for your attention
Syslog-ng receives good data, as it seems (when no modification on output is done, all data is written to log file).
If I use template t_IIS { template("$MESSAGE\n"); template_escape(no); }
Then the output in logfile is : IISWebLog 0 2008-09-30 14:02:40 GET (...)
This is not standard syslog format, thus syslog-ng parses the complete string into the "MESSAGE" part of the syslog (both the IISWebLog header and the date are part of the message, no date header is present). MSGONLY in this case assumes that the message is in the format: program[pid]: payload Thus it searches for the first colon and expands to "payload". In your case you cannot use MSGONLY as the message is not in syslog format. I almost finished syslog-ng OSE 3.0 backports, and that has features that you could use.
If I use $MSGONLY instead of $MESSAGE then I get : 02:40 GET (...)
What I should have is : 2008-09-30 14:02:40 GET (...)
Date and hours gets truncated.
-- Bazsi
Ok, good news then. In the meantime I use $MESSAGE and "cut -c13-5010" does the trick. Regards Steven -----Message d'origine----- De : syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] De la part de Balazs Scheidler Envoyé : mercredi 1 octobre 2008 11:16 À : Syslog-ng users' and developers' mailing list Objet : Re: [syslog-ng] $MSGONLY bad output On Tue, 2008-09-30 at 17:44 +0200, Steven wrote:
Thanks for your attention
Syslog-ng receives good data, as it seems (when no modification on output is done, all data is written to log file).
If I use template t_IIS { template("$MESSAGE\n"); template_escape(no); }
Then the output in logfile is : IISWebLog 0 2008-09-30 14:02:40 GET (...)
This is not standard syslog format, thus syslog-ng parses the complete string into the "MESSAGE" part of the syslog (both the IISWebLog header and the date are part of the message, no date header is present). MSGONLY in this case assumes that the message is in the format: program[pid]: payload Thus it searches for the first colon and expands to "payload". In your case you cannot use MSGONLY as the message is not in syslog format. I almost finished syslog-ng OSE 3.0 backports, and that has features that you could use.
If I use $MSGONLY instead of $MESSAGE then I get : 02:40 GET (...)
What I should have is : 2008-09-30 14:02:40 GET (...)
Date and hours gets truncated.
-- Bazsi ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (3)
-
Balazs Scheidler
-
Geller, Sandor (IT)
-
Steven