Problem with syslog-ng version 2
OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2 Hi, syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config) I have the following in my syslog-ng configuration file: source ESRS { file("/opt/esrsve/gateway/xGate.log"); }; destination esrsfile{ file("/var/log/max.log" );}; log { source(ESRS); destination(esrsfile); }; The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines. ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ... Aug 29 14:00:02 hostname C Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname t Aug 29 14:00:02 hostname Aug 29 14:00:02 hostname V Aug 29 14:00:02 hostname a Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname O Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname i Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname e ... ... What am I doing wrong here? Thanks much!
Hi Maxime,
OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2
that's a very, VERY old Syslog NG version. There are current versions around, so why do you stick with that old stuff?
ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ...
Aug 29 14:00:02 hostname C
try flags( no-parse ) on the source. Not sure if that works with 2.0.9, though ... Best regards, Peter.
Well, older syslog-ng versions probably folowed the file as it was written, and if only a partial line was present, it was assumed to be the entire message. Maybe the application is writing that file one-character at a time? But I agree with Peter, 2.0.9 is probably a decade old, so in order to properly support your problems, you should definitely upgrade to something newer. -- Bazsi On Mon, Aug 29, 2016 at 8:48 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca> wrote:
OS: SLes 11 sp4
syslog-ng: syslog-ng-2.0.9-27.34.39.2
Hi,
syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config)
I have the following in my syslog-ng configuration file:
source ESRS {
file("/opt/esrsve/gateway/xGate.log");
};
destination esrsfile{ file("/var/log/max.log" );};
log { source(ESRS);
destination(esrsfile);
};
The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines.
ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus *Current Value::Offline*
And the destination file looiks like this:
... ...
Aug 29 14:00:02 hostname C
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname t
Aug 29 14:00:02 hostname
Aug 29 14:00:02 hostname V
Aug 29 14:00:02 hostname a
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname O
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname i
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname e
... ...
What am I doing wrong here?
Thanks much!
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thanks for the update guys. I know the version is old, problem is I cannot update it since it’s a black box and we would lose support from EMC. Ill see what I can do. De : syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] De la part de Scheidler, Balázs Envoyé : 31 août 2016 05:15 À : Syslog-ng users' and developers' mailing list Objet : Re: [syslog-ng] Problem with syslog-ng version 2 Courriel provenant de l’externe ATTENTION, avant d’accéder à une pièce jointe ou à un lien de ce courriel, assurez-vous que celui-ci provient d’un tiers de confiance. Well, older syslog-ng versions probably folowed the file as it was written, and if only a partial line was present, it was assumed to be the entire message. Maybe the application is writing that file one-character at a time? But I agree with Peter, 2.0.9 is probably a decade old, so in order to properly support your problems, you should definitely upgrade to something newer. -- Bazsi On Mon, Aug 29, 2016 at 8:48 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca<mailto:Alarie.Maxime@hydro.qc.ca>> wrote: OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2 Hi, syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config) I have the following in my syslog-ng configuration file: source ESRS { file("/opt/esrsve/gateway/xGate.log"); }; destination esrsfile{ file("/var/log/max.log" );}; log { source(ESRS); destination(esrsfile); }; The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines. ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ... Aug 29 14:00:02 hostname C Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname t Aug 29 14:00:02 hostname Aug 29 14:00:02 hostname V Aug 29 14:00:02 hostname a Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname O Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname i Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname e ... ... What am I doing wrong here? Thanks much! ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
We saw this behavior with the old 2.x versions as well even when the source was the kernel syslog API. When the messages were written to a file or forwarded to a central syslog server, some messages were unwrapped to a character by character stream. It always seemed to be the same application that had only some of its messages unwrapped. The symptoms went away after we upgraded to 3.x I never tried to fix it, or track it down. It was only 100 messages per day in a stream of 100 million. I just upgraded. Evan. On 08/31/2016 06:25 AM, Alarie, Maxime wrote:
Thanks for the update guys.
I know the version is old, problem is I cannot update it since it’s a black box and we would lose support from EMC. Ill see what I can do.
*De :*syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] *De la part de* Scheidler, Balázs *Envoyé :* 31 août 2016 05:15 *À :* Syslog-ng users' and developers' mailing list *Objet :* Re: [syslog-ng] Problem with syslog-ng version 2
*Courriel provenant de l’externe ATTENTION, avant d’accéder à une pièce jointe ou à un lien de ce courriel, assurez-vous que celui-ci provient d’un tiers de**confiance.*
Well, older syslog-ng versions probably folowed the file as it was written, and if only a partial line was present, it was assumed to be the entire message.
Maybe the application is writing that file one-character at a time?
But I agree with Peter, 2.0.9 is probably a decade old, so in order to properly support your problems, you should definitely upgrade to something newer.
-- Bazsi
On Mon, Aug 29, 2016 at 8:48 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca <mailto:Alarie.Maxime@hydro.qc.ca>> wrote:
OS: SLes 11 sp4
syslog-ng: syslog-ng-2.0.9-27.34.39.2
Hi,
syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config)
I have the following in my syslog-ng configuration file:
source ESRS {
file("/opt/esrsve/gateway/xGate.log");
};
destination esrsfile{ file("/var/log/max.log" );};
log { source(ESRS);
destination(esrsfile);
};
The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines.
ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus *Current Value::Offline*
And the destination file looiks like this:
... ...
Aug 29 14:00:02 hostname C
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname t
Aug 29 14:00:02 hostname
Aug 29 14:00:02 hostname V
Aug 29 14:00:02 hostname a
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname O
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname i
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname e
... ...
What am I doing wrong here?
Thanks much!
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thanks Evan, I have the exact same problem. Not all entries in the file are causing this. Thanks a lot and have a nice day. De : syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] De la part de Evan Rempel Envoyé : 31 août 2016 09:30 À : syslog-ng@lists.balabit.hu Objet : Re: [syslog-ng] Problem with syslog-ng version 2 Courriel provenant de l’externe ATTENTION, avant d’accéder à une pièce jointe ou à un lien de ce courriel, assurez-vous que celui-ci provient d’un tiers de confiance. We saw this behavior with the old 2.x versions as well even when the source was the kernel syslog API. When the messages were written to a file or forwarded to a central syslog server, some messages were unwrapped to a character by character stream. It always seemed to be the same application that had only some of its messages unwrapped. The symptoms went away after we upgraded to 3.x I never tried to fix it, or track it down. It was only 100 messages per day in a stream of 100 million. I just upgraded. Evan. On 08/31/2016 06:25 AM, Alarie, Maxime wrote: Thanks for the update guys. I know the version is old, problem is I cannot update it since it’s a black box and we would lose support from EMC. Ill see what I can do. De : syslog-ng-bounces@lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu] De la part de Scheidler, Balázs Envoyé : 31 août 2016 05:15 À : Syslog-ng users' and developers' mailing list Objet : Re: [syslog-ng] Problem with syslog-ng version 2 Courriel provenant de l’externe ATTENTION, avant d’accéder à une pièce jointe ou à un lien de ce courriel, assurez-vous que celui-ci provient d’un tiers de confiance. Well, older syslog-ng versions probably folowed the file as it was written, and if only a partial line was present, it was assumed to be the entire message. Maybe the application is writing that file one-character at a time? But I agree with Peter, 2.0.9 is probably a decade old, so in order to properly support your problems, you should definitely upgrade to something newer. -- Bazsi On Mon, Aug 29, 2016 at 8:48 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca<mailto:Alarie.Maxime@hydro.qc.ca>> wrote: OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2 Hi, syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config) I have the following in my syslog-ng configuration file: source ESRS { file("/opt/esrsve/gateway/xGate.log"); }; destination esrsfile{ file("/var/log/max.log" );}; log { source(ESRS); destination(esrsfile); }; The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines. ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ... Aug 29 14:00:02 hostname C Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname t Aug 29 14:00:02 hostname Aug 29 14:00:02 hostname V Aug 29 14:00:02 hostname a Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname O Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname i Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname e ... ... What am I doing wrong here? Thanks much! ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, If you don't mind, can you pls share what kind of EMC box is this? -- Bazsi On Wed, Aug 31, 2016 at 3:25 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca> wrote:
Thanks for the update guys.
I know the version is old, problem is I cannot update it since it’s a black box and we would lose support from EMC. Ill see what I can do.
*De :* syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@ lists.balabit.hu] *De la part de* Scheidler, Balázs *Envoyé :* 31 août 2016 05:15 *À :* Syslog-ng users' and developers' mailing list *Objet :* Re: [syslog-ng] Problem with syslog-ng version 2
*Courriel provenant de l’externe ATTENTION, avant d’accéder à une pièce jointe ou à un lien de ce courriel, assurez-vous que celui-ci provient d’un tiers de* *confiance.*
Well, older syslog-ng versions probably folowed the file as it was written, and if only a partial line was present, it was assumed to be the entire message.
Maybe the application is writing that file one-character at a time?
But I agree with Peter, 2.0.9 is probably a decade old, so in order to properly support your problems, you should definitely upgrade to something newer.
-- Bazsi
On Mon, Aug 29, 2016 at 8:48 PM, Alarie, Maxime <Alarie.Maxime@hydro.qc.ca> wrote:
OS: SLes 11 sp4
syslog-ng: syslog-ng-2.0.9-27.34.39.2
Hi,
syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config)
I have the following in my syslog-ng configuration file:
source ESRS {
file("/opt/esrsve/gateway/xGate.log");
};
destination esrsfile{ file("/var/log/max.log" );};
log { source(ESRS);
destination(esrsfile);
};
The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines.
ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus *Current Value::Offline*
And the destination file looiks like this:
... ...
Aug 29 14:00:02 hostname C
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname t
Aug 29 14:00:02 hostname
Aug 29 14:00:02 hostname V
Aug 29 14:00:02 hostname a
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname O
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname i
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname e
... ...
What am I doing wrong here?
Thanks much!
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Alarie, Maxime
-
Evan Rempel
-
Peter Eckel
-
Scheidler, Balázs