Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off
The application is custom in-house. I'm really not sure what RFC they've followed. However your suggestion about using 'no-multi-line' seems to have take care of the split lines. Thanks, Jo --- On Fri, 6/19/09, Balazs Scheidler <bazsi@balabit.hu> wrote: From: Balazs Scheidler <bazsi@balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 4:38 AM On Wed, 2009-06-17 at 12:10 -0700, Joe Hansen wrote:
We're seeing lines in /var/log/messages that are chopped off. They should look like this:
Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> ,to=To: ""Graeme Ross"" <gross@mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858"
But they look like this:
Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net>
Anyone seen this?
Thanks
Well, can you tell us a little bit more on your configuration? What protocol do you use on the source side? Do you happen to use the new RFC5424 style protocol, or the legacy one? What is generating those messages, can you show us a tcpdump/strace snippet that shows how that frame is travelling the network? Also, syslog-ng 3.0 does not remove embedded NL characters by default, maybe the rest of the message continues on the next line? You can reenable the previous behaviour by using the 'no-multi-line' flag for your source or destination (e.g. you can change the multi-line handling not just for a given source, but also handle the same message differently in different destinations). -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html __________________________________________________________________ Get a sneak peak at messages with a handy reading pane with All new Yahoo! Mail: http://ca.promos.yahoo.com/newmail/overview2/
Just when things were going well, syslog-ng started to chop off lines again. It seems to be doing this under heavy load. Will this get fixed in the later version? thanks, Joe --- On Fri, 6/19/09, Joe Hansen <yrmf250@yahoo.com> wrote: From: Joe Hansen <yrmf250@yahoo.com> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 9:32 PM The application is custom in-house. I'm really not sure what RFC they've followed. However your suggestion about using 'no-multi-line' seems to have take care of the split lines. Thanks, Jo --- On Fri, 6/19/09, Balazs Scheidler <bazsi@balabit.hu> wrote: From: Balazs Scheidler <bazsi@balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 4:38 AM On Wed, 2009-06-17 at 12:10 -0700, Joe Hansen wrote:
We're seeing lines in /var/log/messages that are chopped off. They should look like this:
Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> ,to=To: ""Graeme Ross"" <gross@mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858"
But they look like this:
Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net>
Anyone seen this?
Thanks
Well, can you tell us a little bit more on your configuration? What protocol do you use on the source side? Do you happen to use the new RFC5424 style protocol, or the legacy one? What is generating those messages, can you show us a tcpdump/strace snippet that shows how that frame is travelling the network? Also, syslog-ng 3.0 does not remove embedded NL characters by default, maybe the rest of the message continues on the next line? You can reenable the previous behaviour by using the 'no-multi-line' flag for your source or destination (e.g. you can change the multi-line handling not just for a given source, but also handle the same message differently in different destinations). -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html Looking for the perfect gift? Give the gift of Flickr! -----Inline Attachment Follows----- ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html __________________________________________________________________ Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now http://ca.toolbar.yahoo.com.
You can try setting the log_msg_size() to increase the maximum msg size. What kind of source do you use? M On Tue, 2009-08-25 at 15:48 -0700, Joe Hansen wrote:
Just when things were going well, syslog-ng started to chop off lines again. It seems to be doing this under heavy load.
Will this get fixed in the later version?
thanks, Joe
--- On Fri, 6/19/09, Joe Hansen <yrmf250@yahoo.com> wrote:
From: Joe Hansen <yrmf250@yahoo.com> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 9:32 PM
The application is custom in-house. I'm really not sure what RFC they've followed. However your suggestion about using 'no-multi-line' seems to have take care of the split lines.
Thanks, Jo
--- On Fri, 6/19/09, Balazs Scheidler <bazsi@balabit.hu> wrote:
From: Balazs Scheidler <bazsi@balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 4:38 AM
On Wed, 2009-06-17 at 12:10 -0700, Joe Hansen wrote: > We're seeing lines in /var/log/messages that are chopped off. They > should look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> ,to=To: ""Graeme Ross"" <gross@mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858" > > But they look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> > > > Anyone seen this? > > Thanks > > >
Well, can you tell us a little bit more on your configuration? What protocol do you use on the source side? Do you happen to use the new RFC5424 style protocol, or the legacy one?
What is generating those messages, can you show us a tcpdump/strace snippet that shows how that frame is travelling the network?
Also, syslog-ng 3.0 does not remove embedded NL characters by default, maybe the rest of the message continues on the next line?
You can reenable the previous behaviour by using the 'no-multi-line' flag for your source or destination (e.g. you can change the multi-line handling not just for a given source, but also handle the same message differently in different destinations).
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________ Looking for the perfect gift? Give the gift of Flickr!
-----Inline Attachment Follows-----
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________ Looking for the perfect gift? Give the gift of Flickr! ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D
The problem is not the length of the messages, it is the '\r\n' that the application is inserting in the msg. We want syslog-ng to ignore the 'r\n'. --- On Wed, 8/26/09, ILLES, Marton <illes.marton@balabit.hu> wrote: From: ILLES, Marton <illes.marton@balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Wednesday, August 26, 2009, 9:34 AM You can try setting the log_msg_size() to increase the maximum msg size. What kind of source do you use? M On Tue, 2009-08-25 at 15:48 -0700, Joe Hansen wrote:
Just when things were going well, syslog-ng started to chop off lines again. It seems to be doing this under heavy load.
Will this get fixed in the later version?
thanks, Joe
--- On Fri, 6/19/09, Joe Hansen <yrmf250@yahoo.com> wrote: From: Joe Hansen <yrmf250@yahoo.com> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 9:32 PM The application is custom in-house. I'm really not sure what RFC they've followed. However your suggestion about using 'no-multi-line' seems to have take care of the split lines. Thanks, Jo --- On Fri, 6/19/09, Balazs Scheidler <bazsi@balabit.hu> wrote: From: Balazs Scheidler <bazsi@balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.0 seem to chop some of the long lines off To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Received: Friday, June 19, 2009, 4:38 AM On Wed, 2009-06-17 at 12:10 -0700, Joe Hansen wrote: > We're seeing lines in /var/log/messages that are chopped off. They > should look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> ,to=To: ""Graeme Ross"" <gross@mobileemail.vodafone.com.au>,subject=Turn-key call centre for lease. Launceston, Tasmania","worker/CPSendWork.cpp:858" > > But they look like this: > > Jun 17 00:34:36 bxe1001 WORKER[18808]: DEBUG,email header > info,3893336992,1941292826,CPSendWork::ProcessConverterSuccessMessage,"man_id=551797512,srp_id=GBISXAPC00S07,from=From: ""info@callcentres.net"" <info@callcentres.net> > > > Anyone seen this? > > Thanks > > > Well, can you tell us a little bit more on your configuration? What protocol do you use on the source side? Do you happen to use the new RFC5424 style protocol, or the legacy one? What is generating those messages, can you show us a tcpdump/strace snippet that shows how that frame is travelling the network? Also, syslog-ng 3.0 does not remove embedded NL characters by default, maybe the rest of the message continues on the next line? You can reenable the previous behaviour by using the 'no-multi-line' flag for your source or destination (e.g. you can change the multi-line handling not just for a given source, but also handle the same message differently in different destinations). -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html ______________________________________________________________ Looking for the perfect gift? Give the gift of Flickr! -----Inline Attachment Follows----- ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________ Looking for the perfect gift? Give the gift of Flickr! ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html __________________________________________________________________ Make your browsing faster, safer, and easier with the new Internet Explorer® 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
participants (2)
-
ILLES, Marton
-
Joe Hansen