Message length overflow, line is split, log_msg_size=2048
Hi there, For some time I am running syslog-ng as a backend mostly for snare agents on windoze. I get the following in the log from time to time: Jan 28 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 28 12:01:33 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 28 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 29 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 29 12:00:01 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 29 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 30 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 30 12:00:00 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 30 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 31 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 31 12:00:01 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 31 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 1 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 1 12:01:40 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 1 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 00:12:26 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 12:02:20 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 2 12:12:26 svn01 syslog-ng[12377]: STATS: dropped 0 What does this "Message length overflow" ? How can I find/log who(=pid or IP) is sending long messages? Shall I increase log_msg_size? How? I am using syslog-ng-1.6.9 on Gentoo linux. The global part of the config: ### {{{ global options options { # hostname setup chain_hostnames(no); keep_hostname(no); use_dns(no); dns_cache(no); sync(0); stats(43200); create_dirs(yes); dir_owner(root); dir_group(logop); dir_perm(0750); owner(root); group(logop); perm(0640); }; ### global options }}} Some other bits-n-pieces: source s_network { udp(); }; destination d_ext_by_hosts_ALL { file("/var/log/syslog-ng/raw/remote/$YEAR-$MONTH-$DAY/$HOST/ALL" template("$ISODATE $HOST <$FACILITY.$LEVEL> $MSG\n") template_escape(no) ); }; log { source(s_network); destination(d_ext_by_hosts_ALL); }; Googling found only this unanswered mail from this ML: https://lists.balabit.hu/pipermail/syslog-ng/2005-December/008248.html Kalin. -- | A | | D | | J | | P |
On Fri, 2007-02-02 at 14:27 +0900, Kalin KOZHUHAROV wrote:
Hi there,
For some time I am running syslog-ng as a backend mostly for snare agents on windoze.
I get the following in the log from time to time: Jan 28 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 28 12:01:33 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 28 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 29 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 29 12:00:01 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 29 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 30 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 30 12:00:00 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 30 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 31 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 31 12:00:01 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 31 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 1 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 1 12:01:40 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 1 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 00:12:26 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 12:02:20 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 2 12:12:26 svn01 syslog-ng[12377]: STATS: dropped 0
What does this "Message length overflow" ? How can I find/log who(=pid or IP) is sending long messages?
check the logs in your output files that match the timestamp of the "message length overflow" message. if you have internal() and normal messages in a single file, then the split line is right next to the "message length overflow" line.
Shall I increase log_msg_size?
it's your call. if you don't mind to have garbled messages in your log, then not necessarily. increasing log_msg_size() increases memory usage.
How?
log_msg_size() global option. -- Bazsi
Thank you for the quick response! Balazs Scheidler wrote:
On Fri, 2007-02-02 at 14:27 +0900, Kalin KOZHUHAROV wrote:
Hi there,
For some time I am running syslog-ng as a backend mostly for snare agents on windoze.
I get the following in the log from time to time: Jan 28 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 28 12:01:33 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Jan 28 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Jan 29 00:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 [snip] Feb 1 12:01:40 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 1 12:12:25 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 00:12:26 svn01 syslog-ng[12377]: STATS: dropped 0 Feb 2 12:02:20 svn01 syslog-ng[12377]: Message length overflow, line is split, log_msg_size=2048 Feb 2 12:12:26 svn01 syslog-ng[12377]: STATS: dropped 0
What does this "Message length overflow" ? How can I find/log who(=pid or IP) is sending long messages?
check the logs in your output files that match the timestamp of the "message length overflow" message. if you have internal() and normal messages in a single file, then the split line is right next to the "message length overflow" line. Well, there is nothing wrong around these lines (nothing looks truncated) in all the output files and the longest MSG is about 400 characters. Hmm, that is UTF-8 which means... up to 400x3 = 1200bytes, but still less than 2048.
Shall I increase log_msg_size?
it's your call. if you don't mind to have garbled messages in your log, then not necessarily. increasing log_msg_size() increases memory usage.
How?
log_msg_size() global option. I just set "log_msg_size(4096);" and will see. The time is a bit after some cron jobs start around noon, but as I said no message seems to be truncated.
Kalin. -- | A | | D | | J | | P |
On Fri, Feb 02, 2007 at 06:52:35PM +0900, Kalin KOZHUHAROV wrote:
Balazs Scheidler wrote:
check the logs in your output files that match the timestamp of the "message length overflow" message. if you have internal() and normal messages in a single file, then the split line is right next to the "message length overflow" line.
Well, there is nothing wrong around these lines (nothing looks truncated) in all the output files and the longest MSG is about 400 characters. Hmm, that is UTF-8 which means... up to 400x3 = 1200bytes, but still less than 2048.
Just remember that syslog-ng will still put a valid header on the second part of the truncated message, so you're looking for a message that doesn't look right after the normal "Feb 1 12:01:40 svn01" beginning. You might already be aware of this, but thought I'd spell it out. -- Nate Yea, tho I walk thru the valley of the shadow of clues, I shall fear no luser, for Thou lart with me, Thy chicken and Thy manual, they comfort me.
Nate Campi wrote:
On Fri, Feb 02, 2007 at 06:52:35PM +0900, Kalin KOZHUHAROV wrote:
Balazs Scheidler wrote:
check the logs in your output files that match the timestamp of the "message length overflow" message. if you have internal() and normal messages in a single file, then the split line is right next to the "message length overflow" line. Well, there is nothing wrong around these lines (nothing looks truncated) in all the output files and the longest MSG is about 400 characters. Hmm, that is UTF-8 which means... up to 400x3 = 1200bytes, but still less than 2048.
Just remember that syslog-ng will still put a valid header on the second part of the truncated message, so you're looking for a message that doesn't look right after the normal "Feb 1 12:01:40 svn01" beginning.
You might already be aware of this, but thought I'd spell it out. Thank you for the clarification, I think I figured out that at the time, now checking again.
I put 'log_msg_size(4096);' in my config and haven't seen any of the "message length overflow" lines since (over a week). So I am marking this FIXED:WORKAROUND for now, as I wasn't able to pinpoint the offender (syslog-ng is behaving flawlessly, I was trying to see who was logging big MSGs). Kalin.
participants (3)
-
Balazs Scheidler
-
Kalin KOZHUHAROV
-
Nate Campi