On Fri, 2012-08-17 at 14:24 +0000, Daniel Neubacher wrote:
Hello there,
I’ve got a pretty annoying problem with syslog-ng 3.3.5 and log filenames. I think my syslog-ng.cfg is a little unconventional because I’ve configured for each log file a file statement and a uniq program name to keep the filenames from the client on the server. The cfg I’ve attached bellow is only a small part of it because I can’t show you everything. In the original syslog-ng.cfg I’ve configured about 120 individual log files.
Now to the problem. Each day I’m getting log file names like this on my log server:
/log/applogs/2012/08/17/perl-1.hk.fra1.xing.com/t.file(d_applogs#0,/log/applogs/2012/08/17/mx2-2.xing.com/syslog-ng)=0',
/log/applogs/2012/08/15/syslog-2.log.fra2.xing.com/ropped='dst.file(d_syslog#0,/log/syslog
/log/syslog/2012/08/17/ fai-1.ops.fra1 fai?
This is pretty annoying because my script which is compressing the logs is failing on these files. Does anybody know this problem?
These seem to be part of the stats message of syslog-ng, as if the $PROGRAM portion of those wouldn't be properly set. It's interesting too, as these seem not be complete. To diagnose this issue, it'd be nice to pinpoint which server generate the partial messages in the first place. Judging the identifiers in there (d_applogs#0, d_syslog#0), it seems to relate to the server. Are those destination names (d_applogs and d_syslog) used in the client configuration? I'm assuming that no, only the server has these destinations. This would mean, that somehow the stats message, that gets emitted by internal() is mangled. I've reviewed the code that generates the statistics message, and it is manually setting the various portions of the message $MESSSAGE, $PROGRAM, etc. No real parsing occurs, this would rule out the possibility that the statistics message itself would be mangled. Another thought: what if you are reading _back_ a stats message from file? It may be _very_ long and could easily be longer than log-msg-size() which defaults to 8k. That could explain why this happens. Does this ring a bell?
Server cfg:
@version: 3.3
options {
threaded(yes);
owner("root");
group("root");
perm(0660);
dir_owner("root");
dir_group("root");
dir_perm(0770);
create_dirs(yes);
stats_freq(600);
stats_level(2);
chain_hostnames(no);
check_hostname(yes);
keep_hostname(no);
dns_cache(yes);
dns_cache_size(16384);
dns_cache_expire(3600);
dns_cache_expire_failed(60);
use_fqdn(yes);
log_msg_size(128000);
log_fifo_size(1000000);
};
template t_plain {
template("$MSG\n");
template_escape(no);
};
filter f_syslog {
program(^auth.log)
or program(^cron.log)
or program(^daemon.log)
or program(^kern.log)
or program(^lpr.log)
or program(^mail.log)
or program(^syslog.log)
or program(^user.log)
or program(^uucp.log)
or program(^mail.info)
or program(^mail.warn)
or program(^mail.err)
or program(^news.crit)
or program(^news.err)
or program(^news.notice)
or program(^debug.log)
or program(^error.log)
or program(^messages.log)
or program(^ppp.log);
};
filter f_applogs {
not filter(f_syslog)
and program(".*.log");
};
source s_src {
unix-dgram("/dev/log" max-connections(500));
internal();
file("/proc/kmsg" program_override("kernel"));
};
source s_net {
udp(
log_fetch_limit(400)
so_rcvbuf(51200000)
keep_timestamp(yes)
port(514)
);
tcp(
max-connections(1000)
so_rcvbuf(51200000)
so_keepalive(yes)
keep_timestamp(yes)
port(514)
);
syslog();
};
destination d_messages { file("/var/log/messages"); };
destination d_syslog { file("/log/syslog/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST/$PROGRAM" template(t_plain)); };
destination d_syslog_onefile { file("/log/syslog_onefile/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST"); };
destination d_applogs { file("/log/applogs/${R_YEAR}/${R_MONTH}/${R_DAY}/$HOST/$PROGRAM" template(t_plain)); };
destination d_perlhk {tcp("perl-1.hk.fra1.xing.com" port(514));};
log {
source(s_src);
destination(d_messages);
};
log {
source(s_net);
filter(f_syslog);
destination(d_syslog);
};
log {
source(s_net);
filter(f_syslog);
destination(d_syslog_onefile);
};
log {
source(s_net);
filter(f_applogs);
destination(d_applogs);
};
log {
source(s_net);
filter(f_applogs);
destination(d_perlhk);
};
Client:
@version: 3.3
options {
threaded(yes);
use_dns(yes);
use_fqdn(yes);
dns_cache(yes);
dns_cache_size(16384);
dns_cache_expire(3600);
dns_cache_expire_failed(10);
log_msg_size(128000);
log_fifo_size(100000);
normalize_hostnames(yes);
check_hostname(yes);
bad_hostname("^gconfd$");
create_dirs(yes);
owner("root");
group("root");
perm(0640);
stats_freq(3600);
time_reopen(30);
};
# Applogs
source s_perl_applogs {
file(/www/applogs/admin.log follow_freq(1) flags(no-parse) program_override(admin.log));
file(/www/applogs/fcgid.log follow_freq(1) flags(no-parse) program_override(fcgid.log));
};
##Ruby
source s_ruby_applogs {
file("/virtual/cra/shared/log/production.log" follow_freq(1) flags(no-parse) program_override(production.log));
};
source s_syslog {
file("/var/log/auth.log" follow_freq(1) flags(no-parse) program_override("auth.log"));
file("/var/log/cron.log" follow_freq(1) flags(no-parse) program_override("cron.log"));
file("/var/log/daemon.log" follow_freq(1) flags(no-parse) program_override("daemon.log"));
file("/var/log/kern.log" follow_freq(1) flags(no-parse) program_override("kern.log"));
file("/var/log/lpr.log" follow_freq(1) flags(no-parse) program_override("lpr.log"));
file("/var/log/mail.log" follow_freq(1) flags(no-parse) program_override("mail.log"));
file("/var/log/syslog" follow_freq(1) flags(no-parse) program_override("syslog.log"));
file("/var/log/user.log" follow_freq(1) flags(no-parse) program_override("user.log"));
file("/var/log/uucp.log" follow_freq(1) flags(no-parse) program_override("uucp.log"));
file("/var/log/mail/mail.info" follow_freq(1) flags(no-parse) program_override("mail.info"));
file("/var/log/mail/mail.warn" follow_freq(1) flags(no-parse) program_override("mail.warn"));
file("/var/log/mail/mail.err" follow_freq(1) flags(no-parse) program_override("mail.err"));
file("/var/log/news/news.crit" follow_freq(1) flags(no-parse) program_override("news.crit"));
file("/var/log/news/news.err" follow_freq(1) flags(no-parse) program_override("news.err"));
file("/var/log/news/news.notice" follow_freq(1) flags(no-parse) program_override("news.notice"));
file("/var/log/debug" follow_freq(1) flags(no-parse) program_override("debug.log"));
file("/var/log/error" follow_freq(1) flags(no-parse) program_override("error.log"));
file("/var/log/messages" follow_freq(1) flags(no-parse) program_override("messages.log"));
file("/var/log/ppp.log" follow_freq(1) flags(no-parse) program_override("ppp.log"));
};
destination syslog-1.log.fra1 {
udp("syslog-1.log.fra1.xing.com" port(514));
};
destination syslog-2.log.fra1 {
tcp("syslog-2.log.fra1.xing.com" port(514));
};
destination syslog-1.log.fra2 {
tcp("syslog-1.log.fra2.xing.com" port(514));
};
destination syslog-2.log.fra2 {
tcp("syslog-2.log.fra2.xing.com" port(514));
};
log {
source(s_all);
destination(syslog-1.log.fra1);
};
log {
source(s_syslog);
source(s_perl_applogs);
source(s_ruby_applogs);
destination(syslog-2.log.fra1);
destination(syslog-1.log.fra2);
destination(syslog-2.log.fra2);
};
--
Daniel Neubacher, Network Administrator
daniel.neubacher@xing.com
XING AG
Gaensemarkt 43, 20354 Hamburg, Germany
Tel. +49 40 419131-28, Fax +49 40 419131-11
Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807
Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape
Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq