Hello, list.
I've been trying to parse my apache logs in order to separate log messages into different files based on a column provided by the parser, which in my case is $APACHE.SERVER_NAME.
It's pretty much the same as the example available on the Administrator Guide, in section 3.8, p 60.
Looks like I'm close to get it working but there are two problems which I could not solve:
1- The first field of the log (APACHE.CLIENT_IP) vanishes when it reaches its final destination and for that reason APACHE.SERVER_NAME, which is the last column in the parser is empty. That messes up the file creation on the server. Not mentioning the important log data loss.
2- Syslog-ng server only logs for a few seconds and suddenly stops. Whenever the client is reloaded, it starts logging for a few seconds again. Nothing useful for debugging shows up at the server or client internal logs.
Hope you guys can lend me a hand here. Please let me know if you need any other information.
Here are the meaningful config sections on both the server and the client.
(I think it's worth mentioning that the sources, destinations and the parser below are not used by any other log paths.)
########### SERVER
options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
owner("root");
group("adm");
perm(0640);
};
source s_remote_apache {
tcp(ip(0.0.0.0) port(51480));
};
destination d_apache { file("/var/log/httpd/${APACHE.SERVER_NAME:-nouser}.log" template("MSGONLY\n")); };
parser p_apache {
csv-parser(columns("APACHE.CLIENT_IP", "APACHE.IDENT_NAME", "APACHE.USER_NAME",
"APACHE.TIMESTAMP", "APACHE.REQUEST_URL", "APACHE.REQUEST_STATUS",
"APACHE.CONTENT_LENGTH", "APACHE.REFERER", "APACHE.USER_AGENT",
"APACHE.PROCESS_TIME", "APACHE.SERVER_NAME")
flags(escape-double-char,strip-whitespace)
delimiters(" ")
quote-pairs('""[]')
);
};
log { source(s_remote_apache); parser(p_apache); destination(d_apache); };
############ CLIENT
options {
sync (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (yes);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
};
source s_apache {
file ("/var/log/httpd/access_log" flags(no-parse));
};
destination d_apache_remote {
};
log { source(s_apache); destination(d_apache_remote); };
