Problem parsing apache logs
Hello, list. I've been trying to parse my apache logs in order to separate log messages into different files based on a column provided by the parser, which in my case is $APACHE.SERVER_NAME. It's pretty much the same as the example available on the Administrator Guide, in section 3.8, p 60. Looks like I'm close to get it working but there are two problems which I could not solve: 1- The first field of the log (APACHE.CLIENT_IP) vanishes when it reaches its final destination and for that reason APACHE.SERVER_NAME, which is the last column in the parser is empty. That messes up the file creation on the server. Not mentioning the important log data loss. 2- Syslog-ng server only logs for a few seconds and suddenly stops. Whenever the client is reloaded, it starts logging for a few seconds again. Nothing useful for debugging shows up at the server or client internal logs. Hope you guys can lend me a hand here. Please let me know if you need any other information. Here are the meaningful config sections on both the server and the client. (I think it's worth mentioning that the sources, destinations and the parser below are not used by any other log paths.) ########### SERVER options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); owner("root"); group("adm"); perm(0640); }; source s_remote_apache { tcp(ip(0.0.0.0) port(51480)); }; destination d_apache { file("/var/log/httpd/${APACHE.SERVER_NAME:-nouser}.log" template("MSGONLY\n")); }; parser p_apache { csv-parser(columns("APACHE.CLIENT_IP", "APACHE.IDENT_NAME", "APACHE.USER_NAME", "APACHE.TIMESTAMP", "APACHE.REQUEST_URL", "APACHE.REQUEST_STATUS", "APACHE.CONTENT_LENGTH", "APACHE.REFERER", "APACHE.USER_AGENT", "APACHE.PROCESS_TIME", "APACHE.SERVER_NAME") flags(escape-double-char,strip-whitespace) delimiters(" ") quote-pairs('""[]') ); }; log { source(s_remote_apache); parser(p_apache); destination(d_apache); }; ############ CLIENT options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_apache { file ("/var/log/httpd/access_log" flags(no-parse)); }; destination d_apache_remote { tcp("myhost.example.com" port(51480)); }; log { source(s_apache); destination(d_apache_remote); }; Luis Alen www.izap.com.br Ligue com tarifa local de todo o Brasil 4020.3000
Hi, Am 07.01.2013 15:32, schrieb Luis Fernando Alen: i`m not really sure what you would do. I´m running an centralisied syslog-servers and see no need to split the apache-logs on the client - i just forward the messages to the syslog-server. but i think that´s not the reason for
2- Syslog-ng server only logs for a few seconds and suddenly stops. Whenever the client is reloaded, it starts logging for a few seconds again. Nothing useful for debugging shows up at the server or client internal logs.
If you would like to simply add the sending server to the file-name, why don´t you use $HOST inside the destination? regards Florian you can find my setup for storing apache-logs on acentralisied server here: http://blog.schaal-24.de/?p=321&lang=en
participants (2)
-
Luis Fernando Alen
-
system@ra-schaal.de