flags(no-parse) option of the source. To explicitly parse a message as a syslog message, use the syslog parser. For details, see Section 12.1, Parsing syslog messages.Besides the parser itself, the strange part is why the regex returning that extra info at all...Hi Robert,I'm on 3.9.1, I have just tried hat example and it returns:
Error parsing affile, Error compiling template, error=Invalid template function reference, missing function name or inbalanced '(', error_pos='24' in /etc/syslog-ng/conf.d/apache.conf at line 54, column 18:
included from /etc/syslog-ng/syslog-ng.conf line 68, column 1
template("$(format-json .apache.*\n"));
^^^^^^^^^^^^^^^^^^^^^^^^^^^On Tue, Jul 11, 2017 at 10:59 AM, Fekete, Róbert <robert.fekete@balabit.com> wrote:Hi, in OSE 3.9 and later there is a dedicated apache parser: https://www.balabit.com/documents/syslog-ng-ose-late st-guides/en/syslog-ng-ose- guide-admin/html/apache-access -log-parser.html You might want to try it.HTH,RobertOn Tue, Jul 11, 2017 at 3:11 PM, Filipe Cifali <cifali.filipe@gmail.com> wrote:______________________________This log produces an strange behavior:but I think something is not ok, I'm not sure this is the right way to do it.Hi all,reading the docs I got into this config:
source s_apache_access_log {
file(
"/var/logs/apache2/access_log"
follow-freq(1)
flags(no-parse)
);
};
filter f_apache_access_log {
match(
'(.*) (.*) - - \[[0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{ 2} -0300\] \"(.*) (.*) (.*)\" (.*) (.*) \"-\" (.*)'
type("pcre")
flags("store-matches")
);
};
rewrite r_apache_access_log {
set("$1", value("DOMAIN") condition(filter(f_apache_access_log)));
set("$2", value("IP") condition(filter(f_apache_access_log)));
set("$3", value("HTTP_METHOD") condition(filter(f_apache_access_log)));
set("$4", value("URI") condition(filter(f_apache_access_log)));
set("$6", value("HTTP_STATUS") condition(filter(f_apache_access_log)));
set("$7", value("SIZE") condition(filter(f_apache_access_log)));
set("$8", value("USER_AGENT") condition(filter(f_apache_access_log)));
};
destination d_apache_access_log {
mongodb(
# https://docs.mongodb.com/manual/reference/connection-string/
persist-name("apache-access-logs")
uri("mongodb://$server_and_port/syslog?wtimeoutMS=60000&sock etTimeoutMS=60000&connectTimeo utMS=60000")
collection("logs")
retries(3600)
value-pairs(
pair("HOST", "${HOST}")
pair("SERVICE", "APACHE")
pair("DATE", "${DAY}/${MONTH}/${YEAR}")
pair("TIME", "${HOUR}:${MIN}")
pair("MESSAGE", "${MESSAGE}")
pair("DOMAIN", "${DOMAIN}")
pair("HTTP_STATUS", "${HTTP_STATUS}")
pair("HTTP_METHOD", "${HTTP_METHOD}")
pair("USER_AGENT", "${USER_AGENT}")
pair("SIZE", "${SIZE}")
pair("URI", "${URI}")
pair("IP", "${IP}")
)
);
};
log {
source(s_apache_access_log);
filter(f_apache_access_log);
rewrite(r_apache_access_log);
destination(d_apache_access_log);
};
www.cifa.li 127.0.0.1 - - [11/Jul/2017:09:18:56 -0300] "GET / HTTP/1.1" 200 18652 "http://cifa.li/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"but this one doesn't
cifa.li 127.0.0.1 - - [11/Jul/2017:09:18:56 -0300] "GET / HTTP/1.1" 200 18652 "http://cifa.li/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"The behavior is (only for subdomains):corret oneThe subdomain seems like it's adding stuff that I didn't (or want) to add.Am I missing something?Thanks in advance.
--[ ]'s
Filipe Cifali Stangler______________________________ __________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog -ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
--[ ]'s
Filipe Cifali Stangler