Hi, I intend to manage the distribution of the Apache log-files for each virtualhost using syslog-ng. I write the following in Apache vhost configue: <VirtualHost *> SetEnv V3WUSER w_test-l26-apache-_b8649b LogFormat "%{V3WUSER}e %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" xcombined CustomLog /var/log/apache_aux2_worker2/access_pipe.log xcombined </IfModule> as a result I get this kind of format of the log file: w_test-l26-apache-_b8649b 10.0.2.24 - - [24/May/2016:12:41:33 +0300] "GET / HTTP/1.1" 200 - "http://example.com/" "ELinks (0.11.7; Linux 3.14.46-1gb-csm x86_64; 158x45)" where the first field is the unique identifier of the virtualhost. Next step I write the following in syslog-ng: source src_apache_piped_logs { pipe("/var/log/apache_aux2_worker2/access_pipe.log" keep_timestamp(no)); }; destination dst_apache_piped_logs { file("/var/log/virtwww/${V3WUSER}/access.log" template("$MSGONLY\n") template-escape(no)); }; log { source(src_apache_piped_logs); destination(dst_apache_piped_logs); }; Сonsequently I would like to have each log file of the virtualhost placed in its own catalog and the path should contain the first field of the message (${V3WUSER} = w_test-l26-apache-_b8649b) Is it actually possible? Thanks in advance for the help.
Hi, Using the CSV-parser, you can parse Apache log messages as well: https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-g... Or if you can configure the Apache log format to use name=value pairs, that's the easiest to parse: https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-g... An alternative is to use a pattern database. Regards, Robert On Tue, May 24, 2016 at 11:46 AM, Alexey Vlasov <renton@renton.name> wrote:
Hi,
I intend to manage the distribution of the Apache log-files for each virtualhost using syslog-ng.
I write the following in Apache vhost configue:
<VirtualHost *> SetEnv V3WUSER w_test-l26-apache-_b8649b LogFormat "%{V3WUSER}e %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" xcombined CustomLog /var/log/apache_aux2_worker2/access_pipe.log xcombined </IfModule>
as a result I get this kind of format of the log file:
w_test-l26-apache-_b8649b 10.0.2.24 - - [24/May/2016:12:41:33 +0300] "GET / HTTP/1.1" 200 - "http://example.com/" "ELinks (0.11.7; Linux 3.14.46-1gb-csm x86_64; 158x45)"
where the first field is the unique identifier of the virtualhost.
Next step I write the following in syslog-ng:
source src_apache_piped_logs { pipe("/var/log/apache_aux2_worker2/access_pipe.log" keep_timestamp(no)); };
destination dst_apache_piped_logs { file("/var/log/virtwww/${V3WUSER}/access.log" template("$MSGONLY\n") template-escape(no)); };
log { source(src_apache_piped_logs); destination(dst_apache_piped_logs); };
Сonsequently I would like to have each log file of the virtualhost placed in its own catalog and the path should contain the first field of the message (${V3WUSER} = w_test-l26-apache-_b8649b)
Is it actually possible? Thanks in advance for the help.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, yet another alternative is to use the python parser in 3.7.x: https://guest.blogs.balabit.com/2015/09/processing-log-messages-with-python-...
Yes, you have to parse the incoming access.log entries, for example using the apache-accesslog-parser() in SCL, after which you'll have separate name-value pairs for access.log fields that you can later use in templates. -- Bazsi On Tue, May 24, 2016 at 11:46 AM, Alexey Vlasov <renton@renton.name> wrote:
Hi,
I intend to manage the distribution of the Apache log-files for each virtualhost using syslog-ng.
I write the following in Apache vhost configue:
<VirtualHost *> SetEnv V3WUSER w_test-l26-apache-_b8649b LogFormat "%{V3WUSER}e %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" xcombined CustomLog /var/log/apache_aux2_worker2/access_pipe.log xcombined </IfModule>
as a result I get this kind of format of the log file:
w_test-l26-apache-_b8649b 10.0.2.24 - - [24/May/2016:12:41:33 +0300] "GET / HTTP/1.1" 200 - "http://example.com/" "ELinks (0.11.7; Linux 3.14.46-1gb-csm x86_64; 158x45)"
where the first field is the unique identifier of the virtualhost.
Next step I write the following in syslog-ng:
source src_apache_piped_logs { pipe("/var/log/apache_aux2_worker2/access_pipe.log" keep_timestamp(no)); };
destination dst_apache_piped_logs { file("/var/log/virtwww/${V3WUSER}/access.log" template("$MSGONLY\n") template-escape(no)); };
log { source(src_apache_piped_logs); destination(dst_apache_piped_logs); };
Сonsequently I would like to have each log file of the virtualhost placed in its own catalog and the path should contain the first field of the message (${V3WUSER} = w_test-l26-apache-_b8649b)
Is it actually possible? Thanks in advance for the help.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Alexey Vlasov
-
Fabien Wernli
-
Fekete, Róbert
-
Scheidler, Balázs