Basic scenario is we have 5 web servers in a cluster which I wish to centrally log. I was pointed to syslog-ng and the linuxquestions guys 50+ readings and not 1 response so found this list and will give it a try. Basically all the webservers are running FC5, and rpm -qa shows the following version; syslog-ng-1.6.12-1.fc5 I see some newer versions, not sure if I should/have to, but really just want to log the different vhosts apache logs to a central server. I read a few diff things a few places, and so far have the following conf files. *Server.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; *Client.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { udp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; I restart the services and look for that web.log file and never see anything, so any help is appreciated.
Hi, Here is modified configuration file for central logging. I have tested it working in my environment. Server.conf ################################################################# source s_remote { tcp(); }; ## This will create seprate file for each client on central log server and log http messages destination d_clients { file("/var/adm/web.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; ################################################################# options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; ## This will log local http messages to defined file destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; Client.conf options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames(on); use_dns(yes); dns_cache(yes); use_fqdn(no); create_dirs (yes); keep_hostname (yes); perm(0640); dir_perm(0750); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; destination send_http_logs { tcp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; Hope this will work for you. -- Chanchal lance raymond <lance.raymond@gmail.com> Sent by: syslog-ng-bounces@lists.balabit.hu 01/29/2009 02:53 AM Please respond to "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> To syslog-ng@lists.balabit.hu cc Subject [syslog-ng] 1st post on some basic 101 setup Basic scenario is we have 5 web servers in a cluster which I wish to centrally log. I was pointed to syslog-ng and the linuxquestions guys 50+ readings and not 1 response so found this list and will give it a try. Basically all the webservers are running FC5, and rpm -qa shows the following version; syslog-ng-1.6.12-1.fc5 I see some newer versions, not sure if I should/have to, but really just want to log the different vhosts apache logs to a central server. I read a few diff things a few places, and so far have the following conf files. Server.conf options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; Client.conf options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { udp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; I restart the services and look for that web.log file and never see anything, so any help is appreciated. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Finally returned to work and naturally jumped right on this project. Well I used the below code, and tried and it's not working. I am not sure on a few things, but the code below (client / server) are copy/pasted (I do see my primary logging server address there) so nothing else should need to be changed.
From what I read, syslog should not be running, so I did stop syslog and only start syslog-ng, yet on the client side I have the following running; root 1607 0.0 0.0 6216 904 ? Ss 2008 0:46 syslogd -m 0 root 19596 0.0 0.0 7364 584 ? Ss 14:06 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid
the server has the following process running; root 18182 0.0 0.0 7368 684 ? Ss 14:06 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid now there is a web.log file with a Jan23 date (I think I created by hand) but nothing has changed. And... the client machine is still logging locally. The servers all run 5 websites, I don't think I can break out the logs separately in the vhost config accross to 5 different files can I? Thats ok as I can parse them later, but right now I am getting nothing on the server. The vhost logging on the client looks like this; CustomLog /home/mcp/local/apache2/logs/www.domain.com-access_log combined ErrorLog /home/mcp/local/apache2/logs/www.domain.com-error_log Does anything there have to change and/or will the local stop logging? Thanks again. On Thu, Jan 29, 2009 at 2:14 AM, Chanchal Verma <cverma2@csc.com> wrote:
Hi,
Here is modified configuration file for central logging. I have tested it working in my environment.
*Server.conf*
################################################################# source s_remote { tcp(); }; ## This will create seprate file for each client on central log server and log http messages destination d_clients { file("/var/adm/web.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; #################################################################
options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); };
source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); };
## This will log local http messages to defined file destination send_http_logs { file("/var/log/web.log"); };
filter send_http_logs { program("httpd.*"); };
log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; * Client.conf*
options { sync (0); time_reopen (10);
log_fifo_size (1000); long_hostnames(on); use_dns(yes); dns_cache(yes); use_fqdn(no); create_dirs (yes); keep_hostname (yes); perm(0640); dir_perm(0750); };
source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; destination send_http_logs { tcp("192.168.2.54" port(514)); };
filter send_http_logs { program("httpd.*"); };
log { source(s_sys); filter(send_http_logs); destination(send_http_logs); };
Hope this will work for you.
-- Chanchal
*lance raymond <lance.raymond@gmail.com>* Sent by: syslog-ng-bounces@lists.balabit.hu
01/29/2009 02:53 AM Please respond to "Syslog-ng users' and developers' mailing list" < syslog-ng@lists.balabit.hu>
To syslog-ng@lists.balabit.hu cc Subject [syslog-ng] 1st post on some basic 101 setup
Basic scenario is we have 5 web servers in a cluster which I wish to centrally log. I was pointed to syslog-ng and the linuxquestions guys 50+ readings and not 1 response so found this list and will give it a try.
Basically all the webservers are running FC5, and rpm -qa shows the following version; syslog-ng-1.6.12-1.fc5
I see some newer versions, not sure if I should/have to, but really just want to log the different vhosts apache logs to a central server. I read a few diff things a few places, and so far have the following conf files. * Server.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); };
source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); };
destination send_http_logs { file("/var/log/web.log"); };
filter send_http_logs { program("httpd.*"); };
log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; * Client.conf*
options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); };
source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); };
destination send_http_logs { udp("192.168.2.54" port(514)); };
filter send_http_logs { program("httpd.*"); };
log { source(s_sys); filter(send_http_logs); destination(send_http_logs); };
I restart the services and look for that web.log file and never see anything, so any help is appreciated. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (2)
-
Chanchal Verma
-
lance raymond