Syslog-ng + Mysql problems
On loghost as log store I'm using syslog-ng-1.6.5-2 + mysql-4.0.21-4. OS is Suse-9.2. After configuring database and syslog-ng.conf and start script which send logs to mysql from FIFO, I see the following mysql error : ERROR 1040: Too many connections. Part of syslog-ng.conf where the only local log send to FIFO: destination d_mysql_local { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN: $SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(src); destination(d_mysql_local); }; Loghost have about 10-20 local log messages per minute. Database use only for syslog queries and the load of server CPU about 10% . Script which sends log from FIFO to Mysql : #!/bin/bash if [ -e /var/log/mysql.pipe ]; then while [ -e /var/log/mysql.pipe ] do mysql -u syslog --password=secret syslog < /var/log/mysql.pipe done else mkfifo /var/log/mysql.pipe fi server:/bin # ls -la /var/log/mysql.pipe prw------- 1 root root 0 2005-03-23 12:53 /var/log/mysql.pipe I try to increase max_connection to 500 but this don't solve the problem. Have anyone the same problems? I have the same installation on FreeBSD-5.3p5 and have no problems.
What does your mysql client process count look like, while this is running? From the description of the problem, it seems like the clients are spawning, connecting, and then not exiting. Can you include the output of 'show full processlist' and `ps auwx | grep mysql | grep -v mysqld | wc -l`? - billn On Wed, 23 Mar 2005, Vladimir Potapov wrote:
On loghost as log store I'm using syslog-ng-1.6.5-2 + mysql-4.0.21-4. OS is Suse-9.2. After configuring database and syslog-ng.conf and start script which send logs to mysql from FIFO, I see the following mysql error : ERROR 1040: Too many connections. Part of syslog-ng.conf where the only local log send to FIFO:
destination d_mysql_local { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN: $SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(src); destination(d_mysql_local); };
Loghost have about 10-20 local log messages per minute. Database use only for syslog queries and the load of server CPU about 10% .
Script which sends log from FIFO to Mysql : #!/bin/bash if [ -e /var/log/mysql.pipe ]; then while [ -e /var/log/mysql.pipe ] do mysql -u syslog --password=secret syslog < /var/log/mysql.pipe done else mkfifo /var/log/mysql.pipe fi
server:/bin # ls -la /var/log/mysql.pipe prw------- 1 root root 0 2005-03-23 12:53 /var/log/mysql.pipe
I try to increase max_connection to 500 but this don't solve the problem. Have anyone the same problems?
I have the same installation on FreeBSD-5.3p5 and have no problems.
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Problems whas solved after remove from the line "mysql -u syslog --password=secret syslog < /var/log/mysql.pipe &" the "&" simbol.... -----Original Message----- From: syslog-ng-admin@lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu]On Behalf Of Bill Nash Sent: Wednesday, March 23, 2005 8:37 PM To: Syslog-Ng mailing lists Subject: Re: [syslog-ng]Syslog-ng + Mysql problems What does your mysql client process count look like, while this is running? From the description of the problem, it seems like the clients are spawning, connecting, and then not exiting. Can you include the output of 'show full processlist' and `ps auwx | grep mysql | grep -v mysqld | wc -l`? - billn On Wed, 23 Mar 2005, Vladimir Potapov wrote:
On loghost as log store I'm using syslog-ng-1.6.5-2 + mysql-4.0.21-4. OS is Suse-9.2. After configuring database and syslog-ng.conf and start script which send logs to mysql from FIFO, I see the following mysql error : ERROR 1040: Too many connections. Part of syslog-ng.conf where the only local log send to FIFO:
destination d_mysql_local { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN: $SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(src); destination(d_mysql_local); };
Loghost have about 10-20 local log messages per minute. Database use only for syslog queries and the load of server CPU about 10% .
Script which sends log from FIFO to Mysql : #!/bin/bash if [ -e /var/log/mysql.pipe ]; then while [ -e /var/log/mysql.pipe ] do mysql -u syslog --password=secret syslog < /var/log/mysql.pipe & done else mkfifo /var/log/mysql.pipe fi
server:/bin # ls -la /var/log/mysql.pipe prw------- 1 root root 0 2005-03-23 12:53 /var/log/mysql.pipe
I try to increase max_connection to 500 but this don't solve the problem. Have anyone the same problems?
I have the same installation on FreeBSD-5.3p5 and have no problems.
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Vladimir Potapov wrote:
Problems whas solved after remove from the line "mysql -u syslog --password=secret syslog < /var/log/mysql.pipe &" the "&" simbol....
Right - because you only need to start the client once (it reads input continually from the pipe). I use this setup and the mysql client occasionally stops (my mysql server is across the network since I have multiple loggers writing to a database to provide a realtime log view to the NOC). The pipe will buffer data (depending on your log rate), so running the mysql client in a wrapper (which reruns it) or restarting it from a management console or cfengine or similar is a prudent step. -andrew
On 3/23/05, Andrew Fort <afort@choqolat.org> wrote:
I use this setup and the mysql client occasionally stops (my mysql server is across the network since I have multiple loggers writing to a database to provide a realtime log view to the NOC). The pipe will buffer data (depending on your log rate), so running the mysql client in a wrapper (which reruns it) or restarting it from a management console or cfengine or similar is a prudent step.
-andrew
Could you provide some detail on how to run the client in a wrapper? I have the exact problem of client sometimes dying, and needing a manual restart. Thanks Shane
participants (4)
-
Andrew Fort
-
Bill Nash
-
Shane Presley
-
Vladimir Potapov