Hello, I have a problem with remote logging for Squid logs. Our setup is the following: - syslog-ng server: syslog-ng 3.1.3. It is a dedicated server for logging. From now on, the server. - syslog-ng in Squid: syslog-ng 2.0.9. From now on, the client. The Squid process writes three log files: access.log, store.log and cache.log. I have configured the client syslog-ng to send those files to the log server. The problem is that the content of those files are sent only when syslog-ng starts on the client side. So: 1) The syslog-ng client writes all the information to the local files (access, cache and store). So, locally, it works. 2) The information is sent, but only when the client syslog-ng process restarts. So it is not a networking problem. 3) And, I've left the default config for all the other log stuff (like messages, syslog, etc.). That information is also sent to the log server. And, in this case, the information is sent constantly. I mean, I don't have to restart the syslog-ng client to make the Squid machine send the "messages", "syslog", "mail" and other logs. I've googled a while, but the only thing I found was some article about delayed logs, but it was due a bad timestamp. The thing here is Squid writes its logs with a strange timestamp: "1302699202.239", for example. Is there any way to translate that Squid timestamp into a more readable one, so I can probe if that is the problem? In addition, I've started syslog-ng client y debug mode (with -d -F), but nothing appears in the screen, although the logs are being written. Thank you for any answer or comment you might have :-) ----- CeSPI Centro Superior para el Procesamiento de la Información Universidad Nacional de La Plata ------------------------------------------------------------------------------- Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario
Hello, On Wed, Apr 13, 2011 at 2:59 PM, Matias Banchoff <matiasb@cespi.unlp.edu.ar> wrote:
Hello, I have a problem with remote logging for Squid logs. Our setup is the following:
- syslog-ng server: syslog-ng 3.1.3. It is a dedicated server for logging. From now on, the server. - syslog-ng in Squid: syslog-ng 2.0.9. From now on, the client.
Very old version, but should still work.
The Squid process writes three log files: access.log, store.log and cache.log. I have configured the client syslog-ng to send those files to the log server.
The problem is that the content of those files are sent only when syslog-ng starts on the client side. So: 1) The syslog-ng client writes all the information to the local files (access, cache and store). So, locally, it works.
These files are actually written by squid not by syslog-ng, right? syslog-ng should just read this files.
2) The information is sent, but only when the client syslog-ng process restarts. So it is not a networking problem.
I guess you aren't using the follow_freq() option for the incoming files so when syslog-ng reaches EOF it will no longer try to read it. the file offset gets stored so after restarting syslog-ng it will continue reading from where it left before.
3) And, I've left the default config for all the other log stuff (like messages, syslog, etc.). That information is also sent to the log server. And, in this case, the information is sent constantly. I mean, I don't have to restart the syslog-ng client to make the Squid machine send the "messages", "syslog", "mail" and other logs.
Other things work because syslog-ng keeps reading standard sources like /dev/log. for files you need follow_freq() which is enabled only in 3.x versions by default. Regards, Sandor
Hello! I've just probed it and it works perfectly :-) I used follow-freq(10). Thank you, Sandor!!!!! Regards, Matias On 04/13/2011 10:09 AM, Sandor Geller wrote:
Hello,
On Wed, Apr 13, 2011 at 2:59 PM, Matias Banchoff <matiasb@cespi.unlp.edu.ar> wrote:
Hello, I have a problem with remote logging for Squid logs. Our setup is the following:
- syslog-ng server: syslog-ng 3.1.3. It is a dedicated server for logging. From now on, the server. - syslog-ng in Squid: syslog-ng 2.0.9. From now on, the client. Very old version, but should still work.
The Squid process writes three log files: access.log, store.log and cache.log. I have configured the client syslog-ng to send those files to the log server.
The problem is that the content of those files are sent only when syslog-ng starts on the client side. So: 1) The syslog-ng client writes all the information to the local files (access, cache and store). So, locally, it works. These files are actually written by squid not by syslog-ng, right? syslog-ng should just read this files.
2) The information is sent, but only when the client syslog-ng process restarts. So it is not a networking problem. I guess you aren't using the follow_freq() option for the incoming files so when syslog-ng reaches EOF it will no longer try to read it. the file offset gets stored so after restarting syslog-ng it will continue reading from where it left before.
3) And, I've left the default config for all the other log stuff (like messages, syslog, etc.). That information is also sent to the log server. And, in this case, the information is sent constantly. I mean, I don't have to restart the syslog-ng client to make the Squid machine send the "messages", "syslog", "mail" and other logs. Other things work because syslog-ng keeps reading standard sources like /dev/log. for files you need follow_freq() which is enabled only in 3.x versions by default.
Regards,
Sandor
----- CeSPI Centro Superior para el Procesamiento de la Información Universidad Nacional de La Plata ------------------------------------------------------------------------------- Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario
Hi Matias, Squid supports logging directly to syslog. I would set that up and then filter for program "squid". On Wed, Apr 13, 2011 at 6:30 AM, Matias Banchoff <matiasb@cespi.unlp.edu.ar>wrote:
Hello! I've just probed it and it works perfectly :-) I used follow-freq(10).
Thank you, Sandor!!!!!
Regards, Matias
On 04/13/2011 10:09 AM, Sandor Geller wrote:
Hello,
On Wed, Apr 13, 2011 at 2:59 PM, Matias Banchoff <matiasb@cespi.unlp.edu.ar> wrote:
Hello, I have a problem with remote logging for Squid logs. Our setup is the following:
- syslog-ng server: syslog-ng 3.1.3. It is a dedicated server for logging. From now on, the server. - syslog-ng in Squid: syslog-ng 2.0.9. From now on, the client. Very old version, but should still work.
The Squid process writes three log files: access.log, store.log and cache.log. I have configured the client syslog-ng to send those files to the log server.
The problem is that the content of those files are sent only when syslog-ng starts on the client side. So: 1) The syslog-ng client writes all the information to the local files (access, cache and store). So, locally, it works. These files are actually written by squid not by syslog-ng, right? syslog-ng should just read this files.
2) The information is sent, but only when the client syslog-ng process restarts. So it is not a networking problem. I guess you aren't using the follow_freq() option for the incoming files so when syslog-ng reaches EOF it will no longer try to read it. the file offset gets stored so after restarting syslog-ng it will continue reading from where it left before.
3) And, I've left the default config for all the other log stuff (like messages, syslog, etc.). That information is also sent to the log server. And, in this case, the information is sent constantly. I mean, I don't have to restart the syslog-ng client to make the Squid machine send the "messages", "syslog", "mail" and other logs. Other things work because syslog-ng keeps reading standard sources like /dev/log. for files you need follow_freq() which is enabled only in 3.x versions by default.
Regards,
Sandor
----- CeSPI Centro Superior para el Procesamiento de la Información
Universidad Nacional de La Plata
------------------------------------------------------------------------------- Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Matias Banchoff <matiasb@cespi.unlp.edu.ar> wrote:
Is there any way to translate that Squid timestamp into a more readable one, so I can probe if that is the problem? In addition, I've started syslog-ng client y debug mode (with -d -F), but nothing appears in the screen, although the logs are being written.
Others have answered how to get things moving (but they are right and you should just send the logs straight away over syslog) but the following should help you rewrite the squid access logs to make the timestamp more useful: http://www.digriz.org.uk/syslog-ng-integration#Squid_Proxy_Server Cheers -- Alexander Clouter .sigmonster says: Many hands make light work. -- John Heywood
participants (4)
-
Alexander Clouter
-
Lance Laursen
-
Matias Banchoff
-
Sandor Geller