[syslog-ng] squid access.log to syslog-ng - how?

Tue Oct 6 12:47:57 UTC 2020

Hi Victor!

Syslog-ng tries to parse the file sources according to the old BSD syslog protocol (RFC3164). I guess rsyslog doesn't do any parsing by default.
You can disable the default parsing with the "no-parse" flag. This causes that the entire message will be put into the $MESSAGE macro.

Example:
source {
  file("/var/log/squid/access.log" flags("no-parse"))
};

I've googled the log format for squid access logs:
https://wiki.squid-cache.org/Features/LogFormat

Here are some examples:
1265939281.764      1 172.16.167.228 TCP_DENIED/403 734 POST http://lbcore1.metacafe.com/test/SystemInfoManager.php - NONE/- text/html
1556260467.596 8 192.168.45.27 TCP_MISS/200 2037 CONNECT 192.168.70.4:443 - HIER_DIRECT/192.168.70.4 -

In order to parse the unix timestamp from these message, you would need to do some custom parsing additionally.
I can help you with it if you would like to, but maybe you process your logs already on the (receiving) server side.

Regards,
Gabor

________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of victorsugo <victorsugo at protonmail.com>
Sent: Tuesday, October 6, 2020 13:19
To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] squid access.log to syslog-ng - how?

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hi,

I'd like to send all of my squid proxy's log to a syslog-ng server.
But it doesn't work as I want, because when syslog-ng parses the lines from access.log, it interprets the first field (actually UTC timestamp) as a program.
(I get the timestamp value in the PROGRAM field).

But... if I put an rsyslog (installed locally on squid's container) between the squid and syslog-ng servers, I can get correct results.
Could you help me, how can I omit the rsyslog from the logging and get a usable log?
Is there a standard way to do it? Or do I need to create an own template in squid.conf for this log?

regards,
Victor
--------------------- What's the vector Victor? :) ---------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20201006/62f022fc/attachment.html>