[syslog-ng] Multiple dates

[Scheidler, Balázs]

On Jun 8, 2017 08:38, "Sandbox" <sandboxheh at gmail.com> wrote:

Hi,

I just started to test, learn etc  syslog-ng, my server configuration is
really basic:

Q: Can i filter (and mark them on client) the incoming logs, so i dont have
to open multiple ports for different logs?


Sure, you can open one port and have it filtered using source ip (netmask()
filter), embedded hostname (host() filter) or even message content.



The stored log:

Q: Why does it store the date 3 times in every logs?


You seem to have received an rfc5424 formatted message, but it was not
parsed, maybe because you were using the wrong source driver (syslog() is
the one that should handle this format).

Since it wasnt parsed, syslog-ng assumed the entire line is a $MSG, and
prepended its own syslog header. Also, apache itself contains date as well.

The solution depends on your exact use case. If you want to transport
non-syslog data (like apache.log), you'll probably want to dedicate a port
to it (so it doesnt mix syslog), or you make sure you can identify it on
the server side.

E.g.

source { file("/var/log/apache/access.log" host-override("hostname")
program-override("apache-access-log") flags(no-parse))); };

This would read the log file without parsing it, adds $HOST and $PROGRAM
fields, which would otherwise be missing.

Then:

* send it on to the server using whatever means (tcp and syslog both
works), on the wire, the syslog header will be prepended.

* On the server, identify that these are apache logs (based on the $PROGRAM
value), then write a file using a custom template, where you only use $MSG:

file("logfile" template("$MSG\n"));

This would remove the syslog header in your output file.
Hope this helps
Bazsi





Jun  8 08:20:11 192.168.7.30 133 <13>1 2017-06-08T08:20:11+02:00 testweb01
- - - [meta sequenceId="24"] :1 - - [08/Jun/2017:08:20:10 +0200] "GET /
HTTP/1.1" 200 3004
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170608/34ab273f/attachment.html>