[syslog-ng] Forwarding locally stored RAW messages

Evan Rempel erempel at uvic.ca
Thu Apr 13 13:18:40 UTC 2023

It really depends on the format of your stored message files. If any 
information is missing, then you can't expect to recreate the original 
source messages.

We store the messages in a format defined as


which looks like

2023-04-13T06:15:01.613-07:00 my.host.name cron.info CROND[662460]: 
(root) CMD (/usr/local/sbin/fscheck)

These messages can be read back with a no-parse option from a named pipe

source s_replay { pipe("/var/log/syslog.pipes/replay" 
log_iw_size(100000) log_fetch_limit(5000) flags(no-parse) ); };

and parsed with a pattern database (attached) with

parser p_rawsyslog {

log {

Then sent to any destination with templated values from the parsing.

template t_replay     { template("<$pri>$parsedate $parsehost 
$parsemessage\n"); template_escape(no); };

message files can be fed into this by the command

cat log.file.name >> /var/log/syslog.pipes/replay

Hope that helps as a starting point.


On 2023-04-13 04:45, Dragan Zecevic wrote:
> Hi,
> we have syslog-ng Open Source Edition 3.33 and we are storing syslog 
> messages from some systems into log files locally on a partition on 
> syslog-ng server.
> Each day those log files are compressed.
> In order to better analyze some logs if needed we would like to 
> extract some log files and ingest them to SIEM.
> Is it possible to make some forwarder that will read these RAW syslog 
> messages from a log file and send them via syslog to SIEM?
> Thank you.
> Br,
> Dragan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20230413/984dad6b/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: reprocess.xml
Type: text/xml
Size: 47909 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20230413/984dad6b/attachment-0001.xml>

More information about the syslog-ng mailing list