Mike, Thanks. If you wouldn't mind sending a config example it would be much appreciated. Daniel On Wed, November 5, 2008 14:51, Mike wrote:

Heya,

when ever I have to replay old logs back into syslog-ng, I create a FIFO on the file system, and make syslog-ng listen to that pipe.

you can then cat which ever old files you have into that pipe, and syslog-ng will handle them like they are new.

just make sure that you have syslog-ng configured to use that pipe source() when sending it to your database.

I can provide config example of what I just said if you want.

Mike

On Wed, 5 Nov 2008, Daniel L. Spells Sr. wrote:

...

Sirs,

Is there a way to import /var/log/messages, created by syslog-ng, into syslog-ng to be placed into a database?

Daniel

_______________________________________________________________________ _______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_________________________________________________________________________ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

Mike, Thanks. I was close but not sure. Daniel On Wed, November 5, 2008 15:30, Mike wrote:

this is a pretty basic example, but hopefully it helps:

(note: these are only partial config files!)

so lets say right now you have something like this:

source s_UDP { udp(); };

filter f_firewall { match ("iptables") or match ("PIX"); };

destination d_loghost { udp(192.168.1.1 port(514)); };

log { source (s_UDP); filter (f_firewall); destination (d_loghost); };

so now, anything coming in on UDP port 514, and contains either iptables, or PIX will be forwarded on to another server (192.168.1.1).

but if I want to bring in logs from last week that I have in a file /home/operator/old_logs.txt, I would adjust hte syslog-ng.conf to look like this: source s_UDP { udp(); }; source s_pipe { pipe ("/var/syslog_ng_pipe"); };

filter f_firewall { match ("iptables") or match ("PIX"); };

destination d_loghost { udp(192.168.1.1 port(514)); };

log { source (s_UDP); source (s_pipe); filter (f_firewall); destination (d_loghost); };

I would then run the following comands (as the root user): mkfifo /var/syslog_ng_pipe /etc/init.d/syslog-ng restart

cat /home/operator/old_logs.txt > /var/syslog_ng_pipe

if you are concerned about uptime on your syslog-ng process, you could copy the config file to a new file, then fire up a new syslog-ng process to handle the data from the pipe (and not from the network). buut! you gotta make sure that your destination is OK with having two incoming data streams.

cheers, Mike

On Wed, 5 Nov 2008, Daniel L. Spells Sr. wrote:

...

Mike,

Thanks. If you wouldn't mind sending a config example it would be much appreciated.

Daniel

On Wed, November 5, 2008 14:51, Mike wrote:

...

Heya,

when ever I have to replay old logs back into syslog-ng, I create a FIFO on the file system, and make syslog-ng listen to that pipe.

you can then cat which ever old files you have into that pipe, and syslog-ng will handle them like they are new.

just make sure that you have syslog-ng configured to use that pipe source() when sending it to your database.

I can provide config example of what I just said if you want.

Mike

On Wed, 5 Nov 2008, Daniel L. Spells Sr. wrote:

...

Sirs,

Is there a way to import /var/log/messages, created by syslog-ng, into syslog-ng to be placed into a database?

Daniel

___________________________________________________________________ ____ _______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_____________________________________________________________________ ____ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_______________________________________________________________________ _______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_________________________________________________________________________ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

On Wed, Nov 05, 2008 at 04:30:29PM -0500, Mike wrote:

...

but if I want to bring in logs from last week that I have in a file /home/operator/old_logs.txt, I would adjust hte syslog-ng.conf [...]

But it's much simpler to use the "contrib/relogger.pl" script that you can find in the syslog-ng tarball.

hah! nice. and here I have been including that little pipe() hack in all of my syslog-ng configs for quite a while now. I really should remember to check those contrib/ dirs more often and more carefully. looks like it has been in there for *quite* some time now. Mike

I am getting: "syslog-ng[18859]: Message length overflow, line is split, log_msg_size=4096" When I try to cat the log files to the named pipe. It repeats until it completes the lines in the log file. This is on a base install on a Suse box. On Wed, November 5, 2008 15:30, Mike wrote:

this is a pretty basic example, but hopefully it helps:

(note: these are only partial config files!)

so lets say right now you have something like this:

source s_UDP { udp(); };

filter f_firewall { match ("iptables") or match ("PIX"); };

destination d_loghost { udp(192.168.1.1 port(514)); };

log { source (s_UDP); filter (f_firewall); destination (d_loghost); };

so now, anything coming in on UDP port 514, and contains either iptables, or PIX will be forwarded on to another server (192.168.1.1).

but if I want to bring in logs from last week that I have in a file /home/operator/old_logs.txt, I would adjust hte syslog-ng.conf to look like this: source s_UDP { udp(); }; source s_pipe { pipe ("/var/syslog_ng_pipe"); };

filter f_firewall { match ("iptables") or match ("PIX"); };

destination d_loghost { udp(192.168.1.1 port(514)); };

log { source (s_UDP); source (s_pipe); filter (f_firewall); destination (d_loghost); };

I would then run the following comands (as the root user): mkfifo /var/syslog_ng_pipe /etc/init.d/syslog-ng restart

cat /home/operator/old_logs.txt > /var/syslog_ng_pipe

if you are concerned about uptime on your syslog-ng process, you could copy the config file to a new file, then fire up a new syslog-ng process to handle the data from the pipe (and not from the network). buut! you gotta make sure that your destination is OK with having two incoming data streams.

cheers, Mike

On Wed, 5 Nov 2008, Daniel L. Spells Sr. wrote:

...

Mike,

Thanks. If you wouldn't mind sending a config example it would be much appreciated.

Daniel

On Wed, November 5, 2008 14:51, Mike wrote:

...

Heya,

when ever I have to replay old logs back into syslog-ng, I create a FIFO on the file system, and make syslog-ng listen to that pipe.

you can then cat which ever old files you have into that pipe, and syslog-ng will handle them like they are new.

just make sure that you have syslog-ng configured to use that pipe source() when sending it to your database.

I can provide config example of what I just said if you want.

Mike

On Wed, 5 Nov 2008, Daniel L. Spells Sr. wrote:

...

Sirs,

Is there a way to import /var/log/messages, created by syslog-ng, into syslog-ng to be placed into a database?

Daniel

___________________________________________________________________ ____ _______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_____________________________________________________________________ ____ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_______________________________________________________________________ _______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

_________________________________________________________________________ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

I have pushed that number to 90K, and get the same message, except the "log_msg_size=XXXX" changes to the what ever value the XXXX is set to in the log, and nothing is written the the database. On Mon, December 29, 2008 05:36, Balazs Scheidler wrote:

On Tue, 2008-12-16 at 11:44 -0600, Daniel L. Spells Sr. wrote:

...

I am getting:

"syslog-ng[18859]: Message length overflow, line is split, log_msg_size=4096"

When I try to cat the log files to the named pipe. It repeats until it completes the lines in the log file.

This is on a base install on a Suse box.

the message above says that you have messages over 4096 bytes in length. increase log_msg_size().

-- Bazsi

_________________________________________________________________________ _____ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html