[syslog-ng]log archiving

[Hamilton, Andrew Mr RAYTHEON 5 SIG CMD]

Nate,

A lot of people treat their logs differently.  Some folks use a database
they insert their log lines into, some have complicated scripts that process
them and spit out HTML on the statistics, some people grep the logs for
particular lines and delete them if they aren't there.  People use all kinds
of different hardware to run their log servers and really there is no one to
really tell them that what they are doing is a waste of time or resources.
If your logs are important to you and your organization then it really makes
no difference to those of us on the outside what you use.  I use an Ultra 60
with 2 18 gig scsi drives myself and I log around 1000 devices.  So it
really all depends on what you think the logs are worth.  If it works for
you then that is what matters.  The system administrators here use a
separate script to rotate and archive the logs.  And I leave that to them
since they are the ones that use them.  Your use of the macros is nice
though.

Regards,
Drew

-----Original Message-----
From: Nate Campi [mailto:nate@campi.cc]
Sent: Saturday, August 04, 2001 12:54 AM
To: syslog-ng@lists.balabit.hu
Subject: [syslog-ng]log archiving


I thought I'd share something with the list that I discovered recently.

I had been archiving logs for about 70 host using syslog-ng for the last
year, and recently we discovered that our ssh gateway had been rooted. I
archived the logs by host like this:

destination hosts {
file("/var/log/HOSTS/$HOST/$FACILITY/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$
DAY"
owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); };

log { source(src); destination(hosts); };

It's easy to use find and grep for finding pertinent log messages
relating to events, but I found that since I was only logging into
directories by host, I couldn't easily search ALL logs on a certain
date. I could write up scripts to look for certain dates in filenames
but that's not as fast and efficient as just seaching a directory 
with logs from all hosts on just that day.

So I did this too:

destination times {
file("/var/log/TIMES/$YEAR/$MONTH/$DAY/$HOST/$FACILITY$YEAR$MONTH$DAY"
owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); };

log { source(src); destination(times); };

What do people think? I mounted /var/log/HOSTS and /var/log/TIMES each
on their own 18 gig ultra2 SCSI 10k RPM drives, on a machine with a
single PIII 550, 768 RAM. Performance and disk space aren't an issue,
plus I can add another proc and disks (hot swap SCA SCSI).

I just want to hear if people think I'm just wasting disk space and I
should script something up, or if they like the idea of fast searches by
date or hostname.

TIA,
-- 
	Nate

_______________________________________________
syslog-ng maillist  -  syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng