Greetings I'm writing my thesis on the subject of collecting and analysing logfiles. i'm in the information gathering and i'm interested in your opinions on the topic. the primary idea is that hosts log all their syslog (and maybe other files like http-logs) to a dedicated loghost where the log messages are inserted into a DB. additionally there should be a filter program that scans the message stream for certain patterns and triggers actions (information gathering like backfinger, ripe data etc). things to be evaluated now are o how much net load does syslog create? one pitfall could be that logging to the extreme could have an impact on network performance making it impossible to transfer all logging information. my view (not yet supported by numbers) is that the impact on net performance should be rather low, but i'm setting up a test environment to packet count. since i assume thet remote logging is common practice i'm interested in your experiences. o how much data does a host generate? this is of course dependant on which services are running on the machine and the usage of the services. question is how much data a DB (e.g. PostgreSQL) can handle (real time queries should be possible). this will be taken care of in the test environment, too. o common tools i'm aware of several analyzing tools (swatch, logsurfer ... from freshmeat). i like the logsurfer approach on creating contexts (to gather information from different sources into a small report on specified events). are there more tools like that that do a good job? some unsorted ideas: + doing long term analysis via the DB interface, to maybe trace long term patterns (slow portscans or door knob testing come to mind) + central administration of configurations and pushing of configurations to 'agents' running on each host + default filter configurations to handle standard services and their logs (bind, sendmail, ftpd), creation of regular reports as said, i'd be very interested on your thoughts on the topic, your experiences and maybe references to papers/tools. wolfi -- Internet ist das Ding mit den Kabeln. Usenet ist das Ding mit den Menschen. -- Hubert Partl