[syslog-ng] Before the basic 101 questions

lance raymond lance.raymond at gmail.com
Tue Jul 21 21:49:10 CEST 2009 

lol on the signature line!   But thanks.   I am going to end this thread
with this last question (and answer) then move to the next.  I will have to
look at a filter then as I don't want ALL the messages to goto the remote
log, just the apache ones.  Wait as I type and think.  I will in fact have
apache write them to /var/log/messages, but there will be a filter setup
(somehow) to take the web-ones, and based on that filter use a different
destination (that is part 2)..

Right?  Please say right, please say right...  And yes if so, It's more the
apache URL posted a bit earlier to read and undersand to send the logs to
syslog not the actual file.  My head hurts  :(

On Tue, Jul 21, 2009 at 2:23 PM, Fegan, Joe <Joe.Fegan at hp.com> wrote:

>  I think you're missing part of the picture. Apache writes its messages to
> private log files and what you are trying to do (though I'm not sure you
> know that) is feed the live content of those files into the syslog-ng
> logging system in real time. Once you get that feed into syslog-ng working
> then yes forwarding syslog-ng messages to a remote location is fairly
> straightforward and people will be able to give you lots of example of how
> to do that. But you need to get step one working first.
>
> So I think you need to adopt a three phase approach. First get your Apache
> logs feeding into syslog-ng on the local nodes and thereby appearing in
> /var/log/messages. Once you have that working, figure how to filter them out
> from the rest of the syslog traffic and send them to local files written by
> syslog-ng. Third figure out how to send them to a remote server.
>
> Those are my thoughts anyway, your mileage may vary.
>
> Joe.
>
>  ------------------------------
> *From:* syslog-ng-bounces at lists.balabit.hu [mailto:
> syslog-ng-bounces at lists.balabit.hu] *On Behalf Of *lance raymond
> *Sent:* 21 July 2009 16:33
>
> *To:* Syslog-ng users' and developers' mailing list
> *Subject:* Re: [syslog-ng] Before the basic 101 questions
>
> Not sure on that last post.  Each webserver's vhosts name the logs in the
> following manner (ws = webserver);
> ws1.sitename.com-access
> ws1.sitename,com-error
> ws1.othersite.com-access  ... and so on.
>
> So I simply want to send every file (rather than logging local) to goto the
> central.  When I look at the central, I will have only log folder with;
> ws1.sitename.com-access
> ws2.sitename.com-access  ... and so on
>
> The post above noticed I was doing a folder (/var/log/apache2) where I
> don't know if you can say for the source /var/log/apache2/*   or something.
>
> I am also puzzled as this to me is a real basic thing.  Take ALL apache
> logs and send to remote box.  No-one has come and said, here is my config
> and it works fine.  Just something that I can look at an entire server and
> client to see how it's done (which is why I posted mine).
>
> Really thought this was an easy thing.  Even using webmin.  There is no
> clear cut, a server is setup like this, remote client like that.  Fustrating
> is not even the start of how to describe this, but thanks for all replies so
> far.
>
> On Tue, Jul 21, 2009 at 3:44 AM, Siem Korteweg <Siem.Korteweg at qnh.nl>wrote:
>
>> Lance,
>>
>> Depending on how you want to process the collected logfiles, you can also
>> configure Apache:
>>
>> ErrorLog syslog:local1
>> CustomLog "|/usr/bin/logger -t apache -i -p local6.notice" combined
>>
>> This can be done global (for all virtual hosts) or per virtual host. How
>> to
>> add the name of the virtual host to the messages in the access logs can be
>> found here: http://httpd.apache.org/docs/2.0/logs.html
>>
>> All that remains to be done is to forward syslog from the client with the
>> webservers to the syslog-ng server.
>>
>> regards,
>>
>> Siem Korteweg
>>
>> -----Oorspronkelijk bericht-----
>> Van: syslog-ng-bounces at lists.balabit.hu namens lance raymond
>> Verzonden: di 21-7-2009 4:07
>> Aan: Syslog-ng users' and developers' mailing list
>> Onderwerp: Re: [syslog-ng] Before the basic 101 questions
>>
>> Joe, thanks for the update.    Yes, that is the directory name.  Now
>> regarding the "file", I will start a little more reading, but I don't
>> think
>> I would have to do this for each file right?  This server (along with the
>> others in the cluster) have 12 or so virtual sites, each with it's own
>> access and error log, so that would be at least 24 'file' sources.  Is
>> there
>> a way to wildcard it?
>>
>> Also, (I know this too is a basic Q) but are these defined on the
>> server/client or both?  I still don't see how they mesh, but soon, oh soon
>> when that light comes on!
>>
>> Thanks
>>
>> On Mon, Jul 20, 2009 at 8:38 PM, Fegan, Joe <Joe.Fegan at hp.com> wrote:
>>
>> >  I'm no apache expert, but I think /var/log/apache2 is the name of a
>> > directory that contains apache log files, right? But you have defined it
>> as
>> > a unix-stream source:
>> >
>> > source inputs { internal();
>> >                 unix-stream("/var/log/apache2");
>> >                 udp();
>> >                 tcp(max_connections(100)); };
>> >
>> > unix-stream is for reading a socket, not a directory, so this can't
>> work...
>> >
>> > You can use "file" sources for individual files in that directory..
>> >
>> >  ------------------------------
>> > *From:* syslog-ng-bounces at lists.balabit.hu [mailto:
>> > syslog-ng-bounces at lists.balabit.hu] *On Behalf Of *lance raymond
>> > *Sent:* 20 July 2009 21:52
>> > *To:* Syslog-ng users' and developers' mailing list
>> > *Subject:* Re: [syslog-ng] Before the basic 101 questions
>> >
>> > ok, here is the update. I have built a standalone ubuntu box to be the
>> > central server so now have that I can 'play' with.  It's a clean
>> install,and
>> > really not sure what to do as this list seems to be the best resource.
>>  So,
>> > I would think you can specify 'a' logfile, but I need ALL the apache
>> logs
>> > centrally located, so going to say, take everything from
>> /var/log/apache2
>> > and send it to the central log server.
>> >
>> > The central log server as I said is a default setup, due to size, I
>> copied
>> > them up to a play webserver, the server can be seen here;
>> > server.conf <http://www.darkerforce.com/server.conf>
>> > and the client here (the only thing changed is the remote IP)
>> > client.conf <http://www.darkerforce.com/client.conf>
>>  >
>> > When left like that and syslog-ng is started on the client I get the
>> > following;
>> >
>> > Error binding socket; addr='AF_UNIX(/var/log/apache2)', error='Address
>> > already in use (98)'
>> > Error initializing source driver; source='inputs'
>> >
>> > As I said before, I am not looking for anything complex, etc.  Just want
>> > ALL the weblogs to goto one box which is really the function of
>> syslog-ng.
>> > I am sure there is one or two things that need a tweak, and I can go
>> from
>> > there.
>> >
>> > Thanks.
>> >
>> >
>> > On Wed, Jul 15, 2009 at 3:45 AM, Sandor Geller <
>> > Sandor.Geller at morganstanley.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> On Tue, Jul 14, 2009 at 10:06 PM, lance raymond<
>> lance.raymond at gmail.com>
>> >> wrote:
>> >> > What I thought of was to make each file unique;
>> >> > ws = webserver;
>> >> >
>> >> > ws1.domain.com-access_log
>> >> > ws2.domain.com-access_log
>> >> >
>> >> > and just write them each to an nfs share.
>> >>
>> >> It'd not the name of the files which matter. When a single process
>> >> (like syslog-ng) writes to a file then NFS behaves well. The problems
>> >> start when there are multiple processes trying to access the same
>> >> file. Disabling attribute caching in the NFS client could help, but
>> >> this could have a big impact on performance.
>> >>
>> >> > Not flaming the group at all, actually Bazsi your name shows up more
>> >> than
>> >> > any of my normal mail :)   But, I have tried twice with a reply or
>> two,
>> >> and
>> >> > once conf files were sent up and/or shown the thread died.  I see
>> some
>> >> very
>> >> > intersting questions, answers on the group and it would be nice to
>> see
>> >> some
>> >> > of these things, but really, I am talking about a handful of
>> webservers
>> >> > (nothing fancy) just to write to a central log and it's not working.
>> >> The
>> >> > basic syslog @server worked perfect, since the platform updates, just
>> >> not
>> >> > working, but I appreciate the reply.
>> >>
>> >> So could you please tell what is the actual problem?
>> >>
>> >> Regards,
>> >>
>> >> Sandor
>> >>
>> >>
>>
>> _____________________________________________________________________________
>> _
>> >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> >> Documentation:
>> >> http://www.balabit.com/support/documentation/?product=syslog-ng
>> >> FAQ: http://www.campin.net/syslog-ng/faq.html
>> >>
>> >>
>> >
>> >
>> >
>>
>> _____________________________________________________________________________
>> _
>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> > Documentation:
>> > http://www.balabit.com/support/documentation/?product=syslog-ng
>> > FAQ: http://www.campin.net/syslog-ng/faq.html
>> >
>> >
>> >
>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090721/a9079766/attachment-0001.htm

More information about the syslog-ng mailing list