[syslog-ng] Before the basic 101 questions
Fegan, Joe
Joe.Fegan at hp.com
Wed Jul 22 02:28:24 CEST 2009
Yes, first you need to get Apache to send its messages into the syslog system. That's the key. Once the messages are in the system you will be able to instruct syslog-ng to filter them any way you want and send them anywhere you want. It's very flexible.
Joe.
________________________________
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of lance raymond
Sent: 21 July 2009 20:49
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Before the basic 101 questions
lol on the signature line! But thanks. I am going to end this thread with this last question (and answer) then move to the next. I will have to look at a filter then as I don't want ALL the messages to goto the remote log, just the apache ones. Wait as I type and think. I will in fact have apache write them to /var/log/messages, but there will be a filter setup (somehow) to take the web-ones, and based on that filter use a different destination (that is part 2)..
Right? Please say right, please say right... And yes if so, It's more the apache URL posted a bit earlier to read and undersand to send the logs to syslog not the actual file. My head hurts :(
On Tue, Jul 21, 2009 at 2:23 PM, Fegan, Joe <Joe.Fegan at hp.com<mailto:Joe.Fegan at hp.com>> wrote:
I think you're missing part of the picture. Apache writes its messages to private log files and what you are trying to do (though I'm not sure you know that) is feed the live content of those files into the syslog-ng logging system in real time. Once you get that feed into syslog-ng working then yes forwarding syslog-ng messages to a remote location is fairly straightforward and people will be able to give you lots of example of how to do that. But you need to get step one working first.
So I think you need to adopt a three phase approach. First get your Apache logs feeding into syslog-ng on the local nodes and thereby appearing in /var/log/messages. Once you have that working, figure how to filter them out from the rest of the syslog traffic and send them to local files written by syslog-ng. Third figure out how to send them to a remote server.
Those are my thoughts anyway, your mileage may vary.
Joe.
________________________________
From: syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu> [mailto:syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>] On Behalf Of lance raymond
Sent: 21 July 2009 16:33
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Before the basic 101 questions
Not sure on that last post. Each webserver's vhosts name the logs in the following manner (ws = webserver);
ws1.sitename.com-access
ws1.sitename,com-error
ws1.othersite.com-access ... and so on.
So I simply want to send every file (rather than logging local) to goto the central. When I look at the central, I will have only log folder with;
ws1.sitename.com-access
ws2.sitename.com-access ... and so on
The post above noticed I was doing a folder (/var/log/apache2) where I don't know if you can say for the source /var/log/apache2/* or something.
I am also puzzled as this to me is a real basic thing. Take ALL apache logs and send to remote box. No-one has come and said, here is my config and it works fine. Just something that I can look at an entire server and client to see how it's done (which is why I posted mine).
Really thought this was an easy thing. Even using webmin. There is no clear cut, a server is setup like this, remote client like that. Fustrating is not even the start of how to describe this, but thanks for all replies so far.
On Tue, Jul 21, 2009 at 3:44 AM, Siem Korteweg <Siem.Korteweg at qnh.nl<mailto:Siem.Korteweg at qnh.nl>> wrote:
Lance,
Depending on how you want to process the collected logfiles, you can also
configure Apache:
ErrorLog syslog:local1
CustomLog "|/usr/bin/logger -t apache -i -p local6.notice" combined
This can be done global (for all virtual hosts) or per virtual host. How to
add the name of the virtual host to the messages in the access logs can be
found here: http://httpd.apache.org/docs/2.0/logs.html
All that remains to be done is to forward syslog from the client with the
webservers to the syslog-ng server.
regards,
Siem Korteweg
-----Oorspronkelijk bericht-----
Van: syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu> namens lance raymond
Verzonden: di 21-7-2009 4:07
Aan: Syslog-ng users' and developers' mailing list
Onderwerp: Re: [syslog-ng] Before the basic 101 questions
Joe, thanks for the update. Yes, that is the directory name. Now
regarding the "file", I will start a little more reading, but I don't think
I would have to do this for each file right? This server (along with the
others in the cluster) have 12 or so virtual sites, each with it's own
access and error log, so that would be at least 24 'file' sources. Is there
a way to wildcard it?
Also, (I know this too is a basic Q) but are these defined on the
server/client or both? I still don't see how they mesh, but soon, oh soon
when that light comes on!
Thanks
On Mon, Jul 20, 2009 at 8:38 PM, Fegan, Joe <Joe.Fegan at hp.com<mailto:Joe.Fegan at hp.com>> wrote:
> I'm no apache expert, but I think /var/log/apache2 is the name of a
> directory that contains apache log files, right? But you have defined it as
> a unix-stream source:
>
> source inputs { internal();
> unix-stream("/var/log/apache2");
> udp();
> tcp(max_connections(100)); };
>
> unix-stream is for reading a socket, not a directory, so this can't work...
>
> You can use "file" sources for individual files in that directory..
>
> ------------------------------
> *From:* syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu> [mailto:
> syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>] *On Behalf Of *lance raymond
> *Sent:* 20 July 2009 21:52
> *To:* Syslog-ng users' and developers' mailing list
> *Subject:* Re: [syslog-ng] Before the basic 101 questions
>
> ok, here is the update. I have built a standalone ubuntu box to be the
> central server so now have that I can 'play' with. It's a clean
install,and
> really not sure what to do as this list seems to be the best resource. So,
> I would think you can specify 'a' logfile, but I need ALL the apache logs
> centrally located, so going to say, take everything from /var/log/apache2
> and send it to the central log server.
>
> The central log server as I said is a default setup, due to size, I copied
> them up to a play webserver, the server can be seen here;
> server.conf <http://www.darkerforce.com/server.conf>
> and the client here (the only thing changed is the remote IP)
> client.conf <http://www.darkerforce.com/client.conf>
>
> When left like that and syslog-ng is started on the client I get the
> following;
>
> Error binding socket; addr='AF_UNIX(/var/log/apache2)', error='Address
> already in use (98)'
> Error initializing source driver; source='inputs'
>
> As I said before, I am not looking for anything complex, etc. Just want
> ALL the weblogs to goto one box which is really the function of syslog-ng.
> I am sure there is one or two things that need a tweak, and I can go from
> there.
>
> Thanks.
>
>
> On Wed, Jul 15, 2009 at 3:45 AM, Sandor Geller <
> Sandor.Geller at morganstanley.com<mailto:Sandor.Geller at morganstanley.com>> wrote:
>
>> Hi,
>>
>> On Tue, Jul 14, 2009 at 10:06 PM, lance raymond<lance.raymond at gmail.com<mailto:lance.raymond at gmail.com>>
>> wrote:
>> > What I thought of was to make each file unique;
>> > ws = webserver;
>> >
>> > ws1.domain.com-access_log
>> > ws2.domain.com-access_log
>> >
>> > and just write them each to an nfs share.
>>
>> It'd not the name of the files which matter. When a single process
>> (like syslog-ng) writes to a file then NFS behaves well. The problems
>> start when there are multiple processes trying to access the same
>> file. Disabling attribute caching in the NFS client could help, but
>> this could have a big impact on performance.
>>
>> > Not flaming the group at all, actually Bazsi your name shows up more
>> than
>> > any of my normal mail :) But, I have tried twice with a reply or two,
>> and
>> > once conf files were sent up and/or shown the thread died. I see some
>> very
>> > intersting questions, answers on the group and it would be nice to see
>> some
>> > of these things, but really, I am talking about a handful of webservers
>> > (nothing fancy) just to write to a central log and it's not working.
>> The
>> > basic syslog @server worked perfect, since the platform updates, just
>> not
>> > working, but I appreciate the reply.
>>
>> So could you please tell what is the actual problem?
>>
>> Regards,
>>
>> Sandor
>>
>>
_____________________________________________________________________________
_
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>
>
>
_____________________________________________________________________________
_
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090722/8ade9086/attachment.htm
More information about the syslog-ng
mailing list