[syslog-ng] Mass text log files processing over network.

[Baron Von Pietri]

If you do use the file() source driver then what i have done in a
similar situation is use the program() flag to set what destination
filename the log data should be in.  then in the destination i can
just use file("/var/log/customstuff/$PROGRAM.log").

Hope that helps.

-Karl

On Wed, Sep 8, 2010 at 7:08 AM, Joe McDonagh
<joseph.e.mcdonagh at gmail.com> wrote:
> On 09/08/2010 06:26 AM, Tomasz Wrona wrote:
>> Hello,
>>
>> Could somebody point me if is it possible to configure syslog-ng to
>> automatic, mass, text log files processing over network?
>> Which version eventually support following case?
>>
>>
>> My case:
>> Webservers run virtual servers, which logs their own php error logs
>> [text files].
>> I want to send this logs to central log server keeping orginal
>> distribution based on virtual servers in the simplest way possible.
>>
>>
>> I would like to have something like this virtual setup:
>>
>>
>> #*** Client config ***#
>>
>> source s_php {
>>    # Wildcard match [only for commercial edition?]  and TAG it somehow:
>>   file("/var/log/php/*-error.log" follow_freq(1) flags(no-parse)
>> TAG("$FILENAME: "));
>>    # eventually manual config for each file if above not possible:
>>   file("/var/log/php/service1-error.log" follow_freq(1) flags(no-parse)
>> log_prefix("service1: "));
>>   file("/var/log/php/service2-error.log" follow_freq(1) flags(no-parse)
>> log_prefix("service2: "));
>> };
>>
>> destination d_collector_php { udp("collector" port(5501)); };
>>
>> log { source(s_php); destination(d_collector_php); };
>>
>>
>>
>>
>> #*** Collector Server config ***#
>>
>> source s_network_php {
>>      udp(ip(0.0.0.0) port(5501));
>> };
>>
>>   # Can I split log stream using MACRO based on custom information send in
>> log [or syslog header]?
>> destination d_php {
>> file("/var/log/hosts/webservers/php/$HOST/$YEAR$MONTH$DAY/$TAG.log"); };
>>   # or maybe:
>> destination d_php {
>> file("/var/log/hosts/webservers/php/$HOST/$YEAR$MONTH$DAY/$log_prefix.log");
>> };
>>
>> log { source(s_network_php); destination(d_php); };
>>
>>
>>
>> Regards
>> Tom
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
> Probably easier to just pipe the configs to logger, since you can set
> the facility and log level with it. This is what I do for my apache
> stuff, pretty much a standard practice among the guys I know.
>
> --
> Joe McDonagh
> AIM: YoosingYoonickz
> IRC: joe-mac on freenode
> "When the going gets weird, the weird turn pro."
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>