[syslog-ng] Stats on destinations with macro's ?
Scot Needy
scotrn at gmail.com
Wed Feb 19 07:10:29 CET 2014
I can’t post the full output but if I look at the dst output there is no stats for my d_file destination but I do see them for others which are remote targets.
This is the only stats output matching
/opt/syslog-ng/sbin/syslog-ng-ctl stats |grep file
destination;d_file;;a;processed;3780673
On Feb 18, 2014, at 11:34 PM, Evan Rempel <erempel at uvic.ca> wrote:
> you have to stop grepping for "destination"
>
> the detailed lines are of the form
>
> dst.file;d_var_syslog#0;/var/log/syslog.20140218.000000;a;stored;0
>
>
> note the leading dst.{destination type}.....
>
> Evan.
>
>
> ________________________________________
> From: Scot Needy [scotrn at gmail.com]
> Sent: Tuesday, February 18, 2014 12:44 PM
> To: Evan Rempel
> Cc: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Stats on destinations with macro's ?
>
> stats_level makes no difference to the stats output.
>
> Level 1
> [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin
> destination;d_em7;;a;processed;4304
> destination;d_mysql;;a;processed;11711
> destination;d_fifo;;a;processed;11711
> destination;d_file;;a;processed;11715
>
> [root@## ~]# vi /etc/syslog-ng/syslog-ng.conf
> [root@## ~]# /etc/init.d/syslog-ng restart
> Restarting syslog-ng: Stopping syslog-ng: [ OK ]
> Starting syslog-ng: [ OK ]
>
> Level 3
> nohup: appending output to `nohup.out'
> [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin
> destination;d_em7;;a;processed;62
> destination;d_mysql;;a;processed;132
> destination;d_fifo;;a;processed;132
> destination;d_file;;a;processed;136
>
>
>
> On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel at uvic.ca> wrote:
>
>> Try adding the global option
>>
>> stats_level(1);
>>
>> ________________________________________
>> From: Scot Needy [scotrn at gmail.com]
>> Sent: Tuesday, February 18, 2014 11:50 AM
>> To: Evan Rempel
>> Cc: Syslog-ng users' and developers' mailing list
>> Subject: Re: [syslog-ng] Stats on destinations with macro's ?
>>
>> Ok I must have something wrong with my conf then.
>>
>> options { long_hostnames (off);
>> flush_lines (0);
>> use_dns(no);
>> dns_cache(no);
>> use_fqdn(no);
>> # dns_cache_size(2014);
>> check_hostname(no);
>> chain_hostnames(no);
>> keep_hostname(no);
>> };
>>
>> ######
>> # sources
>> source src {
>> unix-dgram("/var/run/log");
>> unix-dgram("/var/run/logpriv" perm(0600));
>> internal();
>> file("/dev/klog");
>> };
>> ### Local sources
>> source s_local {
>> internal();
>> unix-stream("/dev/log" max-connections(20));
>> file("/proc/kmsg" program_override("kernel")); };
>> ### External Network sources
>> source s_net { udp(); tcp(max-connections(50)); };
>> # Relay external sources
>> log { source(s_net);
>> destination (d_mysql); destination (d_fifo); destination (d_file);
>> };
>>
>> #######################################################################
>> destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log"
>> owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); };
>> ….
>>
>>
>> # /opt/syslog-ng/sbin/syslog-ng-ctl stats
>>
>> SourceName;SourceId;SourceInstance;State;Type;Number
>> global;payload_reallocs;;a;processed;1441
>> source;s_net;;a;processed;44079304
>> source;s_local;;a;processed;1035
>> global;msg_clones;;a;processed;0
>> destination;d_mysql;;a;processed;44079304
>> src.internal;s_local#0;;a;processed;737
>> src.internal;s_local#0;;a;stamp;1392752561
>> global;sdata_updates;;a;processed;0
>> center;;received;a;processed;0
>> destination;d_fifo;;a;processed;44079304
>> destination;d_file;;a;processed;44080339
>> center;;queued;a;processed;0
>>
>> On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel at uvic.ca> wrote:
>>
>>> That certainly is not the way it works on 3.4
>>>
>>> I have a file destinations that contain date stamps etc and when I run the
>>>
>>> sudo syslog-ng-ctl stats
>>>
>>> I get each destination as a separate statistic.
>>>
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0
>>> ...
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156
>>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0
>>> ...
>>>
>>>
>>> so that should be what you are loooking for.
>>>
>>> the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
>>>
>>>
>>>
>>> On 02/18/2014 04:33 AM, Scot wrote:
>>>>
>>>> I realized my problem, if a destination contains a macro it’s still defined as one destination.
>>>>
>>>> Looking for direction here….
>>>>
>>>> My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
>>>>
>>>> I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
>>>>
>>>> I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
>>>>
>>>> I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
>>>>
>>>> I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
>>>>
>>>> if VLAN...
>>>> or VLAN…
>>>> or VLAN…
>>>> else everything_else..
>>>>
>>>>
>>>> NOTE: Syntax may be off, this is just from memory.
>>>>
>>>> destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)};
>>>> filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
>>>>
>>>> destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)};
>>>> filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
>>>>
>>>> Sent from my iPad
>>>>
>>>>> On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta at toxcorp.com> wrote:
>>>>>
>>>>>> On 14.02.2014 02:55, Scot wrote:
>>>>>> Is there a trick to get stats on destinations with macros ?
>>>>>>
>>>>>> I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
>>>>>
>>>>> What do you mean by 'destinations with macros'? Does local file()
>>>>> destination (with macros) count? Then it works for me (on 3.5.3):
>>>>>
>>>>> # syslog-ng-ctl stats | grep d_net_test
>>>>> destination;d_net_test;;a;processed;888891
>>>>> # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf
>>>>> destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); };
>>>>> #
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> --
>>>>> Jakub Jankowski|shasta at toxcorp.com|http://toxcorp.com/
>>>>> GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
>>>>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>
>>>
>>> --
>>> Evan Rempel erempel at uvic.ca
>>> Senior Systems Administrator 250.721.7691
>>> Data Centre Services, University Systems, University of Victoria
>>
More information about the syslog-ng
mailing list