[syslog-ng] Stats on destinations with macro's ?
Evan Rempel
erempel at uvic.ca
Wed Feb 19 05:34:06 CET 2014
you have to stop grepping for "destination"
the detailed lines are of the form
dst.file;d_var_syslog#0;/var/log/syslog.20140218.000000;a;stored;0
note the leading dst.{destination type}.....
Evan.
________________________________________
From: Scot Needy [scotrn at gmail.com]
Sent: Tuesday, February 18, 2014 12:44 PM
To: Evan Rempel
Cc: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Stats on destinations with macro's ?
stats_level makes no difference to the stats output.
Level 1
[root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin
destination;d_em7;;a;processed;4304
destination;d_mysql;;a;processed;11711
destination;d_fifo;;a;processed;11711
destination;d_file;;a;processed;11715
[root@## ~]# vi /etc/syslog-ng/syslog-ng.conf
[root@## ~]# /etc/init.d/syslog-ng restart
Restarting syslog-ng: Stopping syslog-ng: [ OK ]
Starting syslog-ng: [ OK ]
Level 3
nohup: appending output to `nohup.out'
[root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin
destination;d_em7;;a;processed;62
destination;d_mysql;;a;processed;132
destination;d_fifo;;a;processed;132
destination;d_file;;a;processed;136
On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel at uvic.ca> wrote:
> Try adding the global option
>
> stats_level(1);
>
> ________________________________________
> From: Scot Needy [scotrn at gmail.com]
> Sent: Tuesday, February 18, 2014 11:50 AM
> To: Evan Rempel
> Cc: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Stats on destinations with macro's ?
>
> Ok I must have something wrong with my conf then.
>
> options { long_hostnames (off);
> flush_lines (0);
> use_dns(no);
> dns_cache(no);
> use_fqdn(no);
> # dns_cache_size(2014);
> check_hostname(no);
> chain_hostnames(no);
> keep_hostname(no);
> };
>
> ######
> # sources
> source src {
> unix-dgram("/var/run/log");
> unix-dgram("/var/run/logpriv" perm(0600));
> internal();
> file("/dev/klog");
> };
> ### Local sources
> source s_local {
> internal();
> unix-stream("/dev/log" max-connections(20));
> file("/proc/kmsg" program_override("kernel")); };
> ### External Network sources
> source s_net { udp(); tcp(max-connections(50)); };
> # Relay external sources
> log { source(s_net);
> destination (d_mysql); destination (d_fifo); destination (d_file);
> };
>
> #######################################################################
> destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log"
> owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); };
> ….
>
>
> # /opt/syslog-ng/sbin/syslog-ng-ctl stats
>
> SourceName;SourceId;SourceInstance;State;Type;Number
> global;payload_reallocs;;a;processed;1441
> source;s_net;;a;processed;44079304
> source;s_local;;a;processed;1035
> global;msg_clones;;a;processed;0
> destination;d_mysql;;a;processed;44079304
> src.internal;s_local#0;;a;processed;737
> src.internal;s_local#0;;a;stamp;1392752561
> global;sdata_updates;;a;processed;0
> center;;received;a;processed;0
> destination;d_fifo;;a;processed;44079304
> destination;d_file;;a;processed;44080339
> center;;queued;a;processed;0
>
> On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel at uvic.ca> wrote:
>
>> That certainly is not the way it works on 3.4
>>
>> I have a file destinations that contain date stamps etc and when I run the
>>
>> sudo syslog-ng-ctl stats
>>
>> I get each destination as a separate statistic.
>>
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0
>> ...
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156
>> dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0
>> ...
>>
>>
>> so that should be what you are loooking for.
>>
>> the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
>>
>>
>>
>> On 02/18/2014 04:33 AM, Scot wrote:
>>>
>>> I realized my problem, if a destination contains a macro it’s still defined as one destination.
>>>
>>> Looking for direction here….
>>>
>>> My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
>>>
>>> I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
>>>
>>> I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
>>>
>>> I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
>>>
>>> I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
>>>
>>> if VLAN...
>>> or VLAN…
>>> or VLAN…
>>> else everything_else..
>>>
>>>
>>> NOTE: Syntax may be off, this is just from memory.
>>>
>>> destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)};
>>> filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
>>>
>>> destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)};
>>> filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
>>>
>>> Sent from my iPad
>>>
>>>> On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta at toxcorp.com> wrote:
>>>>
>>>>> On 14.02.2014 02:55, Scot wrote:
>>>>> Is there a trick to get stats on destinations with macros ?
>>>>>
>>>>> I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
>>>>
>>>> What do you mean by 'destinations with macros'? Does local file()
>>>> destination (with macros) count? Then it works for me (on 3.5.3):
>>>>
>>>> # syslog-ng-ctl stats | grep d_net_test
>>>> destination;d_net_test;;a;processed;888891
>>>> # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf
>>>> destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); };
>>>> #
>>>>
>>>>
>>>> Regards,
>>>>
>>>> --
>>>> Jakub Jankowski|shasta at toxcorp.com|http://toxcorp.com/
>>>> GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>
>>
>> --
>> Evan Rempel erempel at uvic.ca
>> Senior Systems Administrator 250.721.7691
>> Data Centre Services, University Systems, University of Victoria
>
More information about the syslog-ng
mailing list