Stats on destinations with macro's ?
Is there a trick to get stats on destinations with macros ? I get stats on my FIFO, local, net work destinations but not on the destinations with macros. Also I don't see any difference in the stats output using any level level 1,2,3. Syslog-ng restarted after each conf change Sent from my iPad
On 14.02.2014 02:55, Scot wrote:
Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3): # syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; # Regards, -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
Yup, it should work just as Jakub explained. On Feb 14, 2014 2:41 PM, "Jakub Jankowski" <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote:
Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
I realized my problem, if a destination contains a macro it’s still defined as one destination. Looking for direction here…. My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet. I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file. I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example. I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination. I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ? if VLAN... or VLAN… or VLAN… else everything_else.. NOTE: Syntax may be off, this is just from memory. destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)}; destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)}; Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
That certainly is not the way it works on 3.4 I have a file destinations that contain date stamps etc and when I run the sudo syslog-ng-ctl stats I get each destination as a separate statistic. dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ... so that should be what you are loooking for. the "o" in the last three lines indicates that the destination is old (closed due to idle timeout) On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
Ok I must have something wrong with my conf then. options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); }; ###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); }; ####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; …. # /opt/syslog-ng/sbin/syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0 On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
Try adding the global option stats_level(1); ________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 11:50 AM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ? Ok I must have something wrong with my conf then. options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); }; ###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); }; ####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; …. # /opt/syslog-ng/sbin/syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0 On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
stats_level makes no difference to the stats output. Level 1 [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;4304 destination;d_mysql;;a;processed;11711 destination;d_fifo;;a;processed;11711 destination;d_file;;a;processed;11715 [root@## ~]# vi /etc/syslog-ng/syslog-ng.conf [root@## ~]# /etc/init.d/syslog-ng restart Restarting syslog-ng: Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ] Level 3 nohup: appending output to `nohup.out' [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;62 destination;d_mysql;;a;processed;132 destination;d_fifo;;a;processed;132 destination;d_file;;a;processed;136 On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel@uvic.ca> wrote:
Try adding the global option
stats_level(1);
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 11:50 AM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
Ok I must have something wrong with my conf then.
options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); };
###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); };
####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; ….
# /opt/syslog-ng/sbin/syslog-ng-ctl stats
SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0
On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
you have to stop grepping for "destination" the detailed lines are of the form dst.file;d_var_syslog#0;/var/log/syslog.20140218.000000;a;stored;0 note the leading dst.{destination type}..... Evan. ________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 12:44 PM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ? stats_level makes no difference to the stats output. Level 1 [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;4304 destination;d_mysql;;a;processed;11711 destination;d_fifo;;a;processed;11711 destination;d_file;;a;processed;11715 [root@## ~]# vi /etc/syslog-ng/syslog-ng.conf [root@## ~]# /etc/init.d/syslog-ng restart Restarting syslog-ng: Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ] Level 3 nohup: appending output to `nohup.out' [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;62 destination;d_mysql;;a;processed;132 destination;d_fifo;;a;processed;132 destination;d_file;;a;processed;136 On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel@uvic.ca> wrote:
Try adding the global option
stats_level(1);
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 11:50 AM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
Ok I must have something wrong with my conf then.
options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); };
###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); };
####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; ….
# /opt/syslog-ng/sbin/syslog-ng-ctl stats
SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0
On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
I can’t post the full output but if I look at the dst output there is no stats for my d_file destination but I do see them for others which are remote targets. This is the only stats output matching /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep file destination;d_file;;a;processed;3780673 On Feb 18, 2014, at 11:34 PM, Evan Rempel <erempel@uvic.ca> wrote:
you have to stop grepping for "destination"
the detailed lines are of the form
dst.file;d_var_syslog#0;/var/log/syslog.20140218.000000;a;stored;0
note the leading dst.{destination type}.....
Evan.
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 12:44 PM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
stats_level makes no difference to the stats output.
Level 1 [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;4304 destination;d_mysql;;a;processed;11711 destination;d_fifo;;a;processed;11711 destination;d_file;;a;processed;11715
[root@## ~]# vi /etc/syslog-ng/syslog-ng.conf [root@## ~]# /etc/init.d/syslog-ng restart Restarting syslog-ng: Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ]
Level 3 nohup: appending output to `nohup.out' [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;62 destination;d_mysql;;a;processed;132 destination;d_fifo;;a;processed;132 destination;d_file;;a;processed;136
On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel@uvic.ca> wrote:
Try adding the global option
stats_level(1);
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 11:50 AM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
Ok I must have something wrong with my conf then.
options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); };
###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); };
####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; ….
# /opt/syslog-ng/sbin/syslog-ng-ctl stats
SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0
On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5 dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200 dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0 ... dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156 dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0 ...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it’s still defined as one destination.
Looking for direction here….
My intention is to get syslog-ng-ctl to report stats on each VLAN in our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN’s in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet filters something like so…. but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN… or VLAN… else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log”)}; filter VLAN_NAME_HIGH_des { netmask(“192.168.1.0/255.255.255.0”); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file(“/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info”)}; filter VLAN_NAME_LOW_des { netmask(“192.168.1.0/255.255.255.0”); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
On 14.02.2014 02:55, Scot wrote: Is there a trick to get stats on destinations with macros ?
I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
What is your syslog-ng version? On Feb 19, 2014 7:09 AM, "Scot Needy" <scotrn@gmail.com> wrote:
I can't post the full output but if I look at the dst output there is no stats for my d_file destination but I do see them for others which are remote targets.
This is the only stats output matching /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep file
destination;d_file;;a;processed;3780673
On Feb 18, 2014, at 11:34 PM, Evan Rempel <erempel@uvic.ca> wrote:
you have to stop grepping for "destination"
the detailed lines are of the form
dst.file;d_var_syslog#0;/var/log/syslog.20140218.000000;a;stored;0
note the leading dst.{destination type}.....
Evan.
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 12:44 PM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
stats_level makes no difference to the stats output.
Level 1 [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;4304 destination;d_mysql;;a;processed;11711 destination;d_fifo;;a;processed;11711 destination;d_file;;a;processed;11715
[root@## ~]# vi /etc/syslog-ng/syslog-ng.conf [root@## ~]# /etc/init.d/syslog-ng restart Restarting syslog-ng: Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ]
Level 3 nohup: appending output to `nohup.out' [root@## ~]# /opt/syslog-ng/sbin/syslog-ng-ctl stats |grep destin destination;d_em7;;a;processed;62 destination;d_mysql;;a;processed;132 destination;d_fifo;;a;processed;132 destination;d_file;;a;processed;136
On Feb 18, 2014, at 3:09 PM, Evan Rempel <erempel@uvic.ca> wrote:
Try adding the global option
stats_level(1);
________________________________________ From: Scot Needy [scotrn@gmail.com] Sent: Tuesday, February 18, 2014 11:50 AM To: Evan Rempel Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Stats on destinations with macro's ?
Ok I must have something wrong with my conf then.
options { long_hostnames (off); flush_lines (0); use_dns(no); dns_cache(no); use_fqdn(no); # dns_cache_size(2014); check_hostname(no); chain_hostnames(no); keep_hostname(no); };
###### # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); internal(); file("/dev/klog"); }; ### Local sources source s_local { internal(); unix-stream("/dev/log" max-connections(20)); file("/proc/kmsg" program_override("kernel")); }; ### External Network sources source s_net { udp(); tcp(max-connections(50)); }; # Relay external sources log { source(s_net); destination (d_mysql); destination (d_fifo); destination (d_file); };
####################################################################### destination d_file { file("/data/syslog-ng/$R_YEAR/$R_MONTH/$R_DAY/$R_HOUR/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; ....
# /opt/syslog-ng/sbin/syslog-ng-ctl stats
SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1441 source;s_net;;a;processed;44079304 source;s_local;;a;processed;1035 global;msg_clones;;a;processed;0 destination;d_mysql;;a;processed;44079304 src.internal;s_local#0;;a;processed;737 src.internal;s_local#0;;a;stamp;1392752561 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 destination;d_fifo;;a;processed;44079304 destination;d_file;;a;processed;44080339 center;;queued;a;processed;0
On Feb 18, 2014, at 1:33 PM, Evan Rempel <erempel@uvic.ca> wrote:
That certainly is not the way it works on 3.4
I have a file destinations that contain date stamps etc and when I run the
sudo syslog-ng-ctl stats
I get each destination as a separate statistic.
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;dropped;0
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;processed;5
dst.file;d_authorized_unknown#0;/var/syslog/unknown/Windows_Server_Update_Services.unknown.20140218.000000;o;stored;0
dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;dropped;0
dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;processed;200
dst.file;d_authorized_unknown#0;/var/syslog/unknown/flare-event.unknown.20140218.000000;o;stored;0
...
dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;dropped;0
dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;processed;156
dst.file;d_authorized_unknown#0;/var/syslog/unknown/runaway.unknown.20140217.000000;o;stored;0
...
so that should be what you are loooking for.
the "o" in the last three lines indicates that the destination is old (closed due to idle timeout)
On 02/18/2014 04:33 AM, Scot wrote:
I realized my problem, if a destination contains a macro it's still
defined as one destination.
Looking for direction here....
My intention is to get syslog-ng-ctl to report stats on each VLAN in
our environment while logging to a destination such as /var/log//$YYYY/$MM/$DD/$VLAN-Name-$SEVERITY.log . VLAN's in our environment are defined in a IPAM database with a name and subnet.
I can drive a include file for syslog-ng.conf with a script, I just
need guidence on the format of the config file.
I would like to define a unique destination per subnet+severity so
syslog-ng-ctl will give me counters if a subnet start sending large numbers of critical messages for example.
I also feel I need a catch all for any message that does not match a
defined destination. These would be malformed messages from hosts which would need to be corrected so they get to the proper destination.
I think the subnet destinations would be be driven by matching subnet
filters something like so.... but how would one create a filter that defines everything NOT matched by another filter ?
if VLAN... or VLAN... or VLAN... else everything_else..
NOTE: Syntax may be off, this is just from memory.
destination VLAN_NAME_HIGH_des {
file("/var/log/$YYYY/$MM/$DD/$VLAN_NAME.log")};
filter VLAN_NAME_HIGH_des { netmask("192.168.1.0/255.255.255.0"); level(warn..emerg)};
destination VLAN_NAME_LOW_des { file("/var/log/$YYYY/$MM/$DD/$VLAN_NAME.info")}; filter VLAN_NAME_LOW_des { netmask("192.168.1.0/255.255.255.0"); level(info..notice)};
Sent from my iPad
On Feb 14, 2014, at 8:40 AM, Jakub Jankowski <shasta@toxcorp.com> wrote:
> On 14.02.2014 02:55, Scot wrote: > Is there a trick to get stats on destinations with macros ? > > I get stats on my FIFO, local, net work destinations but not on the destinations with macros.
What do you mean by 'destinations with macros'? Does local file() destination (with macros) count? Then it works for me (on 3.5.3):
# syslog-ng-ctl stats | grep d_net_test destination;d_net_test;;a;processed;888891 # grep 'destination d_net_test' /etc/syslog-ng/syslog-ng.conf destination d_net_test { file("/var/log/$HOST/$R_YEAR-$R_MONTH.log"); }; #
Regards,
-- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, I'm new to syslog-ng. please forgive. Can someone please... explain these: processed='center(queued)=2836986327', dropped='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=104396', processed='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=1395253254', stored='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=0', and what needs tweaking? We have: syslog-ng 3.5.3 Installer-Version: 3.5.3 Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5 Compile-Date: Jan 8 2014 13:35:02 Available-Modules: afsocket,afprog,dbparser,system-source,affile,syslogformat,linux-kmsg-format,csvparser,afmongodb,afsocket-tls,confgen,afuser,afstomp,afsocket-notls,basicfuncs,cryptofuncs,afamqp Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: on My config is: options { threaded(yes); flush_lines(1000); time_reopen(10); log_fifo_size(200000); use_dns(yes); dns_cache_hosts(/etc/hosts); use_fqdn(no); keep_hostname(yes); chain_hostnames(no); }; source s_udp { udp( port(514) so_rcvbuf(15000000) log_iw_size(50000) log_msg_size(65535) log_fetch_limit(50000) flags(validate-utf8));}; source s_tcp { tcp( log_msg_size(65535) port(1999) log_fetch_limit(500) max_connections(1000) log_iw_size(500000) so_rcvbuf(15000000) so_keepalive(yes) flags(validate-utf8) ); }; My stats are: Apr 9 14:16:02 sopher1 syslog-ng[5952]: Log statistics; processed='destination(acs)=394697', processed='destination(d_cron)=1755', processed='center(queued)=2836986327', processed='destination(bradx)=0', processed='destination(aruba)=62814613', processed='source(s_tcp)=0', processed='destination(d_mlal)=0', processed='destination(d_syslognglog)=955', processed='destination(clearpass)=2104355', processed='src.none()=0', stamp='src.none()=0', processed='source(s_internal)=955', processed='global(msg_clones)=0', processed='src.internal(s_internal#0)=955', stamp='src.internal(s_internal#0)=1397070362', processed='source(s_udp)=0', processed='destination(paloacp)=1141796', processed='destination(vpn)=344489994', processed='destination(routers)=28167', processed='destination(d_cons)=1863', processed='destination(airwave)=7300', processed='destination(d_boot)=0', processed='destination(routers2)=1180', processed='destination(d_spol)=0', processed='destination(brad)=219125', processed='destination(pix)=1017647488', processed='destination(catch_all)=67', dropped='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=104396', processed='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=1395253254', stored='dst.udp(sopher2udp#0,udp,xxx.xxx.xxx.xxx:514)=0', processed='destination(switches)=2171789', processed='destination(paloborder)=319941', processed='destination(ups)=15', processed='destination(palo)=47922', processed='destination(d_kern)=0', processed='destination(sopher2udp)=1395253254', processed='global(payload_reallocs)=3235', processed='global(sdata_updates)=8', processed='destination(d_mail)=32', processed='destination(d_mesg)=280', processed='center(received)=955', processed='destination(dhcp)=10338454', processed='destination(d_auth)=1241', processed='source(s_system)=0', processed='destination(d_tcp)=44'
participants (6)
-
Balazs Scheidler
-
Evan Rempel
-
Frank Wilkinson
-
Jakub Jankowski
-
Scot
-
Scot Needy