[syslog-ng] dropped udp packets and help with config

13 May 2011

      Something really wrong with syslog-ng or my config. I'm dropping way too
many packets.
I will admit that my configuration is probably really a large part of the
problem
and would appreciate it if someone could take a look at it and offer some
suggestions.
 There is another thread going about a similar problem on a similar
platform.

 We recently upgraded to Solaris 10 from Solaris 9 and I don't recall us
dropping that
many packets before. And we also upgraded from a very older Sylog-ng version
to 3.1.2.
I am basing the dropped packets on the udp stats, not syslog-ng stats.
Syslog-ng stats has NO dropped packets.

UDP     udpInDatagrams      -4599313    udpInErrors         -     0
        udpOutDatagrams     -  3421     udpOutErrors        -     0
        tcpInErrs           -     0     udpNoPorts          -2587612
        udpInCksumErrs      -     0     udpInOverflows      -95806254

The above is a 3 hour sample and it is from our syslog server that does not
get that much traffic.
____________________________

Here is the current version info: Solaris 10,

syslog-ng 3.1.2
Installer-Version: 3.1.2
Revision: ssh+git://bazsi@git.balabit
//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8bf13c304b6ab5fc1a372b49d55c78370efe14ca
Compile-Date: Oct 25 2010 23:56:18
Enable-Threads: off
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-Sun-STREAMS: on
Enable-Sun-Door: on
Enable-IPv6: on
Enable-Spoof-Source: on
Enable-TCP-Wrapper: off
Enable-SSL: on
Enable-SQL: off
Enable-Linux-Caps: off
Enable-Pcre: on
_____________________________

Below is a very small sampling of our syslog-ng.conf. We are filtering on
about
1400 devices most of which are Firewalls and routers. The IPs in the
following
sample have been made up.

One of my questions is "Does the number of devices we are filtering on make
a difference? (1400)"
We have several sites and use just one version of the syslogng.conf file. It
is
a lot easier to maintain one copy:))

Also notice the format:  ("^10\.123\.10\.133$") for the filters. All 1400
are in that format.
 I was hoping this would help a little but don't really know for sure:))

The source statementbelow "...external_Future_tcp" has not yet been
implemented. Since we are dropping so
many packets, I was going to try configuring the devices to log TCP instead
of UDP.

@version: 3.0

# Created: 01 March 2011

#----------------[ GLOBAL OPTIONS ]-------------------------

options {
create_dirs(yes);
use_dns(no);
time_reopen(10);
time_reap(360);
keep_timestamp(yes);
};

#---------------------[ SOURCES ]---------------------------

source s_local  {
sun-stream("/dev/log" door("/etc/.syslog_door"));
internal();
};

source s_external       {
udp();
};

source s_external_tcp      {
tcp(max-connections(50) port(514));
};

source s_external_Future_tcp      {
tcp(max-connections(1400) port(1470));
};

#---------------------[ DESTINATION ]---------------------------

destination d_local     {
file("/var/adm/messages"
perm(0655)
dir_perm(0655));
};

destination d_network_file      {
file("/logs/$YEAR/$MONTH/$DAY/network.log"
perm(0655)
dir_perm(0655));
};

destination d_bacsit        {
udp("10.11.13.114" port(2514) spoof-source(yes));
};

destination d_network_syslogd      {
udp("10.11.13.116" port(1514) spoof-source(yes));
};

destination d_firewall_file     {
file("/logs/$YEAR/$MONTH/$DAY/firewall/$HOST.log"
perm(0655)
dir_perm(0655));
};

destination d_mrv_file     {
file("/logs/$YEAR/$MONTH/$DAY/mrv.log"
perm(0655)
dir_perm(0655));
};

destination d_mail_file      {
file("/logs/$YEAR/$MONTH/$DAY/mail/$HOST.log"
perm(0655)
dir_perm(0655));
};

destination d_f567_file     {
file("/logs/$YEAR/$MONTH/$DAY/f5s/$HOST.log"
perm(0655)
dir_perm(0655));
};

#---------------------[ FILTERS ]---------------------------

filter f_f567 {
host("^10\.123\.10\.133$") or  # Host B
host("^10\.100\.10\.200$") or  # Host A
host("^10\.115\.10\.246$") or  # Host C
host("^10\.121\.10\.102$") or  # Host D
host("^10\.117\.10\.99$");    # Host F
};

filter f_mrv {
host("^10\.68\.69\.100$") or   #
host("^10\.100\166\.10$") or  #
};
.
.
. and so on

#---------------------[ LOGS ]---------------------------

log     {
source(s_local);
destination(d_local);
};

log     {
source(s_external);
filter(f_f567);
destination(d_f5_file);
};

log     {
source(s_external); source(s_external_tcp);
filter(f_firewall);
destination(d_bacsit);
};

log     {
source(s_external);
filter(f_network);
destination(d_bacsit);
};

log     {
source(s_external); source(s_external_tcp);
filter(f_firewall);
destination(d_combo_file);
};

log     {
source(s_external);
filter(f_mail);
destination(d_mail_file);
};

....and so on

I'm grateful for all help and suggestions.

Thanks!!

[syslog-ng] dropped udp packets and help with config

Zeek Anow