[syslog-ng] Some messages are missing

[Sandor Geller]

Hi,

On Mon, Mar 14, 2011 at 3:38 AM, Tinnis G <tinnisg at gmail.com> wrote:
> Hi,
>
> I am not getting all the messages  in the message log files . If someone
> helps me , it will be really appreciated.
> We have remote servers , where we are keeping the log files too.
>
> I feel that the problem is in the filter.  Please correct me.  I want all
> the logs will be reported .

Could you be more specific like what kind of messages are missing? You
config looks like it is sending the same logs  multiple times (auth
messages 3 times, authpriv 2 times) to remote hosts so it would be
surprising when something was still missing...

> ## Auth log
> destination loghost1 { tcp("log1.xx.org" port(514)); };
> destination loghost2 { tcp("log2.xx.org" port(514)); };
> destination loghost3 { tcp("log3.xx.org" port(514)); };
> filter f_auth { facility(auth); };
> log { source(src); filter(f_auth); destination(loghost1); };
> log { source(src); filter(f_auth); destination(loghost2); };
> log { source(src); filter(f_auth); destination(loghost3); };
> #
> ## Authpriv log
> destination loghost1 { tcp("log1.xx.org" port(514)); };
> destination loghost2 { tcp("log2.xx.org" port(514)); };
> destination loghost3 { tcp("log3.xx.org" port(514)); };
> filter f_authpriv { facility(auth, authpriv); };
> log { source(src); filter(f_authpriv); destination(loghost1); };
> log { source(src); filter(f_authpriv); destination(loghost2); };
> log { source(src); filter(f_authpriv); destination(loghost3); };
>
> ##  Everything log
> destination loghost1 { tcp("log1.xx.org" port(514)); };
> destination loghost2 { tcp("log2.xx.org" port(514)); };
> destination loghost3 { tcp("log3.xx.org" port(514)); };
> filter f_everything { level(debug..emerg); };
> log { source(src); filter(f_everything); destination(loghost1); };
> log { source(src); filter(f_everything); destination(loghost2); };
> log { source(src); filter(f_everything); destination(loghost3); };

The above is suboptimal. If the loghosts are actually the same then
you're defining these 3 times, and also do a lot of filtering which
could get avoided. Please note that you're also redefining the
f_authpriv filter later. These definitions aren't local but global so
you should use unique names otherwise the last definition wins.

For example this below part

> destination loghost1 { tcp("log1.xx.org" port(514)); };
> destination loghost2 { tcp("log2.xx.org" port(514)); };
> destination loghost3 { tcp("log3.xx.org" port(514)); };
> filter f_authpriv { facility(auth, authpriv); };
> log { source(src); filter(f_authpriv); destination(loghost1); };
> log { source(src); filter(f_authpriv); destination(loghost2); };
> log { source(src); filter(f_authpriv); destination(loghost3); };

could get written as

destination loghosts {
  tcp("log1.xx.org" port(514));
  tcp("log2.xx.org" port(514));
  tcp("log3.xx.org" port(514));
};

filter f_auth_authpriv {
  facility(auth, authptiv);
};

log {
  source(src);
  filter(f_auth_authpriv);
  destination(loghosts);
};

This way the filter gets evaluated only once per log message instead
of 3 times. You can have the same effect by adding multiple
destinations to a single log{} block:

log {
  source(my_src);
  filter(my_filter);
  destination(my_first_destination);
  destination(my_second_destination);
  ...
};

IMO the f_everything filter is redundant (doesn't exclude anything),
so you could just drop it.

I recommend reading the admin guide, your config could get optimized
further fairly easily. Configs created by tools like syslog2ng are in
need of hand-optimizing...

Regards,

Sandor