[syslog-ng] Some messages are missing

[Robert Fekete]

On 03/14/2011 02:15:14 PM, Sandor Geller wrote:
> Hi,
> 
> On Mon, Mar 14, 2011 at 3:38 AM, Tinnis G <tinnisg at gmail.com> wrote:
> > Hi,
> >
> > I am not getting all the messages  in the message log files . If
> someone
> > helps me , it will be really appreciated.
> > We have remote servers , where we are keeping the log files too.
> >
> > I feel that the problem is in the filter.  Please correct me.  I
> want all
> > the logs will be reported .
> 
> Could you be more specific like what kind of messages are missing? 
> You
> config looks like it is sending the same logs  multiple times (auth
> messages 3 times, authpriv 2 times) to remote hosts so it would be
> surprising when something was still missing...
> 
> > ## Auth log
> > destination loghost1 { tcp("log1.xx.org" port(514)); };
> > destination loghost2 { tcp("log2.xx.org" port(514)); };
> > destination loghost3 { tcp("log3.xx.org" port(514)); };
> > filter f_auth { facility(auth); };
> > log { source(src); filter(f_auth); destination(loghost1); };
> > log { source(src); filter(f_auth); destination(loghost2); };
> > log { source(src); filter(f_auth); destination(loghost3); };
> > #
> > ## Authpriv log
> > destination loghost1 { tcp("log1.xx.org" port(514)); };
> > destination loghost2 { tcp("log2.xx.org" port(514)); };
> > destination loghost3 { tcp("log3.xx.org" port(514)); };
> > filter f_authpriv { facility(auth, authpriv); };
> > log { source(src); filter(f_authpriv); destination(loghost1); };
> > log { source(src); filter(f_authpriv); destination(loghost2); };
> > log { source(src); filter(f_authpriv); destination(loghost3); };
> >
> > ##  Everything log
> > destination loghost1 { tcp("log1.xx.org" port(514)); };
> > destination loghost2 { tcp("log2.xx.org" port(514)); };
> > destination loghost3 { tcp("log3.xx.org" port(514)); };
> > filter f_everything { level(debug..emerg); };
> > log { source(src); filter(f_everything); destination(loghost1); };
> > log { source(src); filter(f_everything); destination(loghost2); };
> > log { source(src); filter(f_everything); destination(loghost3); };
> 
> The above is suboptimal. If the loghosts are actually the same then
> you're defining these 3 times, and also do a lot of filtering which
> could get avoided. Please note that you're also redefining the
> f_authpriv filter later. These definitions aren't local but global so
> you should use unique names otherwise the last definition wins.
> 
> For example this below part
> 
> > destination loghost1 { tcp("log1.xx.org" port(514)); };
> > destination loghost2 { tcp("log2.xx.org" port(514)); };
> > destination loghost3 { tcp("log3.xx.org" port(514)); };
> > filter f_authpriv { facility(auth, authpriv); };
> > log { source(src); filter(f_authpriv); destination(loghost1); };
> > log { source(src); filter(f_authpriv); destination(loghost2); };
> > log { source(src); filter(f_authpriv); destination(loghost3); };
> 
> could get written as
> 
> destination loghosts {
>   tcp("log1.xx.org" port(514));
>   tcp("log2.xx.org" port(514));
>   tcp("log3.xx.org" port(514));
> };
> 
> filter f_auth_authpriv {
>   facility(auth, authptiv);
> };
> 
> log {
>   source(src);
>   filter(f_auth_authpriv);
>   destination(loghosts);
> };
> 
> This way the filter gets evaluated only once per log message instead
> of 3 times. You can have the same effect by adding multiple
> destinations to a single log{} block:
> 
> log {
>   source(my_src);
>   filter(my_filter);
>   destination(my_first_destination);
>   destination(my_second_destination);
>   ...
> };
> 
> IMO the f_everything filter is redundant (doesn't exclude anything),
> so you could just drop it.
> 
> I recommend reading the admin guide, your config could get optimized
> further fairly easily. Configs created by tools like syslog2ng are in
> need of hand-optimizing...
> 
> Regards,
> 
> Sandor

Also, you might want to create an extra log path locally that uses the 
'fallback' flag: this should collect any messages that were not 
processed by your filters.

Regards, 

Robert

> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
> 
> 
>