[syslog-ng] Better performance between udp, unix-stream or pipe ?

[Yann I.]

>
> Does this work?
>
> source s_udp_not_parsed { udp(port(514) flags(no-parse)); };
> source s_udp { udp(port(514)); };
>
> (...)
>
>
I tried that solution few days ago and it didn't work. Two processes
"syslog-ng" listened on port UDP/514 : the messages are not handled
correctly. They would be handled by "s_udp_not_parsed" or "s_udp".

nb : it's strange that two processes can listen on the same port (for the
protocol UDP) isn't it ? This is the same behaviour with "netcat". I tried
with netcat (example : nc -l -u 1234) and I have two processes which listen
the port UDP/1234. Messages send by "nc" on the port 1234 are received by
the last "netcat" which has been started.


For passing yourself back the reparsed message I would recommend
> unix-dgram. AF_UNIX is usually better than pipes for me at least.
>

Thank you ! I'll use AF_UNIX.

Regards,

Yann I.


2010/11/16 Matthew Hall <mhall at mhcomputing.net>

> For passing yourself back the reparsed message I would recommend
> unix-dgram. AF_UNIX is usually better than pipes for me at least.
>
> Matthew.
>
> On Tue, Nov 16, 2010 at 12:14:53PM -0600, Martin Holste wrote:
> > Does this work?
> >
> > source s_udp_not_parsed { udp(port(514) flags(no-parse)); };
> > source s_udp { udp(port(514)); };
> >
> > log {
> >   source(s_udp);
> >   parser(db-parser());
> >   destination(d_parsed);
> > };
> > log {
> >   source(s_udp_not_parsed);
> >   destination(d_not_parsed);
> > };
> >
> > Otherwise, try reassembling a no-parse like message with a different
> > output template.
> >
> > On Tue, Nov 16, 2010 at 11:13 AM, Yann I. <yann.frm at gmail.com> wrote:
> > > Well I'm not sure because of the flag I used for the UDP source which
> is set
> > > to "no-parse".
> > >
> > > Here is my problem. From the UDP source, I may receive logs which are
> not
> > > "syslog compliance". So I'm using the flag 'no-parse' then I rewrite
> the
> > > message. After that rewrite, I forward the new message to the same
> syslog-ng
> > > server.
> > > Then... I can apply filter, parser, etc on that new message which is
> now
> > > "syslog compliance" :-)
> > >
> > > So, I think I can't use log statement. I need to use that mecanism...
> > > There might be another solution but this one seems to be a good
> solution.
> > >
> > >
> > > 2010/11/16 Martin Holste <mcholste at gmail.com>
> > >>
> > >> Ok, then this should be accomplished with a standard log statement
> > >> like you've already begun to write.  What do your destinations look
> > >> like?
> > >>
> > >> On Tue, Nov 16, 2010 at 10:58 AM, Yann I. <yann.frm at gmail.com> wrote:
> > >> > In fact, this is the same process... There is only one process.
> > >> >
> > >> >
> > >> > 2010/11/16 Martin Holste <mcholste at gmail.com>
> > >> >>
> > >> >> Why do you need separate syslog-ng processes running?
> > >> >>
> > >> >> On Tue, Nov 16, 2010 at 10:49 AM, Yann I. <yann.frm at gmail.com>
> wrote:
> > >> >> > Hi !
> > >> >> >
> > >> >> > I have a question about the use of udp, unix-stream or pipe. I
> would
> > >> >> > like to
> > >> >> > forward a syslog message to the same syslog server like this :
> > >> >> >
> > >> >> > |  log {
> > >> >> > |     source (s_r_udp);    (<-- listen on UDP/514)
> > >> >> > |
> > >> >> > |     filter (....);
> > >> >> > |     filter (....);
> > >> >> > |     parser (...);
> > >> >> > |
> > >> >> > |     destination (d_local_syslog);  (<-- send the message to a
> local
> > >> >> > syslog
> > >> >> > by using unix-stream, udp or pipe mecanism)
> > >> >> > |  };
> > >> >> >
> > >> >> > (...)
> > >> >> >
> > >> >> > |  log {
> > >> >> > |     source (s_local_syslog);    (<--- here I receive the
> messages
> > >> >> > sent
> > >> >> > by
> > >> >> > the "d_syslog_loop")
> > >> >> > |
> > >> >> > |     filter (...);
> > >> >> > |     filter (...);
> > >> >> > |     parser (...);
> > >> >> > |
> > >> >> > |     destination (d_remote_syslog);
> > >> >> >
> > >> >> > I'm looking for the better way to send syslog message to the same
> > >> >> > syslog
> > >> >> > server : which mecanism provides the better performances : pipe,
> udp
> > >> >> > (by
> > >> >> > using network) or unix-stream ?
> > >> >> > Maybe the "pipe" is the better solution ?...
> > >> >> >
> > >> >> > I'm using the syslog-ng OSE 3.1.2 on CentOS.
> > >> >> >
> > >> >> > Regards,
> > >> >> >
> > >> >> > Yann I.
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> >
> ______________________________________________________________________________
> > >> >> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > >> >> > Documentation:
> > >> >> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > >> >> > FAQ: http://www.campin.net/syslog-ng/faq.html
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >>
> > >> >>
> > >> >>
> ______________________________________________________________________________
> > >> >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > >> >> Documentation:
> > >> >> http://www.balabit.com/support/documentation/?product=syslog-ng
> > >> >> FAQ: http://www.campin.net/syslog-ng/faq.html
> > >> >>
> > >> >
> > >> >
> > >>
> > >>
> ______________________________________________________________________________
> > >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > >> Documentation:
> > >> http://www.balabit.com/support/documentation/?product=syslog-ng
> > >> FAQ: http://www.campin.net/syslog-ng/faq.html
> > >>
> > >
> > >
> >
> ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20101117/21bf30e5/attachment.htm