[syslog-ng] Better performance between udp, unix-stream or pipe ?

[Yann I.]

You're right Matthew and this is why I won't use that solution (2 UDP
sources). It was just a test. :-)
Instead, I'll use only one UDP source with the flag "no-parse" and a
patterndb.

Regards,

Yann


2010/11/17 Matthew Hall <mhall at mhcomputing.net>

> As a connectionless protocol UDP cannot always detect existing listeners on
> all platforms. However if you think about it it's clear why this wouldn't
> work. How would it know which messages should be parse or noparse? It can't
> read your mind! ;)
>
> Matthew.
>
> Sent from my mobile device
>
> "Yann I." <yann.frm at gmail.com> wrote:
>
> >>
> >> Does this work?
> >>
> >> source s_udp_not_parsed { udp(port(514) flags(no-parse)); };
> >> source s_udp { udp(port(514)); };
> >>
> >> (...)
> >>
> >>
> >I tried that solution few days ago and it didn't work. Two processes
> >"syslog-ng" listened on port UDP/514 : the messages are not handled
> >correctly. They would be handled by "s_udp_not_parsed" or "s_udp".
> >
> >nb : it's strange that two processes can listen on the same port (for the
> >protocol UDP) isn't it ? This is the same behaviour with "netcat". I tried
> >with netcat (example : nc -l -u 1234) and I have two processes which
> listen
> >the port UDP/1234. Messages send by "nc" on the port 1234 are received by
> >the last "netcat" which has been started.
> >
> >
> >For passing yourself back the reparsed message I would recommend
> >> unix-dgram. AF_UNIX is usually better than pipes for me at least.
> >>
> >
> >Thank you ! I'll use AF_UNIX.
> >
> >Regards,
> >
> >Yann I.
> >
> >
> >2010/11/16 Matthew Hall <mhall at mhcomputing.net>
> >
> >> For passing yourself back the reparsed message I would recommend
> >> unix-dgram. AF_UNIX is usually better than pipes for me at least.
> >>
> >> Matthew.
> >>
> >> On Tue, Nov 16, 2010 at 12:14:53PM -0600, Martin Holste wrote:
> >> > Does this work?
> >> >
> >> > source s_udp_not_parsed { udp(port(514) flags(no-parse)); };
> >> > source s_udp { udp(port(514)); };
> >> >
> >> > log {
> >> >   source(s_udp);
> >> >   parser(db-parser());
> >> >   destination(d_parsed);
> >> > };
> >> > log {
> >> >   source(s_udp_not_parsed);
> >> >   destination(d_not_parsed);
> >> > };
> >> >
> >> > Otherwise, try reassembling a no-parse like message with a different
> >> > output template.
> >> >
> >> > On Tue, Nov 16, 2010 at 11:13 AM, Yann I. <yann.frm at gmail.com> wrote:
> >> > > Well I'm not sure because of the flag I used for the UDP source
> which
> >> is set
> >> > > to "no-parse".
> >> > >
> >> > > Here is my problem. From the UDP source, I may receive logs which
> are
> >> not
> >> > > "syslog compliance". So I'm using the flag 'no-parse' then I rewrite
> >> the
> >> > > message. After that rewrite, I forward the new message to the same
> >> syslog-ng
> >> > > server.
> >> > > Then... I can apply filter, parser, etc on that new message which is
> >> now
> >> > > "syslog compliance" :-)
> >> > >
> >> > > So, I think I can't use log statement. I need to use that
> mecanism...
> >> > > There might be another solution but this one seems to be a good
> >> solution.
> >> > >
> >> > >
> >> > > 2010/11/16 Martin Holste <mcholste at gmail.com>
> >> > >>
> >> > >> Ok, then this should be accomplished with a standard log statement
> >> > >> like you've already begun to write.  What do your destinations look
> >> > >> like?
> >> > >>
> >> > >> On Tue, Nov 16, 2010 at 10:58 AM, Yann I. <yann.frm at gmail.com>
> wrote:
> >> > >> > In fact, this is the same process... There is only one process.
> >> > >> >
> >> > >> >
> >> > >> > 2010/11/16 Martin Holste <mcholste at gmail.com>
> >> > >> >>
> >> > >> >> Why do you need separate syslog-ng processes running?
> >> > >> >>
> >> > >> >> On Tue, Nov 16, 2010 at 10:49 AM, Yann I. <yann.frm at gmail.com>
> >> wrote:
> >> > >> >> > Hi !
> >> > >> >> >
> >> > >> >> > I have a question about the use of udp, unix-stream or pipe. I
> >> would
> >> > >> >> > like to
> >> > >> >> > forward a syslog message to the same syslog server like this :
> >> > >> >> >
> >> > >> >> > |  log {
> >> > >> >> > |     source (s_r_udp);    (<-- listen on UDP/514)
> >> > >> >> > |
> >> > >> >> > |     filter (....);
> >> > >> >> > |     filter (....);
> >> > >> >> > |     parser (...);
> >> > >> >> > |
> >> > >> >> > |     destination (d_local_syslog);  (<-- send the message to
> a
> >> local
> >> > >> >> > syslog
> >> > >> >> > by using unix-stream, udp or pipe mecanism)
> >> > >> >> > |  };
> >> > >> >> >
> >> > >> >> > (...)
> >> > >> >> >
> >> > >> >> > |  log {
> >> > >> >> > |     source (s_local_syslog);    (<--- here I receive the
> >> messages
> >> > >> >> > sent
> >> > >> >> > by
> >> > >> >> > the "d_syslog_loop")
> >> > >> >> > |
> >> > >> >> > |     filter (...);
> >> > >> >> > |     filter (...);
> >> > >> >> > |     parser (...);
> >> > >> >> > |
> >> > >> >> > |     destination (d_remote_syslog);
> >> > >> >> >
> >> > >> >> > I'm looking for the better way to send syslog message to the
> same
> >> > >> >> > syslog
> >> > >> >> > server : which mecanism provides the better performances :
> pipe,
> >> udp
> >> > >> >> > (by
> >> > >> >> > using network) or unix-stream ?
> >> > >> >> > Maybe the "pipe" is the better solution ?...
> >> > >> >> >
> >> > >> >> > I'm using the syslog-ng OSE 3.1.2 on CentOS.
> >> > >> >> >
> >> > >> >> > Regards,
> >> > >> >> >
> >> > >> >> > Yann I.
> >> > >> >> >
> >> > >> >> >
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20101118/2068b4d0/attachment.htm