[syslog-ng] Syslog-NG OSE : a more and more difficult choice to make.

[Christophe Brocas]

Le 14/08/2010 14:10, Balazs Scheidler a écrit :
> Hello Christophe,
>   
Hello Balazs

> First of all, thanks for your email. I really appreciate honest
> opinions, and although not all of your points are accurate, messages
> like this actually has an influence on syslog-ng direction.
>   
Thank you for understanding the meaning of my message and sorry for my
mistakes.

> On Thu, 2010-08-12 at 17:00 +0200, Christophe Brocas wrote:
>   
>> Hello everybody,
>>
>> I really enjoy the syntax, the stability, the flexibility and the so
>> clear and accurate documentation of Syslog-NG OSE. This is why I write
>> this post, I love the product, my message is definitively not a troll.
>>
>> Despite above positive aspects, it is more and more difficult to choose
>> Syslog-NG OSE in corporate environment where you have Linux platforms
>> and others Unix flavors. Rsyslog comes with security and performance
>> features (sql driver, disk based bufferring, Solaris port etc) inside
>> whose can only be acquired through Premium Syslog-NG Edition.
>>     
> This is not completely true, the platforms supported by syslog-ng are by
> no means less than the premium edition. We don't build binaries of the
> OSE edition for all of PE's platforms, but the code is the same,
> everyone is free to build it on his/her platform of choice. In fact a
> number of binary download site do have syslog-ng binaries (sunfreeware
> for Solaris, perzl.org for AIX) and we also work together with the
> maintainers of these sites on updating OSE packages in these
> repositories, just like we worked hard to update the syslog-ng package
> in Linux distributions.
>
> syslog-ng OSE had the SQL destination feature since 2.1, first released
> in January 2008.
>   
As Patrick as said before in the thread, totally true. Sorry for my
mistake :(

> The only missing item in your list is disk based buffering. This is
> true, but also quite easy to work around:
>   * write everything to a local file and
>   * set up the same file as a source driver
>
> So while it may seem that rsyslog has more hype around it, it isn't
> true, that it surpasses syslog-ng OSE in all ways.
>   
Ok.

> Also, I feel important to note that syslog-ng has been refocused in
> recent years and now it also cares about the content of the messages. It
> is not merely a transport for syslog messages anymore and I think this
> certainly is ahead of what rsyslog provides.
>   
That is right but it depends how each organization uses its syslog
architecture (transport vs messages understanding). I think Syslog-NG
has a rough battle ahead because messages exploitation leads directly to
SIEM solutions.

A quite hard question to answer : where does a log messaging solution
have to stop its development ?

> This is what those parsers & rewrite rules are about, and also in the
> recent 3.2 release it also introduces support for binary but structured
> source files (it can read Process Accounting logs). Doing things like
> receiving SNMP traps as name-value pairs and polling SQL tables for new
> logs are in the pipe.
>
> I'd like to push out an update to the current syslog-ng OSE roadmap at
> the webpage, but anyway, here are my plans for the near future:
>
> 1) syslog-ng OSE 3.2 is out as an alpha release, but I don't expect too
> much problems there, I guess 3.2.0 can be released latest in a month.
> syslog-ng was rearchitected to be plugin based and other important
> changes were applied (see my last blog posts for more details).
>
> 2) syslog-ng OSE 3.3/syslog-ng PE 4.0 is going to be developed in
> parallel, 
>   * OSE 3.3 will focus on performance
>   * PE 4.0 is going to be the last long-term-support release ("stable"
> as we call it) based on the current, forked syslog-ng OSE codebase
>
> 3) syslog-ng PE and OSE will be merged into PE 4.1, this means that
> existing core (e.g. non-plugin) features of the PE will be migrated to
> the OSE and core-wise they will become equivalent. This will mean that
> the "wildcard log files" and the recent multiline feature will
> definitely go to the OSE version. The disk buffer however is still
> undecided.
>   
Oh, it is a great news !

Of course, it will be great to have disk buffering inside the OSE
because I really think by this way, Syslog-NG would close the story
about syslog transport : Syslog-NG OSE would have all the features
required for log transport : security (authentication, integrity and no
lost of messages), performance and easiness of exploitation (syntax,
wildcard etc).

And then, the debate will go the message exploitation where as you
demonstrated, Syslog-NG is ahead of all others solutions.

One thing :

do you think about switching from OSE and PE editions model to only one
distribution which would be Open Source and selling closed source
plugins which would be usable through this new only open source Syslog-NG ?

IMHO, it would be far more easy to promote in the Open Source community
than open source vs premium editions. But, of course, only you can say
if it is a sufficient model to provide a living for Balabit.
 
>> If in the future, Rsyslog provides an AIX port on PPC architecture, I
>> really think it will be an ended story for Syslog-NG on corporate
>> environment : it will no more exist a technical reason to stay with an
>> open source under powered solution like Syslog-NG OSE or to buy a
>> solution while it exists an opensource solution with same / more features.
>>     
> I would really question that rsyslog has the same or more features. In
> some areas it surpasses syslog-ng, in others it is lacking. 
>   
You are right. The key feature is disk based buffering I think and
that's why I think it would be a major step in Syslog-NG history if you
integrate it inside the Syslog-NG 4.1 OSE.

>> I really understand everybody has to earn its life, really. But the
>> current situation in the open source syslog products area is quite
>> difficult for Syslog-NG, that's why I wanted to point the above facts in
>> corporate environment out to you. I don't know how to do : more
>> appliances, more closed products, more consulting ... but the 2 flavors
>> (free and paid) of Syslog-NG are imho an each day harder choice to defend.
>>     
> Well, don't look at the functionality only. In the PE edition there are:
>   * binaries for 27 platforms (and growing)
>   * thorough testing for each release
>   * long term support
>
> Apart from the few feature differences, PE really makes it easier to
> deploy syslog-ng in enterprise environment. If you have 3 different
> platforms (Solaris, Linux, AIX), possibly multiple versions of these,
> how long does it take to compile syslog-ng on them? And what if there's
> a bug/security issue and you need to rebuild?
>
> It is exactly the same set of incentives that for example RedHat uses in
> its Enterprise Linux offering. The difference is that we also have some
> additional features, because certainly an Operating System is applicable
> to more situations, the market is larger and the number of people
> willing to pay solely for services is larger.
>
> With syslog-ng, this is not true. But, please read my recent blog post
> (also posted to this list).
>   
You have got the point :-)

>> It is the message from a Syslog-NG user that would like to be able to
>> promote and use it in its company for a long time.
>>     
> Hopefully I could at least blur the picture somewhat. It is not black &
> white.
>   
Thank you very much for your answer which is very usefull for users like
us : it gives a good visibility for the future of Syslog-NG.

I really hope that Syslog-NG will be back in the heart of Linux
distributions and users because it deserves it : so clean syntax,
accurate documentation, performance, security and advanced messages parsing.

Bye
Christophe

-- 
Christophe Brocas
keyid  : 0x237E9DB2




*****************************************************
"Le contenu de ce courriel et ses eventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire.

Attention : L'Organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'Organisme sauf s'il en est disposé autrement dans le présent courriel."
******************************************************