[syslog-ng] [PATCH]: Experimental SMTP destination

[Gergely Nagy]

On Mon, 2011-02-14 at 14:37 +0000, Alexander Clouter wrote:
> Gergely Nagy <algernon at balabit.hu> wrote:
> >>
> >> So isn't the built-in MTA a solution in search of a problem? 
> > 
> > Nope, it isn't. I have routers that do not have a reliable MTA, and I
> > don't intend to install one on them for the sole purpose of sending
> > syslog alerts.
> > 
> Then make them reliable (for me it was 'aptitude install nullmailer', 
> not complicated).

nullmailer fails at supporting tls, so even if I install it, I still
need an intermediate relay, or another tool to provide TLS for me.

At which point, I can just add smtp() to syslog-ng, teach it TLS & auth
(both of which is supported by libesmtp, I only need a few lines of glue
code in the destination driver) and I'm better off that way. Less extra
space, less stuff to configure, less points of potential failure.

> How do you plan to send cron job alerts to the 
> sysadmin if there is no local MTA?  All UNIX hosts really should have a 
> way to send email to their sysadmin, especially if they are a crucial 
> part of the organisation's infrastructure.

Those alerts are logged to syslog, and my smtp destination can send the
emails. I don't need a local MTA, nor do I desire one.

> > There might be N other ways to accomplish the same thing, a built-in
> > smtp destination was the easiest for me, so I coded it up. All the other
> > soultions I could think of would have taken me more time and effort.
> > 
> N ways that already exist, and have worked for ~30 years.

N ways already exist, neither of which is suitable for my use-case. I
might be the first guy in 30 years, in which case I feel honored.

> > I had an itch, I scratched it, I'm happy.
> > 
> Well, that's a different and perfectly okay objective.  The issue is 
> that sending mail from a UNIX host is an already solved problem.

Provided that said host has the tools for it, yes. None of the tools
available in debian were found to fit my needs. Unless combined, which
IS the UNIX way, but is something I want to avoid in this case.
 
> >> I'm pretty sure it won't be too hard to configure syslog-ng so, that
> >> it's pretty easy to troubleshoot the MTA that it sends it's messages to.
> > 
> > The thing is, I don't want a full-blown MTA on the same host that
> > syslog-ng runs on. So at best, I can have a program that forwards alerts
> > to an external host.
> >
> Nullmailer is *not* a full-blown MTA but it does reliably deliver mail 
> to a smarthost.  It's a 500kB (100kB binary) install with *all* the man 
> pages and Debian documentation.

It does not support TLS, though, as far as I saw. Thus, I'd still need
an intermediate relay, or another program to provide TLS for me.

> > Having an built-in smtp destination, that will have tls & auth real
> > soon, does not have these issues.
> > 
> These are problems that do not exist.  Can you tell me what problems you 
> have seen (not theorised) there are by just forking sendmail 
> (wrapper)/nbsmtp/whatever?

Problem #1: I need a suitable sendmail on the host.
Problem #2: Depending on the MTA, I will likely need something else to
provide TLS support for it.
Problem #3: External tools do not provide anything over my smtp driver,
at least not in my use-case, while my smtp driver solves my immediate
issue of not having to install external tools to get the job done.

Yes, the problem can be solved with program() in most cases. My case is
not the common case, though, and I do not see harm in catering to a few
corner-cases when it does not disrupt anything else.

> >> But the main point is, that I don't think that an internal SMTP
> >> destination will do any good for misconfiguration. I.e. misconfiguration
> >> is a problem in itself, and no matter how many destinations you add to
> >> syslog-ng, you won't be able to prevent misconfiguration.
> > 
> > Correct. But as far as I'm concerned, I find it easier to debug one
> > program, on one host, than go through N hops to find which one is
> > misbehaving.
> > 
> It's far more difficult to ram the SMTP RFC's into a syslog daemon.

Which is why I didn't do that, but rely on libesmtp, which does the bulk
of the job for me. I don't need all of the SMTP RFC, only a subset,
which libesmtp implements.

If I ever need more than that, I will either extend libesmtp, or find a
way to use program(). Right now, program() would require more work from
me, than the smtp destination did.

> > With the program() destination, in order for my alerts to leave my
> > system, it would go through several hops.
> > 
> > Assuming a sendmail that can do TLS and auth (best case):
> > 
> >        syslog-ng -> sendmail -> external SMTP host
> >
> 'best' = 'typical'

Yes. But I see no harm in making corner-cases easy too. Typical installs
can still use program() if so they wish, and they need the extra
features (if any).

> > With a bit of luck, the sendmail program will do proper logging, so I
> > can troubleshoot it easily. Hopefully, it will also do queuing, either
> > in memory, or on disk, so that when the SMTP host is unreachable, my
> > alerts will not get lost.
> >
> 'luck' = 'typical'
>
> 'queue' is down to personal preference, but as I keep hammering, 
> nullmailer and nbsmtp both do queuing and have done so for *years*.  I 
> am guessing syslog-ng will not queue a 10kB email for me (as it would be 
> nice to call on mailx's '~!').

nullmailer doesn't have TLS, so I'd have to install yet another tool to
fix that, or use an intermediate relay. I'd rather not do either. 

I tried to check what nbsmtp can do, but it's not available on neither
Lucid, nor Squeeze, and the homepage returns a default Debian Apache
page.

As for queuing: syslog-ng will queue any log messages that it couldn't
send via the smtp driver (the message, not the email), and will
reconstruct the mail at a later point.

> > There's too many ifs in there, and I'm really not interested in finding
> > a small enough forwarder that can do all that. I know busybox can't, nor
> > can nullmailer. Dragonfly Mail Agent (dma) might be a good candidate,
> > but I'd need to review it. Esmtp is relay-only, and does not queue.
> > 
> nullmailer can and does queue, I use it on my laptop to get mail off to 
> a smarthost.  It also logs.

But, as I said, it doesn't do TLS. All of the listed solutions do part
of what I want, but neither does it all. Thus, if I go with either, I
will have to install something else aswell, which just complicates
things.

A built-in smtp() driver is far easier for me. It can do everything I
want, in one place.

> > With a built-in smtp destination, the setup would look like this:
> > 
> >        syslog-ng -> external SMTP host
> > 
> > Boom. The smtp driver will detect if there's a network outage, and queue
> > the mail, and retry later. It does this using syslog-ng's built-in
> > queuing mechanism. I don't need to go an extra mile to make sure that
> > any message sent from syslog-ng will be retried at least until the queue
> > fills up.
> >
> nullmailer....nullmailer....nullmailer. :-/

TLS... TLS... TLS.

> If we say, "yes there is a void in lightweight MTA's that can do 
> queuing, auth and TLS" the fix is to dedicate your efforts to produce 
> one.  Then *everything* can use it...not just syslog-ng.  Putting a SMTP 
> transport in syslog-ng helps no one, but the syslog-ng users.

I'm fine with helping syslog-ng users only. Producing a suitable MTA
that can do everything I want would require far more effort than
producing the smtp destination took.

> I am forced to use program to open 'psql' as syslog-ng cannot do simple 
> "SELECT function('foobar', 'cheese', 'toys');" and also a Perl script to 
> process squid log files too[1] as syslog-ng cannot do xyz; tasks which 
> are syslog-ng specific...  These work *extremely* well.

So sql() has next to no use, let's remove it, it's useless.

(I'd actually agree with that - sql-specific drivers, which exploit the
features specific to said sql server would be of much more value, in my
opinion)

> Comparing smtp() to sql() is not quite like for like.  You get 
> additional functionality when accessing it natively such as prepared 
> statements and such.  Natively speaking SMTP does not buy you anything 
> that I am aware of?

I already listed why I prefer a native driver over using program(), but
let me reiterate:

* I want TLS, auth and queuing, preferably in one place. None of the
existing lightweight solutions I know of provide that in themselves.
Yes, nullmailer comes close, but it does lack TLS, which forces me to
setup or install yet another tool.
* program() was designed with a long-running process in mind, and even
though syslog-ng will restart a program that exists, that's not the
intended usage. Yes, one can write a wrapper that doesn't exist, but
then I could just use the smtp destination. It also restarts the
program, if it exits, but personally, I find that an abuse, and if I'd
use program(), I'd do that with a long-running wrapper instead. And then
it's easier to use smtp()...
* A built-in driver suited my immediate needs best, it's optional,
noone's forced to use it if they're happy with something else.

> >> All in all, it sounds like the perfect solution for a certain
> >> widely-used proprietary operating system, that has no proper built-in
> >> SMTP possibilities. And because the SMTP destination is configurable
> >> with ENABLE_SMTP, Linux builds can easily avoid the new dependency ;)
> > 
> > If you're talking about the OS I think you're talking about, let me
> > assure you, I wouldn't touch it with a ten feet pole if I can avoid it
> > (not to mention, that libestmp is most probably not available for said
> > platform).
> > 
> Ironically libesmtp is larger than nullmailer on my amd64 Debian laptop.

That's interesting. libesmtp.so is ~72k on my amd64 lucid, while
nullmailer (/usr/lib/nullmailer/* + /usr/bin/nullmailer-inject
+ /usr/sbin/nullmailer*) is ~172k. I'm not sure how much of that I'd
need for simple forwarding, but I assume I need at least
nullmailer-queue and nullmailer-send, which are 60k.

And indeed, 60k is smaller than 72k, but libesmtp also does TLS, which
nullmailer does not.

> I think the reference you are missing here is that with Windows it is 
> *very* difficult (unsurprisingly) to apply the original UNIX philosophy 
> of stringing together many small simple tools that do their singular 
> task very well to perform the job you need.  In Windows, if the 
> application you are using does not support what you need it to do...you 
> are generally out of luck.

I suppose you're right. I do not know Windows well enough to make an
informed opinion (and I'd prefer to keep it that way).

> What do you do when given the task of parsing a CSV file (overlook 
> escaping) to summing up the numbers in the fourth column and producing 
> the average?  Under UNIX would you reach for OpenOffice or awk?  Under 
> Windows you typically have no choice but to do the task in Excel.

I'd go with awk, yes. But if I had an application that would need this
data, and I had a library to parse CSV files, I probably wouldn't call
out to awk, but use the parser library instead.

It does depend on the use case, though. If I'm writing a shell script,
then awk it is. If I'm writing a GUI application, I'd rather use a
library instead of calling out to awk.

> This is what I'm trying to communicate over, possibly poorly.  I really 
> am just very curious what the actual *real* problem is, and where all 
> this supposed unreliability in calling sendmail comes from?

The actual problem is, that I need TLS + Auth. I could do that with
program(), if I install an MTA that can do at least auth and queuing,
and a wrapper for TLS. However, that's something I want to avoid for
various reasons, ranging from limiting the installed and running
programs to a minimum to finding it easier to debug & maintain when it's
built into syslog-ng itself.

If the smtp driver fails, I'll see that immediately from the logs, and
can start debugging right away. If an external program fails, I will see
the fail in the logs, then will have to start figuring out WHY it failed
(which is something I already know from the smtp driver's logs), and can
only start debugging then. Using an external program also means I will
have to learn - sooner or later - how to efficiently debug it. I already
know how to handle syslog-ng.

It's a corner case, indeed, highly motivated by personal preferences. I
do not think that this makes it a bad itch to scratch, though. It's
merely an itch that few people have.

> On a passing note, what would be handy is some v6 netmask() 
> action...much obliged if you find and scratch that itch :)

Hnnngnhn. And I was so happy that I scratched this itch, you find me
another one! Argh. (Added to my TODO list :P).

-- 
|8]