7. Fetchmail Grows Up
There I was with a neat and innovative design, code that I knew worked
well because I used it every day, and a burgeoning beta list. It
gradually dawned on me that I was no longer engaged in a trivial
personal hack that might happen to be useful to few other people. I
had my hands on a program every hacker with a Unix box and a SLIP/PPP
mail connection really needs.
With the SMTP forwarding feature, it pulled far enough in front of the
competition to potentially become a ``category killer'', one of those
classic programs that fills its niche so competently that the
alternatives are not just discarded but almost forgotten.
I think you can't really aim or plan for a result like this. You have
to get pulled into it by design ideas so powerful that afterward the
results just seem inevitable, natural, even foreordained. The only
way to try for ideas like that is by having lots of ideas -- or by
having the engineering judgment to take other peoples' good ideas
beyond where the originators thought they could go.
Andrew Tanenbaum had the original idea to build a simple native Unix
for the 386, for use as a teaching tool. Linus Torvalds pushed the
Minix concept further than Andrew probably thought it could go -- and
it grew into something wonderful. In the same way (though on a
smaller scale), I took some ideas by Carl Harris and Harry Hochheiser
and pushed them hard. Neither of us was `original' in the romantic
way people think is genius. But then, most science and engineering
and software development isn't done by original genius, hacker
mythology to the contrary.
The results were pretty heady stuff all the same -- in fact, just the
kind of success every hacker lives for! And they meant I would have
to set my standards even higher. To make fetchmail as good as I now
saw it could be, I'd have to write not just for my own needs, but also
include and support features necessary to others but outside my orbit.
And do that while keeping the program simple and robust.
The first and overwhelmingly most important feature I wrote after
realizing this was multidrop support -- the ability to fetch mail from
mailboxes that had accumulated all mail for a group of users, and then
route each piece of mail to its individual recipients.
I decided to add the multidrop support partly because some users were
clamoring for it, but mostly because I thought it would shake bugs out
of the single-drop code by forcing me to deal with addressing in full
generality. And so it proved. Getting
RFC 822
parsing right took me a remarkably long time, not because any
individual piece of it is hard but because it involved a pile of
interdependent and fussy details.
But multidrop addressing turned out to be an excellent design decision
as well. Here's how I knew:
14. Any tool should be useful in the expected way, but a
truly great tool lends itself to uses you never
expected.
The unexpected use for multi-drop fetchmail is to run mailing lists with the
list kept, and alias expansion done, on the client side of
the SLIP/PPP connection. This means someone running a personal
machine through an ISP account can manage a mailing list without
continuing access to the ISP's alias files.
Another important change demanded by my beta testers was support for
8-bit MIME operation. This was pretty easy to do, because I had been
careful to keep the code 8-bit clean. Not because I anticipated the
demand for this feature, but rather in obedience to another rule:
15. When writing gateway software of any kind, take pains to disturb
the data stream as little as possible -- and *never*
throw away information unless the recipient forces you to!
Had I not obeyed this rule, 8-bit MIME support would have been
difficult and buggy. As it was, all I had to do is read
RFC 1652
and add a trivial bit of header-generation logic.
Some European users bugged me into adding an option to limit the
number of messages retrieved per session (so they can control costs
from their expensive phone networks). I resisted this for a long
time, and I'm still not entirely happy about it. But if you're
writing for the world, you have to listen to your customers --
this doesn't change just because they're not paying you in money.