Subject: Re: Lisp is neither (was Re: Ousterhout and Tcl lost the plot)
From: Erik Naggum <>
Date: 1997/04/20
Newsgroups: comp.lang.scheme,comp.lang.scheme.scsh,comp.lang.lisp,comp.lang.tcl,comp.lang.functional,comp.lang.c++,comp.lang.perl.misc,comp.lang.python,comp.lang.eiffel
Message-ID: <>

* Scott Schwartz
| The catch, as you well know, is that those implementations also have
| huge, expensive runtime systems and slow startup times, so to be usable
| you have to live inside them.

have to?  not really.  since you implicitly involve the Common Lisp system
I use, let me illustrate with a real life example: I used to run all
incoming mail and manage all mailing lists from within Emacs, including
increasingly advanced spam-filters.  as part of moving to an entire mail
delivery system written in Allegro Common Lisp, I rewrote the Emacs Lisp
code into Common Lisp.  to handle one message, the ACL image is started
from sendmail (just like Emacs was), and proceeds to deliver the message
(if accepted) and return an (error) exit code to sendmail after having
decided what to do.  it outputs its new decisons, if any, as Lisp code to a
file, which is loaded by the next instantiation.  (the Emacs version loaded
all the decisions every time it was started.)  every once in a while, I
bring up a fresh Lisp image, load the decisions to date, and dump a new
image to save on startup time.  it is really this image that is executed.

on my 50MHz SPARC 2 with 64M RAM, this solution handles ~15 messages per
second at full throttle, but after having written ~600 new decisions to
disk, that is reduced to ~10 message per second.  however, I receive 100 to
150 messages a day, so this is not at all a problem for me.  the load on
the system from handling a message is not noticable.  I don't trust the
SMTP server I have written enough to let it handle the entire mail process,
but once I do, the system will be able to handle more than 400 rejections
and around 35 delivered messages per second.  the bottle-neck is the fork
system call, which is unfortunately much slower than it should be on my
system.  if I write to the mailbox directly, I can deliver 10 times more
messages.  in practice, however, the bottle-neck is my Internet link, which
is enough for my needs, but it couldn't handle more then 5 messages per
second, anyway.  my aim is to make this fast and flexible enough to work
for a large ISP that can offer spam-blocking as a service to their users.

| On this topic, you yourself have argued that the lisp repl should be
| one's shell and that it should be judged in that context.

I fail to see the contradiction you imply.  if the "scripts" are in Lisp,
as functions inside the Lisp image, run directly from the shell/repl, they
will be vastly faster than a fork/exec with a lot of dynamic libraries and
interpreters being loaded alongside it, no matter how well you do it.  by
using Lisp for the scripting language, you will need to run fewer external
programs.  the net effect is that instead of starting up the a shell to run
the entire script and starting up new shells to handle other scripts in the
process, you will run them all inside the first Lisp.  there's much to be
saved with this approach, and unless the first action of the "script" is
the most important, the speed of the whole script should increase.

| While that might be to your taste, you should acknowledge that it is
| totally at odds with how things are done in the (much nicer) underlying
| system, and pretty much proves Ousterhout's point.

you know, what I realized when I started to use a real Lisp (to a lesser
degree when I had only CMUCL, but I realize now that I could have done more
with CMUCL than I did) was that I used to think of Unix and its shell and
all the shell scripts and programs in a model that fits much better with
working within Emacs with a Lisp underneath than with a shell and standard
I/O as the communications vehicle.  since almost as far back as when I
first used a Unix system in 1980, I have wanted programs to return their
values to some environment, not just print something on standard output.  I
have wanted to control how the output of these programs were formatted and
reused, I have wanted persistent default options to programs, without
revamping the entire system or use thousands of aliases that other programs
don't know about.  I won't discount the possibility that I'm a natural born
Lisper, but it seems that Unix and its plethora of tools and languages are
really trying hard to become what Lisp systems could already offer.

so, no, I don't think this is totally at odds with how things are done in
the (much uglier) underlying system (Unix/MS-DOS/whatever), I think it does
the job that Unix and its tools and shells try to do much _better_, and
that effectively disproves Ousterhout's bogus points.

now, don't forget that Unix wasn't really a _new_ idea, either.  almost all
of the good stuff came from the MULTICS project, with which the Bell Labs
team became very impatient and they did much the same to MULTICS in Unix
that CP/M did to TOPS-10, after which it was modeled; that is, reducing it
to only bits and pieces, retaining a flavor of the previous systems' glory.

sometimes, I wonder how those large systems could be built with the meager
hardware resources they had at their disposal.  maybe it's _because_ they
had less powerful hardware, and had to think more and waste less.  the Bell
Labs team really wanted a PDP-10, according to the history written down by
Peter Salus in A Quarter Century of Unix.  imagine what the world would
have been if they did.  (on the other hand, Bill Gates got his hands on a
PDP-10 at Harvard and Microsoft used PDP-10's for their development for a
long time.  look where that brought us.)

when I gave my annual guest lecture on Unix and its history (this time
focusing on Unix as a networked, social development, and MS-DOS as a
non-network, asocial development, with software in source form being what
built and could save an industry that consisntely refuses to learn from its
own discoveries and mistakes) at a college this past week, and pointed out
that the Unix tools philosophy was "one job, one tool", a refreshingly
bright student came up and asked me if Emacs violated the "one job, one
tool" philosophy.  it is a good question.  I answered that Unix itself
provides an environment in which those tools work together and that that is
its job, exposed to the user through the shell.  Emacs (and by extension,
any Lisp system) provides just such an environment, a more powerful and
portable one than the Unix shells do, yet it can work with the other tools
and be part of the system, too.  (I'm not sure I'm satisfied with that
answer, but I think she was...)

Bastard Sex Therapist from Hell: "Read the F*cking Manual!"