Subject: Re: Lisp is neither (was Re: Ousterhout and Tcl lost the plot) From: Erik Naggum <erik@naggum.no> Date: 1997/04/20 Newsgroups: comp.lang.scheme,comp.lang.scheme.scsh,comp.lang.lisp,comp.lang.tcl,comp.lang.functional,comp.lang.c++,comp.lang.perl.misc,comp.lang.python,comp.lang.eiffel Message-ID: <3070566458689207@naggum.no> * Scott Schwartz | The catch, as you well know, is that those implementations also have | huge, expensive runtime systems and slow startup times, so to be usable | you have to live inside them. have to? not really. since you implicitly involve the Common Lisp system I use, let me illustrate with a real life example: I used to run all incoming mail and manage all mailing lists from within Emacs, including increasingly advanced spam-filters. as part of moving to an entire mail delivery system written in Allegro Common Lisp, I rewrote the Emacs Lisp code into Common Lisp. to handle one message, the ACL image is started from sendmail (just like Emacs was), and proceeds to deliver the message (if accepted) and return an (error) exit code to sendmail after having decided what to do. it outputs its new decisons, if any, as Lisp code to a file, which is loaded by the next instantiation. (the Emacs version loaded all the decisions every time it was started.) every once in a while, I bring up a fresh Lisp image, load the decisions to date, and dump a new image to save on startup time. it is really this image that is executed. on my 50MHz SPARC 2 with 64M RAM, this solution handles ~15 messages per second at full throttle, but after having written ~600 new decisions to disk, that is reduced to ~10 message per second. however, I receive 100 to 150 messages a day, so this is not at all a problem for me. the load on the system from handling a message is not noticable. I don't trust the SMTP server I have written enough to let it handle the entire mail process, but once I do, the system will be able to handle more than 400 rejections and around 35 delivered messages per second. the bottle-neck is the fork system call, which is unfortunately much slower than it should be on my system. if I write to the mailbox directly, I can deliver 10 times more messages. in practice, however, the bottle-neck is my Internet link, which is enough for my needs, but it couldn't handle more then 5 messages per second, anyway. my aim is to make this fast and flexible enough to work for a large ISP that can offer spam-blocking as a service to their users. | On this topic, you yourself have argued that the lisp repl should be | one's shell and that it should be judged in that context. I fail to see the contradiction you imply. if the "scripts" are in Lisp, as functions inside the Lisp image, run directly from the shell/repl, they will be vastly faster than a fork/exec with a lot of dynamic libraries and interpreters being loaded alongside it, no matter how well you do it. by using Lisp for the scripting language, you will need to run fewer external programs. the net effect is that instead of starting up the a shell to run the entire script and starting up new shells to handle other scripts in the process, you will run them all inside the first Lisp. there's much to be saved with this approach, and unless the first action of the "script" is the most important, the speed of the whole script should increase. | While that might be to your taste, you should acknowledge that it is | totally at odds with how things are done in the (much nicer) underlying | system, and pretty much proves Ousterhout's point. you know, what I realized when I started to use a real Lisp (to a lesser degree when I had only CMUCL, but I realize now that I could have done more with CMUCL than I did) was that I used to think of Unix and its shell and all the shell scripts and programs in a model that fits much better with working within Emacs with a Lisp underneath than with a shell and standard I/O as the communications vehicle. since almost as far back as when I first used a Unix system in 1980, I have wanted programs to return their values to some environment, not just print something on standard output. I have wanted to control how the output of these programs were formatted and reused, I have wanted persistent default options to programs, without revamping the entire system or use thousands of aliases that other programs don't know about. I won't discount the possibility that I'm a natural born Lisper, but it seems that Unix and its plethora of tools and languages are really trying hard to become what Lisp systems could already offer. so, no, I don't think this is totally at odds with how things are done in the (much uglier) underlying system (Unix/MS-DOS/whatever), I think it does the job that Unix and its tools and shells try to do much _better_, and that effectively disproves Ousterhout's bogus points. now, don't forget that Unix wasn't really a _new_ idea, either. almost all of the good stuff came from the MULTICS project, with which the Bell Labs team became very impatient and they did much the same to MULTICS in Unix that CP/M did to TOPS-10, after which it was modeled; that is, reducing it to only bits and pieces, retaining a flavor of the previous systems' glory. sometimes, I wonder how those large systems could be built with the meager hardware resources they had at their disposal. maybe it's _because_ they had less powerful hardware, and had to think more and waste less. the Bell Labs team really wanted a PDP-10, according to the history written down by Peter Salus in A Quarter Century of Unix. imagine what the world would have been if they did. (on the other hand, Bill Gates got his hands on a PDP-10 at Harvard and Microsoft used PDP-10's for their development for a long time. look where that brought us.) when I gave my annual guest lecture on Unix and its history (this time focusing on Unix as a networked, social development, and MS-DOS as a non-network, asocial development, with software in source form being what built and could save an industry that consisntely refuses to learn from its own discoveries and mistakes) at a college this past week, and pointed out that the Unix tools philosophy was "one job, one tool", a refreshingly bright student came up and asked me if Emacs violated the "one job, one tool" philosophy. it is a good question. I answered that Unix itself provides an environment in which those tools work together and that that is its job, exposed to the user through the shell. Emacs (and by extension, any Lisp system) provides just such an environment, a more powerful and portable one than the Unix shells do, yet it can work with the other tools and be part of the system, too. (I'm not sure I'm satisfied with that answer, but I think she was...) #\Erik -- Bastard Sex Therapist from Hell: "Read the F*cking Manual!"