Subject: Re: How Lisp's Nested Notation Limits The Language's Utility
From: rpw3@rpw3.org (Rob Warnock)
Date: Sat, 05 May 2007 21:52:51 -0500
Newsgroups: comp.lang.lisp
Message-ID: <BJqdnd9R65ee3qDbnZ2dnUVZ_vGinZ2d@speakeasy.net>
Malcolm McLean <regniztar@btinternet.com> wrote:
+---------------
| "Rainer Joswig" <joswig@lisp.de> wrote in message 
| > When I write code for a batch system, I type to an
| > editor and the objects I work with are characters,
| > words, sentences. Functions, declarations, etc.
| > The running application is far away. I have to imagine
| > what my code will do once it runs. Not so in Lisp.
| >
| > When I write Lisp code, I talk to an object system via code
| > on the screen. There is something alive on the other side.
| > It is a running Lisp image fully loaded with functions, objects,
| > Lisp code. I type a bit code. Then I give that code to the Lisp...
| > I talk to a debugger who gives me access to the objects,
| > functions and the code. So Lisp code has a feel associated to it. ...
|
| This makes a lot of sense. I know that I ought to integrate Lisp
| with emacs, but I can't get it to work. So I'm writing it like C -
| write the script, and submit it. So it works, but not nicely, and
| I am always counting parentheses...
+---------------

1. Get an editor -- *ANY* editor, not necessarily Emacs -- which
   at the *very* least *balances* [not counts -- nobody counts]
   parentheses for you. For example, in most versions of Vi[1]
   ":set showmatch matchtime=2" will "flash" the cursor back on
   the matching open parenthesis for ~200ms whenever you're in
   insert mode and type a closing parenthesis. And when sitting
   with the cursor on a parenthesis [either flavor], the "%" command
   will move you to the matching parenthesis, and when used as
   the "motion" command of a any of the commands that take motion
   commands to specify their scope, will select the entire s-expr
   as the argument. That is, "ady (or "ad%) will copy an s-expr
   into Q-register "a (or delete it, respectively); the "ap commmand
   will "paste" it back, >% and <% will shift all of the lines
   containing the s-expr right or left, respectively. [Note: When
   using Vi for Lisp, it helps to ":set shiftwidth=1". Then you
   can do >%... to shift some form right four spaces, etc.]

2. You speak of "scripts": {Develop,Choose,Use consistently}
   *some* stylized way of writing your scripts that lets you
   load them into a running CL image *without* executing them,
   so you can debug them at the REPL rather than in "batch" mode
   as you seem to be doing [and as Rainer rightfully criticizes].
   That lets you *use* the powerful debugging tools that exist
   inside most Lisp systems.

   For example [not to be blindly copied, but just so you know
   that I *do* practice what I preach here], I have the CL that
   I most often use [CMUCL] set up so that when I run it with a
   REPL, the keyword :REPL ends up on a *SCRIPT-EXIT-HOOKS* list,
   but *not* if it's run from a script[2]. So I write all of my
   "#!/usr/local/bin/cmucl -script" scripts[3] with a MAIN function
   that does all the work, and then at the very bottom of each
   script file contains this:

       (unless (find :repl *script-exit-hooks*) ; Debugging?
	 (apply 'main *script-args*))

   or this [depending on the needs of the particular script]:

       (unless (find :repl *script-exit-hooks*) ; Debugging?
	 (apply 'main (mapcar #'read-from-string *script-args*)))

   [...where *SCRIPT-ARGS* ends up being a list of lightly-massaged
   *COMMAND-LINE-STRINGS*.]

   The point of this is that if you run the script from the shell,
   it "just runs". But if you start up your REPL and LOAD the script,
   it just sits there and waits for you to poke at it from the REPL,
   e.g., by calling MAIN with a list of manually-constructed command-
   line arguments, or by calling some of the internal functions of
   your script, etc. If something goes wrong, you drop into the
   internal Lisp debugger, where you have access to the entire
   environment *and* a full CL implementation at your command,
   including the compiler!

3. *Use* the Lisp environment you make your life easier!!
   Define as many little personal "conveniece" functions
   and/or macros as you find you want or need, and arrange
   that they get loaded into your default REPL. Here's just
   a small subset of mine:

   ;;; More shortcuts & conveniences:
   (defun ap (string &optional package)    ; be more liberal about 2nd arg
     (apply #'apropos string (when package (list (find-package package)))))
   (defun de (&rest rest) (apply #'describe rest))
   (defun dis (&rest rest) (apply #'disassemble rest))  
   (defun mxp (&rest rest) (pprint (apply #'macroexpand rest)))
   (defun mxp0 (&rest rest) (apply #'macroexpand rest))	; same, but w/o PP
   (defun mxp1 (&rest rest) (apply #'macroexpand-1 rest))
   (defun mxp* (&rest rest) (apply #'walker:macroexpand-all rest)) ; CMUCL only

   ;;; For REPL compiles of functions with non-NIL lexical environments,
   ;;; e.g, (COMPILE* (let ((y 5)) (defun add5 (x) (+ x y)))).
   (defmacro compile* (&body body)
     `(funcall (compile nil (lambda () ,@body))))

   ;;; I don't know why I find myself needing this so often, but I do...
   (defun hash-table-alist (hash-table &key (test (constantly t)))
     (loop for key being each hash-key in hash-table using (hash-value value)
       when (funcall test key value)
	 collect (cons key value)))

4. *Use* the Lisp environment you make your life easier!!
   If you can't (or don't want to) use ASDF or MK:DEFSYSTEM
   or equiv., at least whip yourself up a little function
   that's loaded into your default REPL that lets you reload
   and/or recompile all the files you're currently working on --
   something short [e.g., "REDO"] that's data-driven [e.g., a
   global named *USER-LOAD-LIST*, maybe]. I have one[4] that I
   call LOAD*, that does the following:

   - Remembers when it was last called.
   - If called with args, adds them to the end of *USER-LOAD-LIST*
     (suppressing duplicates).
   - For each file in the [possibly-just-modified] *USER-LOAD-LIST*,
     if the FILE-WRITE-DATE of either the file or [if a compiled
     version already exists] the compiled version of it is newer
     than the last LOAD* time, re-LOAD that file [recompiling it
     first, iff a compiled version already existed].

   Then say (DEFUN REDO () (LOAD*) (current-unit-test args...)),
   and your "edit/compile/test" cycle becomes:

   - Type "Save" in all the editor windows in which you've made changes.
   - Type (REDO) to the REPL.
   - [Optional:] If what you're working on is a persistent web
     applications server, hit your web browser's "Reload" or "Refresh".

   That's all the basic "IDE" you need to be *very* productive
   in Lisp [CL, Scheme, whatever].

Now that you know all this, time to start coding.  *NOW!!*
[Or I'll sic Kenny on you...  ;-} ]


-Rob

[1] Some Lispers may consider me a heretic because I don't use Emacs
    when writing Lisp. Tough. [I've *tried* to, several times, but
    my fingers just don't bend that way. And besides, I've been using
    "moded" editors like TECO, Bravo, Ed, & Vi for more than half my
    total lifetime.] Frankly, I spend such a *small* percentage of
    my programming time on "editor issues" that I simply don't buy
    the "I don't have (or can't use) the perfect editor (or IDE)"
    obstacle as a legitimate excuse for not simply getting on with
    Lisp programming [though as noted in the text above it *does*
    help a lot if you have an editor that at least *shows* you the
    balancing paren, even better if it can delete/shift an s-expr
    as a group -- which almost *all* can, even if not with a single
    keystroke].

    So if someone says, "I can't use Lisp... no {Emacs,SLIME,IDE}..."
    I just say, "FIDO!" [An acronym of supposed military origin;
    the last two letters stand for "Drive On"...]

[2] Unless the script explicitly chooses to push it onto the list.  ;-}
    Which is how I normally run CMUCL -- with a script named "cmu"
    that sets up a bunch of stuff, REQUIREs the usual suspects,
    pushes :REPL onto *SCRIPT-EXIT-HOOKS*, and falls off the bottom.
    This causes "cmucl -script"[3] to start a REPL and *not* silently
    exit [which is what a good script should normally do otherwise].

[3] (*sigh*) I've been promising to put my "-script" extension
    to CMUCL on the net for entirely too long. Soon, soon...
    [Hint for home hobbyists who don't want to wait: It's implemented
    by adding a few lines to "/usr/local/lib/cmucl/lib/site-init.lisp"...]

[4] No, I'm not going to clutter this article any more than
    it already is with my version of LOAD*. It's simple enough
    [mine's only ~40 lines, could be even shorter], and is a
    good beginner exercise for one to get used to building
    their own personalized "conveniences".

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607