Subject: Re: lambda-returning defmacro, capture
From: rpw3@rpw3.org (Rob Warnock)
Date: Mon, 18 Dec 2006 06:44:13 -0600
Newsgroups: comp.lang.lisp
Message-ID: <75WdnVkSW_GAEBvYnZ2dnUVZ_oS3nZ2d@speakeasy.net>
<to9sn2r02@sneakemail.com> wrote:
+---------------
| > +---------------
| > | Then there's the proliferation of multiple deflex macros...
| > +---------------
| >
| > This statement verges on FUD. Since DEFLEX is not part of the
| > standard, of *course* there are multiple versions. But that's
| > no different than any other useful abstraction that is not part
| > of the standard, such as socket libraries.
| 
| Sorry, didn't mean to sound that way. But to give you a little context
| -- I'm having a hard time explaining this whole thread to other
| developers involved in the project, and conveying any feeling of
| confidence... firstly, I don't entirely understand what's going on
| (yet); secondly, you found a bug in your first version -- and then
| provided two more versions, one of which you characterized as "I
| *don't* know that it works reliably, either, but it passes a few quick
| tests". Untill I completely grok symbol macros and packages I am still
| worried.
+---------------

To give you a little context as well, if I remember the order
correctly the "first" [expands to uninterned symbol] and "second"
[expands to SYMBOL-VALUE call] versions of DEFLEX were things
I posted in the long sub-thread about the hack of using LOCALLY
to avoid declaring a variable globally special. That whole sub-thread
was sort of wild & wooly, IMHO, and my contributions were as
half-baked as anyone else's, but were attempts to show how the
LOCALLY hack might help provide a DEFLEX definition that was
considerably shorter/simpler than the usual mangled-named version.
Obviously, my first try was *too* simple, but the second should
probably work fine [though I have no long experience with it].

The chronologically "third" version of DEFLEX [expands to mangled-named
and declared special variables] -- that was in the "p.s." of the message
with the second one in it -- is actually the one I have been using for
several years, and the one in which I therefore have the most confidence.
[Especially now that the CMUCL bug has finally been found/fixed -- see
my just-posted patch!] The third version also generates better code
*in CMUCL* than the second. To repeat it again:

(defmacro deflex (var val &optional (doc nil docp))    
  (let ((backing-var (intern (concatenate 'string
                                          (symbol-name '#:*deflex-var-)
                                          (symbol-name var)
                                          (symbol-name '#:*))
                             (symbol-package var))))
    `(progn
       (defparameter ,backing-var ,val ,doc)
       ,@(when docp `((setf (documentation ',var 'variable) ,doc)))
       (define-symbol-macro ,var ,backing-var))))

+---------------
| Another question: can I
|   (defun f (x) (+ *global* x))
| and only then declare *global*  with deflex? Contrary to my intuition
| (which is that macroexpansion takes place when the defun is read) this
| does seem to work in SBCL.
+---------------

Several things about that seem rather questionable to me, though since
I don't use SBCL at all perhaps I'm probably not the best person to
address this:

1. In ANSI Common Lisp, macros need to be defined before they
   are referenced. See CLHS 3.2.2.3 "Semantic Constraints" and
   also 3.1.2.1.1 "Symbols as Forms":

      The symbol names a symbol macro if there is a binding of the
      symbol as a symbol macro in the current lexical environment...

   Thus, in general, you would need to DEFLEX any such variables
   *before* their first use.

2. I thought SBCL compiled eveything as it was entered. If so, why
   didn't it complain about the undefined free variable? CMUCL will
   certainly complain about it if you compile F:

      cmu> (compile *)
      ; Compiling LAMBDA (X): 
      ; Compiling Top-Level Form: 

      ; In: LAMBDA (X)

      ;   (+ *GLOBAL* X)
      ; Warning: Undefined variable *GLOBAL*
      ...[trimmed]...

3. In CMUCL, at least, doing a (DEFLEX *GLOBAL* 37) *after* F has been
   compiled certainly does *NOT* work [and I wouldn't expect it to]:

      cmu> (deflex *global* 37)

      *GLOBAL*
      cmu> (f 12)

      Error in KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER:  the variable *GLOBAL* is unbound.
	 [Condition of type UNBOUND-VARIABLE]
      ...[trimmed]...

   It will sometimes "work" (*unreliably!!*) in CMUCL if, when
   the DEFLEX is done, the function has not yet been compiled
   *or* "converted", an internal CMUCL operation in which a function
   is "half-compiled" [including macroexpansion] into the form used
   by the interpreter. I said "unreliably", because such "conversion"
   [and the attendant macroexpansion] can be triggered even if the
   function is never called, simply by appearing in a position where
   it *might* be called, e.g.:

      cmu> (defun f (x) (+ *global* x))

      F
      cmu> (if nil (f 13) 456)
      ; 
      ; Warning: This variable is undefined:
      ;   *GLOBAL*
      ; 
      456
      cmu> 

   Doing a DEFLEX of *GLOBAL* after this "conversion" has occurred
   will not prevent an error if F is subsequently called.

4. IMHO the *VAR* convention should be reserved for variables which
   are declared to be special, and *not* used for "global lexicals".
   Doing so will seriously confuse a Lisp-aware reader. Besides, the
   whole *point* of DEFLEX is to allow one to safely used undistinguished
   names without worrying about accidentally forcing dynamic scope on
   some inner binding.

Summing up: No, you should not count on being able to define functions
which reference undefined free variables which you *later* define as
symbol macros (e.g., with DEFLEX or equiv.).


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607