Subject: Re: special variables
From: Erik Naggum <erik@naggum.no>
Date: 09 Sep 2002 02:59:28 +0000
Newsgroups: comp.lang.lisp
Message-ID: <3240529168697173@naggum.no>

* Johan Ur Riise
| Certainly I would stand no chance at all with the documentation only.

  I think you need to examine the reasons for this.  I read it to mean that
  you have not achieved predictability for what you read in the documentation,
  which is a sign that there is some underlying issue or concept that is still
  very unclear to you.  It is also very hard to help you with something you
  struggle with at too high a level, which is the impression I got from reading
  your list of experiments, as well.

  Perhaps it may be helpful to understand lexical vs special bindings by
  examining the situation before, during, and after each form of binding.
  First, let us take a lexical binding of the variable `foo´.  Suppose first
  that `foo´ is not bound.  The lexical binding introduces a new variable to
  the scope of the binding, which is not accessible by any means outside that
  scope, even if you have the name.  Because a lexical bindings is unique, a
  lexical binding may survive its scope by returning a closure from the scope.
  A lexical binding is therefore an object.  This will become important soon.
  Now, suppose instead that `foo´ was already bound when we entered this scope
  where a new variable with an existing name is introduced.  The name refers
  to the lexically enclosing scope, and there is no way to refer to any other
  scope.  One may therefore argue that the binding from the outer scope is
  shadowed, but it would be equally valid to argue that the object that is the
  binding has been pushed on a stack when entering the scope and gets popped
  off that stack when exiting the scope, making the name refer to the same
  binding it did before the intervening scope.  However, since lexical bindings
  are inaccessible from without, these binding objects may be compile-time
  constructs unless they are closed over by a closure that is passed to other
  functions or returned from its scope.  Typically, lexical bindings are held
  in registers and are shuttled back and forth between memory at will, using a
  stack or a predetermined slot in the stack frame and their nature as objects
  is only seen when understanding the true nature of closures.  An explanation
  of their implementation is outside the scope of this discussion, however.

  Let us now examine the special binding of the variable `foo´.  What makes a
  special binding special is that there is only binding object, so it cannot
  be captured by a closure.  A re-binding of `foo´ must explicitly save and
  restore the value of the binding because the single binding object cannot be
  replaced by a new object as it is accessible everywhere the name of the
  special variable is known.  The great value of special variables as global
  variables with clearly defined binding behavior instead of the usual setting
  behavior in inferior languages, is thus achieved by using the same syntax
  for all bindings to cause save and restore to become automatic when the
  variable is special and shadowing when the variable is "normal", but through
  exactly the same underlying principle of pushing and popping bindings.

  Now, what may seem strange, but really is very useful, the variable value of
  a symbol /is/ that single binding object for the special variable named by
  that symbol.  The reason this may confuse people is pretty obvious, though:
  There are several ways to reference the value of the single binding of a
  special variable.  A top-level `(setq foo bar)´ will behave like `(setf
  (symbol-value 'foo) bar)´ as this is indeed what the operation actually
  amounts to anywhere a special variable is referenced.  Therefore, if you do
  a top-level `setq´, you exploit a feature in many Common Lisp
  implementations where an unknown or "free" variable is assumed to be special
  without being pervasively declared to be so.  This can come in very handy,
  but it is also known to confuse people, so there should probably be a switch
  to signal an error in this case to help new users understand what they are
  doing.  In any case, we have a similar issue with functional arguments,
  where the single function-binding of a symbol is inferred when you name the
  symbol in a "function context" such as passing a symbol to `funcall´.

  A convenience to experienced programmers can become confusing for new
  programmers.  The ability to reference the single binding object of a special
  binding with `symbol-value´ should really be thought of as a low-level access
  mechanism that is useful when you wish to develop interpreters for your own
  languages.  Likewise, `boundp´ answers two questions depending on how much
  you know when you ask.  If the argument is a symbol you know names a special
  variable, you ask it whether the single binding it names has a value, i.e.,
  whether an enclosing binding exists for a variable that has no global value.
  If the argument is a symbol you know /should/ name a special variable that
  would always have a value if some file was loaded, such as by `defvar´, you
  know that that has not yet happened.  Other than this, I think programmers
  should stay away from `boundp´ and `makunbound´ unless they intend to
  communicate through the boundness of variables like one may communicate
  through the boundness of slots in CLOS instances.

  I find special variables and their binding behavior exceptionally intuitive,
  so I may have some problems explaining them to someone who has not yet seen
  what causes me to find it intuitive, but I believe that thinking of bindings
  as objects should be helpful in realizing the deeper nature of variables in
  Common Lisp.  Understanding variables in C/C++ requires a very different
  approach, but, these languages also have binding objects, with the difference
  that you can actually refer directly to the binding object through pointer
  that has the machine address of the binding, even when it is on the stack.
  Of course, this can be a major drag on the efficiency of code that must sync
  the value of the binding with the actual storage in memory if its address
  has been taken, and the `register´ keyword may be used to inform the
  compiler that the address of the binding will never be taken.  A Common Lisp
  program that uses closures will face similar syncronization needs for the
  bindings that are closed over when they may be referenced elsewhere at the
  same time, but will normally be able to avoid similar syncronization costs.

  I hope this has been helpful.

-- 
Erik Naggum, Oslo, Norway

Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.