From ...
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news.tele.dk!small.news.tele.dk!193.213.112.26!newsfeed1.ulv.nextra.no!nextra.com!uio.no!nntp.uio.no!ifi.uio.no!not-for-mail
From: Erik Naggum <erik@naggum.no>
Newsgroups: comp.lang.lisp
Subject: Re: enumerating constants
Date: 14 Oct 2002 03:55:32 +0000
Organization: Naggum Software, Oslo, Norway
Lines: 40
Message-ID: <3243556532240005@naggum.no>
References: <87bs5ye3ba.fsf@bob.intern>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Trace: maud.ifi.uio.no 1034567732 6301 129.240.65.5 (14 Oct 2002 03:55:32 GMT)
X-Complaints-To: abuse@ifi.uio.no
NNTP-Posting-Date: 14 Oct 2002 03:55:32 GMT
Mail-Copies-To: never
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2
Xref: archiver1.google.com comp.lang.lisp:43838

* Tim Daly, Jr.
| I'm going through Holub's book on compilers, and working out the code in
| lisp.  Listing 1.1 starts out

  I am not familiar with the book, but the "usual" way to return tokens
  from the tokenizer, which I presume this is all about, is to fake dynamic
  types pretty badly.  E.g., your tokenizer would return (the integer) SEMI
  and possibly some other value or write something into a pointer you
  passed the tokenizer.  The caller would dispatch upon the "type" of the
  returned value to do something interesting with the token.

  In Common Lisp, we have real dynamic types and you may return a real
  object from the tokenizer.  Normally, you would return an instance of any
  of the token classes.  In the case of trivial things like symbols, return
  the symbol.  That is, return the symbol for + and ; and whatever else
  your tokenizer has produced.  Call `intern´ on the string you collect
  from the input source in the tokenization phase in the appropriate
  package.  (If your vocabulary of symbols is restricted, you may want to
  use `find-symbol´ and signal or handle a syntax error at this point.)

  Note that you are reinventing some of the fundamentals of Common Lisp in
  this part of your project.  Common Lisp already has the function `read´
  that tokenizes and returns Common Lisp objects.  You may want to study
  how it works before you try to do something like it.  Lots of data types
  are available in Common Lisp that should be usable directly, such as
  symbol names, integers, floating-point numbers, etc.  Much of the work
  you are about to do, if my guess about the purpose of your question is
  correct, will be to build the lexer/tokenizer/reader functions for your
  new language.  If you really want to do this in Common Lisp instead of C,
  you should investigate the higher-level purpose of the C code.  You may
  be very surprised at how much smaller the Common Lisp code is compared to
  the C code.

  Somebody should have written a book on compiler design in Common Lisp.

-- 
Erik Naggum, Oslo, Norway

Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.