Subject: Re: beginner question on input from large files
From: Erik Naggum <erik@naggum.no>
Date: 2000/02/10
Newsgroups: comp.lang.lisp
Message-ID: <3159210673593055@naggum.no>

* Joseph Dale <jdale@uclink4.berkeley.edu>
| I'm trying to write a function that takes the name of a dictionary file
| containing one word per line and returns the contents of that file as a
| list of strings.

  first off, it's a reasonable thing to do.

| The latest of several versions is this one (I know it's not very elegant):

  au contraire, it's _too_ elegant.

| (defun get-words (filename)
|   (let ((*words* '()))
|     (defun read-loop (stream)
|       (setf *words* (cons (read-line stream nil 0) *words*))
|       (if (numberp (first *words*))
| 	  (rest *words*)
| 	(read-loop stream)))
|     (with-open-file (stream filename :direction :input)
| 		    (read-loop stream))))

  whether this is compiled or not, this is needlessly complex and "elegant"
  only in the Scheme sense, which is not elegant at all in Common Lisp.

(defun stream-lines (input-stream)
  (do ((lines nil)
       (line #1=(read-line input-stream nil :eof) #1#))
      ((eq line :eof) (nreverse lines))
    (push line lines)))

  if you're not as silly as to have an allergic reaction to LOOP, this is
  even better:

(defun stream-lines (input-stream)
  (loop for line = (read-line input-stream nil :eof)
	until (eq line :eof)
	collect line))

  using a number for magic purposes and then testing for any number is
  _really_ bad style.  where did you pick this up?  a Perl class?

| The function works with files containing less than around 10,000 words,
| but no matter what I do, whenever I call this function with a file
| containing more words, I get a stack overflow or a core dump. How can I
| do this with larger files? I feel like there must be a better way.

  recursion has its uses.  using recursion for iteration is not one of
  them.  Scheme educators got that one _seriously_ confused, and your code
  is simply such Scheme code in Common Lisp.  this is _not_ a reasonable
  thing to do, not even in Scheme.

  my suggestions:

1 get rid of the internal define.
2 use iteration forms when that's what you do.
3 use recursion for inherently recursive tasks, only.
4 learn Common Lisp if you want to think in Common Lisp.
5 use Scheme if you think in Scheme.

  writing Scheme code in Common Lisp is also really bad for your karma.

#:Erik