Subject: Re: how to tokenize a string in Lisp
From: rpw3@rpw3.org (Rob Warnock)
Date: Tue, 03 Feb 2009 22:46:26 -0600
Newsgroups: comp.lang.lisp
Message-ID: <uJKdnXw5dfk_gBTUnZ2dnUVZ_juWnZ2d@speakeasy.net>
cartercc  <cartercc@gmail.com> wrote:
+---------------
| (2) I don't have cl-ppcre. If I have trouble getting it, I'll
| certainly ask for help.
+---------------

See <http://www.cliki.net/CL-PPCRE>.

Also look at the following CL functions, which you *do* already have:

    POSITION
    SEARCH
    MISMATCH
    PARSE-INTEGER
    SUBSEQ
    REPLACE
    CONCATENATE

To get the most from these, you will need to read & understand
the sections in the CLHS about "bounding index designators"
and the :START and :END [and sometimes :START2 and :END2]
keyword arguments which nearly all sequence functions take.
Also learn about the :KEY and :TEST keyword arguments, again,
which nearly all sequence functions take. [Oh, and :FROM-END, too.]

Additional hints:

- Coming from a C or Perl world, you may find the following bits
  of syntactic sugar helpful:

    (defun strcat (&rest strings)
      (apply #'concatenate 'string strings))

    (define-compiler-macro strcat (&rest strings)
      `(concatenate 'string ,@strings))

    (defun join (delimiter &rest strings)
      (apply #'concatenate 'string
	     (if (zerop (length delimiter)) ; If explicit "" or NIL.
	       strings                      ;  do short-circuit optimization.
	       (loop for s on strings       ; Long way.
		 collect (car s)
		 when (cdr s)
		   collect delimiter))))

- MISMATCH is one of more underappreciated string-bashing functions
  in CL, since it actually tells you how much *was* matched.  ;-}
  Very useful [especially with the :START2/:END2 options] to
  tell whether a (possibly-abbreviated) fixed substring exists
  at some specific location in a string, *without* having to do
  a SUBSEQ first to extract the portion to be tested. [Avoids
  unnecessary consing.]


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607