From: Erik Naggum

Subject: Re: Need select equivalent

Date: 1999-5-31 14:53

* Jeff Dalton
| By "select", I mean the Unix operation for "synchronous I/O
| multiplexing".  I have some Common Lisp software that relies heavily on
| select in Another Common Lisp (GCL -- I provided the select myself in C),
| and I would very much like to port this to Allegro.

  unless it would be too much work to port/rework, I would highly recommend
  using the multiprocessing facilities in Allegro CL instead of doing your
  own select-and-dispatch.

  my current application ran into a lot of problems because I didn't even
  think twice about doing the dispatching myself.  it works, but it runs
  into deadlocks and annoying blocking states every once in a while, mostly
  because I have some fancy stream layers which report input available at
  the operating-system level, but causes READ-CHAR to consume the input
  before the real input is available, like a TELNET stream with TELNET
  options processed in STREAM-READ-CHAR, but as time went on and added more
  and more code to cope with this, I finally realized that the overhead of
  one connection = one process and the process scheduling for I/O were
  worth is, despite initial tests that showed this to be _way_ expensive.

| However, the on-line documentation for wait-for-input-available says that
| "have input available" means that a call to read-char or read-byte will
| immediately return without blocking, and I also need to wait for socket
| connections (at which point I would call accept, which I take it is in
| Allegro called accept-connection, rather than read-char or read-byte).
| (I want to wait for both sorts of readiness at once.)

  John added this at my request, and this is in my view the most useful in
  "user code" -- except I ran into a problem here as well: it is sometimes
  useful to have masters and slaves, where masters do the work and the
  slaves do the I/O with the external world, connected with "simpler"
  sockets than the full-fledged means that need to cater to all kinds of
  evil socket error conditions.  my system is designed to have one master
  and any number of slaves, but they have to be separate Unix processes if
  run on the same machine.  in my absence, one guy at operations started a
  slave in a master process, and it promptly got into a seriously wedged
  state: since the incoming connections were all handled by the single
  WAIT-FOR-INPUT-AVAILABLE handling all servers and dispatching to
  functions that might service the request or let a new process service the
  request (essentially via PROCESS-RUN-FUNCTION), it was no longer waiting
  for input if the request would "forward" a connection from a slave to the
  master.  this would not have been a problem if each socket in listen
  state had been a separate process or each request had always been handled
  by a new process.

| So my question is: can wait-for-input-available do this, and if not, what
| incredible contortions do I have to go through to get something like that
| to happen?

  the code that John added might be available as a patch if you ask for it
  (it's in 5.0.1.beta), but if you have a list of streams, extract the file
  descriptor from each with SOCKET-OS-FD, wait for those, and re-associate
  the file descriptors you get back with the stream:

(loop
  with fds = (mapcar #'socket-os-fd <list-of-streams>)
  for (fd) = (wait-for-input-available fds :whostate "connection-wait")
  for stream = (find fd <list-of-streams> :key #'socket-os-fd) do
    ...whatever...)

  this is more efficient than the built-in stuff, which more or less does
  (find-if #'stream-listen <list-of-stream>)  instead.

  also note that waiting for input on  (lognot fd)  waits for _output_ to
  become possible on that fd, instead.  yeah, I dabbled in that, too, for a
  while, before coming to my senses and I started to use one process per
  connection.

| Do I have to create separate threads?

  you don't have to, but my guess is you will benefit greatly from it, and
  unless you get the select-lookalike working in no time, 

| (If so, I probably will just give up on Allegro for this, because I don't
| have time to deal with threads right now.)

  unless you need a whole lot of process locks (WITH-PROCESS-LOCK is very
  cheap), this is no worse than calling PROCESS-RUN-FUNCTION on a function,
  and it will take care of itself.  amazingly convenient and a lot less
  work than to take care of implicit multithreading yourself.  the ability
  to use READ or just plain READ-LINE on a stream instead of carefully
  assembling an object in a stateless WAIT-FOR-INPUT-AVAILABLE loop might
  save you a lot of time.

  of course, you don't _have_ to, and the above code snippet is included to
  let you use WAIT-FOR-INPUT-AVAILABLE just like you use select today.

#:Erik