<RobertMaas@YahooGroups.Com> wrote:
+---------------
| {{Date: Wed, 13 Aug 2003 07:43:50 -0500
| From: rpw3@rpw3.org (Rob Warnock)
| then I found one, and then a second, small consulting task that could
| easily be done in a Lisp-based persistent application server ...
| between a web server and a database}}
|
| Do I understand correctly? The actual Web server-application clones a
| new copy (with most pages shared) for each HTTP connection...
+---------------
Actually, Apache [at least, and probably other current production web
servers] *doesn't* work quite that way. First, it pre-forks a bunch
of server instances, depending on recent average load [this number may
be adjusted up or down over time, as needed]. Second, it re-uses those
processes over & over for successive HTTP requests. [E.g., on my own
machine the set of "httpd" process IDs hasn't changed since *May*!]
+---------------
| ...but there's another program that just sits there interfacing the
| cloned server-applications with the database?
+---------------
Sort of. There's another (Lisp) program that just sits there and *is*
"the application", and contains all the "business logic" for handling
each HTTP request, the SQL access code (client side), and the result
HTML generation code. As far as the web server (Apache, say) is
concerned, it's just another "CGI script", except it's magically
persistent. [Some people call this "middleware".]
+---------------
| If so, what advantages do you have this way, compared to each
| cloned server-application working the database directly?
+---------------
*Not* cloning a whole copy of the application per request, mainly!!
That is, *much* less overhead per request, if you're comparing it
to classic fork/exec per HTTP request "CGI".
But if you're comparing it to things that put the application language
*into* the web server, such as mod_perl or mod_php or mod_python, well,
the advantage is that you don't have to shoehorn all of Common Lisp
into Apache (or AOLserver or IIS or whatever), and that you don't have
as many copies of the Lisp image as you have web server processes.
If the application/middleware server is in Common Lisp and is persistent
(long-lived), you can talk to it through a REPL, see what it's doing,
load patches, fix bugs, all without taking it down. [And you don't have
to fork/exec an entire Lisp image for each request!]
See <URL:http://www.cliki.net/Web> for a lot more on this general
approach, especicially these items:
Araneida A small Lisp-based server designed to sit behind
a caching proxy such as Apache. Used to implement
CLiki itself.
mod_lisp An small Apache module that connects to a
persistent Lisp server via sockets. (Or any
*other* language, actually. People have used it
with Python.)
cl-modlisp A useful library for the Lisp server side of mod_lisp.
[And HTOUT & CL-Who & many others, for the HTML generation from Lisp.
But that's another topic altogether...]
Now for various off-topic reasons, I'm not actually using mod_lisp
per se at the moment, but I'm doing something very similar: There's
a *tiny* C trampoline that runs as a classic CGI program and connects
to the Lisp application server over a Unix-domain socket, and otherwise
does essentially the same thing that mod_lisp would have done.
[Yes, it still fork/execs the trampoline once per request. But on
my 1.4GHz Athlon, "apachebench" says that a CGI + Lisp-server-computed
page only takes ~12ms (assuming no SQL accesses), which is more than
fast enough for the current applications, especially since that time
is swamped by the database overheads. Plus, I can always move to
mod_lisp later, if necessry.]
+---------------
| Does the persistent interface application do a better job of queuing
| cases when several cloned server applications try to get into the
| database at about the same time?
+---------------
I actually have no idea, since I *DON'T* have "several cloned server
applications...at about the same time" -- one application server.
At worst there are multiple HTTP requests at about the same time,
each of which results in a CMUCL MAKE-PROCESS (green thread creation).
[Though see CL-Modlisp, above, which can run either that way or with
a pre-created CMUCL thread pool.]
+---------------
| Or is opening and closing access to the database really expensive
| so you want to do it just once in a while instead of for each HTTP
| transaction?
+---------------
It *would* be, if the transaction rates were a lot higher. In that case
SQL connection pooling would become quite important. But for the current
(and anticipated for quite some time) loads, opening an SQL connection
per HTTP request is acceptable. [Heretical, perhaps, but "good enough".]
However, I was careful to write the server with hooks for SQL connection
pooling, so if it's needed later...
+---------------
| {{any little weird, off-the-wall, non-standard thingy somebody needs
| done quickly... and that *can't* be done with off-the-shelf software
| [at least, not easily].}}
|
| How do people like you and I, having skills to hack together new tools
| and applications, and potential customers, who are bothered by tricky
| problems they can't easily do themselves, get matched up together to
| find each other, so they can get the task done and we can get paid for
| doing it?
+---------------
Dunno. I'm still working on that one. Both of my current clients
were people I already knew. [And see next answer below.]
+---------------
| I posted my ad to ba.jobs.resumes a couple times in 2002...
+---------------
You need to buy/read a 2003 edition of "What Color Is My Parachute".
Seriously. No kidding. The best $25(?) you could spend right now.
There are some very interesting statistics in there on what works
for finding a job and what doesn't. Posting a resume on the Internet
is all the way at the *BOTTOM* of the list. [Less than 4% of people
who do that find a job that way.] Much, much better is "working your
network" -- personal contacts with former employers & co-workers,
friends, family, old school chums, people you hike/bike/camp/play
with, people you work with on charities, political parties, etc.
+---------------
| {{Yes, I'm quite familiar with and take encouragement from these:
| <URL:http://www.paulgraham.com/road.html>}}
|
| Yes, I came to the same conclusion three years ago, that Server-based
| software was the way to go... Also, I can write most of my
| software in LISP, and none of the users ever need care about that.
...
| Yeah, that's my market too. I wish I could find customers.
+---------------
1. See above comments re "jobs". Same thing. It's all word-of-mouth.
2. Go to ILC. Get known in person. [Netnews personas are often different.]
3. Come to Bay Area Lispniks meetings. Peter Seibel <peter@javamonkey.com>
is putting together the next one as we speak, probably at the end of
August/beginning of September.
4. Just *do* it. That is, go ahead and start coding whatever infrastructure
you're going to want to use; put together a web site that uses it;
talk it up with people you meet [see #1]; and even consider doing
(small amounts of) real work using it for free [but ask for "resume
credits"], just to show people you can.
-Rob
-----
Rob Warnock, PP-ASEL-IA <rpw3@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607