Subject: source access vs dynamism
From: Erik Naggum <erik@naggum.no>
Date: 1999/08/23
Newsgroups: comp.lang.lisp
Message-ID: <3144404199547949@naggum.no>

  once upon a time, I was very much opposed to Free Software on a variety
  of political grounds.  then I thought that it could do a lot of good,
  partially motivated by seeing a lot of problems that came with lack of
  access to source code.  these days, I largely think both Free Software
  and Open Source are very serious mistakes, because they don't solve the
  core problems, they make them much, much worse.

  suppose one operating system (or language) lets you open an already open
  file by calling OPEN or whatever on the open stream.  since this is
  clearly beneficial to anyone who can do it, let's assume people start to
  do it and demand the same functionality elsewhere.  one easy way to do
  this is to use a function other than OPEN to do this, but this does not
  scale well: every request for a feature would need a new OPEN, and you
  would have to decide _which_ OPEN to call both in your code and in the
  implementation of the new functionality.  clearly, this is a very messy
  situation and migration from one system that did support a feature to one
  that did not would be a function of the number of features people might
  have added.  this problem would be solved if people had the source and
  could fix things themselves and then channel the fixes back to a central
  place that could "take over the world" with better versions.  consider
  the Unix way, which used to have the function call DUP, but recently has
  /dev/fd/<N> -- you can fake /dev/fd/<N> with DUP, but to do that requires
  getting under the skin in OPEN and parsing a file name and all sorts of
  cruft that the file system code would normally do.  ugly, but it would be
  necessary, right?

  the reason that source access is needed is the same as the cause of the
  problem: the inability to modify system behavior non-intrusively.  if one
  could modify OPEN on any given system to acquire new behavior _without_
  compiling a new OPEN or, even if you did, without re-linking the entire
  system (thus the need for dynamic linking and similar solutions, too),
  source code access would be entirely immaterial.

  so suppose you wanted to wanted to add DUP functionality when OPEN was
  called on /dev/fd/<N>.  you could grab the old OPEN definition and call
  it anonymously, while redefining OPEN to be your particular function,
  which acted only on a match to such pathnames, or you could redefine OPEN
  once and for all and add your stuff via a well-defined hook system, or
  advice or whatever.  whether you would try the new thing first or only
  after trying the argument the old way had failed would be an orthogonal
  design decision, subject to how easy error handling is and the expected
  frequency of failures.  this way of doing things would in any case cause
  a few simple improvements over static source: (1) the core system could
  upgrade its OPEN without the need to migrate the changes.  (2) if you did
  it right, you could also inspect, add, remove, and reorder your changes,
  and you wouldn't need the source for either of these operations, either.
  (3) bugs introduced by changes could be fixed the same way the changes
  were put in place, as opposed to rebuilding the system with the previous
  version of the dynamic library or whatever didn't fail.  (4) you could
  communicate your desire for a system-wide improvement to someone who
  could put it in the official sources and you could add your changes
  conditionally.

  compare this to a world where no dynamism is available and people don't
  think in dynamic terms, anyway.  you need to make OPEN change behavior so
  of course you need access to the source!  you want this functionality
  everywhere, not just in particular programs, so of course you need the
  ability to modify it and rebuild from scratch!  since this creates a
  maintenance nightmare, of course you want your changes to be part of the
  core system, too.  to get that to happen, of course you need to be part
  of a huge movement where people compete with eachother for features they
  think they need and the best way to implement them.  this is incredibly
  exhilarating, so of course this is the solution to all the world's many
  problems with missing features in software.

  suppose you drive down the highway and you suddenly want to go some nice
  place you saw just before you pass a forest.  you veer off the road,
  plunge into the wilderness and promptly decide that you need four-wheel
  drive, a huge cutting device in front of your car, much better shock
  absorbers, a bigger engine that could actually run on swamp water instead
  of getting all drowned, and then need an amphibious vehicle to get across
  the river on the far side of the forest.  you can't get all that right
  away after you made up your mind to veer off the road, and This Is Surely
  Somebody's Fault, so now you start this big protest movement to have auto
  manufacturers use Open Car Design by making your own cars and giving them
  away so people can modify them.  the obvious morale of this story is that
  it would have been a little smarter to see if there were another way to
  get there over existing roads or ferries or whatever, even if that would
  cost a lot less and be faster if not Your Way Right Now, and which didn't
  need all this add-on stuff to your car, and perhaps you would just get
  there instead of getting all worked up about all the neat stuff that
  other people had done to their cars while claiming that if it weren't for
  Open Car Design, they would actually never have gotten where they are,
  and they sure wouldn't have built this very versatile car, which they
  talk about a lot more than where they have used it and for what.  suppose
  someone adorns a commercial vehicle with a proprietary navigation system
  that let you discover any and all ways to get from point A to point B by
  talking to it, including scheduling and cost estimates so you could
  decide whether it was worth it, but refused to give it away to Open Car
  Designers.  why would anyone who weren't more interested in building cars
  than Getting There insist on home-made Swiss Army Cars with Open Car
  Design when the _real_ goal had become attainable?  why would Open Car
  Designers _not_ call the producer of the navigation system and find a way
  to deploy it almost everywhere?  and would they insist that a home-made
  navigation system would be better than buying it from whoever invented
  it?  although a ridiculous example, I don't see why anyone would insist
  on a way to do things that does not solves the real problem, but rather
  solve an ancilliary problem that is a problem only because someone is
  very stubborn.

  I keep wondering when and under which conditions the models employed by
  Free Software and Open Source make sense.  I don't think it makes sense
  when the design is _already_ flexible and open and so malleable that it's
  used as an argument against it by the very same people who want to change
  everything, but only Their Way.  in other words, if all you have is Unix,
  certain problems are so staggeringly tall that it takes an extra special
  genius to see what the _real_ problem is because all the people who are
  trying to climb it make it look as if doing something else is stupid.  if
  you have, say, Common Lisp and good access to the exposable parts of the
  internals, with advice support, hooks, etc, why do you _need_ the source
  code?  not that it wouldn't be useful, but would the demand be as great
  if the Smart Route to Yonder Place was at least available to intelligent
  people instead of requiring everybody to scale every mountain just to
  prove a point about inaccessibility?

  the problem is that scaling such mountains is much more fun than actually
  doing anything useful with computers.  mostly, computers are used to do
  Really Boring Stuff, especially if you don't have any dynamism, either,
  and if you're young and getting into computers because it is supposedly
  exciting, you discover that writing quality software for real people is
  tremendously hard and the real rewards are pretty far apart in real life,
  so it's a lot more fun to spend one's time cutting another cycle off an
  interrupt handler or bolt another semi-dynamic feature onto a system as
  flexible as a block of granite.

  the solution is dynamic _software_, but Open Source (etc) is the best way
  to ensure that this never happens on a large scale, because a pale shadow
  of the dynamism people actually need is immediately achievable using the
  static model that people accept without question through fixing a source
  file and recompiling.  ever better ways to overcome this inherent problem
  are created, such as Java and interpreted languages, dynamic linking and
  lots of nifty features.  to help people _find_ the solution, however, we
  need to show them that the problem is aggravated by the way they think
  they are solving the problem today: by making the static software model
  even more entrenched and unmaintainable.

  I think Free Software and Open Source addicts will get down from their
  current high and become tremendously frustrated when the obvious problem
  of having millions of pieces of source code available and they actually
  can't move in any direction because the ability to make modifications
  that they have so lauded in the past turn into a very serious problem
  when much bigger problems require modifications to much larger systems
  than the ones people today think are usefully modified locally.

  I predict that "migrability" will be the one factor that causes people to
  choose dynamic software over static software, however modifiable, because
  while the software has been the most valuable for a long while (because
  of its cost of development), the data we stuff into the programs and ask
  them to safekeep for us will be so much more valuable than the software.
  static software is totally unable to deal with graceful migration and the
  need for the cost of changes to be linear in the size of the changes.

  however, writing "migrable" software is not a task for novices or people
  who are not fully aware of the impact of their decisions, and I think
  that too much access to source code will cause a lot of people to be
  exceedingly good at stuff that is antithetical to long-term solutions.
  it seems that programmers need to be made as aware of the problem of not
  killing the patient while fixing some problem as the medical profession,
  because as long as the static software world considers "crash and reboot"
  to be an option, we won't get anywhere worth going.

  Free Software and Open Source may well be the last gasp of the static
  software world and its "success" may well cause companies who believe in
  static software to fail while users betray their own long-term needs in
  exchange for the smidgeon of dynamism that source code affords.  in the
  long run, software that adapts and can migrate from one half-expressed
  need to another as real users discover what they actually want, _should_
  win.  resisting the sometimes attractive non-solution of unrestricted
  source access and full modifiability may be the only way to get what we
  really need.

#:Erik
-- 
  save the children: just say NO to sex with pro-lifers