Subject: Re: MD5 in LISP and abstraction inversions
From: Erik Naggum <>
Date: Thu, 01 Nov 2001 13:27:56 GMT
Newsgroups: comp.lang.lisp
Message-ID: <>

* Lieven Marchand <>
| You've done the work on md5 and I haven't so this is guess work, but my
| first thought where the problem would lie is that a lot of these
| algorithms assume efficient 32 bit operations, which in CL will
| cons. Given sufficient declarations the character->(unsigned-byte 8)
| conversions with char-code should be negligable.

  MD5 seems to be designed with hardware implementations in mind.  This is
  not a bad idea, but it tends to make software implementations weirder.

  I have written an MD5 function in Allegro CL, using several low-level
  features (which are still at a higher level than C) and decided to split
  the 32-bit numbers in two 16-bit parts.  Emacs has done the same thing,
  but in a slightly different way.  I wanted to add support for a bitwise
  rotate function that would end up using such instructions if available,
  but it is probably easier to write MD5 in assembly on each platform than
  to optimize it better.  In the application I used this, MD5 hashes were a
  serious bottleneck, so it had to be better than using FFI to a C function.

  After having worked with the MD5 functions on and off for a month or so,
  I came to conclude that it _should_ be written in assembly, and started
  to do that, but it turned out to be extremely time-consuming work.  The
  Intel processors are shy too many registers, so all the fun in trying to
  make it superfast was replaced by increasing frustration over the design
  of those processors.  Still, initial estimates indicated that a hand-
  tuned assembly version would be about twice as fast as the code that gcc
  produced for the naive implementation found in the RFC, so it would be
  worth it at significant cost.

  I also think the new streams design from Franz Inc makes for a good way
  to deal with the 64-byte buffers.  It is wrong to try to work with MD5 at
  the character level, and the typical use of MD5 is to ensure that some
  data stream is intact.  If the application that consumes the stream can
  do both MD5 hashing and its real work at the same time, and can roll back
  the work if the MD5 hash turns out bad, much will be saved compared to
  making two passes, since we must assume that the MD5 hash is usually good.

  Norway is now run by a priest from the fundamentalist Christian People's
  Party, the fifth largest party representing one eighth of the electorate.
  Carrying a Swiss Army pocket knife in Oslo, Norway, is a criminal offense.