From ... From: Erik Naggum Subject: Re: Benchmark Results for CL's onWin32 Date: 1999/12/02 Message-ID: <3153166270117604@naggum.no>#1/1 X-Deja-AN: 555984313 References: <823vj1$ska$1@nnrp1.deja.com> <01bf3d04$ad901e10$35ee58c0@cutler> mail-copies-to: never X-Complaints-To: newsmaster@eunet.no X-Trace: oslo-nntp.eunet.no 944177470 5504 195.0.192.66 (2 Dec 1999 23:31:10 GMT) Organization: Naggum Software; +47 8800 8879 or +1 510 435 8604; fax: +47 2210 9077; http://www.naggum.no NNTP-Posting-Date: 2 Dec 1999 23:31:10 GMT Newsgroups: comp.lang.lisp * "Martin Simmons" | For best performance, you should also turn off all debugging support. all of these benchmarks seem to be testing fairly unusual conditions. the arguments we see here for "tuning" seem to me to imply that we have to chuck all the Lispy features we can think of to get a comparison. I think this is completely bogus. I'm not going to trust _myself_ enough to run my code with the kinds of optimization settings people advocate for these benchmarks. I _want_ a lot of dynamism, debugging support, etc, and I'm not usually ready to run my code in production without these things, except in particular functions that I have determined to need it, and which have been specially written to be blindingly fast. however, in the latter case, I'd be pretty damn stupid if I thought I could have 100% portable ANSI Common Lisp code _and_ super-duper maximal performance at the same time. _sheesh_, guys. if I had such needs as you imply with these declarations for "best performance", I would even hack the compiler output and look for instruction scheduling opportunities myself. using a special function to turn off debugging, for instance, is just _one_ such measure to squeeze the bejeezus out of the system, and we're already outside of the portable ANSI Common Lisp domain. so in my view, you either want super-fast code and profile it and get the optimal results with hand-tweaked code that may not work anywhere else, or you use fully portable code under _normal_ condtions. anything else is very close to cheating, rendering the benchmark results evidence of how good you are at cheating at random in each implementation with no way of comparing your skills and then _nothing_ else. if you let instead consciously and openly do your extreme best to write fast code for each implementation, and _then_ compare the results, you at the very least get comparable results for comparable _people_ (i.e., experts). I suggest such benchmarks limit themselves to the narrowest possible type declarations, the highest SPEED and the lowest DEBUG and SAFETY, and either full local tweaking or no local tweaking. if a declaration isn't enough, that's a factor that you need to take into account when and if you engage in _full_ local tweaking, and not at all until then. #:Erik