BBN of Cambridge, Massachusetts was a venerable high-tech institution, pioneers in acoustics and electronics, at the heart of the development of both sonar and the internet. Their supercomputing venture was a failed sideline, the demise of which was not fatal to the parent. Nevertheless, the company no longer exists independently, having been bought out by GTE.
The following is pieced together from anecdotes and various references on the web and in the literature. Any BBN project veterans are kindly requested to write up their recollections and send them to kevink@acm.org.
The BBN Butterfly was so named for the "butterfly" multi-stage switching network around which it was built. Up to 512 CPUs, each with local memory, could be connected to allow every CPU access to every other CPUs memory, albeit with a substantially greater latency (roughly 15:1) than for its own. The CPUs were commodity microprocessors. The earlier, GP-1000 models used Motorola 68020's and scaled to 256 CPUs. The later, TC-2000 models used Motorola 88100's, and scaled to 512 CPUs.
The Butterfly did not employ particularly exotic technology in electronics or packaging. This reduced the development and manufacturing costs, but may have contributed to reports of the system running hot and having reliability issues.
The Butterfly began life, poetically enough, under a proprietary OS called Chrysalis, but moved to a MACH-based operating system in 1989. While the memory access time was distinctly non-uniform, the machine still had SMP memory semantics, and could be operated as a symmetric multiprocessor.
TotalView[tm], the parallel program debugger developed for the Butterfly outlived the platform, to be ported to a number of other massively parallel machines.
These machines were physically large and very expensive for the performance they delivered. The commodity microprocessors of the period did not support multiple outstanding memory requests, so while it was technically possible to run SMP parallel programs on the machine, the remote memory access penalty killed performance unless good data distribution was achieved. And if one has solved the data distribution problem, why pay for global memory access when a message-passing machine will deliver similar parallelism at a lower cost. Case in point: Oracle did some of their early massively-parallel database server experiments on the TC-2000, but abandoned it in favor of the Ncube-2.
A spiffy interconnect by itself will not really allow the construction of a supercomputer out of common household appliances.
Does anyone out there have one?
Mark Riordan's reminiscences on the GP-1000.
"Subgraph Isomorphism on the BBN Butterfly Multiprocessor", by J. Costanzo, L. Crowl, L. Sanchis, and M. Srinivas, University of Rochester Department of Computer Science project report, October 1986. (gzipped PostScript file)
BBN Butterfly in the Wikipedia