Friday, April 08, 2005

Software for Brains: Metcalfe's Law is Alive and Well

Minor update: April 12/05

The famous "Metcalfe's Law" states the value of any communications network doubles with every new node that is added. So, why doesn't the Internet suddenly double in value every time a new server goes online?

A recent post on Slashdot March of '05 raised the question with the usual geek-speak and this heading:

"Cnet News reports on a powerful refutation of Metcalfe's Law (that the value of a network goes up with n^2 in the number of members). The academic PDF is available at Southwest Missouri State University. Basically, the thesis is that not all the links in a network are equally valuable, so Metcalfe's argument that everyone can connect to everyone (n(n-1)/2 links, roughly n^2) is irrelevant. The authors propose nlog(n) instead, a much smaller increase."

The mistake is to assume that the usage we presently draw from networks is a complete measure of it's value.

The Untapped Power of Networks


The hallmark feature of a network is that a new "whole" is created along with the emergence of the network. It is, at the same time, both a honey producing hive, New York or Einstein's brain and a group of bees, people and neurons. Today's computer networks are still quite dumb waiting for software brains.

SETI@home is an example of how software can extract latent network power. Simply using the idle time from a network of hundreds of thousands of volunteer computers, well over a million years of otherwise wasted computation time has been harnessed from this virtual supercomputer. It earned the Guinness World Record for the largest computation ever done on earth.

Grid computing is a related and emerging form of distributed high performance networking which shares unused clock cycles, memory, hard drive space and bandwidth.

Ever reallocating unused network resources as they become available and better used elsewhere it is essentially supercomputing on demand. Well designed software would allow a grid to be self-healing, automatically directing the combined computational powers to deal with network attacks.

Imagine a Seti-like application stitching together the computational force of all connected home desktops in North America. Like a global magnifying glass, it could focus awesome computational intelligence onto a single problem. A fraction of a second would provide more digital power than all of todays computers seperately muster in a year. It would be vastly superior than any singular supercomputer, and owned by everyone.

Emergent Intelligence from Networks


Networks have the potential for emergent behavior, including intelligence. Consider a network of dumb ants which give rise to the spectacular collective intelligence which choreographs the design, engineering and production of colonies. Some types of ants actually farm. Computer networks are capable of vastly more powerful emergent intelligence.

We're presently accelerating towards a global computing grid which will extract unimaginable power from hundreds of thousands of separate computers each with the processing capabilities of our brain. The collective intelligence which emerges will possibly rival our fantasies of artificial intelligence.

Metcalfe's Law is quite alive. We're just getting started.