[cd34]

grub.org used to do something similar.

There are quite a few distributed search spider programs -- almost like a peer to peer search engine that crawls the web and submits indexes to some master site. Looksmart sponsored grub and then shut it down a while back.

I would bet this is probably a spider related to Peter Nordin's work. (His name came up in a google search and he has quite a few peer reviewed distributed projects under his belt)