ever look at thunderstone's webinator? thats for installing on your server. but you can even have TS do the spidering for you from their service. In anycase, they have a nice collection of products and services.
http://www.thunderstone.com/texis/site/pages