Script To Spider My Server?

[Bunnyhop]

The problem with doing searches directly on the server is the script will only be able to find the index pages relative to the server structure...it won't know the domain setup. Of course one could add to the script an ability to parse your web server settings to try to translate all that.

If your structure is relatively simple though you could get around it using Cleo's suggestion

find / -name "index.html" -print

or

find / -name "index.html" -print > filenames.txt

to have the results put into a file you can download.

Then use a global search/replace on the file on your favorite word processor changing the server relative directories to the url equivalent...example if you had

/home/websites/foobar.com/site1/index.html
/home/websites/foobar.com/site2/index.html

Replace /home/websites/foobar.com/
with
http://foobar.com/

to become

http://foobar.com/site1/index.html
http://foobar.com/site2/index.html