Script To Spider My Server?

[3xTom]

You guys are close to what im looking for i think cleo is closer but im not wanting to spider a site or a database...

cleo do you know if that script will pull the html pages they arent in a database they are just on my server.....

I want to spider my entire server.... there are like 30 domain names and because of years of building and submitting gallerys there are literally millions of gallerys on my server in each of the domains so i want to spider my server for all the webpages that are on it then if that isnt enough...lol i want it to place everything into a nice format so that 1 i can see what all i have out there and 2 i could even take that one page that has all my galleries in it and submit it to search engines etc etc....