Quote:
Originally posted by cd34
ia_archiver is alexa
a9.com by amazon uses alexa to calculate relevency to supplement its google backend results.
|
ia_archiver is actually the archiving robot from
www.archive.org. If you don't want your site archived there, leave this in or dissallow it in your robots.txt file.
Mr. H.