 |
|
|
|
|
|
|
|
|
|
|||||||
|
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
2007-11-07, 10:57 PM
|
#1 |
|
Internet! Is that thing still around?
Join Date: Sep 2006
Posts: 9
|
robots.txt vs. meta name="ROBOTS" content="ALL, INDEX, FOLLOW
I have seen many sites with either one or the other.. some with both. I am curious as to which is better and if the sites with no robot.txt is doing fine why is it needed(other than to prohibit bots from indexing certain areas). And if the site with meta robot text is doin fine why the robot txt?
|
|
|
|
|
Threaded Mode