Quote:
Originally Posted by Useless Warrior
Ok, you convinced me. I'll give it a shot. I just edited my robots.txt to disallow both the detail and tag pages.
|
Well...apparently you don't have to keep the door open very long before the hungry vermin find their way in. Ever since I put the disallow back in place for my tag pages, Googlebot and Yahoo! Slurp have been munching almost exclusively on my tag pages. That disallow was missing less than 24 hours. Go figure.
I can't promise traffic to my submitters
, but I can guarantee that I will direct the bots in your general direction (from many, many pages)
.