About the robots.txt file

LD · 2008-03-15, 05:13 PM

Thanks Halfdeck, good info if I can digest it..

The issue I am having is that my main index page is being indexed and crawled fine ("domain/index.php" ), but I'm having problems with my cat pages "domain/linklist/oral.php" and so forth. I submitted a site map, and G tells me my pages are indexed, yet when I type the url of most of my catetory pages in the G search box, I get nothing. They do show up in Yahoo and msn, and probably others...but no G.

Perhaps I should just wait a while..? I mean after they are indexed, should I allow for a certain amount of time before they show up in the search? They were not even indexed before I submitted the site map, and that was only a week or so ago.

The interesting thing about all this is my shaved pussy cat DOES show up, yet it gets very little traffic and I have no trades there,,,(?)

2008-03-15, 05:13 PM	#7
LD wtfwjd? Join Date: May 2007 Posts: 2,103	Thanks Halfdeck, good info if I can digest it.. The issue I am having is that my main index page is being indexed and crawled fine ("domain/index.php" ), but I'm having problems with my cat pages "domain/linklist/oral.php" and so forth. I submitted a site map, and G tells me my pages are indexed, yet when I type the url of most of my catetory pages in the G search box, I get nothing. They do show up in Yahoo and msn, and probably others...but no G. Perhaps I should just wait a while..? I mean after they are indexed, should I allow for a certain amount of time before they show up in the search? They were not even indexed before I submitted the site map, and that was only a week or so ago. The interesting thing about all this is my shaved pussy cat DOES show up, yet it gets very little traffic and I have no trades there,,,(?) __________________ Artisteer Wordpress Theme Generator Create Custom Themes! My Little Network Last edited by LD; 2008-03-15 at 05:15 PM..