Why wont Google crawl my damn site?

b1ng0 · 2005-06-11, 04:14 PM

Yahoo seems to like my site just fine but Google wont seem to crawl it for anything. Even though I've put a few inbound links on other domains/pages that Google occasionally visits, and the site is updated pretty regularly. I do use .htaccess to make the page appear static, which means there's a lot of 302's, especially to exit to sponsors.. could this be turning Google off to my site? Should I exclude the exit 302's via robots.txt? The site is only a 5-6 weeks old but still.. Yahoo seems to like the site pretty well, it just doesn't index the links that redirect off site, obviously. Any suggestions?

Bill · 2005-06-11, 07:39 PM

Add a batch of plain html pages with unique text content and photos. Interlink these with your existing pages with natural and interesting text links. Get some new exterior links coming to these new pages.

And wait a bit more. 6 weeks ain't that long for google, if you don't have good strong incoming links.

graphicsbytia · 2005-06-11, 07:55 PM

yes.. I have a suggestion, post the url here that'll get it spidered

MadMax · 2005-06-11, 09:48 PM

Quote:

Originally Posted by graphicsbytia

yes.. I have a suggestion, post the url here that'll get it spidered

Very true, G shows several inbound links for my sites from natural board postings.

SEO_Konsul · 2005-06-12, 03:12 AM

so what's the url of your site?

b1ng0 · 2005-06-12, 02:16 PM

http://www.e-hotties.com is the url. Looks like googlebot's been back 2 days in a row so maybe were getting somewhere.

MadMax · 2005-06-12, 02:19 PM

Make sure you've got a robots.txt file in your main directory as well. I've noticed that when I slap up a site quickly and don't put that up right away Gbot is slower to take notice. Don't know if that's by design or coincidence, but it's still a good idea to have a robots.txt on every site

b1ng0 · 2005-06-12, 09:59 PM

Thanks for all the suggestions so far, I'll implement them and see where that gets me.

neveremail · 2005-06-13, 05:13 AM

If you get the time make a site map. Google looks for a site map and it should help raise your internal page rank and using the right linking text will help raise you keyword ranking.

Read http://www.google.co.uk/intl/en/webm...uidelines.html

May give you some ideas.

Also note google always seems slower to pick up on things than yahoo. Personally i found yahoo indexes new pages quicker, indexes page changes quicker, my rankings for keywords respond quicker. My rankings shot up on yahoo a month ago, only this weekend did the same thing happen on google.

Linkster · 2005-06-14, 10:43 PM

noticed you put a disallow on a directory called /exit/ but didnt see that directory on your site?

b1ng0 · 2005-06-15, 12:14 AM

Well, the exit links ( sponsor links ) go through this non-existent directory but an htaccess rule rewrites them to a script anyway so... I have it as exit but really it could be anything. I just chose exit so that those who know that they can look at the status bar to see where a link is going, realize that they will be leaving the site ( some of the links are a tad blind ). I'm also working under the assumption that search engines see my fake static pages and not the scripts they 301 or 302 to, or at least thats what they index.. otherwise whats the point. Basically I don't want them to try'n index pages that are nothing but redirects and flag my site as being shady or whatever. Do you mean to say that not having an exit directory defeats the purpose of having the robots.txt? I've never even attempted seo, and haven't done a whole lot of reading, so I'm kind of learning as I go along..

Linkster · 2005-06-15, 06:40 AM

didnt mean to confuse you on the exit directory in the robots - I just thought it was unusual to see a virtual directory listed in there - but definitely if you have rewrites/virtual links being created that is the way to go

I noticed that your description tag in the head section of the site could probably use some work - make it something that will reach out and grab the surfers once you do get listed - since Google will normally use what is in the description tag if the dont find a good desc on the page - also Yahoo will use both the desc and keywords tag so make sure those are good keywords for you (although they seem to be whats on the page)

I guess you know that your 2257 link doesnt work - once it does you might want to have that in a directory that also doesnt get allowed in the robots.txt file

2005-06-11, 04:14 PM	#1
b1ng0 I'm going to the backseat of my car with the woman I love, and I won't be back for TEN MINUTES Join Date: Jan 2005 Posts: 85	Why wont Google crawl my damn site? Yahoo seems to like my site just fine but Google wont seem to crawl it for anything. Even though I've put a few inbound links on other domains/pages that Google occasionally visits, and the site is updated pretty regularly. I do use .htaccess to make the page appear static, which means there's a lot of 302's, especially to exit to sponsors.. could this be turning Google off to my site? Should I exclude the exit 302's via robots.txt? The site is only a 5-6 weeks old but still.. Yahoo seems to like the site pretty well, it just doesn't index the links that redirect off site, obviously. Any suggestions?

2005-06-12, 03:12 AM	#5
SEO_Konsul Lord help me, I'm just not that bright Join Date: Dec 2004 Location: Au Posts: 107	so what's the url of your site? __________________ Affordable SEO, Icq 335 997 125

2005-06-13, 05:13 AM	#9
neveremail Oh no, I'm sweating like Roger Ebert Join Date: Apr 2005 Posts: 504	If you get the time make a site map. Google looks for a site map and it should help raise your internal page rank and using the right linking text will help raise you keyword ranking. Read http://www.google.co.uk/intl/en/webm...uidelines.html May give you some ideas. Also note google always seems slower to pick up on things than yahoo. Personally i found yahoo indexes new pages quicker, indexes page changes quicker, my rankings for keywords respond quicker. My rankings shot up on yahoo a month ago, only this weekend did the same thing happen on google. __________________ Please Re-Read The Rules For Sig Files

2005-06-14, 10:43 PM	#10
Linkster NO! Im not a female - but being a dragon, I do eat them. Join Date: Mar 2003 Location: Sex Delta Posts: 5,084	noticed you put a disallow on a directory called /exit/ but didnt see that directory on your site? __________________ Pussy Chompers Porn Links NSCash

2005-06-15, 06:40 AM	#12
Linkster NO! Im not a female - but being a dragon, I do eat them. Join Date: Mar 2003 Location: Sex Delta Posts: 5,084	didnt mean to confuse you on the exit directory in the robots - I just thought it was unusual to see a virtual directory listed in there - but definitely if you have rewrites/virtual links being created that is the way to go I noticed that your description tag in the head section of the site could probably use some work - make it something that will reach out and grab the surfers once you do get listed - since Google will normally use what is in the description tag if the dont find a good desc on the page - also Yahoo will use both the desc and keywords tag so make sure those are good keywords for you (although they seem to be whats on the page) I guess you know that your 2257 link doesnt work - once it does you might want to have that in a directory that also doesnt get allowed in the robots.txt file __________________ Pussy Chompers Porn Links NSCash

2005-06-11, 07:39 PM	#2
Bill Selling porn allows me to stay in a constant state of Bliss - ain't that a trip! Join Date: Apr 2003 Posts: 3,914	Add a batch of plain html pages with unique text content and photos. Interlink these with your existing pages with natural and interesting text links. Get some new exterior links coming to these new pages. And wait a bit more. 6 weeks ain't that long for google, if you don't have good strong incoming links.

2005-06-11, 07:55 PM	#3
graphicsbytia Banned Join Date: Oct 2003 Posts: 204	yes.. I have a suggestion, post the url here that'll get it spidered

2005-06-12, 02:16 PM	#6
b1ng0 I'm going to the backseat of my car with the woman I love, and I won't be back for TEN MINUTES Join Date: Jan 2005 Posts: 85	http://www.e-hotties.com is the url. Looks like googlebot's been back 2 days in a row so maybe were getting somewhere.

2005-06-12, 02:19 PM	#7
MadMax "Without evil there can be no good, so it must be good to be evil sometimes" ~ Satan Join Date: Aug 2004 Location: Motor City, baby, where carjacking was invented! Now GIMME THOSE SHOES! Posts: 2,385	Make sure you've got a robots.txt file in your main directory as well. I've noticed that when I slap up a site quickly and don't put that up right away Gbot is slower to take notice. Don't know if that's by design or coincidence, but it's still a good idea to have a robots.txt on every site

2005-06-12, 09:59 PM	#8
b1ng0 I'm going to the backseat of my car with the woman I love, and I won't be back for TEN MINUTES Join Date: Jan 2005 Posts: 85	Thanks for all the suggestions so far, I'll implement them and see where that gets me.

2005-06-15, 12:14 AM	#11
b1ng0 I'm going to the backseat of my car with the woman I love, and I won't be back for TEN MINUTES Join Date: Jan 2005 Posts: 85	Well, the exit links ( sponsor links ) go through this non-existent directory but an htaccess rule rewrites them to a script anyway so... I have it as exit but really it could be anything. I just chose exit so that those who know that they can look at the status bar to see where a link is going, realize that they will be leaving the site ( some of the links are a tad blind ). I'm also working under the assumption that search engines see my fake static pages and not the scripts they 301 or 302 to, or at least thats what they index.. otherwise whats the point. Basically I don't want them to try'n index pages that are nothing but redirects and flag my site as being shady or whatever. Do you mean to say that not having an exit directory defeats the purpose of having the robots.txt? I've never even attempted seo, and haven't done a whole lot of reading, so I'm kind of learning as I go along..