Google Bot Info request

[Mattinblack]

Originally Posted by Allfetish

This is what I have heard and experienced as well. Some people say the sandbox is about 6-9 months however.

My Guru who gets paid megabucks (not by me!) for seo told me that for a new site OR new index page in a subdirectory or a new subdomain (Tip dont submit index pages in sub-directories to google! Have the index pages in case people nav to the directory by accident but never link to them from anywhere and dissalow them in Robots Txt.)

Within 1-2 Weeks - crawled complete site, indexed and cached top level only. You will be able to see site name, and description snippet if you search for the URL.

From here on site gets crawled every 7-14 days to see how often/whether its updated. Until:

Depending on demand/supply for site theme as assessed by googlebot.
6wks -12wks site fully indexed and incoming/outgoing links checked - note not ranked! Will appear in searches for minority keywords ("suckmyass" and "pornogirl" etc) when search engine gets desperate.
Certain subjects such as Medical, Computer Security and Scientific sites seem to be immune to this.

From here-on an aging/updating algorithm seems to be applied. Updating frequently seems to reduce the waiting time till the algorithm evaporates as does your site being repeatedly found by surfers
on the SE.

Time to full ranking 5-9 months UNLESS your page is in exceptionally high demand ie you ARE Brittany Spears or Harry Potter. OR possibly if your main keeywords are so obscure google only has a few sites on those searches.

Yeah it sucks. Plus I did a google search on one of my more critical keywords (ukescort) on google yesterday and found so many obvious spammers added since xmas that I reckon its not working! It was so bad I spent about a half hour going through the first six pages and emailed google with a long list of obviously auto generated spamsites.

For an Old Site then it will get re-indexed as often as Google has observed it changing. If you want to force a re-index then if you have an XML sitemap on Google Sitemaps resubmitting with the last changed date equal to the day of submission seems to get you re-indexed within 24 hours so long as you already have a sitemap with Google. Otherwise manually resubmitting your site works within one to two weeks.

[Allfetish]

VERY nice info! Thank you for sharing those details, I'm learning.

Originally Posted by Mattinblack

Time to full ranking 5-9 months UNLESS your page is in exceptionally high demand ie you ARE Brittany Spears or Harry Potter. OR possibly if your main keeywords are so obscure google only has a few sites on those searches.

Sounds like our sources mostly are in agreeance then on that.

Homage to the Google gods- order finally within the seeming neverending chaos!

[Halfdeck]

Originally Posted by Mattinblack

(Tip dont submit index pages in sub-directories to google! Have the index pages in case people nav to the directory by accident but never link to them from anywhere and dissalow them in Robots Txt.)

True, linking to index.htm/html can cause a havoc.

Note: Don't rely solely on robots.txt since Google likes to ignore it in some cases.

[Allfetish]

Originally Posted by Halfdeck

True, linking to index.htm/html can cause a havoc.

Note: Don't rely solely on robots.txt since Google likes to ignore it in some cases.

What is the reason for this? Possibly the rewriting of the url in regards to duplication? e.g.

www.somedomain.com/dir/
www.somedomain.com/dir/index.html

This is interesting if I read it right as I know many linkslists prefer an index.html reciprical link, so as to help somewhat with discouraging doorway submissions.

[DJilla]

Originally Posted by Allfetish

This is interesting if I read it right as I know many linkslists prefer an index.html reciprical link, so as to help somewhat with discouraging doorway submissions.

Yeah! This was what I was thinking too!

[Mattinblack]

It still is fine to use index pages in subdirectories and subdomains its just that Google seems to see these as being new sites for the purposes of its aging algorithm. Just means they will now take longer to appear in Google search results. If they are OLD pages that have been there a while and Google knows about them then no issues... except see below.

A radical change in page content has the same effect. I had a PR5 non-adult adwords site which despite the traffic was not making me too much money. In November I gave the site a major makeover, kept the keys the same, the links the same, the page scanned the same for keyword density and relevance however the graphics and written content did totally change on each page.

Uploaded the site and three days later saw it go from PR6 and in the top 5 searches to unlisted! At first I thought I had been banned but typing in the URL brought up the site and description and a cached page. Gradually over the last few weeks I have seen a trickle of SE traffic from Google. This compares to quite a flood from MSN since I changed it.

This is what caused me to consult my Guru (see previous post). He said I should have changed my sub pages at the rate of one a week and my index page a little bit at a time over the same sort of period. He said that an index page missing on two consecutive googlebot runs could reset aging as well, but I have no experience of that.

[Halfdeck]

Originally Posted by Mattinblack

It still is fine to use index pages in subdirectories

This is a personal call, but keep in mind that linking to "www.domain.com/index.html" and "www.domain.com/" will usually result in Google indexing both urls, in some cases causing one or the other to go supplemental. This is similar to the www/non-www supplemental issue resulting in dupes. In the worst case scenario, the duplicates may lower your ranking in the SERPs; in the best case scenario they merely make a mess out of your Google listing by inflating (doubling) the number of pages listed in Google. As you know, supplementals can be a bitch to get rid of --- a good enough reason for me to take extra measures to make sure Google listings link to me using / and not /index.html.

One way to avoid the problem of course is if you link to "www.domain.com/" and never to "www.domain.com/index.html". You can also 404 /index.html, or use a base ahref= tag.

For free sites with multiple doorways, index.html, index.htm, warning.htm will probably be seen as unique pages (due to unique recip tables, provided you link to enough LLs), but say if you submit a teen-slut/index.html url and link from your hub to that page using /teen-slut/ or if your malicious competitor links to you using both urls, then you're going to end up with a supplemental listing in Google.

[DJilla]

Originally Posted by Halfdeck

As you know, supplementals can be a bitch to get rid of --- a good enough reason for me to take extra measures to make sure Google listings link to me using / and not /index.html.

One way to avoid the problem of course is if you link to "www.domain.com/" and never to "www.domain.com/index.html". You can also 404 /index.html, or use a base ahref= tag.

Ya this part just caught up with me a few months ago, now I'm really careful about that too!

BTW.. Lot of board chatter re: "BIG DADDY" new algo for google rankings messing stuff up again.

Basic info: http://www.mattcutts.com/blog/

[DJilla]

Originally Posted by Mattinblack

new index page in a subdirectory or a new subdomain (Tip dont submit index pages in sub-directories to google! Have the index pages in case people nav to the directory by accident but never link to them from anywhere and dissalow them in Robots Txt.)

OOOOppps! Didn't know, never heard ths one! Any more info on how google sees index pages in subdirectories. Why they have a problem with this since it is a well know method to keep surfers where they belong?