CGI pages in Google Serps
 

I gotta site going back a few years now which has a few hundred cgi generated pages.

These pages were not created for Google to spider, but all of a sudden I'm getting hits for them and they're showing up in the serps.

I was considering changing them all to html pages in order to get Google to find them, but it seems that Google upgraded before I did.

Are cgi pages a new thing with Google? And is it a bad thing that the site (in Google's eyes) has more than doubled in the serps overnight?

Maybe I should insert a robots.txt ?

Re: CGI pages in Google Serps
 

No, let them alone. :)

I've never used a robots.txt but if you are getting good SERPs then I would just leave well enough alone.

Thanks Surfn & Cleo :)

It's just quite strange that Google never found these pages before... they've been there since 2001 without any SE traffic whatsoever.

My link checker spiders cgi pages and I always wondered why Googlebot would not.

No dont change them

No they are not a new thing - spidering and listing of them is usually dependent on PR and incoming PR and the no. of queries in the url string

No its never a bad thing to get more results

Why? put a robots txt in? do you want them not spidered?

DD