CGI pages in Google Serps

urb · 2004-07-28, 12:43 PM

I gotta site going back a few years now which has a few hundred cgi generated pages.

These pages were not created for Google to spider, but all of a sudden I'm getting hits for them and they're showing up in the serps.

I was considering changing them all to html pages in order to get Google to find them, but it seems that Google upgraded before I did.

Are cgi pages a new thing with Google? And is it a bad thing that the site (in Google's eyes) has more than doubled in the serps overnight?

Maybe I should insert a robots.txt ?

Surfn · 2004-07-28, 01:01 PM

Quote:

Originally posted by urb
Maybe I should insert a robots.txt ?

No, let them alone.

Cleo · 2004-07-28, 01:04 PM

I've never used a robots.txt but if you are getting good SERPs then I would just leave well enough alone.

urb · 2004-07-28, 01:16 PM

Thanks Surfn & Cleo

It's just quite strange that Google never found these pages before... they've been there since 2001 without any SE traffic whatsoever.

My link checker spiders cgi pages and I always wondered why Googlebot would not.

DangerDave · 2004-07-28, 04:53 PM

No dont change them

No they are not a new thing - spidering and listing of them is usually dependent on PR and incoming PR and the no. of queries in the url string

No its never a bad thing to get more results

Why? put a robots txt in? do you want them not spidered?

DD

2004-07-28, 12:43 PM	#1
urb All the way from Room 101 Join Date: Aug 2003 Posts: 3,557	CGI pages in Google Serps I gotta site going back a few years now which has a few hundred cgi generated pages. These pages were not created for Google to spider, but all of a sudden I'm getting hits for them and they're showing up in the serps. I was considering changing them all to html pages in order to get Google to find them, but it seems that Google upgraded before I did. Are cgi pages a new thing with Google? And is it a bad thing that the site (in Google's eyes) has more than doubled in the serps overnight? Maybe I should insert a robots.txt ? __________________

2004-07-28, 01:04 PM	#3
Cleo Subversive filth of the hedonistic decadent West Join Date: Mar 2003 Location: Southeast Florida Posts: 27,936	I've never used a robots.txt but if you are getting good SERPs then I would just leave well enough alone. __________________ Free Rides on Uber and Lyft Uber Car: uberTzTerri Lyft Car: TZ896289

2004-07-28, 01:16 PM	#4
urb All the way from Room 101 Join Date: Aug 2003 Posts: 3,557	Thanks Surfn & Cleo It's just quite strange that Google never found these pages before... they've been there since 2001 without any SE traffic whatsoever. My link checker spiders cgi pages and I always wondered why Googlebot would not. __________________

2004-07-28, 04:53 PM	#5
DangerDave Bonged Join Date: Mar 2003 Location: BrisVegas, AUSTRALIA Posts: 4,882	No dont change them No they are not a new thing - spidering and listing of them is usually dependent on PR and incoming PR and the no. of queries in the url string No its never a bad thing to get more results Why? put a robots txt in? do you want them not spidered? DD __________________ Old Dollars >>>> Now with over 90 Hosted Free Sites <<<< DangerDave.com.au - Adult Links to Free Porn

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: