| |||||||||||
search engines and question marks Posted by wilko (wilko), 27 November 2003 I have built a web site using Perl that is not being indexed by Google and I understand this is because it doesn't like my use of variables in URLs.Before I look into rewrite engines and modifying my scripts I thought I'd share this with you to see if you have any pointers. A typical url from my site: http://www.trintech.com/index.cgi?n0=NAE&n1=213122241451005836515 I guess I would be looking to rewrite this to something like: http://www.trintech.com/NAE/213122241451005836515.html Any tips welcome. Posted by Custard (Custard), 27 November 2003 Hi, not sure I'm going to be much use here, but..How does the search engine find the url you have listed.. Ie. is there a HREF= link to that precise url. as far as I understand, the crawler will look at links with HREFs and index them. If the url is the result of a form, then I would imagine it had little chance of getting there. Also if the link is embedded in javascript it may not find it. I think google etc will follow links into CGI scripts though. I have seen quite a lot indexed, but where the real page has now disappeared because it was dynamicaly generated at the time of the crawl. HTH B Posted by admin (Graham Ellis), 27 November 2003 My understanding is that Google will visit pages with Get information in the URL (i.e. with a ?), but won't then follow any further links from those pages. This, apparently, is to avoid recursive indexing where a whole lot of URLs all really refer to the same page. Certainly there are may pages on www.wellho.net that are indexed including a get string and bring us many visitors.Posted by wilko (wilko), 2 December 2003 The urls are all created on the fly but do remain consistent between visits so can be book-marked and SEs won't have broken links. One point raised that I have been choosing to ignore is dynamically generated links in menus. I am going to have ensure that the site can be fully navigated with JavaScript off, through use of a site map, just for the benefit of the crawler.Then there is the issue of getting good rankings. It's a minefield and I guess this is why we have consultants who specialise in the area. Thanks for your input. This page is a thread posted to the opentalk forum
at www.opentalk.org.uk and
archived here for reference. To jump to the archive index please
follow this link.
|
| ||||||||||
PH: 01144 1225 708225 • FAX: 01144 1225 793803 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho |