My web log files, with careful analysis, tell me what search terms brought people to our pages. But what I would really like is to find commonly searched terms which should bring people to us, but do not ... so that I can optimise the site for them
. And of course there's nothing in the log files to tell you what these underrated/unrated terms are - or so you might have thought.
With the combined log file format, the Referer string includes the query that the user made (encoded) and also a parameter showing the start point in the results set
. Without that parameter inthe referer, you are loking at the start of the result set ... with that parameter, you have a user who has moved on from the first page to a later page of results.
Here are a couple of analyses from yesterday:
4 arrival(s) to /resources/ex.php4?item=h110/mcheck.php
1 - php select radio
1 - php radio exemple
1 - php template for radio
1 - php select from (page starts 10)
That's three users who came to the page in question because their search presented our site on the first page, and one who searched for "php selelct from"
who wasn't satisfied with what (s)he found on the first page, and moved on to the second page.
6 arrival(s) to /resources/ex.php4?item=j907/index.jsp
2 - shopping cart jsp
1 - jsp source code for online shop (page starts 30)
1 - shopping cart example jsp
1 - download sample code for shopping cart in jsp
1 - shopping cart jsp example
And that's a page which our patient user, searching for "jsp source code for online shop"
, didn't reach until the fourth page.
This page start information is a huge clue to lost traffic. With only 4% of our visits being as result of our links being found on 2nd or subsequent pages (see here
), we're been shown the tip of an iceberg - and we could do well to trawl through all our second (and later) page arrivals and see if we should be making more effective use of the words that were used in them.
If you're interested in what subjects users have found recently that are not on the first page, look here for Python
and other topics (written 2009-08-05, updated 2009-08-06)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articlesG911 - Well House Consultants - Search Engine Optimisation 
Implementing an effective site search engine - (2005-01-01) 
The Melksham train - a button is pushed - (2005-08-28) 
Search engine placement - long term strategy and success - (2006-12-30) 
Our search engine placement is dropping. - (2007-01-11) 
Catching up on indexing our resources - (2007-09-10) 
Which country does a search engine think you are located in? - (2008-09-11) 
Search Engines. Getting the right pages seen. - (2009-01-01) 
Telling Google which country your business trades in - (2009-01-02) 
Cooking bodies and URLs - (2009-01-08) 
Site24x7 prowls uninvited - (2009-01-10) 
2000th article - Remember the background and basics - (2009-01-18) 
Baby Caleb and Fortune City in your web logs? - (2009-01-31) 
Does robots.txt actually work? - (2009-02-16) 
Static mirroring through HTTrack, wget and others - (2009-03-03) 
Learning to Twitter / what is Twitter? - (2009-03-28) 
How to tweet automatically from a blog - (2009-03-28) 
Reaching the right people with your web site - (2009-04-23) 
Update - Automatic feeds to Twitter - (2009-08-09) 
Diluting History - (2009-09-27) 
Web site traffic - real users, or just noise? - (2009-12-26) 
Tuning the web site for sailing on through this year - (2010-01-03) 
Freedom of Information - consideration for web site designers - (2010-03-20) 
Monitoring the success and traffic of your web site - (2010-05-01) 
Reading Google Analytics results, based on the relative populations of countries - (2012-03-24) 
Google Analytics and the new UK Cookie law - (2012-06-02) 
Has your Twitter feed stopped working? Switching to their new API - (2013-06-23)
Some other Articles
Melksham - no trains, no southbound buses through the townPlanning!Learn a new programming language this summer.Apache, Tomcat, mod_proxyWhat search terms FAIL to bring visitors to our site, when they should?Java Collection Objects in the java.util packageLooking for a practical standards courseUploading and Downloading files - changing names (Perl and PHP)Helping new arrivals find out about source code examplesGraphics in Lua - an example using the gd library