My web log files, with careful analysis, tell me what search terms brought people to our pages.
But what I would really like is to find commonly searched terms which should bring people to us, but do not ... so that I can optimise the site for them. And of course there's nothing in the log files to tell you what these underrated/unrated terms are - or so you might have thought.
With the combined log file format, the Referer string includes the query that the user made (encoded) and also
a parameter showing the start point in the results set. Without that parameter inthe referer, you are loking at the start of the result set ... with that parameter, you have a user who has moved on from the first page to a later page of results.
Here are a couple of analyses from yesterday:
4 arrival(s) to /resources/ex.php4?item=h110/mcheck.php
1 - php select radio
1 - php radio exemple
1 - php template for radio
1 - php select from (page starts 10)
That's three users who came to the page in question because their search presented our site on the first page, and one who searched for
"php selelct from" who wasn't satisfied with what (s)he found on the first page, and moved on to the second page.
6 arrival(s) to /resources/ex.php4?item=j907/index.jsp
2 - shopping cart jsp
1 - jsp source code for online shop (page starts 30)
1 - shopping cart example jsp
1 - download sample code for shopping cart in jsp
1 - shopping cart jsp example
And that's a page which our patient user, searching for
"jsp source code for online shop", didn't reach until the fourth page.
This page start information is a huge clue to lost traffic. With only 4% of our visits being as result of our links being found on 2nd or subsequent pages (see
here), we're been shown the tip of an iceberg - and we could do well to trawl through all our second (and later) page arrivals and see if we should be making more effective use of the words that were used in them.
If you're interested in what subjects users have found recently that are not on the first page, look here for
Python -
Perl -
PHP -
MySQL -
Tcl -
Ruby -
Lua -
Java -
Tomcat -
Apache -
Linux -
Shell -
Expect -
Database -
Unix and
other topics (written 2009-08-05, updated 2009-08-06)
Associated topics are indexed under
G911 - Well House Consultants - Search Engine Optimisation [2748] Monitoring the success and traffic of your web site - (2010-05-01)
[2686] Freedom of Information - consideration for web site designers - (2010-03-20)
[2562] Tuning the web site for sailing on through this year - (2010-01-03)
[2552] Web site traffic - real users, or just noise? - (2009-12-26)
[2428] Diluting History - (2009-09-27)
[2330] Update - Automatic feeds to Twitter - (2009-08-09)
[2137] Reaching the right people with your web site - (2009-04-23)
[2107] How to tweet automatically from a blog - (2009-03-28)
[2106] Learning to Twitter / what is Twitter? - (2009-03-28)
[2065] Static mirroring through HTTrack, wget and others - (2009-03-03)
[2045] Does robots.txt actually work? - (2009-02-16)
[2019] Baby Caleb and Fortune City in your web logs? - (2009-01-31)
[2000] 2000th article - Remember the background and basics - (2009-01-18)
[1984] Site24x7 prowls uninvited - (2009-01-10)
[1982] Cooking bodies and URLs - (2009-01-08)
[1971] Telling Google which country your business trades in - (2009-01-02)
[1969] Search Engines. Getting the right pages seen. - (2009-01-01)
[1793] Which country does a search engine think you are located in? - (2008-09-11)
[1344] Catching up on indexing our resources - (2007-09-10)
[1029] Our search engine placement is dropping. - (2007-01-11)
[1015] Search engine placement - long term strategy and success - (2006-12-30)
[427] The Melksham train - a button is pushed - (2005-08-28)
[165] Implementing an effective site search engine - (2005-01-01)
Some other Articles
Melksham - no trains, no southbound buses through the townPlanning!Learn a new programming language this summer.Apache, Tomcat, mod_proxyWhat search terms FAIL to bring visitors to our site, when they should?Java Collection Objects in the java.util packageLooking for a practical standards courseUploading and Downloading files - changing names (Perl and PHP)Helping new arrivals find out about source code examplesGraphics in Lua - an example using the gd library