Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
For 2021 - online Python 3 training - see ((here)).

Our plans were to retire in summer 2020 and see the world, but Coronavirus has lead us into a lot of lockdown programming in Python 3 and PHP 7.
We can now offer tailored online training - small groups, real tutors - works really well for groups of 4 to 14 delegates. Anywhere in the world; course language English.

Please ask about private 'maintenance' training for Python 2, Tcl, Perl, PHP, Lua, etc.
What proportion of our web traffic is robots?

We welcome search engines to our site - to index our content and point their visitors back to us where appropriate, but such search engines are a means to an end and not an end in themselves. How much traffic to our web site is true visitor traffic, and how much is automata? An interesting pointer is a graph of log file size for the last four weeks - you'll see it at the top of this page. We noticed a distinct weekly "cyclic" flow in this graph (it will automatically update as this blog archives, so you may not see it if you come back here in 2008 or later!) and as automata run 24 x 7 for the most part, we're pretty sure that the peaks and troughs are caused by real visitors.

Our Current Visitors page allows us to take a snapshot of the HTML and PHP pages that use our standard template that have been called up, and from where, in the last fifteen minutes. The database records which browser was in use, the referring page and the country of origin and makes a fascinating read.


Our Most Popular Pages display looks back at yesterday's log files and tells us where people have visited - and a bias towards a particular page will tend to indicate heave real traffic as automata tend to spider evenly.

I started with the question "What proportion of our web traffic is robots? when perhaps I should have asked "What proportion of our web traffic is generated by real human beings browsing at the time?". From the current peaks and troughs on the graph, and the other evidence, I'm guestimating that the figure is somewhere between 40% and 65%, with a very high proportion of that being business rather than home users.

Update - December 2009 The weekly cycle continues ... there's an update on this story [here] although little has changed except the total traffic volume.
(written 2007-06-19, updated 2010-01-05)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
G998 - Well House Consultants - Newsletter Highlighted Box
  [1011] Well House Manor and Beechfield House, Hotels, Melksham - (2006-12-29)
  [1068] ls -l report, Linux / Unix - types and permssions - (2007-02-06)
  [1132] Well House Manor, Melksham, Art Gallery - (2007-04-02)
  [1307] Troy, up state New York - (2007-08-17)
  [1375] Python v Ruby - (2007-10-02)
  [1475] Tcl/Tk - updating your display while tasks are running - (2007-12-16)
  [1521] Evening drive across the roof of Wiltshire - (2008-01-27)
  [1621] Linux and Java Course in London - (2008-04-24)
  [1629] A short introduction to our courses - (2008-05-03)
  [1751] Public Training Course Dates until July 2009 - (2008-08-13)
  [1843] How many cups of coffee? - (2008-10-17)
  [1967] LinkedIn - Thrice Asked, and joined. - (2008-12-30)
  [2050] Why the Pony Tail? - (2009-02-21)
  [2125] We have lost a regular business guest - (2009-04-10)
  [2222] A (biased?) comparison of PHP courses in the UK - (2009-06-07)
  [2385] Reading all our recent news from a single source - (2009-08-29)
  [2400] Are you wanting to learn PHP? - (2009-09-08)
  [2520] Global and Enable - two misused words! - (2009-11-30)
  [3168] Web Sites - Subject to Advertising Standards from 1st March - check your sites - (2011-02-13)

G902 - Well House Consultants - Web site techniques, utility and visibility
  [23] Skills and responsibilities - (2004-08-22)
  [32] Web design platoon - (2004-08-29)
  [98] No more 'Error 404' pages. Something better. - (2004-10-24)
  [109] URLs - a service and not a hurdle - (2004-11-04)
  [117] A case of case - (2004-11-14)
  [142] Colour for access - (2004-12-06)
  [165] Implementing an effective site search engine - (2005-01-01)
  [173] Data Mining - (2005-01-09)
  [179] The hunt for unique words - (2005-01-16)
  [182] Your personal Google ranking - (2005-01-19)
  [197] Allow for peak traffic on your web site - (2005-02-01)
  [202] Searching for numbers - (2005-02-04)
  [222] Who are all these visitors? - (2005-02-20)
  [259] Responding to spam - (2005-03-27)
  [261] Putting a form online - (2005-03-29)
  [268] Information request forms, cleaning up spam - (2005-04-05)
  [274] Our most popular resources - (2005-04-10)
  [276] An apology to Mr Boneparte - (2005-04-11)
  [278] Cover all the options - (2005-04-13)
  [284] The Iconish language - (2005-04-19)
  [288] Colour blindness for web developers - (2005-04-22)
  [311] Growth pains - (2005-05-14)
  [314] What language is this written in? - (2005-05-17)
  [320] Ordnance Survey - using a 'Get a map' - (2005-05-22)
  [322] More maps - (2005-05-23)
  [347] Frightening and from-friend viruses and spams - (2005-06-14)
  [348] Graveyard pages - (2005-06-15)
  [369] CMS - the minefield of Choices - (2005-07-05)
  [376] What brings people to my web site? - (2005-07-13)
  [414] Form Madness - (2005-08-14)
  [492] New Navigation Aid - Launch of My Wellho - (2005-11-11)
  [510] Dynamic Web presence - next generation web site - (2005-11-29)
  [528] Getting favicon to work - avoiding common pitfalls - (2005-12-14)
  [533] Bigger Box Campaign - (2005-12-18)
  [649] Denial of Service ''attack'' - (2006-03-17)
  [658] Keeping the visitors happy and browsing - (2006-03-26)
  [681] Mirroring a dynamic site - (2006-04-12)
  [718] Protecting images from theft - (2006-05-12)
  [732] Where is a web site visitor browsing from - (2006-05-24)
  [757] Horse and Python training - (2006-06-12)
  [767] Finding the language preference of a web site visitor - (2006-06-18)
  [800] Effective web campaign? - (2006-07-12)
  [893] Visibility - (2006-10-14)
  [916] Driving customers away - (2006-11-07)
  [976] Santa at the station - (2006-12-09)
  [994] Training on Cascading Style Sheets - (2006-12-17)
  [1015] Search engine placement - long term strategy and success - (2006-12-30)
  [1029] Our search engine placement is dropping. - (2007-01-11)
  [1055] Above the fold - (2007-01-28)
  [1104] Drawing dynamic graphs in PHP - (2007-03-09)
  [1177] Sorting out for a site map - (2007-05-05)
  [1184] Finding resources - some pointers - (2007-05-13)
  [1186] Two new pages / sites - (2007-05-14)
  [1198] From Web to Web 2 - (2007-05-21)
  [1207] Simple but effective use of mod_rewrite (Apache httpd) - (2007-05-27)
  [1212] What brought YOU to our web site? - (2007-06-01)
  [1297] Stuffing content into a web page - easy maintainance - (2007-08-09)
  [1437] Above the fold with First Great Western - (2007-11-19)
  [1494] A time to update pictures - (2008-01-03)
  [1505] Script to present commonly used images - PHP - (2008-01-13)
  [1506] Ongoing Image Copyright Issues, PHP and MySQL solutions - (2008-01-14)
  [1513] Perl, PHP or Python? No - Perl AND PHP AND Python! - (2008-01-20)
  [1534] Where in the world / country is my visitor from? - (2008-02-07)
  [1541] Colour, Composition or Content - (2008-02-16)
  [1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
  [1610] PHP course dot co, dot uk - (2008-04-13)
  [1630] To provide external links, or not? - (2008-05-04)
  [1634] Kiss and Book - (2008-05-07)
  [1653] How do Google Ads work? - (2008-05-25)
  [1711] Rapid growth leads to server move - (2008-07-17)
  [1747] Who is watching you? - (2008-08-10)
  [1756] Ever had One of THOSE mornings? - (2008-08-16)
  [1793] Which country does a search engine think you are located in? - (2008-09-11)
  [1797] I have been working hard but I do not expect you noticed - (2008-09-14)
  [1833] Web Bloopers - good form design - avoiding pitfalls - (2008-10-11)
  [1856] A few of my favourite things - (2008-10-26)
  [1888] Find the link - (2008-11-16)
  [1955] How to avoid duplicating web page maintainance - (2008-12-20)
  [1961] Making our things easier to find - (2008-12-26)
  [1970] Plagarism - who is copying my pages? - (2009-01-02)
  [1982] Cooking bodies and URLs - (2009-01-08)
  [2056] Web Site Loading - experiences and some solutions shared - (2009-02-26)
  [2065] Static mirroring through HTTrack, wget and others - (2009-03-03)
  [2225] How important is a front page ranking on a search engine? - (2009-06-09)
  [2332] Formation, des langages Open Source - (2009-08-09)
  [2333] Formaci[83][c2]ón, de los lenguajes de c[83][c2]ódigo abierto - (2009-08-09)
  [2334] Formazione, Open Source computer lingue - (2009-08-09)
  [2335] Ausbildung, die Open-Source-Sprachen - (2009-08-09)
  [2336] Forma[83][c2]ç[83][c2]ão, Open Source computador l[83][c2]ínguas - (2009-08-09)
  [2337] Opleiding, Open Source computertalen - (2009-08-09)
  [2338] Uddannelse, Open Source computer sprog - (2009-08-09)
  [2339] Oppl[83][c2]æring, Open Source datamaskinen spr[83][c2]åk - (2009-08-09)
  [2340] ldning, Open Source dator spr[83][c2]åk - (2009-08-09)
  [2341] Koulutus, Open Source tietokone kielill[83][c2]ä - (2009-08-09)
  [2389] Writing with our customers words - (2009-09-01)
  [2410] Removal of technical resources from this site - (2009-09-19)
  [2519] Status Page / breaks of service in early December - (2009-11-30)
  [2532] Analysing Google arrivals by country of origin - (2009-12-10)
  [2552] Web site traffic - real users, or just noise? - (2009-12-26)
  [2569] How to run a successful online poll / petition / survey / consultation - (2010-01-10)
  [2668] Is it worth it? - (2010-03-09)
  [2981] How to set up short and meaningfull alternative URLs - (2010-10-02)
  [3022] Retaining web site visitors - reducing the one page wonders - (2010-10-31)
  [3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
  [3149] Looking back at www.wellho.net - (2011-01-28)
  [3197] Finding and diverting image requests from rogue domains - (2011-03-08)
  [3367] Google +1 - what is it? - (2011-07-22)
  [3426] Automed web site testing scripted in Ruby using watir-webdriver - (2011-09-09)
  [3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
  [3532] Sharing the user experience - designing a form with the customer in mind - (2011-11-29)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3563] How big is a web page these days? Does the size of your pages matter? - (2011-12-26)
  [3589] Promoting a single one of your domains on the search engines - (2012-01-22)
  [3623] Some TestWise examples - helping use Ruby code to check your web site operation - (2012-02-24)
  [3734] QR codes with marketing logos embedded - (2012-05-16)
  [3744] Short Web Addresses for Melksham - (2012-05-30)
  [3745] Legal change - You need to obtain user consent if you use cookies on your website - (2012-06-01)
  [3776] Some traps it's so easy to fall into in designing your web site - (2012-06-23)
  [3896] An email marathon - (2012-10-15)
  [3974] TV show appearance - how does it effect your web site? - (2013-01-13)
  [4001] Helping search engines with appropriate 400 error codes - (2013-02-11)
  [4076] Web site - fully back! - (2013-04-29)
  [4115] More or less back - what happened to our server the other day - (2013-06-14)
  [4136] How do I post automatically from a PHP script to my Twitter account? - (2013-07-10)
  [4239] Facebook marketing - early experiences - (2014-01-19)
  [4376] Well House Consultants, Well House Manor, First Great Western Coffee shop, TransWilts / 2014 web site reports - (2015-01-01)
  [4401] Selecting RECENT and POPULAR news and trends for your web site users - (2015-01-19)
  [4474] Effect on external factors on traffic to our web sites - an update - (2015-04-26)
  [4492] Almost so wrong, but perhaps it's right for some? - (2015-05-11)

A606 - Web Application Deployment - Apache httpd - log files and log tools
  [1503] Web page (http) error status 405 - (2008-01-12)
  [1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
  [1656] Be careful of misreading server statistics - (2008-05-28)
  [1761] Logging Cookies with the Apache httpd web server - (2008-08-20)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [1796] libwww-perl and Indy Library in your server logs? - (2008-09-13)
  [3015] Logging the performance of the Apache httpd web server - (2010-10-25)
  [3019] Apache httpd Server Status - monitoring your server - (2010-10-28)
  [3027] Server logs - drawing a graph of gathered data - (2010-11-03)
  [3443] Getting more log information from the Apache http web server - (2011-09-16)
  [3447] Needle in a haystack - finding the web server overload - (2011-09-18)
  [3670] Reading Google Analytics results, based on the relative populations of countries - (2012-03-24)
  [3984] 20 minutes in to our 15 minutes of fame - (2013-01-20)
  [4307] Identifying and clearing denial of service attacks on your Apache server - (2014-09-27)
  [4404] Which (virtual) host was visited? Tuning Apache log files, and Python analysis - (2015-01-23)
  [4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)


Back to
Trying things in Python
Previous and next
or
Horse's mouth home
Forward to
The kind spirit of Melksham
Some other Articles
If I say 'I am fine', what do I mean?
Fancy going to Glastonbury?
End of File on a Java BufferedReader
The kind spirit of Melksham
What proportion of our web traffic is robots?
Trying things in Python
Outputting numbers as words - MySQL with Perl or PHP
A review of a week and a trip to Brugge
Horses of Brugge
Bathtub example
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2021: 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/1237_Wha ... bots-.html • PAGE BUILT: Sun Oct 11 16:07:41 2020 • BUILD SYSTEM: JelliaJamb