Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
Python, Lua, Tcl, C and C++ training - public course schedule [here]
Private courses on your site - see [here]
Please ask about maintenance training for Perl, PHP, Java, Ruby, MySQL and Linux / Tomcat systems
 
Which (virtual) host was visited? Tuning Apache log files, and Python analysis

We host a number of domains on our main server, and in order to avoid fragmentation of log files, we keep a sinle composite log. Rather than use a standard logfile format henceforth, I've changed the second field to carry the virtual host name accessed for the request, as that was missing up until this morning.

So in my server config
  LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
has become
  LogFormat "%h %v %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

Gone - Remote logname (from identd, if supplied).
Added - The canonical ServerName of the server serving the request.

I've written a program (in Python) to take a look at the log file - see [here] - and run from the command line that gives:
  -bash-4.1$ /home/wellho/trainee/y202/pytop
   13358 - www.wellho.net
    2719 - www.firstgreatwestern.info
     223 - www.melkshamchamber.org.uk
     100 - melksh.am
      62 - www.twcrp.org.uk
      39 - www.savethetrain.org.uk
      39 - www.across-the-pond.co.uk
      33 - www.wellhousemanor.co.uk
      30 - twhc.org.uk
      16 - transwilts.org.uk
       5 - thebutlerdidit.info
       1 - railcustomer.info
  -bash-4.1$


The program's also got a web wrapper - if called up on the web, it uses a different formatter:
  output = '{0:6d} - {1:s}'
  try:
    web = sys.argv[1] == "-w"
    if web: output = '<tr><td>{0:d}</td><td><a href="http://{1:s}" target="avh">{1:s}</a></td></tr>'
  except:
    pass

and later in my code:
  print output.format(counter[site],site)

And you can see the current results [here].

P.S. There's another quick demo web analysis program (showing its age) [here].
(written 2015-01-23)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y202 - Python on the Web
  [4536] Json load from URL, recursive display, Python 3.4 - (2015-10-14)
  [4089] Quick and easy - showing Python data hander output via a browser - (2013-05-15)
  [2365] Counting Words in Python via the web - (2009-08-18)
  [2238] Handling nasty characters - Perl, PHP, Python, Tcl, Lua - (2009-06-14)
  [1745] Moodle, Drupal, Django (and Rails) - (2008-08-08)
  [903] Pieces of Python - (2006-10-23)
  [433] FTP - how to make the right transfers - (2005-09-01)
  [426] Robust checking of data entered by users - (2005-08-27)
  [237] Crossfertilisation, PHP to Python - (2005-03-06)

A604 - Web Application Deployment - Apache httpd - virtual hosts
  [3568] Telling which ServerAlias your visitor used - useful during merging domains - (2012-01-04)
  [3132] Virtual Hosting with Apache http server - an overall scheme, and avoiding common pitfalls - (2011-01-14)
  [1896] Virtual Hosts and Virtual Servers - (2008-11-20)
  [1687] Virtual Hosting on Apache httpd - a primer example - (2008-06-25)
  [690] Adding Virtual Hosts - (2006-04-19)

A606 - Web Application Deployment - Apache httpd - log files and log tools
  [4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)
  [4307] Identifying and clearing denial of service attacks on your Apache server - (2014-09-27)
  [3984] 20 minutes in to our 15 minutes of fame - (2013-01-20)
  [3974] TV show appearance - how does it effect your web site? - (2013-01-13)
  [3670] Reading Google Analytics results, based on the relative populations of countries - (2012-03-24)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
  [3447] Needle in a haystack - finding the web server overload - (2011-09-18)
  [3443] Getting more log information from the Apache http web server - (2011-09-16)
  [3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
  [3027] Server logs - drawing a graph of gathered data - (2010-11-03)
  [3019] Apache httpd Server Status - monitoring your server - (2010-10-28)
  [3015] Logging the performance of the Apache httpd web server - (2010-10-25)
  [1796] libwww-perl and Indy Library in your server logs? - (2008-09-13)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [1761] Logging Cookies with the Apache httpd web server - (2008-08-20)
  [1656] Be careful of misreading server statistics - (2008-05-28)
  [1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
  [1503] Web page (http) error status 405 - (2008-01-12)
  [1237] What proportion of our web traffic is robots? - (2007-06-19)
  [376] What brings people to my web site? - (2005-07-13)


Back to
The unbalanced relationship between customer and provider
Previous and next
or
Horse's mouth home
Forward to
Backup procedures - via backup server
Some other Articles
Additional Python courses added to our schedule
Python - even named code blocks are objects
Fixing damaged MySQL tables - Error 1712 and Error 2013
Backup procedures - via backup server
Which (virtual) host was visited? Tuning Apache log files, and Python analysis
The unbalanced relationship between customer and provider
Finding sum, minimum, maximum and average in Python (and Ruby)
Selecting RECENT and POPULAR news and trends for your web site users
Commenting out an echo killed my bash backup script
The Campus in Melksham - name, logo, and livery choice
4755 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2017: 404 The Spa • Melksham, Wiltshire • United Kingdom • SN12 6QL
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/4404_Whi ... lysis.html • PAGE BUILT: Sat May 27 16:49:10 2017 • BUILD SYSTEM: WomanWithCat