Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
For 2021 - online Python 3 training - see ((here)).

Our plans were to retire in summer 2020 and see the world, but Coronavirus has lead us into a lot of lockdown programming in Python 3 and PHP 7.
We can now offer tailored online training - small groups, real tutors - works really well for groups of 4 to 14 delegates. Anywhere in the world; course language English.

Please ask about private 'maintenance' training for Python 2, Tcl, Perl, PHP, Lua, etc.
Getting more log information from the Apache http web server

The Apache httpd server is preconfigured when you download it to define common and combined logging formats - and those are the standards accepted by so many tools. But there's so much more information available if you choose to log it. In order to get a better understanding of metrics of the traffic on one of our dedicated servers for a few days, I've added the following log lines to my httpd.conf:

  LogFormat "%a %v %{%Y%m%d.%H%M%S}t %s %>s %B %T %P %X \"%{Accept-language}i\" \
      \"%{Accept-encoding}i\" \"%r\" \"%{If-modified-since}i\" \"%{Referer}i\" \
      \"%{User-agent}i\"" extras
  CustomLog "/home/wellho/logs/extra_log" extras


Here are what those fields call for:

%aremote host ip
%vYour virtual host serving the request
%{%Y%m%d.%H%M%S}tThe time the request was made
(In our own specified time format)
%sThe returned Status from the request
%>sThe returned Status in the response
%Breturn size (0 for no content)
%TTime taken to serve the request (seconds)
%PThe child process ID that served the request
%XCompletion Status
%{Accept-language}iLanguage preference of user
%{Accept-encoding}iCompressions accepted by user
%rThe request that was made
%{If-modified-since}iWhether request was time stamp conditional
%{Referer}iPage from which this request was called
%{User-agent}iThe type of browser that's making the request


There's more available too - see http://httpd.apache.org/docs/current/mod/mod_log_config.html

Here are some examples:

203.99.217.10 www.wellho.net 20110916.085049 304 304 0 0 10541 + "en-us,en-securid" "gzip, deflate" "GET /commonimages/f_logo.jpg HTTP/1.1" "Sun, 20 Mar 2011 17:47:06 GMT" "http://www.wellho.net/regex/javare.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.2; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MS-RTC LM 8)"

- Microsoft Internet Explorer 7, Under Windows XP ... requesting an image only if it had changed (which it hadn't). Called up from a page on Java Regular Expressions, and the request took less that a second to serve. Requester prefers answer in US English, and will accept compressed content that's gzipped or deflated.




95.108.240.250 www.wellho.net 20110916.085314 200 200 116845 0 10651 + "ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01" "gzip,deflate" "GET /forum/The-Tcl-programming-language/Re-expect-script-aborting-in-the-middle.html HTTP/1.1" "-" "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

- A request from the Yandex Spider for a forum page. Prefers a response in Russian (?) ...




27.107.186.224 www.wellho.net 20110916.085921 200 200 12344 18 11098 + "en-US,en;q=0.8" "gzip,deflate,sdch" "GET /bing/images/WHClogoBW.jpg HTTP/1.1" "-" "http://www.wellho.net/net/search.php4?search=use+of+structures" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.0 Safari/535.2"

- User running the Chrome browser on Windows Vista, calling for an image. For some reason (to be further investigated) it took 18 seconds for our server to send out the response.




180.76.5.136 twcrp.org.uk 20110916.090108 200 200 2320 0 11132 - "tr-TR" "gzip" "GET /community/index.php?action=printpage;topic=79.0 HTTP/1.1" "Tue, 16 Aug 2011 16:23:28 GMT" "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

- Another Spider request - this time with a preference for a response in Turkish. The page requested was from the twcrp.org.uk web site and not from the www.wellho.net site - our server looks after multiple domains. And the "-" 9th field indicates that the server is not to keep the thread alive when the request has been completed; you'll see a "+" on other sample records indicating that the thread is to be kept alive for a few seconds for follow up requests; an "X" would indicate a request that wasn't completed because the browser went away.




77.88.27.25 www.wellho.net 20110916.090659 200 404 16210 0 11844 + "ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01" "gzip,deflate" "GET /dcms/index.php HTTP/1.1" "-" "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

- This final example shows a page that was initially anticipated to be a good request (i.e. was going to send back a status 200), but our server took a further look at the request and decided to send a "page not found" error back instead. We have a number of URL patterns that are rewritten by mod_rewrite which run our user scripts, but respond with code 404. This shows the difference between %s and %>s in the format specification line.




We cover the configuration and deployment of the Apache httpd web server on our Deploying LAMP course.
(written 2011-09-16)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
A606 - Web Application Deployment - Apache httpd - log files and log tools
  [376] What brings people to my web site? - (2005-07-13)
  [1237] What proportion of our web traffic is robots? - (2007-06-19)
  [1503] Web page (http) error status 405 - (2008-01-12)
  [1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
  [1656] Be careful of misreading server statistics - (2008-05-28)
  [1761] Logging Cookies with the Apache httpd web server - (2008-08-20)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [1796] libwww-perl and Indy Library in your server logs? - (2008-09-13)
  [3015] Logging the performance of the Apache httpd web server - (2010-10-25)
  [3019] Apache httpd Server Status - monitoring your server - (2010-10-28)
  [3027] Server logs - drawing a graph of gathered data - (2010-11-03)
  [3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
  [3447] Needle in a haystack - finding the web server overload - (2011-09-18)
  [3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3670] Reading Google Analytics results, based on the relative populations of countries - (2012-03-24)
  [3974] TV show appearance - how does it effect your web site? - (2013-01-13)
  [3984] 20 minutes in to our 15 minutes of fame - (2013-01-20)
  [4307] Identifying and clearing denial of service attacks on your Apache server - (2014-09-27)
  [4404] Which (virtual) host was visited? Tuning Apache log files, and Python analysis - (2015-01-23)
  [4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)


Back to
A demonstration of how many Python facilities work together
Previous and next
or
Horse's mouth home
Forward to
Take the dog on a lead - do not carry her. Perl references.
Some other Articles
Awk v Perl
Perl and CGI - simple form, and monitoring script.
Take the dog on a lead - do not carry her. Perl references.
Getting more log information from the Apache http web server
A demonstration of how many Python facilities work together
Pressing ^C in a Python program. Also Progress Bar.
Research is exciting. But should routine be automated?
Python for loops - applying a temporary second name to the same object
Melksham to become a part of Trowbridge?
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2021: 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/3443_Get ... erver.html • PAGE BUILT: Sun Oct 11 16:07:41 2020 • BUILD SYSTEM: JelliaJamb