Our web site traffic rose from 37000 hits last Wednesday to 64000 hits on Thursday. Good sales and marketing activity on our part? No - it's a potential problem; all the extra traffic came from a single location and my immediate concerns included:
* Possible denial of service, where all the bandwidth was being eaten up by the visitor.
* What use were they making of what was approaching a complete copy of the site?
* Potential extra costs if we were to hit our traffic ceiling
On this occasion, I noticed that we had a problem on Friday when I looked at the previous day's log analysis and spotted the problem sticking out like a sore thumb. I spent a couple of hours yesterday investigating more thoroughly, and putting a filter in place to cap aggressive browsing as it happens - a piece of PHP some 30 or 40 lines long. PHP's a powerful language, so in those lines I'm also able to add a permanent record of the potential abuse to a database ...
If you want to read more, I've put a
technical analysis and sample PHP script in our
solution centre.
(written 2005-06-12, updated 2008-05-04)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
W512 - Web and Intranet - Site Design Aspects [229] A fortunate accident - (2005-02-27)
[261] Putting a form online - (2005-03-29)
[288] Colour blindness for web developers - (2005-04-22)
[319] Accommodation and landing pages - (2005-05-21)
[352] Improved mining techniques! - (2005-06-19)
[391] One mans pleasure is another mans poison - (2005-07-26)
[510] Dynamic Web presence - next generation web site - (2005-11-29)
[556] Colour doesn't have to mean colourful - (2006-01-06)
[649] Denial of Service ''attack'' - (2006-03-17)
[718] Protecting images from theft - (2006-05-12)
[795] Remember a site's non-technical issues too - (2006-07-07)
[823] An excellent use for a visitor count? - (2006-08-05)
[859] Put the answer in context - it may be printed - (2006-09-08)
[918] Databases needn't be frightening, hard or expensive - (2006-11-08)
[1015] Search engine placement - long term strategy and success - (2006-12-30)
[1047] Maintainable code - some positive advice - (2007-01-21)
[1054] UK legal requirements for your commercial web site - (2007-01-27)
[1353] Mood shots - (2007-09-16)
[1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
[2214] Global Index to help you find resources - (2009-06-01)
[3517] Tags used in writing this blog - (2011-11-12)
[3563] How big is a web page these days? Does the size of your pages matter? - (2011-12-26)
[3589] Promoting a single one of your domains on the search engines - (2012-01-22)
H117 - Security in PHP [426] Robust checking of data entered by users - (2005-08-27)
[920] A lion in a cage - PHP - (2006-11-10)
[947] What is an SQL injection attack? - (2006-11-27)
[1052] Learning to write secure, maintainable PHP - (2007-01-25)
[1086] Injection attacks - safeguard your PHP scripts - (2007-02-20)
[1323] Easy handling of errors in PHP - (2007-08-27)
[1387] Error logging to file not browser in PHP - (2007-10-11)
[1396] Using PHP to upload images / Store on MySQL database - security questions - (2007-10-19)
[1482] A story about benchmarking PHP - (2007-12-23)
[1542] Are nasty programs looking for security holes on your server? - (2008-02-17)
[1679] PHP - Sanitised application principles for security and useability - (2008-06-16)
[1694] Defensive coding techniques in PHP? - (2008-07-02)
[1747] Who is watching you? - (2008-08-10)
[1779] Injection Attacks - avoiding them in your PHP - (2008-08-31)
[2025] Injection Attack if register_globals in on - PHP - (2009-02-04)
[2628] An example of an injection attack using Javascript - (2010-02-08)
[2688] Security considerations in programming - what do we teach? - (2010-03-22)
[2939] Protecting your images from use out of context - (2010-08-29)
[3210] Catchable fatal error in PHP ... How to catch, and alternative solutions such as JSON - (2011-03-22)
[3698] How to stop forms on other sites submitting to your scripts - (2012-04-15)
[3747] An easy way to comply with the new cookie law if your site is well designed - (2012-06-02)
[3813] Injection Attacks - PHP, SQL, HTML, Javascript - and how to neutralise them - (2012-07-22)
[4642] A small teaching program - demonstration of principles only - (2016-02-08)
H112 - PHP - Further Web Page and Network Handling [220] When to use Frames - (2005-02-19)
[314] What language is this written in? - (2005-05-17)
[356] Sudoku helper or sudoku cheat - (2005-06-23)
[372] Time calculation in PHP - (2005-07-08)
[376] What brings people to my web site? - (2005-07-13)
[410] Reading a news or blog feed (RSS) in your PHP page - (2005-08-12)
[425] Caching an XML feed - (2005-08-26)
[443] Server side scripting of styles to suit the browser - (2005-09-12)
[451] Accessing a page via POST from within a PHP script - (2005-09-26)
[484] Setting the file name for a downloaded document - (2005-11-03)
[537] Daily Image Santafied - (2005-12-22)
[542] Morning image, afternoon image - (2005-12-26)
[565] Using PHP to output images, XML, Style sheets, etc - (2006-01-15)
[603] PHP - setting sort order with an associative array - (2006-02-13)
[675] Adding PHP tags to an old cgi program - (2006-04-08)
[767] Finding the language preference of a web site visitor - (2006-06-18)
[789] Hot answers in PHP - (2006-07-02)
[847] Image maps for navigation - a straightforward example - (2006-08-28)
[904] Of course I'll tell you by email - (2006-10-25)
[936] Global, Superglobal, Session variables - scope and persistance in PHP - (2006-11-21)
[1009] Passing GET parameters through Apache mod_rewrite - (2006-12-27)
[1114] PHP Image upload script - (2007-03-21)
[1183] Improving searches - from OR to AND? - (2007-05-11)
[1187] Updating a page strictly every minute (PHP, Perl) - (2007-05-14)
[1210] PHP header() function - uses and new restrictions - (2007-05-30)
[1355] .php or .html extension? Morally Static Pages - (2007-09-17)
[1379] Simple page password protection - PHP - (2007-10-04)
[1485] Copyright and theft of images, bandwidth and members. - (2007-12-26)
[1495] Single login and single threaded models - Java and PHP - (2008-01-04)
[1496] PHP / Web 2 logging - (2008-01-06)
[1505] Script to present commonly used images - PHP - (2008-01-13)
[1515] Keeping staff up to date on hotel room status - (2008-01-22)
[1518] Downloading data for use in Excel (from PHP / MySQL) - (2008-01-25)
[1549] http, https and ajp - comparison and choice - (2008-02-22)
[2632] Shipping a test harness with your class in PHP - (2010-02-12)
[2679] How to build a test harness into your PHP - (2010-03-16)
[2729] Uploading a document or image to its own URL via a browser - (2010-04-18)
[2918] Downloading a report from the web for further local analysis - (2010-08-13)
[3036] Sending out an email containing HTML from within a PHP page - (2010-11-07)
[3432] 3 digit HTTP status codes - what are they, which are most common, which should be a concern? - (2011-09-11)
[3540] Easy session example in PHP - keeping each customers data apart - (2011-12-06)
[3568] Telling which ServerAlias your visitor used - useful during merging domains - (2012-01-04)
[3918] Multiple page web applications - maintaining state - PHP - (2012-11-10)
[4070] Passing variable between PHP pages - hidden fields, cookies and sessions - (2013-04-26)
[4483] Moving from mysql to mysqli - simple worked example - (2015-05-03)
A603 - Web Application Deployment - Further httpd Configuration [466] Separating 'per instance' data from binaries and web sites - (2005-10-16)
[526] Apache httpd - serving web documents from different directories - (2005-12-12)
[550] 2006 - Making business a pleasure - (2006-01-01)
[631] Apache httpd to Tomcat - jk v proxy - (2006-03-03)
[662] An unhelpful error message from Apache httpd - (2006-03-30)
[755] Using different URLs to navigate around a single script - (2006-06-11)
[853] To list a directory under httpd on a web server, or not? - (2006-09-02)
[934] Clustering, load balancing, mod_rewrite and mod_proxy - (2006-11-21)
[1080] httpd.conf or .htaccess? - (2007-02-14)
[1121] Sharing the load with Apache httpd and perhaps Tomcat - (2007-03-29)
[1207] Simple but effective use of mod_rewrite (Apache httpd) - (2007-05-27)
[1351] Compressing web pages sent out from server. Is it worth it? - (2007-09-14)
[1377] Load Balancing with Apache mod_jk (httpd/Tomcat) - (2007-10-02)
[1381] Using a MySQL database to control mod_rewrite via PHP - (2007-10-06)
[1551] Which modules are loaded in my Apache httpd - (2008-02-23)
[1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
[1564] Default file (MiMe types) for Apache httpd and Apache Tomcat - (2008-03-04)
[1566] Strange behaviour of web directory requests without a trailing slash - (2008-03-06)
[1619] User and Group settings for Apache httpd web server - (2008-04-22)
[1636] What to do if the Home Page is missing - (2008-05-08)
[1707] Configuring Apache httpd - (2008-07-12)
[1762] WEB-INF (Tomcat) and .htaccess (httpd) - (2008-08-20)
[1767] mod_proxy and mod_proxy_ajp - httpd - (2008-08-22)
[1778] Pointing all the web pages in a directory at a database - (2008-08-30)
[1939] mod_proxy_ajp and mod_proxy_balancer examples - (2008-12-13)
[1954] mod_rewrite for newcomers - (2008-12-20)
[1955] How to avoid duplicating web page maintainance - (2008-12-20)
[1974] Moving a directory on your web site - (2009-01-03)
[2060] Database connection Pooling, SSL, and command line deployment - httpd and Tomcat - (2009-03-01)
[2272] Monitoring and loading tools for testing Apache Tomcat - (2009-07-07)
[2478] How did I do THAT? - (2009-10-26)
[2900] Redirecting a page - silent, temporary or permanent? - (2010-08-03)
[3133] An image from a website that occasionally comes out as hyroglyphics - (2011-01-14)
[3449] Apache Internal Dummy Connection - what is it and what should I do with it? - (2011-09-19)
[3635] Parse error: parse error, unexpected T_STRING on brand new web site - why? - (2012-03-03)
[3862] Forwarding a whole domain, except for a few directories - Apache http server - (2012-09-17)
[3955] Building up from a small PHP setup to an enterprise one - (2012-12-16)
[4001] Helping search engines with appropriate 400 error codes - (2013-02-11)
[4307] Identifying and clearing denial of service attacks on your Apache server - (2014-09-27)
Some other Articles
Comments in TclGraveyard pagesFrightening and from-friend viruses and spamsNo Smoking PubsSpotting a denial of service attackWalk in BathShould I use structured or object oriented?The evening after the courseHappy Birthday, PHPCode and code maintainance efficiency