Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))
Helping search engines with appropriate 400 error codes

Our web site's done its best over the years to help people who enter URLs that have slight errors in them - misspellings, wrong directory names, etc ... and we have quite a number of scrips that take parameters which can be modified to give different results - for example
  http://www.wellho.net/net/quote.html?where=L10+1RF
will get you a quotation for a course in Liverpool and
  http://www.wellho.net/net/quote.html?where=17013
will get you a quote for a course in Carlisle, Pennsylvania!

That's very flexible, but it means that there's a wide range of requests that can be made of our servers which in turn can result in the search engines and otherds finding a lot of pages that really aren't wanted / shouldn't be indexed. It just takes one person to put a link somewhere to set off a whole chain, or one malicious spider to start propogating valid but unwanted URLs.

I've spent some time this weekend cleaning up - making sure that we signal rather better to the search engines (and to other visitors too) which URLs are inappropriate. Firstly, some extras in our robots.txt file:
  Disallow: /net/recents.htm
  Disallow: /resources/recents.htm
And that asks any wellbehaved crawler (most are!) not to go into either or these URLs, or any URLs that start with that text. So as well as .html files (which were previously disallowed), we now request that .htm files are not indexed.

And here are some of the error codes that we now generate ...

400 - Your request is a bad one. Most probably you have tried to call up a script with an inappropriate parameter, such as giving us a URL when we expect a number of simple directory name. (Is this an injection attack going on?)

403 - Forbidden. You are not allowed access to this. We may have identified you as some sort of automatic process trying to mirror out site, and that's not appropriate as this is a dynamic site and the output changes.

404 - Not found. Most likely a typo in the URL or a missing page on our site. Possibly an automated probe to see if a page exists. If you get this from a link, please let us know!

410 - Gone. The resource you're looking for does not exist, and is never likely to exist. Please clear your cache of it and don't ask again. When an automated link has accidentally (or maliciously) been generated and points to a whole family of resources, we'll tell you to clear them out via a 410.

Some of these are generated via lines in the .htaccess files:

  # Any crap on the end of "recents" URLs - kill the page!
  RewriteRule ^recents\.html. /err/tiny410.html [R=410,L]
  RewriteRule ^recents\.htm/. /err/tiny410.html [R=410,L]

  # Get rid of query strings with long (injection?) paths
  RewriteCond %{QUERY_STRING} /.*/.*/
  RewriteRule ^smap /err/tiny400.html [R=400,L]
  RewriteRule ^smap.*%3 /err/tiny400.html [R=400,L]

  # If there's more than just a digit on the end, naughty
  RewriteCond %{QUERY_STRING} full=..
  RewriteRule ^landings\.htm /err/tiny403.html [R=403,L]

And sometimes one of our PHP scripts itself sends out a different header:

  header("HTTP/1.1 404 Not Found");

Covered on our PHP Techniques and Linux Web Server courses when appropriate to the delegate group. Also on private courses.
(written 2013-02-11)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
G902 - Well House Consultants - Web site techniques, utility and visibility
  [23] Skills and responsibilities - (2004-08-22)
  [32] Web design platoon - (2004-08-29)
  [98] No more 'Error 404' pages. Something better. - (2004-10-24)
  [109] URLs - a service and not a hurdle - (2004-11-04)
  [117] A case of case - (2004-11-14)
  [142] Colour for access - (2004-12-06)
  [165] Implementing an effective site search engine - (2005-01-01)
  [173] Data Mining - (2005-01-09)
  [179] The hunt for unique words - (2005-01-16)
  [182] Your personal Google ranking - (2005-01-19)
  [197] Allow for peak traffic on your web site - (2005-02-01)
  [202] Searching for numbers - (2005-02-04)
  [222] Who are all these visitors? - (2005-02-20)
  [259] Responding to spam - (2005-03-27)
  [261] Putting a form online - (2005-03-29)
  [268] Information request forms, cleaning up spam - (2005-04-05)
  [274] Our most popular resources - (2005-04-10)
  [276] An apology to Mr Boneparte - (2005-04-11)
  [278] Cover all the options - (2005-04-13)
  [284] The Iconish language - (2005-04-19)
  [288] Colour blindness for web developers - (2005-04-22)
  [311] Growth pains - (2005-05-14)
  [314] What language is this written in? - (2005-05-17)
  [320] Ordnance Survey - using a 'Get a map' - (2005-05-22)
  [322] More maps - (2005-05-23)
  [347] Frightening and from-friend viruses and spams - (2005-06-14)
  [348] Graveyard pages - (2005-06-15)
  [369] CMS - the minefield of Choices - (2005-07-05)
  [376] What brings people to my web site? - (2005-07-13)
  [414] Form Madness - (2005-08-14)
  [492] New Navigation Aid - Launch of My Wellho - (2005-11-11)
  [510] Dynamic Web presence - next generation web site - (2005-11-29)
  [528] Getting favicon to work - avoiding common pitfalls - (2005-12-14)
  [533] Bigger Box Campaign - (2005-12-18)
  [649] Denial of Service ''attack'' - (2006-03-17)
  [658] Keeping the visitors happy and browsing - (2006-03-26)
  [681] Mirroring a dynamic site - (2006-04-12)
  [718] Protecting images from theft - (2006-05-12)
  [732] Where is a web site visitor browsing from - (2006-05-24)
  [757] Horse and Python training - (2006-06-12)
  [767] Finding the language preference of a web site visitor - (2006-06-18)
  [800] Effective web campaign? - (2006-07-12)
  [893] Visibility - (2006-10-14)
  [916] Driving customers away - (2006-11-07)
  [976] Santa at the station - (2006-12-09)
  [994] Training on Cascading Style Sheets - (2006-12-17)
  [1015] Search engine placement - long term strategy and success - (2006-12-30)
  [1029] Our search engine placement is dropping. - (2007-01-11)
  [1055] Above the fold - (2007-01-28)
  [1104] Drawing dynamic graphs in PHP - (2007-03-09)
  [1177] Sorting out for a site map - (2007-05-05)
  [1184] Finding resources - some pointers - (2007-05-13)
  [1186] Two new pages / sites - (2007-05-14)
  [1198] From Web to Web 2 - (2007-05-21)
  [1207] Simple but effective use of mod_rewrite (Apache httpd) - (2007-05-27)
  [1212] What brought YOU to our web site? - (2007-06-01)
  [1237] What proportion of our web traffic is robots? - (2007-06-19)
  [1297] Stuffing content into a web page - easy maintainance - (2007-08-09)
  [1437] Above the fold with First Great Western - (2007-11-19)
  [1494] A time to update pictures - (2008-01-03)
  [1505] Script to present commonly used images - PHP - (2008-01-13)
  [1506] Ongoing Image Copyright Issues, PHP and MySQL solutions - (2008-01-14)
  [1513] Perl, PHP or Python? No - Perl AND PHP AND Python! - (2008-01-20)
  [1534] Where in the world / country is my visitor from? - (2008-02-07)
  [1541] Colour, Composition or Content - (2008-02-16)
  [1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
  [1610] PHP course dot co, dot uk - (2008-04-13)
  [1630] To provide external links, or not? - (2008-05-04)
  [1634] Kiss and Book - (2008-05-07)
  [1653] How do Google Ads work? - (2008-05-25)
  [1711] Rapid growth leads to server move - (2008-07-17)
  [1747] Who is watching you? - (2008-08-10)
  [1756] Ever had One of THOSE mornings? - (2008-08-16)
  [1793] Which country does a search engine think you are located in? - (2008-09-11)
  [1797] I have been working hard but I do not expect you noticed - (2008-09-14)
  [1833] Web Bloopers - good form design - avoiding pitfalls - (2008-10-11)
  [1856] A few of my favourite things - (2008-10-26)
  [1888] Find the link - (2008-11-16)
  [1955] How to avoid duplicating web page maintainance - (2008-12-20)
  [1961] Making our things easier to find - (2008-12-26)
  [1970] Plagarism - who is copying my pages? - (2009-01-02)
  [1982] Cooking bodies and URLs - (2009-01-08)
  [2056] Web Site Loading - experiences and some solutions shared - (2009-02-26)
  [2065] Static mirroring through HTTrack, wget and others - (2009-03-03)
  [2225] How important is a front page ranking on a search engine? - (2009-06-09)
  [2332] Formation, des langages Open Source - (2009-08-09)
  [2333] Formaci[83][c2]ón, de los lenguajes de c[83][c2]ódigo abierto - (2009-08-09)
  [2334] Formazione, Open Source computer lingue - (2009-08-09)
  [2335] Ausbildung, die Open-Source-Sprachen - (2009-08-09)
  [2336] Forma[83][c2]ç[83][c2]ão, Open Source computador l[83][c2]ínguas - (2009-08-09)
  [2337] Opleiding, Open Source computertalen - (2009-08-09)
  [2338] Uddannelse, Open Source computer sprog - (2009-08-09)
  [2339] Oppl[83][c2]æring, Open Source datamaskinen spr[83][c2]åk - (2009-08-09)
  [2340] ldning, Open Source dator spr[83][c2]åk - (2009-08-09)
  [2341] Koulutus, Open Source tietokone kielill[83][c2]ä - (2009-08-09)
  [2389] Writing with our customers words - (2009-09-01)
  [2410] Removal of technical resources from this site - (2009-09-19)
  [2519] Status Page / breaks of service in early December - (2009-11-30)
  [2532] Analysing Google arrivals by country of origin - (2009-12-10)
  [2552] Web site traffic - real users, or just noise? - (2009-12-26)
  [2569] How to run a successful online poll / petition / survey / consultation - (2010-01-10)
  [2668] Is it worth it? - (2010-03-09)
  [2981] How to set up short and meaningfull alternative URLs - (2010-10-02)
  [3022] Retaining web site visitors - reducing the one page wonders - (2010-10-31)
  [3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
  [3149] Looking back at www.wellho.net - (2011-01-28)
  [3197] Finding and diverting image requests from rogue domains - (2011-03-08)
  [3367] Google +1 - what is it? - (2011-07-22)
  [3426] Automed web site testing scripted in Ruby using watir-webdriver - (2011-09-09)
  [3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
  [3532] Sharing the user experience - designing a form with the customer in mind - (2011-11-29)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3563] How big is a web page these days? Does the size of your pages matter? - (2011-12-26)
  [3589] Promoting a single one of your domains on the search engines - (2012-01-22)
  [3623] Some TestWise examples - helping use Ruby code to check your web site operation - (2012-02-24)
  [3734] QR codes with marketing logos embedded - (2012-05-16)
  [3744] Short Web Addresses for Melksham - (2012-05-30)
  [3745] Legal change - You need to obtain user consent if you use cookies on your website - (2012-06-01)
  [3776] Some traps it's so easy to fall into in designing your web site - (2012-06-23)
  [3896] An email marathon - (2012-10-15)
  [3974] TV show appearance - how does it effect your web site? - (2013-01-13)
  [4076] Web site - fully back! - (2013-04-29)
  [4115] More or less back - what happened to our server the other day - (2013-06-14)
  [4136] How do I post automatically from a PHP script to my Twitter account? - (2013-07-10)
  [4239] Facebook marketing - early experiences - (2014-01-19)
  [4376] Well House Consultants, Well House Manor, First Great Western Coffee shop, TransWilts / 2014 web site reports - (2015-01-01)
  [4401] Selecting RECENT and POPULAR news and trends for your web site users - (2015-01-19)
  [4474] Effect on external factors on traffic to our web sites - an update - (2015-04-26)
  [4492] Almost so wrong, but perhaps it's right for some? - (2015-05-11)

A603 - Web Application Deployment - Further httpd Configuration
  [345] Spotting a denial of service attack - (2005-06-12)
  [466] Separating 'per instance' data from binaries and web sites - (2005-10-16)
  [526] Apache httpd - serving web documents from different directories - (2005-12-12)
  [550] 2006 - Making business a pleasure - (2006-01-01)
  [631] Apache httpd to Tomcat - jk v proxy - (2006-03-03)
  [662] An unhelpful error message from Apache httpd - (2006-03-30)
  [755] Using different URLs to navigate around a single script - (2006-06-11)
  [853] To list a directory under httpd on a web server, or not? - (2006-09-02)
  [934] Clustering, load balancing, mod_rewrite and mod_proxy - (2006-11-21)
  [1009] Passing GET parameters through Apache mod_rewrite - (2006-12-27)
  [1080] httpd.conf or .htaccess? - (2007-02-14)
  [1121] Sharing the load with Apache httpd and perhaps Tomcat - (2007-03-29)
  [1351] Compressing web pages sent out from server. Is it worth it? - (2007-09-14)
  [1355] .php or .html extension? Morally Static Pages - (2007-09-17)
  [1377] Load Balancing with Apache mod_jk (httpd/Tomcat) - (2007-10-02)
  [1381] Using a MySQL database to control mod_rewrite via PHP - (2007-10-06)
  [1551] Which modules are loaded in my Apache httpd - (2008-02-23)
  [1564] Default file (MiMe types) for Apache httpd and Apache Tomcat - (2008-03-04)
  [1566] Strange behaviour of web directory requests without a trailing slash - (2008-03-06)
  [1619] User and Group settings for Apache httpd web server - (2008-04-22)
  [1636] What to do if the Home Page is missing - (2008-05-08)
  [1707] Configuring Apache httpd - (2008-07-12)
  [1762] WEB-INF (Tomcat) and .htaccess (httpd) - (2008-08-20)
  [1767] mod_proxy and mod_proxy_ajp - httpd - (2008-08-22)
  [1778] Pointing all the web pages in a directory at a database - (2008-08-30)
  [1939] mod_proxy_ajp and mod_proxy_balancer examples - (2008-12-13)
  [1954] mod_rewrite for newcomers - (2008-12-20)
  [1974] Moving a directory on your web site - (2009-01-03)
  [2060] Database connection Pooling, SSL, and command line deployment - httpd and Tomcat - (2009-03-01)
  [2272] Monitoring and loading tools for testing Apache Tomcat - (2009-07-07)
  [2478] How did I do THAT? - (2009-10-26)
  [2900] Redirecting a page - silent, temporary or permanent? - (2010-08-03)
  [3133] An image from a website that occasionally comes out as hyroglyphics - (2011-01-14)
  [3449] Apache Internal Dummy Connection - what is it and what should I do with it? - (2011-09-19)
  [3635] Parse error: parse error, unexpected T_STRING on brand new web site - why? - (2012-03-03)
  [3862] Forwarding a whole domain, except for a few directories - Apache http server - (2012-09-17)
  [3955] Building up from a small PHP setup to an enterprise one - (2012-12-16)
  [4307] Identifying and clearing denial of service attacks on your Apache server - (2014-09-27)


Back to
9 years, and 4000 articles on
Previous and next
or
Horse's mouth home
Forward to
Images of Melksham you havent seen before
Some other Articles
New guest ... becoming returning guest
Annual Accounts - a big job for a small business
Web and console - same principle, same code - Ruby example
Images of Melksham you havent seen before
Helping search engines with appropriate 400 error codes
9 years, and 4000 articles on
Handling failures / absences of your backend server nicely
Travelogue - from the Netherlands to London via Hook and Harwich
Servlet v JSP (Java Server Page). What is the difference?
Tips on Tomcat - moving applications around
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2024: 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/4001_Hel ... codes.html • PAGE BUILT: Sun Oct 11 16:07:41 2020 • BUILD SYSTEM: JelliaJamb