Here's an extract (reprinted with permission) from an email I received from James - a fellow web site owner, trying to identify visitors to his web site ... he had found and used our IP lookup, and was impressed that we got it right ....
I have just bought a csv database from [[supplier deleted]]. But their database says an ip is ntl when yours says its Virgin. His ip is a virgin ip, as its my friends.
"I think I have been robbed. Now I am looking to find the correct or accurate database. I am using a program written in php to read the users ip and then cross reference the true IP number (long) with a mysql database.
Some interesting points raised ... my answer ..
James, none of these databases / techniques is going to be 100% accurate. I have customers in Surrey who pop up as being in Ireland, and customers in Bristol claimed for France. In Cambridge last week, my hotel internet connection defaulted to Google in German. Expect between 98% and 99.5% accuracy, and provide a page that lets your user change his country.
To identify systems by country, we use
the Maxmind database. There's an open source varient there, updated monthly, but it's now quite hard to find on their site. We use that for our
user tracking page and, yes, that page is in PHP.
To identify the visiting host more closely, I would start with a reverse DNS lookup - the Apache web server can be set up to do this itself and it means that you don't need to hold a detailed local database at all. This comes up with a name for around 80% of host computers worldwide, from which you can deduce a lot more information, even though the strings are a bit inconsistent and tricky to work with at times. Simply change
HostnameLookups Off
to
HostnameLookups On
in the httpd.conf file and your log lines will contain strings such as
client-82-3-85-219.manc.adsl.virgin.net and
ata01cs603.americas.hp.net
instead of
82.3.85.219 and
212.118.35.6
The other technique I would look at, and I think this is our page that you may have found, uses a whois database lookup. Once again, we're not holding a local database but going out to the authoritative version on the web. Be careful of automating this one too often though as it's intended for human (non-robotic) use and you could get yourself banned if you pile in thousands of requests. If you're using a Linux / Unix server, call up the man page on
whois to get you started.
Where you SHOULD use a database locally is in cacheing your results. There's little chance that an IP address will move except rarely, so if you hold on to your results for - say - a week, then you only have to check back with the reverse DNS / whois / maxmind very occasionally - great for efficiency, and also avoids you irritating them to the extent that your traffic gets questioned.
Further useful links -
Our IP lookup script - public available page
Change country - page on our site
Source code - whois access in PHP
Source code - generating "select country" pulldown (written 2006-05-24)
4ed5
Associated topics are indexed under
G902 - Well House Consultants - Web site techniques, utility and visibility [4076] Web site - fully back! - (2013-04-29)
[4001] Helping search engines with appropriate 400 error codes - (2013-02-11)
[3974] TV show appearance - how does it effect your web site? - (2013-01-13)
[3896] An email marathon - (2012-10-15)
[3776] Some traps it's so easy to fall into in designing your web site - (2012-06-23)
[3745] Legal change - You need to obtain user consent if you use cookies on your website - (2012-06-01)
[3744] Short Web Addresses for Melksham - (2012-05-30)
[3734] QR codes with marketing logos embedded - (2012-05-16)
[3623] Some TestWise examples - helping use Ruby code to check your web site operation - (2012-02-24)
[3589] Promoting a single one of your domains on the search engines - (2012-01-22)
[3563] How big is a web page these days? Does the size of your pages matter? - (2011-12-26)
[3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
[3532] Sharing the user experience - designing a form with the customer in mind - (2011-11-29)
[3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
[3426] Automed web site testing scripted in Ruby using watir-webdriver - (2011-09-09)
[3367] Google +1 - what is it? - (2011-07-22)
[3197] Finding and diverting image requests from rogue domains - (2011-03-08)
[3149] Looking back at www.wellho.net - (2011-01-28)
[3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
[3022] Retaining web site visitors - reducing the one page wonders - (2010-10-31)
[2981] How to set up short and meaningfull alternative URLs - (2010-10-02)
[2668] Is it worth it? - (2010-03-09)
[2569] How to run a successful online poll / petition / survey / consultation - (2010-01-10)
[2552] Web site traffic - real users, or just noise? - (2009-12-26)
[2532] Analysing Google arrivals by country of origin - (2009-12-10)
[2519] Status Page / breaks of service in early December - (2009-11-30)
[2410] Removal of technical resources from this site - (2009-09-19)
[2389] Writing with our customers words - (2009-09-01)
[2341] Koulutus, Open Source tietokone kielillä - (2009-08-09)
[2340] ldning, Open Source dator språk - (2009-08-09)
[2339] Opplæring, Open Source datamaskinen språk - (2009-08-09)
[2338] Uddannelse, Open Source computer sprog - (2009-08-09)
[2337] Opleiding, Open Source computertalen - (2009-08-09)
[2336] Formação, Open Source computador línguas - (2009-08-09)
[2335] Ausbildung, die Open-Source-Sprachen - (2009-08-09)
[2334] Formazione, Open Source computer lingue - (2009-08-09)
[2333] Formación, de los lenguajes de código abierto - (2009-08-09)
[2332] Formation, des langages Open Source - (2009-08-09)
[2225] How important is a front page ranking on a search engine? - (2009-06-09)
[2065] Static mirroring through HTTrack, wget and others - (2009-03-03)
[2056] Web Site Loading - experiences and some solutions shared - (2009-02-26)
[1982] Cooking bodies and URLs - (2009-01-08)
[1970] Plagarism - who is copying my pages? - (2009-01-02)
[1961] Making our things easier to find - (2008-12-26)
[1955] How to avoid duplicating web page maintainance - (2008-12-20)
[1888] Find the link - (2008-11-16)
[1856] A few of my favourite things - (2008-10-26)
[1833] Web Bloopers - good form design - avoiding pitfalls - (2008-10-11)
[1797] I have been working hard but I do not expect you noticed - (2008-09-14)
[1793] Which country does a search engine think you are located in? - (2008-09-11)
[1756] Ever had One of THOSE mornings? - (2008-08-16)
[1747] Who is watching you? - (2008-08-10)
[1711] Rapid growth leads to server move - (2008-07-17)
[1653] How do Google Ads work? - (2008-05-25)
[1634] Kiss and Book - (2008-05-07)
[1630] To provide external links, or not? - (2008-05-04)
[1610] PHP course dot co, dot uk - (2008-04-13)
[1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
[1541] Colour, Composition or Content - (2008-02-16)
[1534] Where in the world / country is my visitor from? - (2008-02-07)
[1513] Perl, PHP or Python? No - Perl AND PHP AND Python! - (2008-01-20)
[1506] Ongoing Image Copyright Issues, PHP and MySQL solutions - (2008-01-14)
[1505] Script to present commonly used images - PHP - (2008-01-13)
[1494] A time to update pictures - (2008-01-03)
[1437] Above the fold with First Great Western - (2007-11-19)
[1297] Stuffing content into a web page - easy maintainance - (2007-08-09)
[1237] What proportion of our web traffic is robots? - (2007-06-19)
[1212] What brought YOU to our web site? - (2007-06-01)
[1207] Simple but effective use of mod_rewrite (Apache httpd) - (2007-05-27)
[1198] From Web to Web 2 - (2007-05-21)
[1186] Two new pages / sites - (2007-05-14)
[1184] Finding resources - some pointers - (2007-05-13)
[1177] Sorting out for a site map - (2007-05-05)
[1104] Drawing dynamic graphs in PHP - (2007-03-09)
[1055] Above the fold - (2007-01-28)
[1029] Our search engine placement is dropping. - (2007-01-11)
[1015] Search engine placement - long term strategy and success - (2006-12-30)
[994] Training on Cascading Style Sheets - (2006-12-17)
[976] Santa at the station - (2006-12-09)
[916] Driving customers away - (2006-11-07)
[893] Visibility - (2006-10-14)
[800] Effective web campaign? - (2006-07-12)
[767] Finding the language preference of a web site visitor - (2006-06-18)
[757] Horse and Python training - (2006-06-12)
[718] Protecting images from theft - (2006-05-12)
[681] Mirroring a dynamic site - (2006-04-12)
[658] Keeping the visitors happy and browsing - (2006-03-26)
[649] Denial of Service ''attack'' - (2006-03-17)
[533] Bigger Box Campaign - (2005-12-18)
[528] Getting favicon to work - avoiding common pitfalls - (2005-12-14)
[510] Dynamic Web presence - next generation web site - (2005-11-29)
[492] New Navigation Aid - Launch of My Wellho - (2005-11-11)
[414] Form Madness - (2005-08-14)
[376] What brings people to my web site? - (2005-07-13)
[369] CMS - the minefield of Choices - (2005-07-05)
[348] Graveyard pages - (2005-06-15)
[347] Frightening and from-friend viruses and spams - (2005-06-14)
[322] More maps - (2005-05-23)
[320] Ordnance Survey - using a 'Get a map' - (2005-05-22)
[314] What language is this written in? - (2005-05-17)
[311] Growth pains - (2005-05-14)
[288] Colour blindness for web developers - (2005-04-22)
[284] The Iconish language - (2005-04-19)
[278] Cover all the options - (2005-04-13)
[276] An apology to Mr Boneparte - (2005-04-11)
[274] Our most popular resources - (2005-04-10)
[268] Information request forms, cleaning up spam - (2005-04-05)
[261] Putting a form online - (2005-03-29)
[259] Responding to spam - (2005-03-27)
[222] Who are all these visitors? - (2005-02-20)
[202] Searching for numbers - (2005-02-04)
[197] Allow for peak traffic on your web site - (2005-02-01)
[182] Your personal Google ranking - (2005-01-19)
[179] The hunt for unique words - (2005-01-16)
[173] Data Mining - (2005-01-09)
[165] Implementing an effective site search engine - (2005-01-01)
[142] Colour for access - (2004-12-06)
[117] A case of case - (2004-11-14)
[109] URLs - a service and not a hurdle - (2004-11-04)
[98] No more 'Error 404' pages. Something better. - (2004-10-24)
[32] Web design platoon - (2004-08-29)
[23] Skills and responsibilities - (2004-08-22)
H401 - Some extra PHP modules [3454] Your PHP website - how to factor and refactor to reduce growing pains - (2011-09-24)
[3453] Reading and using emails including enclosures on your web server. - (2011-09-23)
[2682] Adding extensions to PHP Open Source applications - callbacks - (2010-03-17)
[2343] World Flags in your PHP pages - (2009-08-10)
W603 - Web and Intranet - Server Side Technologies [3915] How does PHP work? - (2012-11-07)
[3705] Django Training Courses - UK - (2012-04-23)
[2282] Checking robots.txt from Python - (2009-07-12)
[2055] Effect on server when memory runs out and swapping starts - (2009-02-26)
[1749] Using server side and client side programming together - (2008-08-11)
[1615] PHP training courses every month - (2008-04-18)
[1365] Korn Shell scripts on the web - (2007-09-25)
[1355] .php or .html extension? Morally Static Pages - (2007-09-17)
[1031] robots.txt - a clue to hidden pages? - (2007-01-13)
[1020] Parallel processing in PHP - (2007-01-03)
[653] Easy feed! - (2006-03-21)
[642] How similar are two words - (2006-03-11)
5304
Some other Articles
New Tape LibrarianBoys will be boys, saved by UbuntuKeeping customers informed by emailPerl for Systems Admin - suid scriptsWhere is a web site visitor browsing fromHotel Technology RequirementsReading the newspaper and working with other restrictionsCareer development adviceLooking ahead and behind in a Regular ExpressionA journey, an arrival, a people