Databases needn't be frightening, hard or expensive

Have you written a web site using plain old HTML (perhaps with style sheets and images) and then extended it to more interactive content using a little PHP, JSP, Perl, cgi or similar? Are you keeping the data associated with your PHP - your on line programs - in plain text files? If you are, that may be the right decision, or it may be the wrong one. Database engines such as MySQL are available for free download and install, are provided as a standard part of the package by most WSPs (Web Service Provider)s, and can provide - in the right circumstances - an easy, effective and efficient way of storing, accessing and updating data. And those "right circumstances" are so common that they'll apply to elements of most modern web sites.

If you keep data in plain files on your web site / within your application, it'll work fine while the application and the data is small and doesn't change a great deal, but plain files have a number of issues you need to consider:

If you edit a file, you have to rewrite the whole file.
You need to write code to understand the format of the file - strip out comma separators, but allowing for commas within fields, etc.
File locking isn't nice - great care needs to be taken if there's any chance of two concurrent changes being made.
As data grows, the file / directory structure with a very large number of files becomes inefficient and unmanageable.
Searching a file / looking for data becomes all the more inefficient as the data volume grows.

You'll notice that many of the limitations I mention relate to data volume and you might thing to yourself "but I don't have a high traffic level - I don't need to bother". You might be surprised at just how quickly data volumes increase and tip the balance in favour of databases even for a small organisation. At Well House Consultants, we're a team of just 4 full time staff ... and yet without databases and tables, navigating our web site would be like running through treacle.

Even this article, when I post it, will be in a database. As a user of the system, I wouldn't need to know that or to take any special action, but as a user I'll see it filed efficiently. And because it's in a database it will be very easy for me to correct spelling mistakes later without having to pull in all the past articles too, and it also helps make it very, very easy to provide search tools. Have a look at this demo that searches through the title of old entries and see just easy the code is.

I train people on using plain text files. And I train people on databases too. So I have a foot in both camps. When would I recommend a switch to a database for a particular table of information?

When there's a large amount of data
When there's a possible need for concurrent edits
When the data is structured and elements would otherwise be repeated
When there's a need for searching and summarising

... oh - you should consider switching to a database when any one of those conditions applies. And the beauty is that you don't have to put ALL you data in databases - you can still use plain text files where they're appropriate. Our "Message of the day" which appears on our home page is a plain text file. Short, simple, easy to edit, unlikely to be searched and only edited by one person at a time.

Examples of database use on this site

This daily diary link
Our rank and review system link
Our user editiable pages link
Our forum link
our current user history link
Our accessibility preferences link
Our library link

(written 2006-11-08, updated 2006-11-12) 229b

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles

S151 - Introduction to SQL and MySQL
  [3928] Storing your intermediate data - what format should you you choose? - (2012-11-20)
  [3269] Files or Databases? MySQL, SQLite, or Oracle? - (2011-04-28)
  [2786] Factory methods and SqLite in use in a Python teaching example - (2010-05-29)
  [2567] Extra MySQL course dates (2 day course, UK) - (2010-01-08)
  [2561] The future of MySQL - (2010-01-03)
  [2134] Oracle take over Sun who had taken over MySQL - (2009-04-21)
  [924] The LAMP Cookbook - Linux, Apache, MySQL, PHP / Perl - (2006-11-13)
  [691] Testing you Perl / PHP / MySQL / Tcl knowledge - (2006-04-19)
  [591] Key facts - SQL and MySQL - (2006-02-04)
  [515] MySQL - an FAQ - (2005-12-03)
  [444] Database or Progamming - which to learn first? - (2005-09-13)
  [382] Central London Courses - Perl, PHP, Python, Tcl, MySQL - (2005-07-18)
  [175] Worthwhile - (2005-01-11)
  [85] Present and future MySQL - (2004-10-12)
  [84] MySQL - nuggets - (2004-10-11)

S154 - MySQL - Designing an SQL Database System
  [3494] Databases - when to treat the rules as guidelines - (2011-10-23)
  [3361] Blowing our own trumpet - MySQL resources - (2011-07-18)
  [3270] SQL - Data v Metadata, and the various stages of data selection - (2011-04-29)
  [2749] Delegate Question - defining MySQL table relationships as you create the tables - (2010-05-02)
  [2204] Images in a database? How big is a database? (MySQL) - (2009-05-28)
  [2085] MySQL - licensing issues, even with using the name - (2009-03-16)
  [2053] What a difference a MySQL Index made - (2009-02-25)
  [1771] More HowTo diagrams - MySQL, Tomcat and Java - (2008-08-24)
  [1575] Database design for a shopping application (MySQL) - (2008-03-15)
  [1423] MySQL - table design and initial testing example - (2007-11-06)
  [945] Code quality counts - (2006-11-26)
  [937] Display an image from a MySQL database in a web page via PHP - (2006-11-22)
  [666] Database design - get it right from first principles - (2006-04-02)
  [494] MySQL - a score of things to remember - (2005-11-12)
  [375] Oops - I got my initial database design wrong - (2005-07-12)
  [361] Binary Large Objects or bars - (2005-06-27)
  [59] MySQL - Pivot tables - (2004-09-22)

W512 - Web and Intranet - Site Design Aspects
  [3589] Promoting a single one of your domains on the search engines - (2012-01-22)
  [3563] How big is a web page these days? Does the size of your pages matter? - (2011-12-26)
  [3517] Tags used in writing this blog - (2011-11-12)
  [2214] Global Index to help you find resources - (2009-06-01)
  [1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
  [1353] Mood shots - (2007-09-16)
  [1054] UK legal requirements for your commercial web site - (2007-01-27)
  [1047] Maintainable code - some positive advice - (2007-01-21)
  [1015] Search engine placement - long term strategy and success - (2006-12-30)
  [859] Put the answer in context - it may be printed - (2006-09-08)
  [823] An excellent use for a visitor count? - (2006-08-05)
  [795] Remember a site's non-technical issues too - (2006-07-07)
  [718] Protecting images from theft - (2006-05-12)
  [649] Denial of Service ''attack'' - (2006-03-17)
  [556] Colour doesn't have to mean colourful - (2006-01-06)
  [510] Dynamic Web presence - next generation web site - (2005-11-29)
  [391] One mans pleasure is another mans poison - (2005-07-26)
  [352] Improved mining techniques! - (2005-06-19)
  [345] Spotting a denial of service attack - (2005-06-12)
  [319] Accommodation and landing pages - (2005-05-21)
  [288] Colour blindness for web developers - (2005-04-22)
  [261] Putting a form online - (2005-03-29)
  [229] A fortunate accident - (2005-02-27)

Q916 - Object Orientation and General technical topics - Database design principles
  [572] Giving the researcher power over database analysis - (2006-01-22)

240a

Back to
Syntax checking in PHP

Previous and next
or
Horse's mouth home

Forward to
Freedom for X is denial of privacy for Y

Some other Articles

Staying at your own hotel
French Exchange
A lion in a cage - PHP
Freedom for X is denial of privacy for Y
Databases needn't be frightening, hard or expensive
Syntax checking in PHP
Driving customers away
Paging through hundreds of entries
A practical example of roles
Python - A list of methods