If you're going to be doing a
complex select in MySQL, how should you formulate it to make it run efficiently?
This sounds like a simple question, but is actually very complex!
Firstly, consider creating indexes on any columns that are likely to be used in the selection of joining of records; choose a UNIQUE INDEX if a field is known to be unique, and a FULLTEXT INDEX if you want to be able to quickly search a text field for a particular value. Use just an INDEX for non-unique fields. Here are some example commands that create indeexes:
create unique index piidindex on b_ptab (piid(5));
create index piidindex on b_btab (piid(5));
create fulltext index synindex on b_btab (synopsis);
Your data structure is now optimised to allow for selects searching on the indexed fields.
Secondly, when you specify your tables to be joined, try to specify the tables at the "heart" of the join operation first before the tables that tag data onto the structure; you may want to specify a STRAIGHT JOIN to force these tables to be read first.
Third, if you're selecting based on text word content
- use MATCH in preference to LIKE
- and use LIKE in preference to RLIKE
Using these techniques, a well written query can run thousands of times faster that a poor one. I'm writing this note on a laptop running MySQL and I ran a following query to join:
* A table of books
* A table of Authors
* A pivot table to link authors to books
* A table of book subject
* and a table of book publishers
Out of some 550 books (with a total of around 700 different authors), I selected the 53 records that included the word "focus" in their Synopsis, using the command:
select sql_no_cache title,fullname,b_btab.biid,b_atab.aiid,pvid,subject,pubname from b_btab, b_atab, b_pivot, b_stab, b_ptab where b_btab.biid = b_pivot.biid and b_atab.aiid = b_pivot.aiid and b_stab.biid = b_btab.biid and b_ptab.piid = b_btab.piid and match (synopsis) against ('focus');
and it took 0.02 seconds. The same join, without indexes and any optimisation, took 3 minutes and 18 seconds.
(written 2004-12-21, updated 2006-06-05)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
S157 - More MySQL commands [158] MySQL - LEFT JOIN and RIGHT JOIN, INNER JOIN and OUTER JOIN - (2004-12-20)
[279] Getting a list of unique values from a MySQL column - (2005-04-14)
[449] Matching in MySQL - (2005-09-24)
[494] MySQL - a score of things to remember - (2005-11-12)
[502] SELECT in MySQL - choosing the rows you want - (2005-11-22)
[513] MySQL - JOIN or WHERE to link tables correctly? - (2005-12-01)
[515] MySQL - an FAQ - (2005-12-03)
[517] An occasional chance, and reducing data to manageable levels - (2005-12-04)
[567] Combining similar rows from a MySQL database select - (2006-01-17)
[572] Giving the researcher power over database analysis - (2006-01-22)
[581] Saving a MySQL query results to your local disc for Excel - (2006-01-29)
[591] Key facts - SQL and MySQL - (2006-02-04)
[673] Helicopter views and tartans - (2006-04-06)
[1213] MySQL - the order of clauses and the order of actions - (2007-06-01)
[1235] Outputting numbers as words - MySQL with Perl or PHP - (2007-06-17)
[1331] MySQL joins revisited - (2007-09-03)
[1574] Joining MySQL tables revisited - finding nonmatching records, etc - (2008-03-15)
[1735] Finding words and work boundaries (MySQL, Perl, PHP) - (2008-08-03)
[1904] Ruby, Perl, Linux, MySQL - some training notes - (2008-11-23)
[2110] MySQL - looking for records in one table that do NOT correspond to records in another table - (2009-03-31)
[2259] Grouping rows for a summary report - MySQL and PHP - (2009-06-27)
[2448] MySQL - efficiency and other topics - (2009-10-10)
[2643] Relating tables with joins in MySQL - (2010-02-21)
[2644] Counting rows in joined MySQL tables - (2010-02-22)
[2645] Optimising and caching your MySQL enquiries - (2010-02-22)
[2647] Removing duplicates from a MySQL table - (2010-02-22)
[3061] Databases - why data is split into separate tables, and how to join them - (2010-11-20)
[3270] SQL - Data v Metadata, and the various stages of data selection - (2011-04-29)
[4481] Extracting data from backups to restore selected rows from MySQL tables - (2015-05-01)
Some other Articles
Introduction fees, bonuses, commissions, kickbacksA Change is as good as a restChristmas breakReview of the AutumnMySQL - Optimising SelectsAutomatic service upgradesSignageRailway train service, Melksham stationLinux - where to put swap space