Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
Python and Tcl - public course schedule [here]
Private courses on your site - see [here]
Please ask about maintenance training for Perl, PHP, Lua, etc
 
Extracting data from backups to restore selected rows from MySQL tables

We take backups from our live server, using mysqldump for the databases, and tar / gzip for most of the file systems. And when we had a server failure which meant a reload (and a new underlying OS too), we were able to be back online in a couple of hours, with minimal data loss.

But our backups are not only taken as a guarantee against complete failure, but also to allow us to step back to a specific checkpoint if we need to, or indeed to do a partial restore. And with the backups being huge data files, the extraction and management of what we want to pull back, and how to merge it onto the live and continuing system, can be an interesting challenge. And it's a challenge we have very very rarely had to face. However, it came up yesterday when one of our moderators accidentally deleted a thread - some 35 posts - on a very active board that continued to roll after the trip-up and couldn't simply be wound back.

So - here's the fix / mechanism used for selective restore:

1. Take another backup before you start, and warn users the may see a service dip.

2. Create a new database and restore the latest backup into it

  create database test_fgw
  use test_fgw
  source latestbackupfile.sql


3. locate the missing records, and copy them into new tables with a structure that mirrors the original structure

  create table ref_15686 select * from smf_messages where id_topic = 15686;
  create table topic_15686 select * from smf_topics where id_topic = 15686;


4. Dump the newly created tables which have the structure of the whole table, but only the missing records in them

  /usr/local/mysql/bin/mysqldump -ufgw -pxxxx -h127.0.0.1 test_fgw ref_15686 topic_15686 > partial.sql

5. Edit the dump, changing 'insert into ref_15686' into 'insert into smf_messages', and so on

6. merge in the records being restored

  use fgw
  source partial.sql


7. Test thoroughly, and when you are sure it's worked delete the test batabase.

This worked for us. But beware - if the tables / records being restored aren't uniquely keyed, or if your software reallocated keys from deleter records for new records, it won't work. Note also the need to ensure that joining keys, etc, may need to be restored - in my case this was the second table in my example above.

Please understand what youre doing when following a procedure like this - the opportunity to really screw up is ever-present!
(written 2015-05-01, updated 2015-05-02)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
S161 - Data Access and Security in MySQL
  [4493] Forgotten / lost MySQL root password - (2015-05-16)
  [4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)
  [4406] Fixing damaged MySQL tables - Error 1712 and Error 2013 - (2015-01-25)
  [4390] Checking MySQL database backups have worked (not failed) - (2015-01-10)
  [3270] SQL - Data v Metadata, and the various stages of data selection - (2011-04-29)
  [2647] Removing duplicates from a MySQL table - (2010-02-22)
  [2263] Mysqldump fails as a cron job - a work around - (2009-06-30)
  [2204] Images in a database? How big is a database? (MySQL) - (2009-05-28)
  [1131] MySQL - Password security (authentication protocol) - (2007-04-02)
  [947] What is an SQL injection attack? - (2006-11-27)
  [647] Checking for MySQL errors - (2006-03-15)
  [535] MySQL permissions and privileges - (2005-12-20)
  [401] What is an SQL injection attack? - (2005-08-02)
  [193] The wrong MySQL - (2005-01-29)
  [192] Current MySQL and PHP paths and upgrades - (2005-01-28)

A162 - Web Application Deployment - Backups and File System Management
  [4405] Backup procedures - via backup server - (2015-01-24)
  [4400] Commenting out an echo killed my bash backup script - (2015-01-19)
  [4115] More or less back - what happened to our server the other day - (2013-06-14)
  [4063] Backups by crossover between network centres - setting up automatic scp transfers - (2013-04-13)
  [4056] An overpractical test of our backup strategy! - (2013-03-30)
  [2299] How much space does my directory take - Linux - (2009-07-20)
  [1893] Some Linux and Unix tips - (2008-11-18)
  [1801] Will your backups work if you have to restore them? - (2008-09-18)
  [1765] Dialects of English and Unix - (2008-08-21)
  [1648] The tourists guide to Linux - (2008-05-20)
  [1439] Linux / Unix - layout of operating system files - (2007-11-20)
  [1288] Linux run states, shell special commands, and directory structures - (2007-08-03)
  [1023] Finding public writeable things on your linux file system - (2007-01-06)
  [1013] Copy multiple files - confusing error message from cp - (2006-12-30)
  [754] tar, jar, war, ear, sar files - (2006-06-10)
  [735] Boys will be boys, saved by Ubuntu - (2006-05-27)
  [703] Copying files and preserving ownership - (2006-04-28)
  [593] Finding where the disc space has gone - (2006-02-06)
  [554] What backup is adequate? - (2006-01-04)
  [334] Symbolic links and hard links - (2005-06-02)
  [153] Linux - where to put swap space - (2004-12-16)

S157 - More MySQL commands
  [3061] Databases - why data is split into separate tables, and how to join them - (2010-11-20)
  [2645] Optimising and caching your MySQL enquiries - (2010-02-22)
  [2644] Counting rows in joined MySQL tables - (2010-02-22)
  [2643] Relating tables with joins in MySQL - (2010-02-21)
  [2448] MySQL - efficiency and other topics - (2009-10-10)
  [2259] Grouping rows for a summary report - MySQL and PHP - (2009-06-27)
  [2110] MySQL - looking for records in one table that do NOT correspond to records in another table - (2009-03-31)
  [1904] Ruby, Perl, Linux, MySQL - some training notes - (2008-11-23)
  [1735] Finding words and work boundaries (MySQL, Perl, PHP) - (2008-08-03)
  [1574] Joining MySQL tables revisited - finding nonmatching records, etc - (2008-03-15)
  [1331] MySQL joins revisited - (2007-09-03)
  [1235] Outputting numbers as words - MySQL with Perl or PHP - (2007-06-17)
  [1213] MySQL - the order of clauses and the order of actions - (2007-06-01)
  [673] Helicopter views and tartans - (2006-04-06)
  [591] Key facts - SQL and MySQL - (2006-02-04)
  [581] Saving a MySQL query results to your local disc for Excel - (2006-01-29)
  [572] Giving the researcher power over database analysis - (2006-01-22)
  [567] Combining similar rows from a MySQL database select - (2006-01-17)
  [517] An occasional chance, and reducing data to manageable levels - (2005-12-04)
  [515] MySQL - an FAQ - (2005-12-03)
  [513] MySQL - JOIN or WHERE to link tables correctly? - (2005-12-01)
  [502] SELECT in MySQL - choosing the rows you want - (2005-11-22)
  [494] MySQL - a score of things to remember - (2005-11-12)
  [449] Matching in MySQL - (2005-09-24)
  [279] Getting a list of unique values from a MySQL column - (2005-04-14)
  [159] MySQL - Optimising Selects - (2004-12-21)
  [158] MySQL - LEFT JOIN and RIGHT JOIN, INNER JOIN and OUTER JOIN - (2004-12-20)


Back to
Crossing between continents
Previous and next
or
Horse's mouth home
Forward to
People in New York
Some other Articles
Misusing statistics? - the seedy side of election campaigning
TransAtlantic crossing - the starter
Moving from mysql to mysqli - simple worked example
People in New York
Extracting data from backups to restore selected rows from MySQL tables
Crossing between continents
Pictures from around Rekjavik
Home in Melksham to Gatwick Airport - easy by public transport
Yesterday - off on leg one of our holiday
Upcoming election - Chippenham Constituency - the major candidates compared
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2019: 404 The Spa • Melksham, Wiltshire • United Kingdom • SN12 6QL
PH: 01225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/4481_Ext ... ables.html • PAGE BUILT: Sat May 27 16:49:10 2017 • BUILD SYSTEM: WomanWithCat