We take backups from our live server, using
mysqldump for the databases, and tar / gzip for most of the file systems. And when we had a server failure which meant a reload (and a new underlying OS too), we were able to be back online in a couple of hours, with minimal data loss.
But our backups are not only taken as a guarantee against complete failure, but also to allow us to step back to a specific checkpoint if we need to, or indeed to do a partial restore. And with the backups being huge data files, the extraction and management of what we want to pull back, and how to merge it onto the live and continuing system, can be an interesting challenge. And it's a challenge we have very very rarely had to face. However, it came up yesterday when one of our moderators accidentally deleted a thread - some 35 posts - on a very active board that continued to roll after the trip-up and couldn't simply be wound back.
So - here's the fix / mechanism used for selective restore:
1. Take another backup before you start, and warn users the may see a service dip.
2. Create a new database and restore the latest backup into it
create database test_fgw
use test_fgw
source latestbackupfile.sql
3. locate the missing records, and copy them into new tables with a structure that mirrors the original structure
create table ref_15686 select * from smf_messages where id_topic = 15686;
create table topic_15686 select * from smf_topics where id_topic = 15686;
4. Dump the newly created tables which have the structure of the whole table, but only the missing records in them
/usr/local/mysql/bin/mysqldump -ufgw -pxxxx -h127.0.0.1 test_fgw ref_15686 topic_15686 > partial.sql
5. Edit the dump, changing 'insert into ref_15686' into 'insert into smf_messages', and so on
6. merge in the records being restored
use fgw
source partial.sql
7. Test thoroughly, and when you are sure it's worked delete the test batabase.
This worked for us. But beware - if the tables / records being restored aren't uniquely keyed, or if your software reallocated keys from deleter records for new records, it won't work. Note also the need to ensure that joining keys, etc, may need to be restored - in my case this was the second table in my example above.
Please understand what youre doing when following a procedure like this - the opportunity to really screw up is ever-present!
(written 2015-05-01, updated 2015-05-02)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
S157 - More MySQL commands [158] MySQL - LEFT JOIN and RIGHT JOIN, INNER JOIN and OUTER JOIN - (2004-12-20)
[159] MySQL - Optimising Selects - (2004-12-21)
[279] Getting a list of unique values from a MySQL column - (2005-04-14)
[449] Matching in MySQL - (2005-09-24)
[494] MySQL - a score of things to remember - (2005-11-12)
[502] SELECT in MySQL - choosing the rows you want - (2005-11-22)
[513] MySQL - JOIN or WHERE to link tables correctly? - (2005-12-01)
[515] MySQL - an FAQ - (2005-12-03)
[517] An occasional chance, and reducing data to manageable levels - (2005-12-04)
[567] Combining similar rows from a MySQL database select - (2006-01-17)
[572] Giving the researcher power over database analysis - (2006-01-22)
[581] Saving a MySQL query results to your local disc for Excel - (2006-01-29)
[591] Key facts - SQL and MySQL - (2006-02-04)
[673] Helicopter views and tartans - (2006-04-06)
[1213] MySQL - the order of clauses and the order of actions - (2007-06-01)
[1235] Outputting numbers as words - MySQL with Perl or PHP - (2007-06-17)
[1331] MySQL joins revisited - (2007-09-03)
[1574] Joining MySQL tables revisited - finding nonmatching records, etc - (2008-03-15)
[1735] Finding words and work boundaries (MySQL, Perl, PHP) - (2008-08-03)
[1904] Ruby, Perl, Linux, MySQL - some training notes - (2008-11-23)
[2110] MySQL - looking for records in one table that do NOT correspond to records in another table - (2009-03-31)
[2259] Grouping rows for a summary report - MySQL and PHP - (2009-06-27)
[2448] MySQL - efficiency and other topics - (2009-10-10)
[2643] Relating tables with joins in MySQL - (2010-02-21)
[2644] Counting rows in joined MySQL tables - (2010-02-22)
[2645] Optimising and caching your MySQL enquiries - (2010-02-22)
[2647] Removing duplicates from a MySQL table - (2010-02-22)
[3061] Databases - why data is split into separate tables, and how to join them - (2010-11-20)
[3270] SQL - Data v Metadata, and the various stages of data selection - (2011-04-29)
A162 - Web Application Deployment - Backups and File System Management [153] Linux - where to put swap space - (2004-12-16)
[334] Symbolic links and hard links - (2005-06-02)
[554] What backup is adequate? - (2006-01-04)
[593] Finding where the disc space has gone - (2006-02-06)
[703] Copying files and preserving ownership - (2006-04-28)
[735] Boys will be boys, saved by Ubuntu - (2006-05-27)
[754] tar, jar, war, ear, sar files - (2006-06-10)
[1013] Copy multiple files - confusing error message from cp - (2006-12-30)
[1023] Finding public writeable things on your linux file system - (2007-01-06)
[1288] Linux run states, shell special commands, and directory structures - (2007-08-03)
[1439] Linux / Unix - layout of operating system files - (2007-11-20)
[1648] The tourists guide to Linux - (2008-05-20)
[1765] Dialects of English and Unix - (2008-08-21)
[1801] Will your backups work if you have to restore them? - (2008-09-18)
[1893] Some Linux and Unix tips - (2008-11-18)
[2299] How much space does my directory take - Linux - (2009-07-20)
[4056] An overpractical test of our backup strategy! - (2013-03-30)
[4063] Backups by crossover between network centres - setting up automatic scp transfers - (2013-04-13)
[4115] More or less back - what happened to our server the other day - (2013-06-14)
[4390] Checking MySQL database backups have worked (not failed) - (2015-01-10)
[4400] Commenting out an echo killed my bash backup script - (2015-01-19)
[4405] Backup procedures - via backup server - (2015-01-24)
S161 - Data Access and Security in MySQL [192] Current MySQL and PHP paths and upgrades - (2005-01-28)
[193] The wrong MySQL - (2005-01-29)
[401] What is an SQL injection attack? - (2005-08-02)
[535] MySQL permissions and privileges - (2005-12-20)
[647] Checking for MySQL errors - (2006-03-15)
[947] What is an SQL injection attack? - (2006-11-27)
[1131] MySQL - Password security (authentication protocol) - (2007-04-02)
[2204] Images in a database? How big is a database? (MySQL) - (2009-05-28)
[2263] Mysqldump fails as a cron job - a work around - (2009-06-30)
[4406] Fixing damaged MySQL tables - Error 1712 and Error 2013 - (2015-01-25)
[4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)
[4493] Forgotten / lost MySQL root password - (2015-05-16)
Some other Articles
Misusing statistics? - the seedy side of election campaigningTransAtlantic crossing - the starterMoving from mysql to mysqli - simple worked examplePeople in New YorkExtracting data from backups to restore selected rows from MySQL tablesCrossing between continentsPictures from around RekjavikHome in Melksham to Gatwick Airport - easy by public transportYesterday - off on leg one of our holidayUpcoming election - Chippenham Constituency - the major candidates compared