Conversion of objects into serial data, such that it can be stored in a file or passed over a network, and restoring it when read back, is a vital topic within any serious object oriented application. It's all very well working with an object on the heap (i.e. in memory while your program runs), but if you dump it out and then restore it, you're going to need to do so in such a way that it can go to a different memory location, as chances are that the addresses it was saved from are going to be in other use when you restore. Thus a serialised format, and
json has beceome very much the standard - it's compact, character based rather than binary, quite flexible, and widely supported.
In Python, you may still have reasons to use
marshall,
pickle and
cpickle, or
xml, but 9 times out of 10 you'll do better using Json.
This morning I have been working with a 1 Gbyte data flow, and rather than repeat the analysis of my raw data (136 seconds elapsed time) to create the dicts and lists I want to go on and graph, I've chosen to do the extraction once and dump out the Json data to a file. Subsequent runs of my program will check for the file, and read it if present (well under a second!) - see
source code here. It's going to save me an awful lot of time during the course this week (and will stop my delegates getting bored) as I'm using Json as my cache. Note the "pretty print" option I have added too - allowing my Json objects to be written raw (which would be the norm), or well formatted (which I've used to show you on our web site - data is
[here].
(written 2015-02-22)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y115 - Additional Python Facilities [183] The elegance of Python - (2005-01-19)
[208] Examples - Gadfly, NI Number, and Tcl to C interface - (2005-02-10)
[239] What and why for the epoch - (2005-03-08)
[463] Splitting the difference - (2005-10-13)
[663] Python to MySQL - (2006-03-31)
[672] Keeping your regular expressions simple - (2006-04-05)
[753] Python 3000 - the next generation - (2006-06-09)
[901] Python - listing out the contents of all variables - (2006-10-21)
[1043] Sending an email from Python - (2007-01-18)
[1136] Buffering output - why it is done and issues raised in Tcl, Perl, Python and PHP - (2007-04-06)
[1149] Turning objects into something you can store - Pickling (Python) - (2007-04-15)
[1305] Regular expressions made easy - building from components - (2007-08-16)
[1336] Ignore case in Regular Expression - (2007-09-08)
[1337] A series of tyre damages - (2007-09-08)
[1876] Python Regular Expressions - (2008-11-08)
[2407] Testing code in Python - doctest, unittest and others - (2009-09-16)
[2435] Serialization - storing and reloading objects - (2009-10-04)
[2462] Python - how it saves on compile time - (2009-10-20)
[2655] Python - what is going on around me? - (2010-02-28)
[2721] Regular Expressions in Python - (2010-04-14)
[2745] Connecting Python to sqlite and MySQL databases - (2010-04-28)
[2746] Model - View - Controller demo, Sqlite - Python 3 - Qt4 - (2010-04-29)
[2764] Python decorators - your own, staticmethod and classmethod - (2010-05-14)
[2765] Running operating system commands from your Python program - (2010-05-14)
[2786] Factory methods and SqLite in use in a Python teaching example - (2010-05-29)
[2790] Joining a MySQL table from within a Python program - (2010-06-02)
[3089] Python regular expressions - repeating, splitting, lookahead and lookbehind - (2010-12-17)
[3442] A demonstration of how many Python facilities work together - (2011-09-16)
[3469] Teaching dilemma - old tricks and techniques, or recent enhancements? - (2011-10-08)
[4085] JSON from Python - first principles, easy example - (2013-05-13)
[4211] Handling JSON in Python (and a csv, marshall and pickle comparison) - (2013-11-16)
[4298] Python - an interesting application - (2014-09-18)
[4451] Running an operating system command from your Python program - the new way with the subprocess module - (2015-03-06)
[4536] Json load from URL, recursive display, Python 3.4 - (2015-10-14)
[4593] Command line parameter handling in Python via the argparse module - (2015-12-08)
[4709] Some gems from Intermediate Python - (2016-10-30)
Some other Articles
Seventh stay away this year - and it's still only February!Mutable v Immuatble objects in Python, and the implicationReading command line parameters in PythonA first graph with Matplotlib in PythonJson is the new marshall, pickle and cPickle / PythonLoving programming in Python - and ready to teach YOU howAdding a PHP build option, rotating an image based on camera data, and a new look at thumbnails in PHPAccessing a MySQL database from Python with mysql.connectorImages of our rail promotion campaign Public training courses - upcoming dates