Fixed Length Record to CSV

File Handling example from a Well House Consultants training course
More on File Handling [link]

This example is described in the following article(s):
• Handling JSON in Python (and a csv, marshall and pickle comparison) - [link]

Source code: flr Module: Y110

""" Fixed Length Record handling in Python ...

You can read from a file in Python using
        read to read a specific number of bytes
        readline to read a line
        readlines to read all the lines to a list
        xreadlines to read all the lines through a generator
(in Python 3, xreadlines becomes readlines ...)

Fixed length records tend to have differing amounts of spaces
between fields, so a regular expression of "one or more spaces" is
often a good way to split the lines up (you will need to use a
different technique if the fields contain "real" spaces as data)

You can then rejoin the lines using a space character - or something
more complex is you want to produce a CSV file as I did in the
following example, which reads a 60 character-per-record server log
file, echoes it as a series of lines to STDOUT, and writes a CSV
file called csv.txt. """

import re
space = re.compile(r"\s+")

fh = open("fixed_60.txt")
fho = open("csv.txt","w")

while (True):
        record = fh.read(60)
        if not record: break
        print record

        values = space.split(record)
        fho.write('"' + '","'.join(values) + '"\n')

fho.close()

""" Samples ----------------------

Echoed to the screen:

83.170.93.77 09/Jun/2010:03:30:02 OPTIONS 200 0
83.170.93.77 09/Jun/2010:03:30:02 OPTIONS 200 0
83.170.93.77 09/Jun/2010:03:30:02 OPTIONS 200 0
38.99.98.17 09/Jun/2010:03:30:04 GET 200 36003
203.47.199.254 09/Jun/2010:03:30:04 GET 200 13533
203.47.199.254 09/Jun/2010:03:30:03 GET 200 65501
66.249.71.119 09/Jun/2010:03:30:06 GET 200 7684
77.88.27.25 09/Jun/2010:03:30:06 GET 200 2826
67.218.116.163 09/Jun/2010:03:30:09 GET 200 10113
67.195.112.180 09/Jun/2010:03:30:14 GET 200 112096
66.249.71.119 09/Jun/2010:03:30:20 GET 200 12688
212.76.47.211 09/Jun/2010:03:30:19 GET 200 15501

Output file:

"83.170.93.77","09/Jun/2010:03:30:02","OPTIONS","200","0"
"83.170.93.77","09/Jun/2010:03:30:02","OPTIONS","200","0"
"83.170.93.77","09/Jun/2010:03:30:02","OPTIONS","200","0"
"38.99.98.17","09/Jun/2010:03:30:04","GET","200","36003"
"203.47.199.254","09/Jun/2010:03:30:04","GET","200","13533"
"203.47.199.254","09/Jun/2010:03:30:03","GET","200","65501"
"66.249.71.119","09/Jun/2010:03:30:06","GET","200","7684"
"77.88.27.25","09/Jun/2010:03:30:06","GET","200","2826"
"67.218.116.163","09/Jun/2010:03:30:09","GET","200","10113"
"67.195.112.180","09/Jun/2010:03:30:14","GET","200","112096"
"66.249.71.119","09/Jun/2010:03:30:20","GET","200","12688"
"212.76.47.211","09/Jun/2010:03:30:19","GET","200","15501"

"""

Learn about this subject

This module and example are covered on the following public courses:
* Learning to program in Python
* Python Programming
Also available on on site courses for larger groups

Books covering this topic

Yes. We have over 700 books in our library. Books covering Python are listed here and when you've selected a relevant book we'll link you on to Amazon to order.

Other Examples

This example comes from our "File Handling" training module. You'll find a description of the topic and some other closely related examples on the "File Handling" module index page.

Full description of the source code

You can learn more about this example on the training courses listed on this page, on which you'll be given a full set of training notes.

Many other training modules are available for download (for limited use) from our download centre under an Open Training Notes License.

Other resources

• Our Solutions centre provides a number of longer technical articles.
• Our Opentalk forum archive provides a question and answer centre.
• The Horse's mouth provides a daily tip or thought.
• Further resources are available via the resources centre.
• All of these resources can be searched through through our search engine
• And there's a global index here.

Web site author

This web site is written and maintained by Well House Consultants.

Purpose of this website

This is a sample program, class demonstration or answer from a training course. It's main purpose is to provide an after-course service to customers who have attended our public private or on site courses, but the examples are made generally available under conditions described below.

Conditions of use

Past attendees on our training courses are welcome to use individual examples in the course of their programming, but must check the examples they use to ensure that they are suitable for their job. Remember that some of our examples show you how not to do things - check in your notes. Well House Consultants take no responsibility for the suitability of these example programs to customer's needs.

This program is copyright Well House Consultants Ltd. You are forbidden from using it for running your own training courses without our prior written permission. See our page on courseware provision for more details.

Any of our images within this code may NOT be reused on a public URL without our prior permission. For Bona Fide personal use, we will often grant you permission provided that you provide a link back. Commercial use on a website will incur a license fee for each image used - details on request.