Very frequently, IT system users find they've got a data file in one format and they need to filter it and transform it. On Linux and Unix systems, utilities such as
grep and
cut were the tradtions, perhaps with some
awk.
Perl provided a single-program replacement for shell scripts that bolted together the Unix utilites, and it can still do a great job. But Perl code can often be very hard to modify and extend later on, and very often similar reformatting jobs will recur. And that's where, these days, I would recommend another language - perhaps
Pyton rather than Perl.
From this week's course - a short Python example that reads a space delimited data file, checks whether the number in the third field is greater than 40, and if it is outputs the record tab delimited:
for line in open("melksham.txt"):
fields = line.split()
if int(fields[2]) > 40:
newline = "\t".join(fields)
print newline
Complete file
[here].
In Perl, it's rather shorter but harder to follow:
FH = open("melksham.txt);
while () {
@f = split(/\s+/);
print (join("\t",@f),"\n") if ($f[2] > 40);
}
See our full course schedule - including both Python and Perl courses -
[here].
(written 2012-06-14, updated 2012-06-16)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y110 - Python - File Handling [114] Relative or absolute milkman - (2004-11-10)
[183] The elegance of Python - (2005-01-19)
[1442] Reading a file multiple times - file pointers - (2007-11-23)
[2011] Conversion of OSI grid references to Eastings and Northings - (2009-01-28)
[2282] Checking robots.txt from Python - (2009-07-12)
[2870] Old prices - what would the equivalent price have been in 1966? - (2010-07-14)
[3083] Python - fresh examples from recent courses - (2010-12-11)
[3442] A demonstration of how many Python facilities work together - (2011-09-16)
[3465] How can I do an FTP transfer in Python? - (2011-10-05)
[3558] Python or Lua - which should I use / learn? - (2011-12-21)
[4438] Loving programming in Python - and ready to teach YOU how - (2015-02-22)
[4451] Running an operating system command from your Python program - the new way with the subprocess module - (2015-03-06)
[4593] Command line parameter handling in Python via the argparse module - (2015-12-08)
[4663] Easy data to object mapping (csv and Python) - (2016-03-24)
[4708] Scons - a build system in Python - building hello world - (2016-10-29)
[4717] with in Python - examples of use, and of defining your own context - (2016-11-02)
Q102 - Choosing your language [76] Learning to program in - (2004-10-07)
[1990] Speaking all the languages - (2009-01-12)
[2001] I have not programmed before, and need to learn - (2009-01-19)
[2048] Learning to program in PHP, Python, Java or Lua ... - (2009-02-19)
[2507] Admission - (2009-11-19)
[2535] When should I use Java, Perl, PHP, or Python? - (2009-12-13)
[2536] All the Cs ... and Java too - (2009-12-13)
[2700] The same very simple program in many different programming languages - (2010-03-31)
[2866] Ruby - how does it compare and where is it the right language? - (2010-07-11)
[3169] Rekeying a table - comparison in #Ruby #Perl and #Python - (2011-02-14)
[3619] Ruby v Perl - a comparison example - (2012-02-21)
[3785] Programming languages - what are the differences between them? - (2012-06-27)
P669 - Perl - Data Munging [597] Storing a regular expression in a perl variable - (2006-02-09)
[1316] Filtering and altering Perl lists with grep and map - (2007-08-23)
[1509] Extracting information from a file of records - (2008-01-16)
[1947] Perl substitute - the e modifier - (2008-12-16)
[2129] Nothing beats Perl to solve a data manipulation requirement quickly - (2009-04-14)
[2702] First and last match with Regular Expressions - (2010-04-02)
[3335] Practical Extraction and Reporting - (2011-06-26)
[3707] Converting codons via Amino Acids to Proteins in Perl - (2012-04-25)
[4620] Perl 6 - a Practical Extraction and Reporting example! - (2016-01-11)
A166 - Web Application Deployment - Linux Utilities [63] Almost like old times - (2004-09-26)
[71] Comparators in Linux and Unix - (2004-10-03)
[1361] Korn shell course - (2007-09-22)
[1366] awk - a powerful data extraction and manipulation tool - (2007-09-25)
[1690] Conversion of c/r line ends to l/f line ends - (2008-06-28)
[2145] Using the internet to remotely check for power failure at home (PHP) - (2009-04-29)
[2320] Helping new arrivals find out about source code examples - (2009-08-03)
[2484] Finding text and what surrounds it - contextual grep - (2009-10-30)
[2638] Finding what has changed - Linux / Unix - (2010-02-17)
[3446] Awk v Perl - (2011-09-18)
[3902] Shell - Grep - Sed - Awk - Perl - Python - which to use when? - (2012-10-22)
[4586] Extending your bash shell with aliases, functions and extra commands - (2015-11-28)
[4682] One line scripts - Awk, Perl and Ruby - (2016-05-20)
Some other Articles
Melksham Chamber of Commerce - looking to our future shape. Pivotal meeting next TuesdayHow well do you know Melksham?Python timing - when to use a list, and when to use a generatorChristmas in June? Melksham hotel bookings and Santa train Shell, Awk, Perl of Python?Spike solutions and refactoring - a Python exampleLearning to program - the if statement. Python.Melksham - placed 2254 out of 2255. What can be done about it?Why you should use objects even for short data manipulation programs in RubyThe five oldest blogs and the horses mouth