Python dictionaries - reaching to new uses

If it sounds easy, it should be easy.

"Go through a file and echo the lines ... but if a line's duplicated, echo it only once". This was a sample application requested on today's Python course ... but one which has been troubling the folks for a while as they tried to filter the data in other languages.

The solution? Set up a dictionary, and as each line is read pretend that the whole line is a key ... and see if the key exists (i.e. if the line occurred previously). If it didn't, then output the line and add a member to the dictionary to indicate that you've done so - keyed on the line, and with a value of "1" which we're not really interested in - we're just looking at the presence or absence of a member. Source code [here] and it's very short!

Road and Rail bridges from Devon to Cornwall

Road and Rail bridges from Devon to Cornwall

Such a use of dictionaries has provided a big leap forward for this application. What was previously a nighmare task is now just a few lines of code, and the hashing algorithm used by the dictionary means that it's going to be efficient even as the data set grows.

Royal Albert Bridge, Saltash

But there are lots of other ways that dictionaries can be used in this "unique line" application - just as there are lots of ways to get across the water once you find the narrowest and easiest crossing point ...

[here] is the application expanded to give a count of the number of times each line occurs, sorted by the number of occurrences.

[here] is an exended report, telling you all the line numbers on which a particular string occurs (makes up the line)

[here] is the same report, but sorted by the number of occurrences of each line - so you have all the unique lines together, for example.

Oh - the sample data is [here].

Further pictures at St Budeaux / the Saltash ferry ...

The old Saltash Ferry builing at St Budeaux

The old Saltash Ferry builing at St Budeaux

Train from Gunnislake passes under Tamar Bridges

(written 2010-10-05) 2259

Associated topics are indexed under

Y107 - Python - Dictionaries
  [4029] Exception, Lambda, Generator, Slice, Dict - examples in one Python program - (2013-03-04)
  [4027] Collections in Python - list tuple dict and string. - (2013-03-04)
  [3934] Multiple identical keys in a Python dict - yes, you can! - (2012-11-24)
  [3662] Finding all the unique lines in a file, using Python or Perl - (2012-03-20)
  [3555] Football league tables - under old and new point system. Python program. - (2011-12-18)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3488] Python sets and frozensets - what are they? - (2011-10-20)
  [3464] Passing optional and named parameters to python methods - (2011-10-04)
  [2994] Python - some common questions answered in code examples - (2010-10-10)
  [2915] Looking up a value by key - associative arrays / Hashes / Dictionaries - (2010-08-11)
  [2368] Python - fresh examples of all the fundamentals - (2009-08-20)
  [1145] Using a list of keys and a list of values to make a dictionary in Python - zip - (2007-04-13)
  [1144] Python dictionary for quick look ups - (2007-04-12)
  [955] Python collections - mutable and imutable - (2006-11-29)
  [103] Can't resist writing about Python - (2004-10-29)

Back to
Right place, right season

Previous and next
or
Horse's mouth home

Forward to
Memorial to a day in 1999

Some other Articles

What are numpy and scipy?
Oddballs in Plymouth
Not mugged in London!
Memorial to a day in 1999
Python dictionaries - reaching to new uses
Right place, right season
Customer Review - Hotel Room, Melksham
Strong arm tactics in the bus industry? Poor result for the customer!
Travel, Transport and Tourism - A vision for Melksham, 2026
How to set up short and meaningfull alternative URLs

4090 posts, page by page

Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).