Finding the latest log file and analysing it (popen, ls)

Input / Output example from a Well House Consultants training course
More on Input / Output [link]

This example is described in the following article(s):
• New Lua Examples - for last weeks delegates - [link]

This example references the following resources:
http://owned-nets.blogspot.com/

Source code: filescan Module: U110

--[[
This example looks though all the files in our log file directory
and finds the most recent log files (our log files are automatically
named ac_ followed by the date on which they were created
]]

-- Use the operating system's commands to parse a directory

stream = io.popen("ls ../data","r")
while true do
        line = stream:read()
        if not line then break end
        year, month, day = string.match(line,"ac_(%d%d%d%d)(%d%d)(%d%d)")
        if year then
                -- print (day .. "/" .. month .."/" .. year .."\n")
                lastest = line
        end
end

-- Parse the last data file that matched

fh = io.open("../data/" .. lastest)
problem = {}
while true do
        line = fh:read()
        if not line then break end

-- here is a sample data line (some line breaks added for clarity)

-- 71.205.187.35 - - [27/Jul/2009:03:30:27 +0100]
-- "GET /resources/ex.php4?item=http://owned-nets.blogspot.com/
-- 2009/05/pro0f3th1sddbluelinebe.html? HTTP/1.1" 200 12555 "-"
-- "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9)
-- Gecko/2009040821 Firefox/3.0.9"

-- Extract the needed fields from the access log line.

-- note the use of the ".-" count in pattern matching - the
-- equivalent of ".*?" in regular expressions, meaning that
-- we are matching as few characters as possible, rather than
-- as many as possible. So we are moving up to the NEXT double
-- quote character, and not to the LAST double quote, in this next line

        ip, request, status, size =
                string.match(line, "^(%S+).-\"(.-)\"%s+(%S+)%s+(%S+)")
        statuscode = tonumber(status)

-- trap non-numeric value in status code first

-- the lazy and operator allows this to be done as a single
-- test, with no worry that the program will crash if the
-- status code wasn't a number

        if statuscode and statuscode >= 400 then
                print (ip, status, size)
                problem[#problem+1] = {ip, status, size}
        end
        end

Learn about this subject

This module and example are covered on the following public courses:
* Learning to program in Lua
* Lua Programming
* Lua Programming
Also available on on site courses for larger groups

Books covering this topic

Yes. We have over 700 books in our library. Books covering Lua are listed here and when you've selected a relevant book we'll link you on to Amazon to order.

Other Examples

This example comes from our "Input / Output" training module. You'll find a description of the topic and some other closely related examples on the "Input / Output" module index page.

Full description of the source code

You can learn more about this example on the training courses listed on this page, on which you'll be given a full set of training notes.

Many other training modules are available for download (for limited use) from our download centre under an Open Training Notes License.

Other resources

• Our Solutions centre provides a number of longer technical articles.
• Our Opentalk forum archive provides a question and answer centre.
• The Horse's mouth provides a daily tip or thought.
• Further resources are available via the resources centre.
• All of these resources can be searched through through our search engine
• And there's a global index here.

Purpose of this website

This is a sample program, class demonstration or answer from a training course. It's main purpose is to provide an after-course service to customers who have attended our public private or on site courses, but the examples are made generally available under conditions described below.

Web site author

This web site is written and maintained by Well House Consultants.

Conditions of use

Past attendees on our training courses are welcome to use individual examples in the course of their programming, but must check the examples they use to ensure that they are suitable for their job. Remember that some of our examples show you how not to do things - check in your notes. Well House Consultants take no responsibility for the suitability of these example programs to customer's needs.

This program is copyright Well House Consultants Ltd. You are forbidden from using it for running your own training courses without our prior written permission. See our page on courseware provision for more details.

Any of our images within this code may NOT be reused on a public URL without our prior permission. For Bona Fide personal use, we will often grant you permission provided that you provide a link back. Commercial use on a website will incur a license fee for each image used - details on request.