I have a program (originally written by Graham), which processes a text file, extracts data and writes it to a mySQL db. The text file can get quite big - the file with real data has  2M lines in it. My program runs fine on short examples, but once the text file gets > 5000 lines I get the above error message. Not a timing problem, as I set the time limit. I have a progress counter which prints (using print) a counter every 100 lines, and I output diagnostics to a file. As I say, it all works fine on 5k lines, but dies at 7k, though the file output indicates it reached the end of input successfully. However, if I alter the diagnostic to print the counter every  line it reads, it works OK, albeit too slowly to be much use, for up to 200k lines.
Any ideas what's going wrong?


I think it probably IS some sort of timing problem - caused by your browser timing out.  The way to test this is to run the thing as a standalone PHP program (and I suspect it will work), and to also test it by running it from a telnet client rather than from a regular browser (which I suspect may also work).   You might also want to change the timeout of your web server .. that's the Timeout line in your httpd.conf file; if stand alone works, but telnet fails, that's probably your problem.

---