[KLUG Members] Best way to use temp files in PHP

[Adam Williams]

>>>internall it holds all this information in an array.  So I've got a
>>>>report with 15 columns of various types, and a heck of alot of
>>rows.
>>>>At about ~40,000 rows I hear "Ahhhhhhhhhhhhhhhhhhhhhhhhhhhh!" as a
>>>>temporal rift occurs to some hell dimension and my PHP process
>>topples
>>>>into it.  So I'm thinking I just can't have arrays that big.
>http://marc.theaimsgroup.com/?l=php-general&m=95685671330868&w=2
>Check your PHP.INI file for the following variable:
>max_execution_time = 30     ; Maximum execution time of each script, in
>seconds
>You might want to try to increase the execution time that the PHP
>engine is allowed to process a script (try doubling the
>max_execution_time). With scripts for parsing large amounts of text, or
>for large database queries, PHP will "time out" before the script has
>finished.

I've already slain the time-out beast.  The process resets it's
time-to-run with every iteration.  PHP (running as a "php" process not
in the context of a web server) actually core bombs,  and as the array
gets larger PHP ***REALLY*** slows down.  A process of a data set of
~7000 rows takes 9 minutes.  ~15000 rows takes 11 minutes.  So some
economy of scale exists - for awhile.  A set of 110,000 rows dumps after
54 minutes. with an immense core file,  so something is going a bit
wrong. And it has 768Mb of physical RAM to play in, and 1Gb of swap.