opening a file in perl

[Anthony Wesley]

Nemo - earth native wrote:
> 
> Hi guys, a quick question for the perl guru's around...
> 
> I've a small script which opens a file, reads through it looking for
> various information, uses that to calculate some figures, and closes.
> 
> Pretty simple stuff.
> 
> ...except when the file being opened is the better part of 400meg,
> and so I get Out of Memory errors - the perl seems to wants to read the
> whole file into memory before looking at it at all.
> 
> At the moment is opens the file as such:
>         open( IN, "chunk.txt" ) or die "Hrm. Couldn't open the file./n";
> 
> And then just runs through line by line:
>         foreach my $line ( <IN> )
>         {
> (insert a few possible regexp matches in `if` statements here. simple stuff)
>         }
> 
> So, how do you get perl to not load the whole file into memory?
> 
> .../Nemo


Don't use the (<IN>) construct.

That requests the file be read into memory (in its entirety) and then split into an array of lines. As you can imagine, this may
be a problem on 400Mb files :-)

Instead, do something like this:

while( $line = <IN>) {
   chomp $line;
   do other stuff...

   }


which just processes one line at a time.

cheers, Anthony