I want to know what is the best way to read text files that are 500 megs in size. each file contains more or less 500,000 rows. I need to read this file line by line to extract data from it.

do I load the file to a string list?

do i read the file using end of file and end of line while loops?

or

is there a better way to ready quickly and efficiently with out using so much memroy?

thank you very much.

: I want to know what is the best way to read text files that are 500 megs in size. each file contains more or less 500,000 rows. I need to read this file line by line to extract data from it.
:
: do I load the file to a string list?
:
: do i read the file using end of file and end of line while loops?
:
: or
:
: is there a better way to ready quickly and efficiently with out using so much memroy?
:
: thank you very much.
:
I would use a while-not-eof() loop with a readln() for this. This way you have a memory efficient program, but there is a downside: this only works if you want to read the lines sequentially. If you want to "jump around" then a TStrings object is better. The eoln() function is not necessary, since it is better to read each line individually instead of each character individually. If you want to access the file on a per character basis, then a TFileStream is better.

reading line by line would take hours with 500 mb file.

as larger the blocks you read faster it will be.
read it as larger blocks as you can - try 5 mb block.

check the filesize and then decrement the filesize by 50 mb each time.

after that you will need to seperate the 50 mb data to lines by your self - more work but much faster.

dolev

: : I want to know what is the best way to read text files that are 500 megs in size. each file contains more or less 500,000 rows. I need to read this file line by line to extract data from it.
: :
: : do I load the file to a string list?
: :
: : do i read the file using end of file and end of line while loops?
: :
: : or
: :
: : is there a better way to ready quickly and efficiently with out using so much memroy?
: :
: : thank you very much.
: :
: I would use a while-not-eof() loop with a readln() for this. This way you have a memory efficient program, but there is a downside: this only works if you want to read the lines sequentially. If you want to "jump around" then a TStrings object is better. The eoln() function is not necessary, since it is better to read each line individually instead of each character individually. If you want to access the file on a per character basis, then a TFileStream is better.
:
:::
Thank you very much i really appreciate you help thanks again