'how to read and write file faster?' Thread RSS Feed

how to read and write file faster?

Thu, 17 Mar 2011 05:55:14 -0700

how to read and write file on the disk faster?

This post has been deleted.

Mon, 21 Mar 2011 21:16:31 -0700

This post has been deleted.

Re: how to read and write file faster?

Tue, 22 Mar 2011 17:15:05 -0700

I want to use multi-thread,but I have some questions.
1. If read one large file(500G) with multi-thread,can it accelerate the speed?
2.If read many files and each thread read one,can it accelerate the speed?
3.If the multi-thread program would accelerate the speed,Must it run on multi-CPU? what about on multi-kernel?

This post has been deleted.

Tue, 22 Mar 2011 18:17:43 -0700

This post has been deleted.

Re: how to read and write file faster?

Thu, 24 Mar 2011 17:02:30 -0700

: I want to use multi-thread,but I have some questions.
: 1. If read one large file(500G) with multi-thread,can it accelerate
: the speed?

Assuming that you are working with a multi-core CPU and lock the critical location of what is being read, but if you are reading one file in a concurrent manner are you really able to process the file? Unless the file is created to have X size bytes read concurrently and processed reading a single file on multiple threads might be pointless.

: 2.If read many files and each thread read one,can it accelerate the
: speed?

Yes, reading one file per thread could speed up the processing of the files. Given either a single-core or multi-core threading system.

: 3.If the multi-thread program would accelerate the speed,Must it run
: on multi-CPU? what about on multi-kernel?

Multi-threading on a single core uses the concept of time-slicing to fake multiple-threads, so it can potentially speed up the program depends on what's going on.

Multi-kernal - I don't know how to answer that question.

Re: how to read and write file faster?

Sat, 26 Mar 2011 05:06:16 -0700

When dealing with a single hard drive - multithreading may be slower. No matter how many CPU or cores you have - a single physical drive may cause excessive wear on the drive and slow processing.

Consider this: two threads read two files, separated physically by some amount of sectors. The hard drive heads must move to the 1st file and read the part of it, then next thread comes and heads move to 2nd file sectors, then process repeats - probably a lot of times - depends on a size of the part of the file being loaded. Since there is only one set of drive heads and it cannot be really shared at run-time (like CPU cores) - some thread always will wait until other thread is moving these heads.

This post has been deleted.

Mon, 28 Mar 2011 12:20:07 -0700

This post has been deleted.

Re: how to read and write file faster?

Tue, 29 Mar 2011 01:43:38 -0700

thk u very much.

This post has been deleted.

Tue, 19 Apr 2011 06:25:29 -0700

This post has been deleted.