Why Small Files Take Longer to Copy Than Large Files

Have you ever noticed that it takes longer to copy 200MB of small text files than it does to copy a 200MB video file. This can seem a bit strange, but there is a simple reason for it.

Metadata

Every time you copy a file, the system must also copy over some metadata. Things like the filename, creation date, modification date, filesize etc. When you copy a large file like a video, this information is copied once, and then all the data blocks are copied into place. With tiny text files, new metadata needs to be transferred for each and every file.

In real-world examples, I’ve seen disks capabale of copying at 100MB/s get as low as 1MB/s when copying small files.

This is easier to see in simple animations. In the images beow, the metadata is represented by the blue container and red block. The data blocks are shown in green.

Copy Large Video Files
Copy Large Video Files

 

Copy Tiny Text Files
Copy Tiny Text Files

This behaviour also leads to one of the most annoying things about copying files. The crazy time-estimates! If you copy a mix of large and small files, the computer can’t figure out how long the copy will take, so just adjusts the estimate as it goes along.

Data Recovery Time Remaining

This post assumes you’re copying the files in ideal conditions. In the real world you also need to account for slow networks, slow connections, failing disks and countless other things that can slow down a transfer.

This is the first post in a mini-series about copying files. More coming soon.

Leave a Reply

Your email address will not be published. Required fields are marked *