Have you ever noticed that it takes longer to copy 200MB of small text files than it does to copy a 200MB video file. This can seem a bit strange, but there is a simple reason for it.
Metadata
Every time you copy a file, the system must also copy over some metadata. Things like the filename, creation date, modification date, filesize etc. When you copy a large file like a video, this information is copied once, and then all the data blocks are copied into place. With tiny text files, new metadata needs to be transferred for each and every file.
In real-world examples, I’ve seen disks capabale of copying at 100MB/s get as low as 1MB/s when copying small files.
This is easier to see in simple animations. In the images below, the metadata is represented by the blue container and red block. The data blocks are shown in green.
This behaviour also leads to one of the most annoying things about copying files. The crazy time estimates! If you copy a mix of large and small files, the computer can’t figure out how long the copy will take, so just adjusts the estimate as it goes along.
This post assumes you’re copying the files in ideal conditions. In the real world you also need to account for slow networks, slow connections, failing disks and countless other things that can slow down a transfer.
This is the first post in a mini-series about copying files.
Read Part Two here about why computers always guess the wrong copy time.
Leave a Reply