Disk defragmentation, explained.

What is disk fragmentation?

A file system doesn’t store a file byte by byte. It hands out space in fixed-size chunks called clusters (or allocation units, or blocks): the smallest amount of disk space a file can occupy. A 10 KB file on a disk with 4 KB clusters takes three of them.

Ideally a file’s clusters sit next to each other. When they don’t, and the file is stored in several separate runs spread across the disk, the file is fragmented.

It happens on its own, as a disk is used:

• Deleting leaves holes. Remove a file and its clusters become a gap. The next file to arrive may be bigger than the gap, so it is split: part in the gap, the rest wherever there is room.
• Files grow. Add a page to a document or a day to a log file, and the new clusters often can’t go right after the old ones, because another file is already there.
• The file system goes for speed, not tidiness. FAT, the file system of MS-DOS and Windows 95 and 98, typically carries on from the last cluster it handed out and takes the next free one. It is quick, and over time it scatters files.

There are really two kinds. File fragmentation is a file broken into pieces. Free-space fragmentation is empty space broken into small gaps, and it is the cause of the first: when no gap is big enough, every new file arrives in pieces.

Why fragmentation slowed hard disks down

A hard disk is a stack of spinning platters with a read/write head on a swinging arm. Before it can read anything, the arm has to move the head to the right track (the seek), and then wait for the right spot on the platter to come round under it (the rotational latency). Only then does data flow, and it flows fast as long as it is in one continuous run.

Every extra fragment means another seek and another wait. Engineers at Microsoft called these two delays “the two major culprits”. A file in 50 pieces costs 50 of them instead of one.

A few milliseconds sound like nothing, but in that time the same disk could have read tens or even hundreds of kilobytes in a straight line. That is why a fragmented disk felt sluggish: programs took longer to start, and the disk chattered audibly as the head jumped back and forth.

Where data sits matters too. With zoned bit recording, outer tracks hold more sectors than inner ones, so the start of a disk (the outer edge) is the fastest part. Good defragmenters used that.

What a defragmenter actually does

A defragmenter is a careful mover. For each fragmented file it finds a free run big enough, copies the pieces there in order, checks the copy, points the file system at the new location and only then frees the old clusters. If anything goes wrong midway, the original is still where it was.

Along the way a good defragmenter also:

• Consolidates free space into one large area, so new files have room to arrive in one piece.
• Places files deliberately: system and frequently used files near the fast start of the disk, rarely used ones further away. Windows 98 went as far as ordering program files the way programs actually loaded them.
• Leaves some things alone. In Windows 2000, files the system kept open, such as the paging file and the NTFS Master File Table, couldn’t be moved while Windows was running. Some tools handled them in a special pass at boot time.

Defragmenting doesn’t delete anything, and it doesn’t make a disk bigger. It only moves data so the disk can read it with fewer trips of the head.

The Windows 95 and 98 Disk Defragmenter

For many people this is the defragmenter: a window full of small squares, slowly changing colour as the program worked its way across Drive C. It came with Windows 95 and, like the DEFRAG command in MS-DOS 6, it was made by Symantec, the company behind Norton Utilities and its Speed Disk.

Clicking Show Details opened the famous map. Its legend explained every square:

• Optimized (defragmented) dataFiles that are finished: in one piece and in their place.
• Unoptimized dataData still to be moved, labelled by where it belongs: at the beginning, in the middle or at the end of the drive.
• Free spaceEmpty clusters, waiting to be filled.
• Data that will not be movedFiles the defragmenter must leave exactly where they are.
• Bad (damaged) area of the diskClusters marked as unusable. Windows drew them as a white box with a red line through it.
• Data that’s currently being readThe blocks the head is picking up right now.
• Data that’s currently being writtenWhere those blocks are being put down.

At the bottom sat the line that explains the whole picture: “Each box represents one disk cluster.” The colours here are the ones in Defragment’s Classic 98 theme.

Windows 98 added a checkbox, “Rearrange program files so my programs start faster”. Behind it was the Intel Application Launch Accelerator: a small program called Task Monitor noted which parts of which files each program read as it started, and the defragmenter laid those parts out in exactly that order. Intel’s own test in 1997 had Office programs starting in about 3.7 seconds instead of 11.3.

It also explains a famous annoyance. If another program wrote to the disk while the defragmenter was working (a screen saver was a typical culprit), the map it had built was out of date, and it started over. Many people remember a defrag that never seemed to get past 10%.

A short history of defragmentation

1. 1986

   Diskeeper and DiskExpress

   Executive Software releases Diskeeper for VAX/VMS minicomputers, and Alsoft ships DiskExpress for the Mac.

2. 1987

   Norton Speed Disk

   Speed Disk arrives in Norton Utilities 4.0 Advanced Edition and becomes one of the best-known defragmenters of the DOS era.

3. 1993

   MS-DOS 6.0: DEFRAG

   DOS gets its own DEFRAG command: a slimmed-down Speed Disk, licensed from Symantec.

4. 1995

   Windows 95

   Disk Defragmenter, again Symantec code, brings the cluster map to millions of screens.

5. 1996

   Windows NT 4.0

   No defragmenter in the box, but new system functions let tools such as Diskeeper move clusters safely while Windows runs.

6. 1998

   Windows 98

   Defrag learns to arrange program files in the order they load, with Intel’s Application Launch Accelerator.

7. 2000

   Windows 2000 and XP

   A defragmenter built with Executive Software, the makers of Diskeeper. Its bars show fragmented files in red, contiguous files in blue, unmovable files in green and free space in white.

8. 2003

   Mac OS X 10.3 Panther

   The Mac file system starts defragmenting small, badly fragmented files by itself as they are opened.

9. 2007

   Windows Vista

   Defragmenting moves to a schedule in the background, and the map disappears.

10. 2009

    Windows 7

    Progress is visible again, several drives can be done at once, and SSDs are left out of defragmenting.

11. 2012

    Windows 8: Optimize Drives

    The tool is renamed and learns to treat SSDs differently: instead of moving data it sends them TRIM.

12. 2017

    APFS

    Apple’s new file system is designed for flash storage. On an SSD, defragmenting stops being a question at all.

Defragmenting in Windows 10 and 11

Windows still does it, you just don’t see it. The tool is now called Optimize Drives (search for “Defragment and Optimize Drives” in the Start menu), and by default it runs once a week on its own.

• On a hard disk it analyses the drive and defragments it.
• On an SSD it sends a retrim instead: it tells the drive which blocks are no longer in use.
• On a removable USB stick or SD card formatted with FAT it does nothing.

There is one exception that surprises people. If System Restore or other volume snapshots are on, Windows will also defragment an SSD, about once a month. Microsoft’s storage team explained that a heavily fragmented volume slows down snapshots and can hit a file-system limit on how many fragments a file may have.

For the command-line minded, defrag C: /A analyses a drive, defrag C: /O runs the right optimisation for its type, and defrag C: /L retrims an SSD. A full defrag wants at least 15% of the drive free.

Should you defragment an SSD?

No. An SSD has no head and no spinning platter, so there is no seek and no waiting. Its controller keeps its own map from the addresses the computer uses to the flash cells where the data really is, and it spreads writes around the chip deliberately. A file that looks fragmented to Windows may be anywhere inside the SSD, and it reads just as fast either way.

Defragmenting would only rewrite data that is fine where it is, and flash memory tolerates a limited number of writes. Samsung, one of the largest SSD makers, says plainly that defragmentation is not required, and recommends turning automatic defragmentation off for its drives.

What an SSD does need is TRIM: the operating system telling it which blocks are free, so the drive can clear them in advance and keep writing quickly. Windows and macOS do it automatically, and most Linux distributions run it on a timer.

Do Macs need defragmenting?

Almost never. In the classic Mac OS era people used tools such as Alsoft’s DiskExpress and PlusOptimizer, Norton Utilities’ Speed Disk or TechTool Pro. Since Mac OS X, the system has taken care of most of it itself.

• Mac OS X 10.2 added delayed allocation: the file system waits before choosing where to put new data, so it can find one contiguous place.
• Mac OS X 10.3 Panther added automatic defragmentation. When a file under 20 MB that is split into more than eight pieces is opened, it is quietly moved into one piece.
• Hot File Clustering gathers the startup disk’s most frequently read small files into a reserved “hot” area of the disk.

Apple’s own advice back then was that with Mac OS X you probably wouldn’t need to optimise at all. Today’s Macs use APFS on SSDs, where fragmentation doesn’t matter. For APFS on a spinning hard disk, macOS has a switch: diskutil apfs defragment followed by the volume and status, enable or disable.

Defragmentation on Linux

Linux file systems are designed to avoid fragmentation rather than repair it, so a defrag is rarely needed.

• ext4 keeps files together with extents (one record for a long run of blocks), delayed allocation and by keeping a file’s data near its metadata. For the odd case there is e4defrag.
• XFS comes with xfs_fsr, which reorganises the most fragmented files, by default for up to two hours per run.
• Btrfs has btrfs filesystem defragment and the autodefrag mount option, which is off by default. Careful with snapshots: defragmenting breaks the sharing between a file and its snapshot copies, and the used space can grow.

Why people loved watching it

Somewhere between the chatter of the disk and the squares turning from light to dark, defragmenting became something to watch. A slow, orderly process with a visible end, where every step makes things a little tidier, is oddly calming.

People still seek it out. In 2018 TechCrunch wrote about a Twitch channel that did nothing but defragment a DOS disk around the clock, and hours-long recordings of Windows 98 defragmenting, hard-disk noises and all, are still being watched on YouTube.

That is the whole idea behind Defragment: the calm of watching a disk tidy up, without a real disk that needs it. Your iPhone or iPad shows a pretend disk that is defragmented forever, in six themes including Classic 98, with the time on the side if you like.

Questions people ask