File systems and Storage
File systems and storage is used more in daily life than you could imagine. Perhaps you have saved songs on an MP3, used a USB or played music from a CD. In this crash course I will go through file systems, the anatomy of hard drives, the read/write process, fragmentation and formatting, the booting process in BIOS and the differences between the file systems NTFS and FAT.
A file system is a process/ability to store and read information. It provides a way of separating the data into individual pieces. Imagine a physical archive with categorized shelves which divide books into which genres they are for instance. This way it makes it easier for the system and the user to have a structure of the files and how to save or retrieve them. Without file systems, we wouldn’t be able to know where our files were located. It would be like a messy room without any structure.
There are many different types of file systems. Each one has different structure; speed, security, and other attributes. However, they all have one thing in common; Metadata. Metadata is an additional description of a file like a date and time it was created, the date it was modified or the file size of it.
Anatomy of a hard drive
The hard drive is a storage media device that stores data in a specific pattern. It is composed of platters that are stacked on top of each other to form a cylinder. Between each platter, arms go along to convert signals/magnetism into information onto it.
- A cylinder is a stack of platters lined up one on top of another to form a cylinder shape.
- Each platter is broken into thousands of tightly packed concentric circles, known as tracks. You can imagine these as small vinyl records tracks. Starting from the very outside of the platter, you have track number 0.
- Each track is further broken down into smaller units called sectors. They store a fixed amount of any data which is traditionally 512 bytes. They are the minimum storage unit of a hard drive that you can logically read or write. These are also called blocks.
- A cluster is also called a file allocation unit but is a logical group defined by a file system. It is the smallest unit of space used for storing files. It is made up of one or more sectors, making it larger.
Disk- read/write heads are the small parts of a disk drive which move above the disk platter and transform the platter’s magnetic field into electrical current (read the disk) or, vice versa, transform electrical current into a magnetic field (write the disk).
In order to understand the magnetism process itself to dig deeper into the subject, you need an understanding of binary signals. This is because everything is broken down to 0s and 1s where the input/output is either on or off.
In a hard drive, the read/write heads ‘fly’ above the disk surface with a clearance of as little as 3 nanometers. This enables great risk of a catastrophic head crash in the head hits the disk’s surface. So when you hear weird sounds or buzzing coming from your hard drive, it’s time to back up your data in case something is happening.
When a file system is first initialized on a partition, it only contains a few structures and is a block of empty space. As existing files are created and then deleted, new regions of free space are created. When new files are added, it’s almost impossible to recreate the same space the last file had. This means that the system has to break other places down into fragments to fit the new files into space.
It’s easier to imagine an empty room. Each time you buy something from a store you put it in a box and place it in the room. If you throw the items away they will be deleted from the box, however, the box is still there. After a while, you will end up having old boxes in all sizes located all over the room and you will have to resize them or merge small boxes together to fit bigger things.
Disk formatting is the process of preparing a data storage device for usage. It can be performed on both magnetic platter hard drives and solid-state drives (HDDs and SSDs). It is usually done before the installation of a new operating system as it also offers to create a file system during the process. Any existing files and data would be erased with formatting.
- The first part of the formatting process that performs basic medium preparation is often referred to as “low-level formatting”. It marks the surfaces of the disks with markers indicating the start of a recording block/sector.
- Partitioning is the common term for the second part of the process, making the data storage device visible to an operating system. It divides a disk into one or more regions, writing data structures to the disk to indicate the beginning and end of the regions. This level of formatting often includes checking for defective tracks or defective sectors.
- The third part of the process, usually termed “high-level formatting” most often refers to the process of generating a new file system. Formatting creates the file system format within a disk partition or a logical volume. This formatting includes the data structures used by the OS to identify the logical drive or partition’s contents. This may occur during operating system installation, or when adding a new disk.
Not to be confused with wiping which destroys all data on a disk without creating any structures or partitions.
NTFS & FAT
FAT (File Allocation Table)
An acronym for File Allocation Table, it’s the oldest and the most experienced file system in the history of computing. The FAT32 file system is perfect for storage devices like flash drives but you’ll have to make sure you don’t have an individual file larger than 4GB. In the earlier years, the FAT file system advanced to FAT12, FAT16, and finally, FAT32 that has been a synonym with the word file system when we have to deal external storage media like removable drives (USBs etc.).
NTFS (New Technology File System)
NTFS has several technical improvements over the file systems that it superseded – File Allocation Table (FAT) and High-Performance File System (HPFS) – such as improved support for metadata and advanced data structures to improve performance, reliability, and disk space use.
Other notable features include reparse points, sparse file support, disk usage quotas, distributed link tracking, and file-level encryption. The NTFS file system supports backward compatibility with the previous versions. It’s a journaling file system which proves to be an important component when it comes to reviving a corrupt file system. It maintains a journal, a data structure which keeps track of any possible modifications to the file system and is used to recover the file system.