File allocation Extents/Bitmap | ||
![]() | ||
Developer(s) Mingming Cao, Andreas Dilger, Alex Zhuravlev (Tomas), Dave Kleikamp, Theodore Ts'o, Eric Sandeen, Sam Naghshineh, others Full name Fourth extended file system Introduced Stable: 21 October 2008
Unstable: 10 October 2006 with Linux 2.6.28, 2.6.19 Partition identifier 0x83: MBR/EBR.
EBD0A0A2-B9E5-4433-87C0-68B6B72699C7: GPT Windows BDP.
0FC63DAF-8483-4772-8E79-3D69D8477DE4: GPT Linux filesystem data.
933AC7E1-2EB4-4F13-B844-0E14E2AEF915: GPT /home partition.
3B8F8425-20E0-4F3B-907F-1A25A76F98E8: GPT /srv (server data) partition. Directory contents Linked list, hashed B-tree |
The ext4 or fourth extended filesystem is a journaling file system for Linux, developed as the successor to ext3.
Contents
History
ext4 was born as a series of backward-compatible extensions to ext3, many of them originally developed by Cluster File Systems for the Lustre file system between 2003 and 2006, meant to extend storage limits and add other performance improvements. However, other Linux kernel developers opposed accepting extensions to ext3 for stability reasons, and proposed to fork the source code of ext3, rename it as ext4, and perform all the development there, without affecting the current ext3 users. This proposal was accepted, and on 28 June 2006, Theodore Ts'o, the ext3 maintainer, announced the new plan of development for ext4.
A preliminary development version of ext4 was included in version 2.6.19 of the Linux kernel. On 11 October 2008, the patches that mark ext4 as stable code were merged in the Linux 2.6.28 source code repositories, denoting the end of the development phase and recommending ext4 adoption. Kernel 2.6.28, containing the ext4 filesystem, was finally released on 25 December 2008. On 15 January 2010, Google announced that it would upgrade its storage infrastructure from ext2 to ext4. On 14 December 2010, they also announced they would use ext4, instead of YAFFS, on Android 2.3.
Features
fallocate()
, a new system call in the Linux kernel, can be used. The allocated space would be guaranteed and likely contiguous. This situation has applications for media streaming and databases.
Limitations
In 2008, the principal developer of the ext3 and ext4 file systems, Theodore Ts'o, stated that although ext4 has improved features, it is not a major advance, it uses old technology, and is a stop-gap. Ts'o believes that Btrfs is the better direction because "it offers improvements in scalability, reliability, and ease of management". Btrfs also has "a number of the same design ideas that reiser3/4 had".
The ext4 file system does not honor the "secure deletion" file attribute, which is supposed to cause overwriting of files upon deletion. A patch to implement secure deletion was proposed in 2011, but did not solve the problem of sensitive data ending up in the file system journal.
Delayed allocation and potential data loss
Because delayed allocation changes the behavior that programmers have been relying on with ext3, the feature poses some additional risk of data loss in cases where the system crashes or loses power before all of the data has been written to disk. Due to this, ext4 in kernel versions 2.6.30 and later automatically handles these cases as ext3 does.
The typical scenario in which this might occur is a program replacing the contents of a file without forcing a write to the disk with fsync. There are two common ways of replacing the contents of a file on Unix systems:
fd=open("file", O_TRUNC); write(fd, data); close(fd);
- If the write does not succeed (which may be due to error conditions in the writing program, or due to external conditions such as a full disk), then both the original version and the new version of the file will be lost, and the file may be corrupted because only a part of it has been written.
- If other processes access the file while it is being written, they see a corrupted version.
- If other processes have the file open and do not expect its contents to change, those processes may crash. One notable example is a shared library file which is mapped into running programs.
fd=open("file.new"); write(fd, data); close(fd); rename("file.new", "file");
Using fsync more often to reduce the risk for ext4 could lead to performance penalties on ext3 filesystems mounted with the data=ordered
flag (the default on most Linux distributions). Given that both file systems will be in use for some time, this complicates matters for end-user application developers. In response, ext4 in Linux kernels 2.6.30 and newer detect the occurrence of these common cases and force the files to be allocated immediately. For a small cost in performance, this provides semantics similar to ext3 ordered mode and increases the chance that either version of the file will survive the crash. This new behavior is enabled by default, but can be disabled with the "noauto_da_alloc" mount option.
The new patches have become part of the mainline kernel 2.6.30, but various distributions chose to backport them to 2.6.28 or 2.6.29. For instance, Ubuntu made them part of the 2.6.28 kernel in version 9.04 ("Jaunty Jackalope").
These patches don't completely prevent potential data loss or help at all with new files. The only way to be safe is to write and use software that does fsync when it needs to. Performance problems can be minimized by limiting crucial disk writes that need fsync to occur less frequently.
Implementation
Linux kernel Virtual File System is a subsystem or layer inside of the Linux kernel. It is the result of the very serious attempt to integrate multiple file systems into an orderly single structure. The key idea, which dates back to the pioneering work done by Sun Microsystems employees in 1986, is to abstract out that part of the file system that is common to all file systems and put that code in a separate layer that calls the underlying concrete file systems to actually manage the data.
» All system calls related to files (or pseudo files!) are directed to the Linux kernel Virtual File System for initial processing. These calls, coming from user processes, are the standard POSIX calls, such as open
, read
, write
, lseek
, … .«
Compatibility with Windows and Macintosh
ext4 does not yet have as much support as ext2 and ext3 on non-Linux operating systems. ext2 and ext3 have stable drivers such as Ext2IFS, which are not yet available for ext4. It is possible to create compatible ext4 filesystems by disabling the extents feature, and sometimes specifying an inode size. Another option for using ext4 in Windows is to use Ext2Fsd, an open-source driver that supports writing in ext4 partitions with limited journaling. Viewing and copying files from ext4 to Windows, even with extents enabled, is also possible with the Ext2Read software. More recently Paragon released its ExtFS for Windows which allows read/write capabilities for ext2/3/4.
macOS has full ext2/3/4 read/write capability through the Paragon ExtFS software, which is a commercial product. Free software such as ext4fuse has only read-only support with limited functionality.