Linux kernel

History

In April 1991, Linus Torvalds, at the time a 21-year-old computer science student at the University of Helsinki, Finland, started working on some simple ideas for an operating system. He started with a task switcher in Intel 80386 assembly language and a terminal driver. On 25 August 1991, Torvalds posted the following to comp.os.minix, a newsgroup on Usenet:

I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since April, and is starting to get ready. I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

I've currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I'll get something practical within a few months [...] Yes - it's free of any minix code, and it has a multi-threaded fs. It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(.

[...] It's mostly in C, but most people wouldn't call what I write C. It uses every conceivable feature of the 386 I could find, as it was also a project to teach me about the 386. As already mentioned, it uses a MMU, for both paging (not to disk yet) and segmentation. It's the segmentation that makes it REALLY 386 dependent (every task has a 64Mb segment for code & data - max 64 tasks in 4Gb. Anybody who needs more than 64Mb/task - tough cookies). [...] Some of my "C"-files (specifically mm.c) are almost as much assembler as C. [...] Unlike minix, I also happen to LIKE interrupts, so interrupts are handled without trying to hide the reason behind them.

After that, many people contributed code to the project. Early on, the MINIX community contributed code and ideas to the Linux kernel. At the time, the GNU Project had created many of the components required for a free operating system, but its own kernel, GNU Hurd, was incomplete and unavailable. The BSD operating system had not yet freed itself from legal encumbrances. Despite the limited functionality of the early versions, Linux rapidly gained developers and users.

By September 1991, version 0.01 of the Linux kernel was released on the FTP server (ftp.funet.fi) of the Finnish University and Research Network (FUNET). It had 10,239 lines of code. In October 1991, version 0.02 of the Linux kernel was released.

In December 1991, Linux kernel 0.11 was released. This version was the first to be self-hosted as Linux kernel 0.11 could be compiled by a computer running the same kernel version. When Torvalds released version 0.12 in February 1992, he adopted the GNU General Public License (GPL) over his previous self-drafted license, which had not permitted commercial redistribution.

A newsgroup known as alt.os.linux was started, and on 19 January 1992, the first post to alt.os.linux was made. On 31 March 1992, alt.os.linux became comp.os.linux.

The X Window System was soon ported to Linux. In March 1992, Linux version 0.95 was the first to be capable of running X. This large version number jump (from 0.1x to 0.9x) was due to a feeling that a version 1.0 with no major missing pieces was imminent. However, this proved to be somewhat overoptimistic, and from 1993 to early 1994, 15 development versions of version 0.99 appeared.

On 14 March 1994, Linux kernel 1.0.0 was released, with 176,250 lines of code. In March 1995, Linux kernel 1.2.0 was released, with 310,950 lines of code.

Version 2 of the Linux kernel, released on 9 June 1996, was followed by additional major versions under the version 2 header:

Starting in 2004, the release process changed and new kernels started coming out on a regular schedule every 2–3 months, numbered 2.6.0, 2.6.1, up through 2.6.39.

On 21 July 2011, Torvalds announced the release of Linux kernel 3.0: "Gone are the 2.6.<bignum> days". The version bump is not about major technological changes when compared to Linux 2.6.39; it marks the kernel's 20th anniversary. The time-based release process remained the same.

Version 3.10 of the Linux kernel, released in June 2013, contains 15,803,499 lines of code, while the version 4.1, released in June 2015, has grown to over 19.5 million lines of code contributed by almost 14,000 programmers.

Tanenbaum–Torvalds debate

The fact that Linux is a monolithic kernel rather than a microkernel was the topic of a debate between Andrew S. Tanenbaum, the creator of MINIX, and Linus Torvalds. The debate, started in 1992 on the Usenet discussion group comp.os.minix, was about Linux and kernel architecture in general. Tanenbaum argued that microkernels are superior to monolithic kernels and that therefore Linux is obsolete. Unlike traditional monolithic kernels, device drivers in Linux are easily configured as loadable kernel modules and are loaded or unloaded while running the system. This subject was revisited on 9 May 2006, and on 12 May 2006 Tanenbaum wrote a position statement.

Popularity

The huge rise in popularity of the Android operating system, which includes the Linux kernel, has made the kernel the most popular choice for mobile devices, rivaling the installed base of all other operating systems. Including previous years, three billion Android smartphones were estimated to have been sold by the end of 2014.

Some consumer routers also use the Linux kernel, which is also the case for a wide variety of other devices, such as servers, smart TVs, and other non-Android embedded systems. The kernel's use in traditional (non-server) desktop Linux varieties is dominant, but the usage share of Linux itself is low in comparison to other operating systems.

Licensing terms

Initially, Torvalds released Linux under a license which forbade any commercial use. This was changed in version 0.12 to the GNU General Public License (GPL). This license allows distribution and sale of possibly modified and unmodified versions of Linux but requires that all those copies be released under the same license and be accompanied by the complete corresponding source code.

Torvalds has described licensing Linux under the GPL as the "best thing I ever did."

GPL version 3

The Linux kernel is licensed explicitly only under version 2 of the GPL, without offering the licensee the option to choose "any later version", which is a common GPL extension. There was a debate over how easily it could be changed to use later GPL versions such as version 3 and whether this is even desirable. Torvalds himself specifically indicated upon the release of version 2.4.0 that his own code is only under version 2. However, the terms of the GPL state that if no version is specified, then any version may be used, and Alan Cox pointed out that very few other Linux contributors have specified a particular version of the GPL.

In September 2006, a survey of 29 key kernel programmers indicated that 28 preferred GPLv2 to the then-current GPLv3 draft. Torvalds commented, "I think a number of outsiders... believed that I personally was just the odd man out, because I've been so publicly not a huge fan of the GPLv3." This group of high-profile kernel developers, including Linus Torvalds, Greg Kroah-Hartman and Andrew Morton, commented on mass media about their objections to the GPLv3. They referred to clauses regarding DRM/tivoization, patents, "additional restrictions" and warned a Balkanisation of the "Open Source Universe" by the GPLv3. Linus Torvalds, who decided not to adopt the GPLv3 for the Linux kernel, reiterated his criticism even years later.

Loadable kernel modules

It is debated whether loadable kernel modules (LKMs) are to be considered derivative works under copyright law, and thereby fall under the terms of the GPL.

Torvalds has stated his belief that LKMs using only a limited, "public" subset of the kernel interfaces can sometimes be non-derived works, thus allowing some binary-only drivers and other LKMs that are not licensed under the GPL. A very good example for this is the usage of dma_buf by the proprietary Nvidia graphics drivers. dma_buf is a recent kernel feature (like the rest of the kernel, it is licensed under the GPL) that allows multiple GPUs to quickly copy data into each other's framebuffers. One possible use case would be Nvidia Optimus that pairs a fast GPU with an Intel integrated GPU, where the Nvidia GPU writes into the Intel framebuffer when it is active. But, Nvidia cannot use this infrastructure because it uses a technical means to enforce the rule that it can only be used by LKMs that are also GPL. Alan Cox replied on LKML, rejecting a request from one of their engineers to remove this technical enforcement from the API. Not all Linux kernel contributors agree with this interpretation, however, and even Torvalds agrees that many LKMs are clearly derived works, and indeed he writes that "kernel modules ARE derivative 'by default'".

On the other hand, Torvalds has also said that "one gray area in particular is something like a driver that was originally written for another operating system (i.e. clearly not a derived work of Linux in origin). [...] THAT is a gray area, and _that_ is the area where I personally believe that some modules may be considered to not be derived works simply because they weren't designed for Linux and don't depend on any special Linux behaviour." Proprietary graphics drivers, in particular, are heavily discussed. Ultimately, it is likely that such questions can only be resolved by a court.

Firmware binary blobs

One point of licensing controversy is the use of firmware "binary blobs" in Linux kernel to support several hardware devices. These files are under a variety of licenses, out of which many are restrictive and their exact underlying source code is usually unknown.

In 2002, Richard Stallman stated why, in his point of view, such blobs make the Linux kernel partially non-free software, and that distributing Linux kernel "violates the GPL", which requires "complete corresponding source code" to be available. In 2008, Free Software Foundation Latin America started Linux-libre as a project that creates a completely free variant of the Linux kernel without proprietary objects; it is used by certain completely free Linux distributions, such as those endorsed by the Free Software Foundation, while it can also be used on most distributions.

On 15 December 2010, the Debian Project announced that the next Debian stable version "6.0 Squeeze" would come with a kernel "stripped of all non-free firmware bits". This policy was continued to be applied in later stable Debian releases.

Trademark

Linux is a registered trademark of Linus Torvalds in the United States and some other countries. This is the result of an incident in which William Della Croce, Jr., who was not involved in the Linux project, trademarked the name and subsequently demanded royalties for its use. Several Linux backers retained legal counsel and filed suit against Della Croce. The issue was settled in August 1997 when the trademark was assigned to Linus Torvalds.

SCO litigation

In March 2003, the SCO Group (SCO) filed a lawsuit against IBM claiming that IBM had violated copyrights that SCO claimed to hold over the Unix source code, by contributing portions of that code to Linux. Additionally, SCO sent letters to a number of companies warning that their use of Linux without a license from SCO may be a violation of copyright law, and claimed in the press that they would be suing individual Linux users. IBM then promised to defend its Linux customers on their behalf. This controversy has generated lawsuits by SCO against Novell, DaimlerChrysler (partially dismissed in July 2004), and AutoZone, and retaliatory lawsuits by Red Hat and others against SCO.

In early 2007, SCO filed the specific details of the purported copyright infringement. Despite previous claims that SCO was the rightful owner of 1 million lines of code, they specified 326 lines of code, most of which were uncopyrightable. In August 2007, the court in the Novell case ruled that SCO did not actually own the Unix copyrights to begin with, though the Tenth Circuit Court of Appeals ruled in August 2009 that the question of who owned the copyright properly remained for a jury to answer. The jury case was decided on 30 March 2010 in Novell's favour.

Architecture

The Linux kernel is a monolithic kernel, supporting true preemptive multitasking (both in user mode and, since the 2.6 series, in kernel mode), virtual memory, shared libraries, demand loading, shared copy-on-write executables (via KSM), memory management, the Internet protocol suite, and threading.

Device drivers and kernel extensions run in kernel space (ring 0 in many CPU architectures), with full access to the hardware, although some exceptions run in user space, for example filesystems based on FUSE/CUSE, and parts of UIO. The graphics system most people use with Linux does not run within the kernel. Unlike standard monolithic kernels, device drivers are easily configured as modules, and loaded or unloaded while the system is running. Also, unlike standard monolithic kernels, device drivers can be pre-empted under certain conditions; this feature was added to handle hardware interrupts correctly, and to better support symmetric multiprocessing. By choice, the Linux kernel has no binary kernel interface.

The hardware is also incorporated into the file hierarchy. Device drivers interface to user applications via an entry in the /dev or /sys directories. Process information as well is mapped to the file system through the /proc directory.

Programming language

The Linux kernel is written in the version of the C programming language supported by GCC (which has introduced a number of extensions and changes to standard C), together with a number of short sections of code written in the assembly language (in GCC's "AT&T-style" syntax) of the target architecture. Because of the extensions to C it supports, GCC was for a long time the only compiler capable of correctly building the Linux kernel.

Compiler compatibility

GCC is the default compiler for the Linux kernel source. In 2004, Intel claimed to have modified the kernel so that its C compiler was also capable of compiling it. There was another such reported success in 2009, with a modified 2.6.22 version of the kernel.

Since 2010, effort has been underway to build the Linux kernel with Clang, an alternative compiler for the C language; as of 12 April 2014, the official kernel could almost be compiled by Clang. The project dedicated to this effort is named LLVMLinux after the LLVM compiler infrastructure upon which Clang is built. LLVMLinux does not aim to fork either the Linux kernel or the LLVM, therefore it is a meta-project composed of patches that are eventually submitted to the upstream projects. By enabling the Linux kernel to be compiled by Clang that, among other advantages, is known for faster compilation compared with GCC, kernel developers may benefit from a faster workflow due to shorter compilation times.

Interfaces

Conformance to standards is a general policy for the Linux kernel's internals. Another rule is that a kernel component is not accepted into the Linux kernel mainline if there is only proprietary user-space software using that component.

Kernel-to-userspace API

Source code portability ensures that a C program written by conforming to a standard can be successfully compiled and run on any system that also conforms to the same standard. The relevant standards, aiming to achieve source code portability of programs, that the development of the Linux kernel, the GNU C Library, and associated utilities tries to adhere to, are POSIX and the Single UNIX Specification. However, as of February 2014, no Linux distributions are branded as "UNIX" by The Open Group, mainly because of the costs of the conformance testing.

The Linux kernel API of the Linux kernel, representing the kernel's system call interface, is composed of the available system calls.

Kernel-to-userspace ABI

Binary portability shall guarantee that any program once compiled for a given hardware platform, can be run in its compiled form on any other hardware platform that conforms to the standard. Binary portability is an essential requirement for the commercial viability of independent software vendor (ISV) applications built for the operating systems based on the Linux kernel. Binary compatibility is much more demanding than source code portability; as of February 2014, the only standard concerning itself with binary compatibility is the Linux Standard Base (LSB).

In-kernel API

There are a couple of kernel internal APIs utilized between the different subsystems and subsystems of subsystems. Some of them have been kept stable over several releases, others have not. There are no guarantees regarding the in-kernel APIs. Maintainers and contributors are free to augment or change them at any time.

Examples of in-kernel APIs include software frameworks/APIs for the following classes of device drivers:

In-kernel ABI

Some organizations have strongly supported defining and maintaining of a stable in-kernel ABI over several releases. For example, it would benefit hardware manufacturers which release proprietary kernel modules and distribute binary-only software (e.g. device drivers). However, the Linux kernel developers choose not to maintain a stable in-kernel ABI. This allows Linux kernel development to happen much more quickly.

Preemption

The Linux kernel provides preemptive scheduling under certain conditions. Until kernel version 2.4, only user processes were preemptive, i.e., in addition to time quantum expiration, an execution of current process in user mode would be interrupted if higher dynamic priority processes entered TASK_RUNNING state. Toward 2.6 series of the Linux kernel, an ability to interrupt a task executing kernel code was added, although with that not all sections of the kernel code can be preempted.

The Linux kernel contains different scheduler classes. By default the kernel uses a scheduler mechanism called the Completely Fair Scheduler introduced in the 2.6.23 version of the kernel. Internally this default-scheduler class is also known as SCHED_OTHER, but the kernel also contains two POSIX-compliant real-time scheduling classes named SCHED_FIFO (realtime first-in-first-out) and SCHED_RR (realtime round-robin), both of which take precedence over the default class.

Through the use of the real-time Linux kernel patch PREEMPT_RT, support for full preemption of critical sections, interrupt handlers, and "interrupt disable" code sequences can be supported. Partial mainline integration of the real-time Linux kernel patch already brought some functionality to the kernel mainline. Preemption improves latency, increases responsiveness, and makes Linux more suitable for desktop and real-time applications. Older versions of the kernel had a so-called big kernel lock for synchronization across the entire kernel, which was finally removed by Arnd Bergmann in 2011.

Additional scheduling policy known as SCHED_DEADLINE, implementing the earliest deadline first algorithm (EDF), was added in kernel version 3.14, released on 30 March 2014.

Portability

While not originally designed to be portable, Linux is now one of the most widely ported operating system kernels, running on a diverse range of systems from the ARM architecture to IBM Z/Architecture mainframe computers. The first port beyond Linux's original 386 architecture was to the Motorola 68000 platform by Amiga users, but to do this port they replaced major parts of the kernel, leading Torvalds to call it a fork and a "Linux-like operating system". With this experience in mind, Torvalds led a major restructure of the kernel code to facilitate the next port effort, to the DEC Alpha AXP platform, and support both 386 and Alpha in a single source tree.

Linux runs as the main operating system on IBM's Blue Gene and other fastest supercomputers. As of June 2016, 99+% of the world's 500 fastest supercomputers run some variant of Linux, including the top 280. Linux has also been ported to various handheld devices such as Apple's iPod and iPhone. Some operating systems developed for mobile phones use modified versions of the Linux kernel, including Google Android, Firefox OS, HP webOS, and Nokia Maemo.

Kernel panic and oopses

In Linux, a "panic" is an unrecoverable system error detected by the kernel, as opposed to similar errors detected by user space code. It is possible for kernel code to indicate such a condition by calling the panic function located in the header file sys/system.h. However, most panics are the result of unhandled processor exceptions in kernel code, such as references to invalid memory addresses. These are typically indicative of a bug somewhere in the call chain leading to the panic. They can also indicate a failure of hardware, such as a failed RAM cell or errors in arithmetic functions in the processor caused by a processor bug, overheating/damaged processor, or a soft error.

A report of a non-fatal bug in the kernel is called an "oops"; such deviations from correct behavior of the Linux kernel may allow continued operation with compromised reliability. These crash reports are automatically collected and can be sent upstream by various software, such as kerneloops, ABRT (Fedora) and apport (Ubuntu). KernelOops.org collected these reports and published statistics on their website.

The kernel panic message might not be printed visibly in some conditions, such as when using a graphical desktop. To debug such conditions, other methods such as attaching a serial port console can be used.

Live patching

Rebootless updates can even be applied to the kernel by using live patching technologies such as Ksplice, kpatch and kGraft. Minimalistic foundations for live kernel patching were merged into the Linux kernel mainline in kernel version 4.0, which was released on 12 April 2015. Those foundations, known as livepatch and based primarily on the kernel's ftrace functionality, form a common core capable of supporting hot patching by both kGraft and kpatch, by providing an application programming interface (API) for kernel modules that contain hot patches and an application binary interface (ABI) for the userspace management utilities. However, the common core included into Linux kernel 4.0 supports only the x86 architecture and does not provide any mechanisms for ensuring function-level consistency while the hot patches are applied. As of April 2015, there is ongoing work on porting kpatch and kGraft to the common live patching core provided by the Linux kernel mainline.

Security

Computer security is a much-publicized topic in relation to the Linux kernel because a large portion of the kernel bugs present potential security flaws. For example, they may allow for privilege escalation or create denial-of-service attack vectors. Over the years, numerous such flaws were found and fixed in the Linux kernel. New security features are frequently implemented to improve the Linux kernel's security.

Critics have accused kernel developers of covering up security flaws or at least not announcing them; in 2008, Linus Torvalds responded to this with the following:

I personally consider security bugs to be just "normal bugs". I don't cover them up, but I also don't have any reason what-so-ever to think it's a good idea to track them and announce them as something special...one reason I refuse to bother with the whole security circus is that I think it glorifies—and thus encourages—the wrong behavior. It makes "heroes" out of security people, as if the people who don't just fix normal bugs aren't as important. In fact, all the boring normal bugs are way more important, just because there's a lot more of them. I don't think some spectacular security hole should be glorified or cared about as being any more "special" than a random spectacular crash due to bad locking.

Linux distributions typically release security updates to fix vulnerabilities in the Linux kernel. Many offer long-term support releases that receive security updates for a certain Linux kernel version for an extended period of time.

Feature history

Version 1.0 of the Linux kernel was released on 14 March 1994. This release of the Linux kernel only supported single-processor i386-based computer systems. Portability became a concern, and so version 1.2 (released 7 March 1995) gained support for computer systems using processors based on the Alpha, SPARC, and MIPS architectures.

Version 2.0 was released on 9 June 1996. There were 41 releases in the series. The major feature of 2.0 was SMP support (that is, support for multiple processors in a single system) and support for more types of processors.

Version 2.2, released on 20 January 1999, removed the global spinlock and provided improved SMP support, added support for the m68k and PowerPC architectures, and added new file systems (including read-only support for Microsoft's NTFS).

Version 2.4.0, released on 4 January 2001, contained support for ISA Plug and Play, USB, and PC Cards. It also included support for the PA-RISC processor from Hewlett-Packard. Development for 2.4.x changed a bit in that more features were made available throughout the duration of the series, including: support for Bluetooth, Logical Volume Manager (LVM) version 1, RAID support, InterMezzo and ext3 file systems.

Version 2.6.0 was released on 17 December 2003. The development for 2.6.x changed further towards including new features throughout the duration of the series. Among the changes that have been made in the 2.6 series are: integration of µClinux into the mainline kernel sources, PAE support, support for several new lines of CPUs, integration of Advanced Linux Sound Architecture (ALSA) into the mainline kernel sources, support for up to 2³² users (up from 2¹⁶), support for up to 2²⁹ process IDs (64-bit only, 32-bit arches still limited to 2¹⁵), substantially increased the number of device types and the number of devices of each type, improved 64-bit support, support for file systems which support file sizes of up to 16 terabytes, in-kernel preemption, support for the Native POSIX Thread Library (NPTL), User-mode Linux integration into the mainline kernel sources, SELinux integration into the mainline kernel sources, InfiniBand support, and considerably more. Also notable are the addition of several file systems throughout the 2.6.x releases: FUSE, JFS, XFS, ext4 and more. Details on the history of the 2.6 kernel series can be found in the ChangeLog files on the 2.6 kernel series source code release area of kernel.org.

Version 3.0 was released on 22 July 2011. On 30 May 2011, Torvalds announced that the big change was "NOTHING. Absolutely nothing." and asked "...let's make sure we really make the next release not just an all new shiny number, but a good kernel too." After the expected 6–7 weeks of the development process, it would be released near the 20th anniversary of Linux.

In December 2012, Torvalds decided to reduce kernel complexity by removing support for i386 processors, making the 3.7 kernel series the last one still supporting the original processor. The same series unified support for the ARM processor.

Version 3.11, released on 2 September 2013, adds many new features such as new O_TMPFILE flag for open(2) to reduce temporary file vulnerabilities, experimental AMD Radeon dynamic power management, low-latency network polling, and zswap (compressed swap cache).

There were no meaningful technical reasons for the numbering change from 2.6.39 to 3.0, or from 3.19 to 4.0. The major version number was increased just to avoid large minor numbers.

Developer community

As of 2007, the development of the kernel had shifted from the top 20 most active developers writing 80% of the code to the top 30 writing 30% of the code, with top developers spending more time reviewing changes. Developers can also be categorized by affiliation; in 2007, the top category was unknown while the top corporation was Red Hat with 12% of contributions, and known amateurs at 3.9%. The kernel changes made in year 2007 have been submitted by over 1900 developers, which may be a significant underestimate because developers working in teams usually count as one. It is generally assumed that the community of Linux kernel developers comprises 5000 or 6000 members.

Codebase

As of 2013, the 3.10 release of the Linux kernel had 15,803,499 lines of code. As of 2007, roughly 5% of the code is part of the "core" while 52% is drivers.

Instead of a roadmap, there are technical guidelines. Instead of a central resource allocation, there are persons and companies who all have a stake in the further development of the Linux kernel, quite independently from one another: People like Linus Torvalds and I don’t plan the kernel evolution. We don’t sit there and think up the roadmap for the next two years, then assign resources to the various new features. That's because we don’t have any resources. The resources are all owned by the various corporations who use and contribute to Linux, as well as by the various independent contributors out there. It's those people who own the resources who decide...

Linux is evolution, not intelligent design!

By this statement it is meant that evolution often does odd (and "sub-optimal") things exactly because it does incremental changes which do not break at any point. As a result, any released version of the Linux kernel is fully usable, even if, for example, device drivers do not support all features of the hardware they are written for.

The conceptual architecture of the Linux kernel has proved its success, while essential factors for this success were the provision for the organization of developers, and the provision for system extensibility. The Linux kernel's architecture was required to support many independent volunteer developers, which suggested that the system portions that require the most development‍—‌hardware device drivers, file systems and network protocols‍—‌be implemented in an extensible fashion. The Linux kernel's architecture chose to make these systems extensible using a data abstraction technique – each hardware device driver is implemented as a separate module that supports a common interface. In this way, a single developer can add a new device driver, with minimal interaction required with other developers of the Linux kernel.

Another important extension to the Linux kernel is the addition of more supported hardware platforms. The architecture of the system supports this extensibility by separating all hardware-specific code into distinct modules within each subsystem. In this way, a small group of developers can implement a port of the Linux kernel to a new hardware architecture by re-implementing only the machine-specific portions of the kernel.

Estimated cost to redevelop

The cost to redevelop the Linux kernel version 2.6.0 in a traditional proprietary development setting has been estimated to be US$612 million (€467M, £394M) in 2004 prices using the COCOMO man-month estimation model. In 2006, a study funded by the European Union put the redevelopment cost of kernel version 2.6.8 higher, at €882M ($1.14bn, £744M).

This topic was revisited in October 2008 by Amanda McPherson, Brian Proffitt and Ron Hale-Evans. Using David A. Wheeler's methodology, they estimated redevelopment of the 2.6.25 kernel now costs $1.3bn (part of a total $10.8bn to redevelop Fedora 9). Again, Garcia-Garcia and Alonso de Magdaleno from University of Oviedo (Spain) estimate that the value annually added to kernel was about €100M between 2005 and 2007 and €225M in 2008, it would cost also more than €1bn (about $1.4bn as of February 2010) to develop in the European Union.

As of 7 March 2011, using then-current LOC (lines of code) of a 2.6.x Linux kernel and wage numbers with David A. Wheeler's calculations it would cost approximately $3bn (about €2.2bn) to redevelop the Linux kernel as it keeps getting bigger.

Development model

The current development model of the Linux kernel is such that Linus Torvalds makes the releases of new versions, also called the "vanilla" or "mainline" kernels, meaning that they contain the main, generic branch of development. This branch is officially released as a new version approximately every ten weeks, after Torvalds does an initial round of integrating major changes made by all other programmers, and several rounds of bug-fix pre-releases.

As of 2015, in the current development scheme, the main branch of development is not a traditional "stable" branch; instead, it incorporates all kinds of changes, including both the latest features, and security and bug fixes. For users who do not want to risk updating to new versions containing code that may not be well tested, a separate set of "stable" branches exist, one for each released version, which are meant for people who just want the security and bug fixes, but not a whole new version. These branches are maintained by the stable team (Greg Kroah-Hartman, Chris Wright, and others).

The development model for the 2.6 kernel series was significantly different compared to the 2.5 series. Before the 2.6 series, there was a stable branch (2.4) where only relatively minor and safe changes were merged, and an unstable branch (2.5), where bigger changes and cleanups were allowed. Both of these branches had been maintained by the same set of people, led by Torvalds. This meant that users would always have a well-tested 2.4 version with the latest security and bug fixes to use, though they would have to wait for the features which went into the 2.5 branch. The downside of this was that the "stable" kernel ended up so far behind that it no longer supported recent hardware and lacked needed features. In the late 2.5 kernel series, some maintainers elected to try backporting of their changes to the stable kernel series, which resulted in bugs being introduced into the 2.4 kernel series. The 2.5 branch was then eventually declared stable and renamed to 2.6. But instead of opening an unstable 2.7 branch, the kernel developers decided to continue putting major changes into the 2.6 branch, which would then be released at a pace faster than 2.4.x but slower than 2.5.x. This had the desirable effect of making new features more quickly available and getting more testing of the new code, which was added in smaller batches and easier to test.

As a response to the lack of a stable kernel tree where people could coordinate the collection of bug fixes as such, in December 2005 Adrian Bunk announced that he would keep releasing 2.6.16.y kernels when the stable team moved on to 2.6.17. He also included some driver updates, making the maintenance of the 2.6.16 series very similar to the old rules for maintenance of a stable series such as 2.4. Since then, the "stable team" had been formed, and it would keep updating kernel versions with bug fixes. In October 2008 Adrian Bunk announced that he will maintain 2.6.27 for a few years as a replacement of 2.6.16. The stable team picked up on the idea and as of 2010 they continue to maintain that version and release bug fixes for it, in addition to others.

After the change of the development model with 2.6.x, developers continued to want what one might call an unstable kernel tree, one that changes as rapidly as new patches come in. Andrew Morton decided to repurpose his -mm tree from memory management to serve as the destination for all new and experimental code. In September 2007, Morton decided to stop maintaining this tree. In February 2008, Stephen Rothwell created the linux-next tree to serve as a place where patches aimed to be merged during the next development cycle are gathered. Several subsystem maintainers also adopted the suffix -next for trees containing code which is meant to be submitted for inclusion in the next release cycle.

As of January 2014, the in-development version of the Linux kernel is held in an unstable branch named linux-next.

Relation with Linux distributions

Most Linux users run a kernel supplied by their Linux distribution. Some distributions ship the "vanilla" or "stable" kernels. However, several Linux distribution vendors (such as Red Hat and Debian) maintain another set of Linux kernel branches which are integrated into their products. These are usually updated at a slower pace compared to the "vanilla" branch, and they usually include all fixes from the relevant "stable" branch, but at the same time they can also add support for drivers or features which had not been released in the "vanilla" version the distribution vendor started basing their branch from.

Maintenance

While Linus Torvalds supervises code changes and releases to the latest kernel versions, he has delegated the maintenance of older versions to other programmers. Major releases as old as 2.0 (officially made obsolete with the kernel 2.2.0 release in January 1999) are maintained as needed, although at a very slow pace.

2.6.x.y releases

Versions 2.6.16 and 2.6.27 of the Linux kernel were unofficially supported in a long-term support (LTS) fashion, before a 2011 working group in the Linux Foundation started a formal long-term support initiative.

Revision control

The Linux kernel source code used to be maintained without the help of an automated source code management system, mostly because of Linus Torvalds' dislike of centralized SCM systems.

In 2002, Linux kernel development switched to BitKeeper, an SCM system which satisfied Torvalds' technical requirements. BitKeeper was made available to Linus and several others free of charge, but was not free software, which was a source of controversy. The system did provide some interoperability with free SCM systems such as CVS and Subversion.

In April 2005, however, efforts to reverse-engineer the BitKeeper system by Andrew Tridgell led BitMover, the company which maintained BitKeeper, to stop supporting the Linux development community. In response, Torvalds and others wrote a new source code control system for the purpose, called Git. The new system was written within weeks, and in two months the first official kernel release was made using Git. Git soon developed into a separate project in its own right and gained widespread adoption.

Version numbering

The Linux kernel has had three different numbering schemes.

The first scheme was used in the run-up to "1.0". The first version of the kernel was 0.01. This was followed by 0.02, 0.03, 0.10, 0.11, 0.12 (the first GPL version), 0.95, 0.96, 0.97, 0.98, 0.99 and then 1.0. From 0.95 on there were many patch releases between versions.

After the 1.0 release and prior to version 2.6, the number was composed as "a.b.c", where the number "a" denoted the kernel version, the number "b" denoted the major revision of the kernel, and the number "c" indicated the minor revision of the kernel. The kernel version was changed only when major changes in the code and the concept of the kernel occurred, twice in the history of the kernel: in 1994 (version 1.0) and in 1996 (version 2.0). Version 3.0 was released in 2011, but it was not a major change in kernel concept. The major revision was assigned according to the even–odd version numbering scheme. The minor revision had been changed whenever security patches, bug fixes, new features or drivers were implemented in the kernel.

In 2004, after version 2.6.0 was released, the kernel developers held several discussions regarding the release and version scheme and ultimately Linus Torvalds and others decided that a much shorter "time-based" release cycle would be beneficial. For about seven years, the first two numbers remained "2.6", and the third number was incremented with each new release, which rolled out after two to three months. A fourth number was sometimes added to account for bug and security fixes (only) to the kernel version. The even-odd system of alternation between stable and unstable was gone. Instead, development pre-releases are titled release candidates, which is indicated by appending the suffix '-rc' to the kernel version, followed by an ordinal number.

The first use of the fourth number occurred when a grave error, which required immediate fixing, was encountered in 2.6.8's NFS code. However, there were not enough other changes to legitimize the release of a new minor revision (which would have been 2.6.9). So, 2.6.8.1 was released, with the only change being the fix of that error. With 2.6.11, this was adopted as the new official versioning policy. Later it became customary to continuously back-port major bug-fixes and security patches to released kernels and indicate that by updating the fourth number.

On 29 May 2011, Linus Torvalds announced that the kernel version would be bumped to 3.0 for the release following 2.6.39, due to the minor version number getting too large and to commemorate the 20th anniversary of Linux. It continued the time-based release practice introduced with 2.6.0, but using the second number; for example, 3.1 would follow 3.0 after a few months. An additional number (now the third number) would be added on when necessary to designate security and bug fixes, as for example with 3.0.18; the Linux community refers to this as "3.x.y.z" versioning. The major version number was also later raised to 4, for the release following version 3.19.

In addition to Linus's "-rc" development releases, sometimes the version will have a suffix such as "tip", indicating another development branch, usually (but not always) the initials of a person who made it. For example, "ck" stands for Con Kolivas, "ac" stands for Alan Cox, etc. Sometimes, the letters are related to the primary development area of the branch the kernel is built from, for example, "wl" indicates a wireless networking test build. Also, distributors may have their own suffixes with different numbering systems and for back-ports to their "enterprise" (i.e. stable but older) distribution versions.

Variants

There are certain variants of the Linux kernel that provide additional functionality, but do not belong to the Linux kernel mainline. Such variants of the Linux kernel include Linux-libre, Compute Node Linux, Cooperative Linux, Longene, grsecurity, INK, L4Linux, MkLinux, RTLinux, and User-mode Linux. Some of these variants have been partially merged into the mainline.