Are microkernels the future of secure OS design ?

MINIX 3, and microkernel OSs in general, might show us the way toward the future of secure OS design.

Date: September 6th, 2010
Author: Chad Perrin
Category: Application Security, Security

--------------------------------------------------------------------------------
Andrew S. Tanenbaum created the MINIX operating system in the 1980s as an instructional tool. He wanted to provide an aid to helping people learn about the design of operating systems, and did so by creating an incredibly simple — in some respects, too simple for any serious use — Unix-like OS from scratch, so that his students could examine and modify the source code and even run it on the commodity hardware of the time. He and Al Woodhull co-authored a textbook that focused on the MINIX OS.

Since then, 1.5 and 2.0 versions of MINIX have been created as well, with the same goal: to serve as a study aid. By contrast, the current iteration of the MINIX project, known as MINIX 3, is being developed with the aim of providing a production-ready, general purpose Unix-like operating system. In Tanenbaum-Torvalds debate, Part II, Andrew Tanenbaum describes the relationship of MINIX 3 to its forebears:

MINIX 1 and MINIX 3 are related in the same way as Windows 3.1 and Windows XP are: same first name. Thus even if you used MINIX 1 when you were in college, try MINIX 3; you’ll be surprised. It is a minimal but functional UNIX system with X, bash, pdksh, zsh, cc, gcc, perl, python, awk, emacs, vi, pine, ssh, ftp, the GNU tools and over 400 other programs.

Among Tanenbaum’s key priorities in the development of MINIX 3 is the desire to improve operating system reliability to the level people have come to expect from televisions and automobiles:

It is about building highly reliable, self-healing, operating systems. I will consider the job finished when no manufacturer anywhere makes a PC with a reset button. TVs don’t have reset buttons. Stereos don’t have reset buttons. Cars don’t have reset buttons. They are full of software but don’t need them. Computers need reset buttons because their software crashes a lot.

He acknowledges the vast differences in the types of software required by these different products, but is optimistic that the shortcomings imposed by the necessarily greater complexity of software in a desktop computer can be mitigated to the point where end users will no longer have to deal with the crashing behavior of operating system software. As he puts it, “I want to build an operating system whose mean time to failure is much longer than the lifetime of the computer so the average user never experiences a crash.”

There are already OSs that meet that goal under extremely limited, tightly constrained circumstances, but he intends to provide the same stability and reliability under the harsh, changeable circumstances of general-purpose consumer computer use. It is an ambitious goal, but his approach is well thought out and shows great promise. Given that many of the stability issues we see in operating systems using monolithic kernels result from the fact that a crashing driver can crash the OS kernel itself, his intention to solve the problem by moving drivers out of kernel space may be on the right track.

In fact, if a driver fails to respond to what he calls a “reincarnation server”, which monitors the status of active drivers, the reincarnation server can silently swap out the hung or otherwise failing driver process for a fresh process that picks up where the original left off. In most — if not all — cases, the user may not even know anything has happened, where a monolithic kernel OS would likely crash immediately.

While his primary interest is in reliability, rather than security per se, there are obvious security benefits to his microkernel design for MINIX 3. Without going into too much technical detail, there are mechanisms in place that restrict a driver’s access to memory addresses, limiting it to its own assigned address space in a manner that in theory cannot be circumvented by the driver. Drivers all run in user space, keeping them out of kernel space, whereas kernels loaded as part of a monolithic kernel design can typically touch any memory addresses at all.

A number of countermeasures have been invented and employed in various OS designs over the years, but keeping drivers out of kernel space entirely certainly seems like the most effective and inviolate protection against driver misbehavior. As a result, this may all but eliminate the possibility of buffer overruns and similar problems in kernel space.

Further, mediated contact between drivers and other user space systems in MINIX 3 extends the strict privilege separation of Unix-like systems to protect other processes than the kernel process, as well. In short, whole classes of common system vulnerabilities may become extinct within the family of OSs that may develop in the wake of MINIX 3.

MINIX 3 itself is still in development, but it is currently a working OS with many of Tanenbaum’s intended reliability assurance features already implemented. You can download it from the MINIX 3 Website and boot it from a LiveCD though, as Tanenbaum states, you should install it to a partition on the hard drive of a computer if you want to do anything useful with it.

It will not replace MS Windows, MacOS X, Ubuntu Linux, or FreeBSD right now, but it may well be the future of general purpose OS design — especially secure general
purpose OS design. Sphere: Related Content