Systrace is a computer security utility which limits an application's access to the system by enforcing access policies for system calls. This can mitigate the effects of buffer overflows and other security vulnerabilities. It was developed by Niels Provos and runs on various Unix-like operating systems.
Systrace is particularly useful when running untrusted or binary-only applications and provides facilities for privilege elevation on a system call basis, helping to eliminate the need for potentially dangerous setuid programs. It also includes interactive and automatic policy generation features, to assist in the creation of a base policy for an application.
Systrace used to be integrated into OpenBSD, but was removed in April 2016 It is available for Linux and Mac OS X, although the OS X port is currently unmaintained. It was removed from NetBSD at the end of 2007 due to several unfixed implementation issues. As of version 1.6f Systrace supports 64-bit Linux 2.6.1 via kernel patch.
Systrace supports the following features:
Confines untrusted binary applications: An application is allowed to make only those system calls specified as permitted in the policy. If the application attempts to execute a system call that is not explicitly permitted, an alarm gets raised.
Interactive policy generation with graphical user interface: Policies can be generated interactively via a graphical frontend to Systrace. The frontend shows system calls and their parameters not currently covered by policy and allows the user to refine the policy until it works as expected.
Supports different emulations: GNU/Linux, BSDI, etc..
Non-interactive policy enforcement: Once a policy has been trained, automatic policy enforcement can be used to deny all system calls not covered by the current policy. All violations are logged to Syslog. This mode is useful when protecting system services like a web server.
Remote monitoring and intrusion detection: Systrace supports multiple frontends by using a frontend that makes use of the network, very advanced features are possible.
Privilege elevation: Using Systrace's privilege elevation mode, it's possible to get rid of setuid binaries. A special policy statement allows selected system calls to run with higher privileges, for example, creating a raw socket.
Systrace has had some vulnerabilities in the past, including:
Exploiting Concurrency Vulnerabilities in System Call Wrappers Paper by Robert Watson (computer scientist) from the First USENIX Workshop On Offensive Technologies (WOOT07) analyzing system call wrapper traces across several wrapper platforms including systrace
Google Security discovers local privilege escalation in Systrace
Local root exploit on NetBSD
Vulnerabilities in systrace
