Understanding Linux System Call Mechanisms: Software Interrupts, Fast Syscall Instructions, and vDSO

System calls are the mechanism by which a user‑space program requests services from the operating‑system kernel, such as hardware access, process creation, and scheduling. They form the primary interface between applications and the OS.

Even a trivial "Hello, World!" program triggers dozens of system calls during startup, as illustrated by a strace trace that shows calls like execve , brk , openat , and the final write that outputs the string.

Compared with ordinary function calls, system calls are far more expensive; a benchmark shows that a SYSCALL instruction can be dozens of times slower than a normal C function call. The overhead stems from mode switches, register saving, parameter validation, and kernel‑user memory copying.

Linux implements system calls using three distinct methods:

Software interrupt (e.g., INT 0x80 )

Fast syscall instructions ( SYSENTER / SYSCALL )

Virtual Dynamic Shared Object (vDSO) – a user‑space mapping of selected kernel functions

Software interrupt – On 32‑bit x86 the classic way is to invoke INT 0x80 . The user‑level C library copies arguments to registers, places the syscall number in eax , and executes the interrupt. The CPU switches to kernel mode, the interrupt handler entry_INT80_32 saves the full register state ( SAVE_ALL ), validates the syscall number via ia32_sys_call_table , performs the requested operation, stores the return value back in eax , restores registers, and finally returns to the user program. Errors are reported through errno .

#include <stdio.h>
int main() {
printf("Hello, World!");
return 0;
}
$ gcc hello.c -o hello
$ strace ./hello
execve("./hello", ["./hello"], 0x7ffd64dd8090 /* 23 vars */) = 0
brk(NULL)                               = 0x557b449db000
...
write(1, "Hello, World!", 13)           = 13
exit_group(0)                           = ?
+++ exited with 0 +++

Fast syscall instructions – Modern kernels use SYSENTER / SYSEXIT on 32‑bit and SYSCALL / SYSRET on 64‑bit CPUs. These instructions are designed specifically for system calls, bypassing the interrupt descriptor table and avoiding unnecessary stack manipulation, which reduces the number of clock cycles by roughly 25 % compared with the software‑interrupt path.

The kernel registers the entry points ( entry_SYSCALL_64 for 64‑bit) in model‑specific registers (MSRs) during initialization. When a user program executes SYSCALL , the CPU jumps directly to the registered handler, which reads the syscall number and arguments from registers, looks up the implementation in the syscall table, performs the operation, and returns the result in eax .

void syscall_init(void) {
wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS);
wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64);
...
}

vDSO (Virtual Dynamic Shared Object) – vDSO is a small shared library that the kernel maps into every process’s address space. It contains implementations of a handful of safe, frequently used syscalls (e.g., gettimeofday , clock_gettime , clock_getres , rt_sigreturn ) so that they can be executed entirely in user mode, eliminating the kernel‑mode transition.

During program start‑up, the ELF loader records the vDSO base address in the auxiliary vector entry AT_SYSINFO_EHDR . The dynamic linker reads this entry, and the C library resolves symbols such as __vdso_gettimeofday to the corresponding functions inside the vDSO.

$ ldd /bin/cat
linux-vdso.so.1 (0x00007fff2709c000)
...
$ cat /proc/self/maps
7ffe8ca90000-7ffe8ca92000 r-xp 00000000 00:00 0      [vdso]

Because vDSO only exposes non‑privileged, side‑effect‑free syscalls, the security risk is minimal, yet the performance gain is substantial—time‑critical calls can become as fast as ordinary function calls.

Summary

Software‑interrupt syscalls ( INT 0x80 ) involve full context saving and interrupt‑descriptor lookup, making them the slowest method.

Fast syscall instructions ( SYSENTER / SYSCALL ) are the common modern approach, offering lower latency by skipping unnecessary steps.

vDSO provides the fastest path for a limited set of safe syscalls, effectively flattening the cost to that of a regular function call.

Understanding these mechanisms helps developers diagnose performance issues, choose appropriate benchmarking techniques, and appreciate the tight coupling between hardware, kernel design, and user‑space programming.