How Long Does a System Call Take?

I’m learning about CPU virtualization and the Limited Direct Execution model. As part of that, I got some homework¹ to measure how long a system call takes. The book suggests:

Measuring the cost of a system call is relatively easy. For example, you could repeatedly call a simple system call (e.g., performing a 0-byte read), and time how long it takes; dividing the time by the number of iterations gives you an estimate of the cost of a system call.

I’ve done that using Zig and you can see the code below. This measures the end-to-end cost (libc wrapper + kernel entry/return), not a bare syscall instruction. The numbers also include loop overhead; if you subtract an empty-loop baseline you can better isolate the syscall+wrapper cost:

const std = @import("std");
const print = std.debug.print;

pub fn main() !void {
    const iterations = 1_000_000;
    print("Measuring System Call Cost (read)\n", .{});
    print("Iterations: {}\n", .{iterations});
    var dummy_buffer: [1]u8 = undefined;
    const start = std.time.nanoTimestamp();
    for (0..iterations) |_| {
        const result = std.c.read(std.c.STDIN_FILENO, &dummy_buffer, 0);
        std.mem.doNotOptimizeAway(result);
    }
    const total_ns = std.time.nanoTimestamp() - start;
    const avg_ns: f64 = @as(f64, @floatFromInt(total_ns)) / iterations;
    print("Average cost: {d:.2}ns\n", .{avg_ns});
}

The results are somewhat interesting. On my macOS (Apple M4 Max) machine:

• macOS: 251.22ns
• FreeBSD²: 115.23ns

The results match the C implementation. I found it curious that the syscall was consistently quicker on FreeBSD even though it is virtualized. I wonder if that’s related to the fact that the host is juggling with a bunch of other things, or if FreeBSD is just more efficient for the same hardware.

Environment for these runs:

• Host: macOS on Apple M4 Max; STDIN was a TTY
• Guest: FreeBSD 14.3-RELEASE under QEMU with HVF²

How to run

zig run measure_syscall_time.zig

Note that I’ve also run with optimizations, but the numbers did not change. Compiled with zig 0.16.0-dev.253+4314c9653.