Using the ftrace kernel tracer
Ftrace is an inbuilt kernel feature; its code is deeply ingrained into that of the kernel itself. It provides developers (anyone with root access, really) with a way to look deep within the kernel, perform detailed traces to see exactly what's going on inside, and to even get help with performance/latency issues that may crop up.
A simple way to think about ftrace's functionality is this – if you'd like to see what a process is up to, performing strace on it can be very useful indeed; it will display every system call that the process invokes in a meaningful way, with parameters, return values, and so on. Thus, strace is useful and interesting, as it shows what occurs at the interesting system call point – the boundary between user and kernel space. But that's it; strace cannot show you anything beyond the system call; what does the system call code do within the kernel? What kernel APIs does it invoke, and thus which...
