Infrastructure to add instrumentation to a running kernel:
trace functions, read and write variables, follow pointers, gather statistics...
Eliminates the need to modify the kernel sources to add one's own instrumentation to investigated a functional or performance problem.
Uses a simple scripting language.
Several example scripts and probe points are available.
Based on the Kprobes instrumentation infrastructure.
See Documentation/kprobes.txt in kernel sources.
Linux 2.6.26: supported on most popular CPUs (arm included in 2.6.25).
However, lack of recent support for mips (2.6.16 only!).