ebpf ยท diff
git:20260905.fd53e99 to git:20260912.07a8a1b
228 added, 22 removed. Audit A to A.
---
name: ebpf
- description: 'Use when writing eBPF programs with libbpf or bpftrace, attaching kprobes, tracepoints, or XDP hooks, triaging verifier errors, choosing maps, or porting with CO-RE. Not for Rust: use ebpf-rust.'
+ description: 'Use when writing eBPF programs with libbpf, bpftrace, or Rust with aya-ebpf and aya-log, attaching kprobes, tracepoints, or XDP hooks, declaring maps, sharing data with a Tokio loader, triaging verifier or Aya load errors, or porting with CO-RE. Not for full kernel bypass: use dpdk.'
---
- # eBPF
+ # eBPF with libbpf, bpftrace, and Aya
## Contract
| Field | Bound contract |
|---|---|
- | Trigger | A kernel event needs tracing, a libbpf program needs writing or loading, the verifier rejects a program, kernel and user space need to share data, or an XDP filter needs attaching. |
- | Authority | Reversible local. The write set is the user's source and object files, programs and maps loaded into the running kernel, pins under `/sys/fs/bpf`, and XDP attachments on a named interface. Rollback is detaching the program, unpinning, and `ip link set dev <if> xdp off`. Loading needs `CAP_BPF` or root. No remote mutation. |
- | Side effect | Kernel state changes while a program is loaded. Every load in this skill is paired with its detach. |
- | Done | The program loads, attaches, and produces output for the traced event, or the verifier error is mapped to its cause and the fix is applied. |
+ | Trigger | A kernel event needs tracing, a C/libbpf or Rust/Aya program needs writing or loading, the verifier rejects a program, kernel and user space need to share data, or an XDP filter needs attaching. |
+ | Authority | Reversible local. The write set is the user's C, Rust, or script source and object files, programs and maps loaded into the running kernel, pins under `/sys/fs/bpf`, and XDP attachments on a named interface. Rollback is detaching the program, unpinning, and `ip link set dev <if> xdp off`. Loading needs `CAP_BPF` or root. No remote mutation. |
+ | Side effect | Kernel state changes while a program or loader is loaded. Every load in this skill is paired with its detach; an Aya loader exits cleanly on Ctrl-C. |
+ | Done | The chosen program loads, attaches, and produces output for the traced event, the Aya loader exits cleanly on Ctrl-C, or the verifier/Aya load error is mapped to its cause and the fix is applied. |
## Inputs
- Goal (required): a one-off trace, a production program with a user-space side, an inspection of loaded objects, or packet processing.
- - Kernel (required): the running kernel must expose BTF at `/sys/kernel/btf/vmlinux` for CO-RE. The grounded floor is Linux 7.2 mainline or the 6.18 LTS line; every kernel feature named below exists in both.
- - Toolchain (required for libbpf work): `clang` with the `bpf` target, `bpftool`, and libbpf 1.x with its headers.
+ - Implementation (required for a production program): C with libbpf or Rust with Aya; use bpftrace for a one-off trace.
+ - Program type and attach point (required for a program): tracepoint, kprobe, uprobe, XDP, TC, LSM, or another type from the mode table in step 4.
+ - Kernel (required): the running kernel must expose BTF at `/sys/kernel/btf/vmlinux` for C/libbpf CO-RE and Aya's target-BTF binding generation. The grounded floor is Linux 7.2 mainline or the 6.18 LTS line; every kernel feature named below exists in both.
+ - Toolchain (required for libbpf work): `clang` with the `bpf` target, `bpftool`, and libbpf 1.x with its headers. For Aya work: stable Rust plus a nightly with `rust-src`, `bpf-linker`, `cargo-generate`, `bpftool`, and a log backend such as `env_logger`, using Rust edition 2024.
## Procedure
- 1. Pick the tool by goal. A one-line trace or a script is `bpftrace`. A production program with a loader is libbpf in C (or aya in Rust, see `ebpf-rust`). Inspecting loaded programs and maps is `bpftool`. High-rate packet processing is XDP with libbpf. Done when: one tool is chosen.
+ 1. Pick the tool and implementation mode by goal. A one-line trace or script is `bpftrace`. A production program with a loader is libbpf in C or Aya in Rust. Inspecting loaded programs and maps is `bpftool`. High-rate packet processing is XDP with libbpf or Aya. Done when: one tool and, for a production program, one language mode are chosen.
2. For a quick trace, write the `bpftrace` one-liner and confirm the probe exists. Done when: the probe prints for a live event.
```bash
bpftrace -l 'tracepoint:syscalls:*' # list probes
bpftrace -l 'kprobe:tcp_*'
bpftrace -e 'tracepoint:syscalls:sys_enter_execve { printf("%s %s\n", comm, str(args->filename)); }'
bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'
bpftrace -e '
tracepoint:syscalls:sys_enter_read { @start[tid] = nsecs; }
tracepoint:syscalls:sys_exit_read { @us = hist((nsecs - @start[tid]) / 1000); delete(@start[tid]); }'
```
- 3. For a libbpf program, write the kernel side, generate the skeleton, and write the loader. Done when: the loader attaches and the map fills.
+ 3. For a production program, select C/libbpf or Rust/Aya. In C/libbpf mode, write the kernel side, generate the skeleton, and write the loader. Done when: the loader attaches and the map fills.
```c
/* counter.bpf.c: count read() calls per process. */
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__type(key, u32);
__type(value, u64);
__uint(max_entries, 1024);
} call_count SEC(".maps");
SEC("tracepoint/syscalls/sys_enter_read")
int trace_read(struct trace_event_raw_sys_enter *ctx)
{
u32 pid = bpf_get_current_pid_tgid() >> 32;
u64 *cnt = bpf_map_lookup_elem(&call_count, &pid);
if (cnt) {
__sync_fetch_and_add(cnt, 1);
} else {
u64 one = 1;
bpf_map_update_elem(&call_count, &pid, &one, BPF_ANY);
}
return 0;
}
char LICENSE[] SEC("license") = "GPL";
```
```c
- /* counter.c: open, load, attach, read, destroy. */
+ /* counter.c: open, load, attach, observe, destroy. */
#include "counter.skel.h"
+ #include <bpf/bpf.h>
+ #include <signal.h>
+ #include <stdio.h>
+ #include <unistd.h>
+ static volatile sig_atomic_t stop;
+
+ static void stop_handler(int signum)
+ {
+ (void)signum;
+ stop = 1;
+ }
+
int main(void)
{
struct counter_bpf *skel = counter_bpf__open();
- if (!skel || counter_bpf__load(skel))
- return 1;
+ int status = 1;
+ if (!skel)
+ return status;
+ if (counter_bpf__load(skel))
+ goto cleanup;
if (counter_bpf__attach(skel))
- return 1;
- /* read call_count through bpf_map__fd(skel->maps.call_count) */
+ goto cleanup;
+ if (signal(SIGINT, stop_handler) == SIG_ERR ||
+ signal(SIGTERM, stop_handler) == SIG_ERR)
+ goto cleanup;
+
+ int map_fd = bpf_map__fd(skel->maps.call_count);
+ while (!stop) {
+ __u32 pid;
+ __u64 count;
+ if (bpf_map_get_next_key(map_fd, NULL, &pid) == 0 &&
+ bpf_map_lookup_elem(map_fd, &pid, &count) == 0) {
+ printf("pid %u: %llu reads\n", pid, (unsigned long long)count);
+ status = 0;
+ break;
+ }
+ sleep(1);
+ }
+ if (stop)
+ status = 0;
+ cleanup:
counter_bpf__destroy(skel);
- return 0;
+ return status;
}
```
```bash
bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
clang -g -O2 -target bpf -D__TARGET_ARCH_x86 -c counter.bpf.c -o counter.bpf.o
bpftool gen skeleton counter.bpf.o > counter.skel.h
gcc -o counter counter.c -lbpf -lelf -lz
```
The skeleton's `open`, `load`, `attach`, and `destroy` functions replace the older `bpf_object__open_file` and `bpf_object__load` split, which still exist for loading an object by path.
- 4. Choose the map. Use a hash map for per-key state, an array for fixed-index config, a per-CPU map for a hot counter that must not lock, an LRU hash for bounded connection tracking, a program array for tail calls, and a ring buffer for kernel-to-user events. Prefer `BPF_MAP_TYPE_RINGBUF` over `BPF_MAP_TYPE_PERF_EVENT_ARRAY` for new code: one shared buffer, ordering across CPUs, variable-size records, and a reserve that fails instead of dropping. The full type table, operations, and pinning live in `references/ebpf-map-types.md`. Done when: the map type and its `max_entries` are chosen.
+ **Rust/Aya mode for a production program.** Install the toolchain and generate the project from the Aya template. Done when: `cargo build` succeeds on the fresh template.
+ ```bash
+ rustup toolchain install stable
+ rustup toolchain install nightly --component rust-src
+ cargo install bpf-linker
+ cargo install cargo-generate
+ cargo generate https://github.com/aya-rs/aya-template # prompts for name and program type
+ ```
+
+ The template lays out `<name>/` (user-space crate), `<name>-ebpf/` (kernel-side crate), and `<name>-common/` (shared types). A build script compiles the eBPF crate and embeds the object, so ordinary `cargo build`, `cargo check`, and `cargo run --release` drive both sides; there is no separate build step.
+
+ Write the kernel side with `aya-ebpf` and `aya-log-ebpf`. Done when: the crate compiles for the BPF target.
+
+ ```rust
+ // <name>-ebpf/src/main.rs
+ #![no_std]
+ #![no_main]
+
+ use aya_ebpf::{
+ helpers::bpf_get_current_pid_tgid,
+ macros::{map, tracepoint},
+ maps::PerCpuHashMap,
+ programs::TracePointContext,
+ };
+ use aya_log_ebpf::info;
+
+ #[map]
+ static CALL_COUNT: PerCpuHashMap<u32, u64> = PerCpuHashMap::with_max_entries(1024, 0);
+
+ #[tracepoint]
+ pub fn trace_read(ctx: TracePointContext) -> u32 {
+ let pid = (bpf_get_current_pid_tgid() >> 32) as u32;
+ if let Some(count) = CALL_COUNT.get_ptr_mut(&pid) {
+ unsafe { *count += 1; }
+ } else {
+ let one = 1;
+ let _ = CALL_COUNT.insert(&pid, &one, 0);
+ }
+ info!(&ctx, "read() called by pid {}", pid);
+ 0
+ }
+
+ #[panic_handler]
+ fn panic(_info: &core::panic::PanicInfo) -> ! {
+ // The verifier never lets a panic path run; this satisfies no_std.
+ loop {}
+ }
+ ```
+
+ Write the user-space loader on Tokio. Done when: the loader attaches, prints log lines, and exits on Ctrl-C.
+
+ ```rust
+ // <name>/src/main.rs
+ use aya::{include_bytes_aligned, maps::PerCpuHashMap, programs::TracePoint, Ebpf};
+ use aya_log::EbpfLogger;
+ use tokio::{
+ io::{unix::AsyncFd, Interest},
+ signal,
+ task,
+ };
+
+ #[tokio::main]
+ async fn main() -> anyhow::Result<()> {
+ let mut bpf = Ebpf::load(include_bytes_aligned!(concat!(env!("OUT_DIR"), "/<name>")))?;
+ env_logger::init();
+ let logger = EbpfLogger::init(&mut bpf)?;
+ let mut logger = AsyncFd::with_interest(logger, Interest::READABLE)?;
+ let logger_task = task::spawn(async move {
+ loop {
+ let mut guard = logger.readable_mut().await.expect("logger fd wait");
+ guard.get_inner_mut().flush();
+ guard.clear_ready();
+ }
+ });
+
+ let program: &mut TracePoint = bpf.program_mut("trace_read").unwrap().try_into()?;
+ program.load()?;
+ program.attach("syscalls", "sys_enter_read")?;
+
+ let counts: PerCpuHashMap<_, u32, u64> =
+ PerCpuHashMap::try_from(bpf.map("CALL_COUNT").unwrap())?;
+
+ signal::ctrl_c().await?;
+ logger_task.abort();
+ let _ = logger_task.await;
+ for entry in counts.iter().filter_map(Result::ok) {
+ let (pid, per_cpu) = entry;
+ let count: u64 = per_cpu.iter().copied().sum();
+ println!("pid {pid}: {count} reads");
+ }
+ Ok(())
+ }
+ ```
+
+ The template's build script sets the object path; take it from the generated `main.rs` rather than hard-coding one.
+
+ 4. Choose the program type, attach target, and map. In C/libbpf, use a hash map for per-key state, an array for fixed-index config, a per-CPU map for a hot counter that must not lock, an LRU hash for bounded connection tracking, a program array for tail calls, and a ring buffer for kernel-to-user events. In Rust/Aya, select the program macro from the table below and the corresponding map. Prefer `BPF_MAP_TYPE_RINGBUF` over `BPF_MAP_TYPE_PERF_EVENT_ARRAY` (or `RingBuf` over `PerfEventArray`) for new code: one shared buffer, ordering across CPUs, variable-size records, and a reserve that fails instead of dropping. The full C type table, operations, and pinning live in `references/ebpf-map-types.md`. Done when: the program type, attach target, map type, and its capacity (`max_entries` or ring-buffer byte size) are chosen.
+
+ | Macro | Program type | Attach target |
+ |---|---|---|
+ | `#[tracepoint]` | Tracepoint | category and name, such as `"syscalls"`, `"sys_enter_read"` |
+ | `#[kprobe]`, `#[kretprobe]` | Kernel probe | kernel function name |
+ | `#[uprobe]`, `#[uretprobe]` | User probe | binary path and symbol or offset |
+ | `#[xdp]` | XDP | network interface |
+ | `#[classifier]` | TC | interface and direction |
+ | `#[socket_filter]` | Socket filter | socket fd |
+ | `#[perf_event]` | Perf event | perf event fd |
+ | `#[lsm]` | LSM hook | hook name |
+ | `#[sk_msg]` | Sockmap | socket map |
+
+ Kernel-side Aya maps live in `aya_ebpf::maps`: `HashMap`, `LruHashMap`, `PerCpuHashMap`, `Array`, `PerCpuArray`, `RingBuf`, `PerfEventArray`, `LpmTrie`, `ProgramArray`, `XskMap`, and others. `RingBuf::reserve` returns an `Option`, so the kernel side must handle a full buffer.
+
+ ```rust
+ use aya_ebpf::maps::RingBuf;
+
+ #[map]
+ static EVENTS: RingBuf = RingBuf::with_byte_size(256 * 1024, 0);
+
+ if let Some(mut entry) = EVENTS.reserve::<MyEvent>(0) {
+ entry.write(MyEvent { pid, ts });
+ entry.submit(0);
+ }
+ ```
+
+ ```rust
+ // User space: poll the ring buffer through tokio's AsyncFd.
+ use aya::maps::RingBuf;
+ use tokio::io::unix::AsyncFd;
+
+ let ring = RingBuf::try_from(bpf.take_map("EVENTS").unwrap())?;
+ let mut ring = AsyncFd::new(ring)?;
+ loop {
+ let mut guard = ring.readable_mut().await?;
+ let rb = guard.get_inner_mut();
+ while let Some(item) = rb.next() {
+ let event: &MyEvent = unsafe { &*(item.as_ptr() as *const MyEvent) };
+ println!("event from pid {}", event.pid);
+ }
+ guard.clear_ready();
+ }
+ ```
+
```c
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} rb SEC(".maps");
SEC("kprobe/do_sys_openat2")
int handle_open(struct pt_regs *ctx)
{
struct event *e = bpf_ringbuf_reserve(&rb, sizeof(*e), 0);
if (!e)
return 0;
e->pid = bpf_get_current_pid_tgid() >> 32;
bpf_ringbuf_submit(e, 0); /* bpf_ringbuf_discard(e, 0) on an error path */
return 0;
}
```
User space consumes with `ring_buffer__poll`; the perf buffer equivalent is `perf_buffer__poll` over `bpf_perf_event_output`.
- 5. Triage a verifier rejection. Load with `bpftool` to see the full log, then map the message. Done when: the message has a cause and the program passes.
+ 5. Triage a verifier rejection or Aya load failure. Load with `bpftool` to see the full log, then map the message; Aya surfaces the verifier log as error text. Done when: the message has a cause and the program passes or loads.
```bash
bpftool prog load prog.bpf.o /sys/fs/bpf/prog type kprobe 2>&1 | head -100
bpftool prog list
bpftool prog dump xlated id 42
```
| Verifier message | Cause | Fix |
|---|---|---|
| `invalid mem access 'scalar'` | Dereference of a pointer the verifier could not bound | Null-check the pointer before use |
| `R0 !read_ok` | A path returns without setting `R0` | Return a value on every path |
| `jump out of range` | Branch target beyond the program | Restructure the conditional |
| `back-edge detected` | A loop the verifier cannot bound | Bound the loop with a constant, or use `bpf_loop()` (kernel 5.17 and later; present on the grounded floor) |
| `unreachable insn` | Dead code after a return | Remove the dead branch |
| `invalid indirect read` | Stack bytes read before being written | Zero-initialize the struct: `struct foo x = {};` |
| `misaligned stack access` | Pointer arithmetic off alignment | Align reads to `__u64` |
+ In Rust/Aya mode, inspect the load and verifier output with:
+
+ ```bash
+ RUST_LOG=debug cargo run --release 2>&1 | grep -A 20 verifier
+ bpftool btf dump file /sys/kernel/btf/vmlinux | grep task_struct
+ bpftool prog list
+ bpftool prog dump xlated name trace_read
+ ```
+
+ | Aya error | Cause | Fix |
+ |---|---|---|
+ | `invalid mem access` | Pointer dereferenced without a bound | Check the `Option` or `Result` before reading |
+ | BTF type not found | Bindings generated against a different kernel | Regenerate `vmlinux.rs` on the target kernel |
+ | `Permission denied` | No `CAP_BPF` or `CAP_SYS_ADMIN` | Run as root or grant the capability |
+ | Map already exists | A pinned map from a previous run | Unpin it or rename the map |
+
+ For Tokio patterns in the loader, use `rust-async-internals`. For raw pointer reads on the kernel side, use `rust-unsafe`.
+
6. For packet filtering, write an XDP program and attach it in driver mode where the NIC supports it. Done when: the program is attached and the interface shows it, and the detach command is recorded.
```c
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>
SEC("xdp")
int xdp_filter(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
struct ethhdr *eth = data;
if ((void *)(eth + 1) > data_end)
return XDP_PASS;
if (bpf_ntohs(eth->h_proto) != ETH_P_IP)
return XDP_PASS;
struct iphdr *ip = (void *)(eth + 1);
if ((void *)(ip + 1) > data_end)
return XDP_PASS;
if (ip->protocol == IPPROTO_ICMP)
return XDP_DROP;
return XDP_PASS;
}
char LICENSE[] SEC("license") = "GPL";
```
```bash
ip link set dev eth0 xdpdrv object xdp_drop_icmp.bpf.o section xdp # driver mode
ip link set dev eth0 xdpgeneric object prog.bpf.o section xdp # fallback when the driver lacks XDP
ip link set dev eth0 xdp off # detach
```
Return codes are `XDP_PASS`, `XDP_DROP`, `XDP_TX` (send back out the same port), and `XDP_REDIRECT`. For redirecting into AF_XDP sockets, use `af-xdp`; for a full kernel bypass, use `dpdk`.
- 7. Make the program portable with CO-RE. Include `vmlinux.h` generated from the build host's BTF and read kernel struct fields through `BPF_CORE_READ`, which libbpf relocates to the running kernel's field offsets at load time. Done when: the same object loads on a second kernel version.
+ 7. Make the program portable with CO-RE. In C/libbpf, include `vmlinux.h` generated from the build host's BTF and read kernel struct fields through `BPF_CORE_READ`, which libbpf relocates to the running kernel's field offsets at load time. In Rust/Aya, generate `vmlinux.rs` with `aya-tool` when the program reads kernel structs and read fields through `aya_ebpf::helpers::bpf_probe_read_kernel`; aya-tool's generated structures are portable across Linux kernel versions via BPF CO-RE, provided the target kernel has `CONFIG_DEBUG_INFO_BTF`. Done when: the same C object loads on a second kernel version, and an Aya program has compiling generated bindings on a BTF-enabled target.
```c
#include <vmlinux.h>
#include <bpf/bpf_core_read.h>
SEC("kprobe/tcp_connect")
int trace_connect(struct pt_regs *ctx)
{
struct sock *sk = (struct sock *)PT_REGS_PARM1(ctx);
u16 dport = BPF_CORE_READ(sk, __sk_common.skc_dport);
bpf_printk("connect to port %d\n", bpf_ntohs(dport));
return 0;
}
```
+ In Rust/Aya mode, generate the bindings with:
+
+ ```bash
+ cargo install bindgen-cli
+ cargo install --git https://github.com/aya-rs/aya -- aya-tool
+ aya-tool generate task_struct > <name>-ebpf/src/vmlinux.rs
+ ```
+
+ Aya's loader relocates generated accesses against the running kernel's BTF; the target kernel must have `CONFIG_DEBUG_INFO_BTF`.
+
8. Reach for iterators and atomics when the simple form is too slow. An `SEC("iter/task")` program walks every task once through `bpf_seq_printf` without a probe firing per element; read its output with `bpftool prog tracelog` or the seq file the loader creates. For a counter updated by concurrent probes, use a per-CPU array and sum in user space; when a single shared value is required, increment it with `__sync_fetch_and_add`. The full BPF atomic set (fetch-and-add with a returned value, and, or, xor, exchange, compare-exchange) is available from kernel 5.12; plain add-without-return predates it. Both are on the grounded floor. Done when: the hot path uses per-CPU storage or an atomic, not a plain read-modify-write.
For seccomp filtering, use `kernel-security`. For module development that a probe targets, use `linux-kernel-modules`.
## Failure and recovery
| Failure class | Behavior |
|---|---|
- | `/sys/kernel/btf/vmlinux` missing | The kernel lacks `CONFIG_DEBUG_INFO_BTF`. CO-RE cannot work; report it and fall back to a kernel with BTF. |
+ | `/sys/kernel/btf/vmlinux` missing | The kernel lacks `CONFIG_DEBUG_INFO_BTF`. CO-RE cannot work and Aya cannot generate target-BTF bindings; report it and fall back to a kernel with BTF. |
| `Permission denied` on load | The process lacks `CAP_BPF` (or `CAP_SYS_ADMIN` for some program types). Run as root or grant the capability; do not weaken `kernel.unprivileged_bpf_disabled`. |
| Verifier rejects the program | Map the message with the table in step 5. Fix the program; do not shrink `max_entries` or remove checks to make it pass. |
| XDP attach fails in driver mode | The driver lacks native XDP. Attach with `xdpgeneric` and note the throughput cost. |
- | Ring buffer reserve returns null | The buffer is full. Enlarge `max_entries` or drain faster in user space; count the drops. |
+ | Ring buffer reserve returns null or `None` | The buffer is full. Enlarge `max_entries` or drain faster in user space; count the drops. |
| Map already pinned | A previous run left a pin under `/sys/fs/bpf`. Reuse it or unpin it before reloading. |
+ | `bpf-linker` missing or the wrong LLVM | The Aya eBPF crate fails to link. Install the version required by the `bpf-linker` README for the installed LLVM; do not pin an old nightly to dodge it. |
+ | Loader killed without cleanup | Programs detach when their file descriptors close; pinned objects survive under `/sys/fs/bpf` and are removed by hand. |
## Output
- The program source, the build and load commands, the attach and detach pair, and either the traced output or the verifier message mapped to its fix. Every kernel-side object the run leaves behind is listed with its removal command.
+ The selected program source (including kernel-side and user-space crates for Aya), the build and load commands, the attach and detach pair, and evidence: traced output, log lines, or map entries for a live event, or the verifier message mapped to its fix. Every kernel-side object the run leaves behind is listed with its removal command.