> ## Documentation Index
> Fetch the complete documentation index at: https://filetrace.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshoot Filetrace

> Fix common Filetrace problems: an agent that is not running, missing kernel support, lost events, files that do not appear, and unexpected attributions.

Most Filetrace problems fall into three groups: the agent cannot run on this kernel, the agent is not reachable, or an answer is missing or incomplete. Start with `filetrace doctor`, which checks the kernel, every probe and the running agent's event pipeline in one report, then find the symptom below.

```bash theme={null}
sudo filetrace doctor
```

`doctor` exits with `0` for `READY`, `3` for `NEEDS PRIVILEGES`, `5` for `UNAVAILABLE` and `9` for `DEGRADED` (some probes are missing, or the agent is losing events). Its `FEATURES` section marks each feature `[OK]`, `[WARN]` (in part), `[OFF]` (turned off) or `[NONE]` (not available), with the reason.

## The agent is not running

```text theme={null}
The Filetrace agent is not running (/run/filetrace/agent.sock: No such file or directory).
```

The command exits with code `4`. Start the service, or run the agent in the foreground to see why it stops:

```bash theme={null}
sudo systemctl enable --now filetrace-agent
sudo filetrace agent run --log-level debug
```

If the agent runs but listens elsewhere, point the command at its socket with `--socket` or `$FILETRACE_SOCKET`, or set `[agent] socket` in the configuration file that both read.

The systemd unit does not retry after an exit with code `2`, `3`, `5` or `6`. Check `journalctl -u filetrace-agent` (or `/var/log/filetrace-agent.log` under OpenRC and SysV init) for the reason.

## The service does not start: no BTF

The systemd unit starts only when `/sys/kernel/btf/vmlinux` exists, and the OpenRC script refuses to start without it:

```text theme={null}
This kernel has no BTF (/sys/kernel/btf/vmlinux), which filetrace-agent needs
```

The probes are adapted to the kernel through BTF type information. Use a kernel built with `CONFIG_DEBUG_INFO_BTF`, as the kernels of current major distributions are. Collection needs Linux 6.1 or later.

## doctor reports NEEDS PRIVILEGES

Loading the probes needs privileges that a regular user does not have; `doctor` lists `CAP_BPF` and `CAP_PERFMON` among its checks. Run `doctor`, `agent run`, and `watch` or `events --follow` without an agent, with `sudo`. The systemd unit gives the agent `CAP_BPF`, `CAP_PERFMON`, `CAP_SYS_ADMIN` and `CAP_SYS_PTRACE`.

## A probe does not attach

When a probe is not `[OK]` in the `PROBES` section, `doctor` ends with `DEGRADED` and names it. Kernel functions are traced through BPF trampolines where the kernel supports them and through kprobes otherwise. Check the other set:

```bash theme={null}
sudo filetrace doctor --probes kprobe
```

If the kprobe set attaches in full, select it with `probes = "kprobe"` in `[agent]` or `--probes kprobe`. [Kernel instrumentation](https://filetrace.mintlify.site/concepts/kernel-probes.md) describes the differences between the sets.

## Permission denied or no answer as a regular user

The agent answers only root and the user it runs as, and closes other connections. Run the commands with `sudo` when the agent runs as root, as the packaged service does.

## Exit code 9: events were lost

Exit code `9` means the answer may be incomplete because events were lost inside the queried time range. The output is still printed. `filetrace stats` and the `LOST` counter of `watch` show how much was lost, and the Prometheus counter `filetrace_events_dropped_total` says where, by `filetrace_loss_kind`:

| Loss kind | Cause | Remedy |
| - | - | - |
| `event_queue` | The kernel-to-agent event queue overflowed. | Raise `[agent] ring_buffer_size` (4 MB by default) or `--queue-size`. |
| `counter_table` | The in-kernel counter table was full. | Raise `[limits] kernel_counters` (16384 by default), or shorten `[agent] flush_interval`. |
| `socket_identity` | A transfer's socket had no usable identity. | None in the configuration. |

A transfer during which events were lost is marked with `*` after its confidence in tables and with `"incomplete": true` in JSON. See [Overhead and sizing](https://filetrace.mintlify.site/guides/performance.md).

## A file you expected is not listed

Work through these causes in order:

1. **The agent started later.** History starts when the agent starts and is kept in memory only. Every report says from when it reaches.
2. **The history forgot it.** Every table is bounded and forgets its least recently updated entries when full. Raise the `[limits]` of busy tables.
3. **It is filtered out.** Check `[filters] exclude_paths` and `exclude_processes` in `filetrace config show`, and the options of your command.
4. **It was below the reporting thresholds.** A read-write association is reported once it covers at least `inferred_min_bytes` (1 KB) and `inferred_min_percent` (5%) of what was read. With `confidence_threshold = "exact"`, only sendfile transfers are reported.
5. **The way it was sent is not observed.** Files read with `readv`, io\_uring or a memory mapping, files read by one thread and sent by another, and files passed through a pipe are not associated with the send. `filetrace transfers --confidence unknown` shows the bytes the process sent without a file.
6. **The receive rule ended the association.** A thread that receives between reading a file and sending is not associated with the file. A file read entirely before a fresh TLS connection is opened is not associated, because the handshake's receive ends the association.

`filetrace capabilities` lists every feature this machine supports, in full or in part, with the reason.

## An INFERRED attribution looks wrong

INFERRED means that the same thread read the file and then sent on the socket within the correlation window, with no receive in between. It can be wrong when a thread reads one file and then sends unrelated data of a similar size. Shorten `[correlation] window` to reduce such matches, or use `--confidence exact` to list only transfers the kernel performed itself. [File attribution and confidence](https://filetrace.mintlify.site/concepts/attribution.md) explains the rules.

## Device and inode look wrong in a container

Inside a container on overlayfs, device and inode are those of the overlay mount. On btrfs, files in different subvolumes can share an inode number.

## The agent did not restart after an upgrade

A package upgrade restarts a running agent only when the configuration file is valid for the new version. Run `filetrace config validate`, fix what it reports, and restart the service.

## A method filter is refused

`--method splice`, `--method mmap` and `--method io_uring` are accepted but refused with exit code `5`, because this build does not observe those methods. It does not answer with an empty list.

## Nothing is collected on macOS

On macOS the command-line interface builds and reports diagnostics only. Collection needs Linux 6.1 or later.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.