Audit what a web server sends

See which files nginx, PHP-FPM, Node.js and other servers serve, how each transfer is attributed, and how PHP request scripts tell apart the requests behind one worker.

The question: what is this web server actually sending, from which file, and which request or script is behind it?

A web server is where EXACT and INFERRED differ most, and where the SCRIPT column earns its place.

​

Static files: sendfile, EXACT

nginx serves static files with sendfile, so the kernel moves the file to the socket itself. Those transfers are EXACT:

sudo filetrace files --process nginx --since 1h --sort bytes

FILE                        PROCESS  DESTINATION    BYTES   METHOD    CONFIDENCE  SCRIPT
/var/www/media/video.mp4    nginx    198.51.100.4   1.20 GB  sendfile  EXACT       -
/var/www/media/brochure.pdf nginx    203.0.113.9    12.4 MB  sendfile  EXACT       -
/var/www/downloads/app.tgz  nginx    198.51.100.9   85.0 MB  sendfile  EXACT       -

The BYTES shown is what the socket accepted, which for a full response is close to the file size. Compare it with file.size in --json output to see whether a transfer was cut short.

​

Application responses: read-write, INFERRED

A framework reads a file and writes it into the response, so the bytes pass through user space. Those transfers are INFERRED, and the SCRIPT column names the program that ran:

sudo filetrace files --process 'php-fpm*' --since 1h

FILE                              PROCESS     DESTINATION  BYTES   METHOD      CONFIDENCE  SCRIPT
/var/www/shop/invoices/1042.pdf   php-fpm8.4  127.0.0.1    2.10 MB  read-write  INFERRED    /var/www/shop/download.php
/var/www/shop/exports/orders.csv  php-fpm8.4  127.0.0.1    640 KB   read-write  INFERRED    /var/www/shop/admin/export.php

The SCRIPT is the script of the request, not the worker’s pool, so two endpoints served by the same worker are told apart. This is available from 26.11.6.5 and can be turned off with [process] php_scripts = false.

​

Ask about one route with --script

# Everything sent by the admin export endpoint
sudo filetrace files --script '/var/www/shop/admin/export.php' --since 24h

# Every file sent by any PHP script under one site
sudo filetrace transfers --script '/var/www/shop/*' --since 24h

# What a Node.js service sent, by module
sudo filetrace files --process node --script 'module:*'

--script matches the path of an interpreter’s script, module:<name> for a Python or Node module entry point, or pool:<name> for a PHP-FPM worker outside a request the agent saw.

​

Rank the files a server sends most

sudo filetrace top files --since 1h --limit 20

top files merges transfers of the same file across sockets, so one popular asset is one row with the total bytes, which is the number you want for cache decisions.

​

Split served files from generated responses

Not every byte a server sends comes from a file. Injected HTML, JSON built from a database and proxied responses have no file behind them. Use transfers to see the split:

sudo filetrace transfers --process nginx --since 1h

Rows with FILE unknown are the generated part. In the metrics, the same split is:

# Share of egress that is backed by a file, per process
sum by (process_executable_name) (rate(filetrace_file_sent_bytes_total[5m]))
  / ignoring(job) group_left()
sum by (process_executable_name) (rate(filetrace_network_io_bytes_total{network_io_direction="transmit"}[5m]))

A low share for a static-file server is suspicious; a high share for an API is unusual.

​

Keep the audit trail

Send file_transfer records to a collector so downloads can be reviewed later:

[otlp]
enabled = true
endpoint = "127.0.0.1:4317"
events = ["file_transfer"]
file_paths = true   # set false if paths must not leave the host

To avoid recording the trust stores, modules and libraries a server reads on the way to a connection, Filetrace treats them as background files by default; see [correlation] background_paths in the configuration reference. They are identified and read, but held back from the transfer list unless they are the only thing on the wire.

​

Where the answer can mislead

• Keep-alive and HTTP/2 mix requests. Bytes are attributed to files by thread and timing, so two responses on one connection are separated by evidence, not by request boundaries. INFERRED reflects that.
• A proxy or gateway has no file. Bytes forwarded from an upstream socket have no local file and appear as UNKNOWN.
• --remote on a reverse proxy shows the proxy, not the client, when the proxy is on the same host and connects to itself. Use the socket view to follow it.
• Compression changes the byte count but not the file attribution: the file is named from the pre-compression read or the sendfile call.

​

Related pages

• filetrace files — the columns and options.
• PHP requests and script attribution
• Investigate a destination