API Latency Benchmark (api-latency-benchmark)

Summary

Repeated HTTP requests with percentile latency statistics.

Classification

• Category: API_HEALTH
• Plugin id: api-latency-benchmark
• Version: 1.1.0
• Target types: API
• Weight class: LIGHT
• Risk level: not set on plugin object
• Browser required: No (HTTP / Node / other — see source)
• Tier / group: Tier 4 — API

Schedule

• scheduleHints: { defaultInterval: '0 */3 * * *', priority: 6 }
• Prerequisites: None declared on this plugin.

Configurable inputs

Public keys are derived from inputSchema: z.object({ ... }) (underscore-prefixed keys are runtime-only and omitted here).

Field	Notes
url	Optional override; defaults to target.url.
method	GET, POST, PUT, or HEAD (default GET). No PATCH or DELETE.
headers	Extra headers merged with the TesteBot user agent.
requests	Sample count (5–100, default 20).
warnP95Ms	p95 above this → slow finding (default 800 ms).
criticalP95Ms	p95 above this → high-latency finding (default 4000 ms).
timeout	Per-request timeout in ms (default 10000).

Runtime / injected config

• None documented beyond standard worker context (target, config, logger, reportProgress). Underscore-prefixed keys are internal.

Checks that produce findings

These checks run inside this plugin during a single test run. They are not separate tests in the database — each becomes a Finding linked to the Run for api-latency-benchmark.

Check	Typical severity	Why
High API latency p95 (above criticalP95Ms, default 4000 ms)	WARNING	Tail latency at this level usually means timeouts, queue backlog, or infra stress that will hit real API clients and SLAs.
Slow API latency p95 (above warnP95Ms, default 800 ms)	WARNING	Elevated p95 degrades mobile apps, checkout flows, and chained microservice calls before hard failures appear.
Failed requests during benchmark	WARNING	Partial failures mean the API is unreliable under repeated load even when successful samples look fast.

How each check works

1. Sample loop — runLatencySamples() issues requests sequential HTTP calls (default 20) with a 200 ms pause between each (anti-bot politeness; not configurable from this plugin).
2. Per sample — fetch with redirect: 'follow', AbortSignal.timeout(timeout), TesteBot user agent, and optional method/headers. SSRF-checked via assertSafeUrl.
3. Failed sample handling — On timeout or network error: increments the errors counter and pushes the full timeout value into the timings array (failed requests count as worst-case latency for percentile math).
4. Percentiles — computeLatencyStats() sorts timings and derives avg, p50, p95, p99, min, max (floor index, not interpolation).
5. Latency findings — If p95 > criticalP95Ms → “High API latency” WARNING (takes precedence). Else if p95 > warnP95Ms → “Slow API latency” WARNING. Both use WARNING severity despite the “critical” threshold name.
6. Error finding — Separate WARNING when errors > 0, even if p95 is within thresholds.
7. Score — 100 when p95 ≤ warnP95Ms and errors === 0; 55 when p95 ≤ criticalP95Ms; 25 otherwise. Run status is warning when any finding exists (never failed / CRITICAL).

Caveats and concerns

• Failed requests inflate p95 — Errors add a timing equal to the timeout value, so a few failures can push p95 to the timeout even when successful requests were fast.
• Sequential, not concurrent — Samples run one-by-one with 200 ms gaps; a run of 20 requests takes at least ~4 s plus latency — not a load test.
• No auth — Unlike api-health-check, there is no built-in bearer/basic/api-key support; auth headers must be passed manually via headers.
• Response body ignored — Any HTTP status that completes without throwing counts as success; 500 responses are not counted as errors unless the fetch throws.
• “Critical” threshold is still WARNING — criticalP95Ms only changes finding title/wording and score band; severity stays WARNING for platform severity policy on API latency.
• Methods limited — PATCH and DELETE are not in the schema enum.

Source

packages/test-plugins/src/plugins/api-latency-benchmark.ts