Performance Testing

BLIT386 has CPU micro-benchmarks for hot methods. This guide explains when to use them, how to add a new benchmark, and how to compare results against a baseline locally.

Overview

BLIT386 uses Vitest bench for CPU micro-benchmarks. These measure isolated methods, hot loops, cache lookups, math helpers, and allocation patterns.

For visual correctness (not performance), use the visual regression tests: pnpm run test:visual. They run Playwright with Chromium + WebGPU and produce PNG snapshots. See Testing for details.

For whole-engine sprite throughput in a real browser, rather than one method in Node, open the bunnymark demo: it spawns bouncing sprites by the hundred and counts the ones held back by the WebGPU sprite cap, so a renderer change can be judged by how many bunnies a machine moves before a frame slips. It is a reference point, not a gate - nothing records its numbers.

When to use CPU benchmarks

Use CPU benchmarks when you want to measure a single method, hot loop, cache lookup, math helper, allocation pattern, or batching helper in isolation.

Examples:

• Vector2i.add() vs. Vector2i.addInPlace()
• Color32.toFloat32Array() vs. Color32.writeToFloat32Array()
• BitmapFont.measureText() cold vs. warm cache
• Rect2i.isContainingXY() vs. Rect2i.isContaining()

CPU benchmarks

CPU benchmarks are implemented with Vitest bench and colocated next to the source as *.bench.ts files.

Current benchmark files:

• src/BLIT386.bench.ts
• src/utils/Vector2i.bench.ts
• src/utils/Color32.bench.ts
• src/utils/Random.bench.ts
• src/utils/Rect2i.bench.ts
• src/assets/BitmapFont.bench.ts
• src/assets/Palette.bench.ts
• src/assets/PaletteEffect.bench.ts
• src/assets/SpriteSheet.bench.ts
• src/assets/SystemFont.bench.ts
• src/core/GameLoop.bench.ts
• src/input/GamepadInput.bench.ts
• src/input/KeyboardInput.bench.ts
• src/input/PointerInput.bench.ts
• src/overlay/Overlay.bench.ts
• src/overlay/palette/PaletteView.bench.ts
• src/render/PrimitivePipeline.bench.ts
• src/render/SoftwareRenderer.bench.ts
• src/render/SpritePipeline.bench.ts

Overlay palette grid

These benchmarks guard perf follow-ups for the live palette swatch grid:

They run in the same Vitest bench suite as the rest of the repo and are included in benchmark-results.json when you run pnpm run bench:json. No separate registration step is required.

Engine hot paths (2026 performance audit)

These benchmarks were added ahead of the engine-wide performance optimization pass, so each optimization PR can quote a measured before/after improvement:

Metrics

CPU benchmarks report ops/sec. Higher numbers are better.

If one benchmark shows:

• 20,000,000 ops/sec

and another shows:

• 10,000,000 ops/sec

the first one is roughly twice as fast.

Why this is the default choice

CPU benchmarks are:

• faster to run locally
• easier to write
• easier to reason about
• already wired into the local baseline/compare workflow described below

If you add a new hot method and want regression protection, this is the first tool to use.

Adding a new CPU benchmark

If you add a new method and it can run without a browser, start here.

File location

Create or extend a *.bench.ts file near the code being measured.

Examples:

src/utils/MyType.bench.ts
src/render/SpritePipeline.bench.ts

Basic structure

import { bench, describe } from 'vitest';

import { MyType } from './MyType';

describe('MyType hot paths', () => {
  const instance = new MyType();

  bench('newMethod()', () => {
    instance.newMethod();
  });

  bench('oldMethod()', () => {
    instance.oldMethod();
  });
});

What to compare

Good benchmark comparisons usually measure one meaningful tradeoff:

• new method vs. previous method
• allocating vs. in-place
• small input vs. large input
• cold cache vs. warm cache
• vector argument vs. raw x, y argument

Benchmark design rules

• keep the benchmark focused on one hot behavior
• use realistic inputs, not absurd synthetic values unless stress testing is intentional
• compare alternatives in the same file so the relative result is obvious
• prefer stable setup outside the bench(...) callback
• if mutation is involved, reset state inside the benchmark or in setup so each iteration is valid

Run CPU benchmarks

pnpm run bench
pnpm run bench:json

pnpm run bench gives the terminal report.

pnpm run bench:json writes benchmark-results.json, the machine-readable file the local compare workflow reads.

Commands

pnpm run bench           # Run all CPU benchmarks
pnpm run bench:json      # Run benchmarks and write benchmark-results.json
pnpm run bench:baseline  # Run benchmarks and write benchmark-baseline.json
pnpm run bench:compare   # Compare benchmark-results.json against benchmark-baseline.json

Which command should I use?

• New hot method: pnpm run bench
• Capturing a baseline on main before starting perf work: pnpm run bench:baseline
• Machine-readable result for a feature branch: pnpm run bench:json
• Comparing a feature branch against the captured baseline: pnpm run bench:compare

Local baseline/compare workflow

Benchmarks run locally only. GitHub-hosted CI runners share CPU, have no GPU, and their run-to-run variance regularly crosses a useful regression threshold, so CI does not gate on benchmark results.

git checkout main
pnpm run bench:baseline

git checkout <feature-branch>
pnpm run bench:json

pnpm run bench:compare

pnpm run bench:compare -- --baseline other-baseline.json --current other-results.json --threshold 10

What the comparison contains

benchmark-comparison.md includes:

• benchmark name
• baseline ops/sec
• current ops/sec
• delta percent
• pass/fail status

Quiet-machine discipline

Local runs replace the CI gate, so run-to-run noise matters more than it used to:

• close other heavy applications before benchmarking
• run on a plugged-in machine, not on battery
• repeat a run that looks marginal before trusting it

Coverage reminder

Nothing forces a benchmark to exist for a given hot path, so a change can slip through with no coverage to compare against. A pre-push hook (scripts/check-bench-coverage.mjs) prints a reminder, never a failure, when a push touches the renderer, input, overlay, core, assets, or utils directories without touching any *.bench.ts file. An agent rule (.claude/rules/bench-coverage.md) covers the same directories for anyone using an AI coding assistant on this repo.

Recommended workflow for new performance work

If you add a new sprite operation, follow this order:

For overlay palette usage or grid draw changes, include the baseline/compare steps so SpriteSheet.bench.ts and PaletteView.bench.ts are compared against main.

Best default

If you are unsure where to start, use CPU benchmarking with Vitest bench. It is simple, fast, and already supported by the local baseline/compare workflow.

See also

Last updated on October 18, 2026