Testing and Benchmarking

js/testing-and-benchmarking.md

Fino includes its own test and benchmark modules so runtime features can be tested without depending on another JavaScript runtime. Tests emit TAP-13. Benchmarks use adaptive measurement loops and print throughput-oriented results.

Writing Tests

Use test for standalone cases or grouped suite tests:

import { suite, test } from 'fino:test/test';

suite('math', () => {
  test('adds numbers', (t) => {
    t.equal(1 + 1, 2);
  });

  test('supports async work', async (t) => {
    const value = await Promise.resolve('ready');
    t.equal(value, 'ready');
  });
});

Use describe and it when lifecycle hooks make the test clearer:

import { beforeEach, describe, it } from 'fino:test/test';

describe('cache', () => {
  let cache: Map<string, string>;

  beforeEach(() => {
    cache = new Map();
  });

  it('stores values', (t) => {
    cache.set('name', 'fino');
    t.equal(cache.get('name'), 'fino');
  });
});

Do not mix the two grouping styles in the same group. Use suite with test, or use describe with it and lifecycle hooks.

The runner provides TAP-13 output, --filter name matching, skip reasons, before/after/beforeEach/afterEach hooks, and captured stdout/stderr for failures. It is not a Node node:test compatibility layer, so only, todo, per-test timeouts, concurrent test scheduling, assertion-object subtests, and pluggable reporters are not included.

Running Tests

Run specific files:

fino test tests/net/serve.test.ts

Run all tests in a directory:

fino test tests/net

Filter by registered test name:

fino test --filter websocket tests/net

Directory arguments expand to *.test.ts files. Keep tests close to the system they cover: networking tests under tests/net, runtime tests under tests/runtime, file tests under tests/file, and so on.

Assertions

The test callback receives an assertion object:

test('response status', async (t) => {
  const response = new Response('created', { status: 201 });

  t.equal(response.status, 201);
  t.ok(response.headers instanceof Headers);
});

Prefer focused assertions that describe the behavior being protected. Broad end-to-end tests are useful for integration paths, but narrow regression tests are easier to diagnose.

Mocks

Use mock helpers when the behavior under test depends on replaceable runtime APIs. Keep the mocked surface small. Scoped helpers restore the original behavior when the callback finishes.

import { test } from 'fino:test/test';
import { mockFetch } from 'fino:test/mock';

test('fetches data', async (t) => {
  await mockFetch(async (mock) => {
    mock
      .get('https://example.com/data')
      .reply(200, JSON.stringify({ ok: true }), {
        headers: { 'content-type': 'application/json' },
      });

    const response = await fetch('https://example.com/data');
    const data = await response.json();

    t.equal(data.ok, true);
    t.equal(mock.calls.length, 1);
  });
});

Writing Benchmarks

Benchmarks register named measurements:

import { bench } from 'fino:test/bench';
import { parse } from 'fino:format/csv';

bench('csv', (b) => {
  const source = 'name,count\nalpha,1\nbeta,2\n';

  b.measure('parse with headers', () => {
    parse(source, { header: true });
  });
});

Use setup and teardown when resource creation should not be part of the timed body:

b.measure('lookup', {
  setup() {
    return new Map([['name', 'fino']]);
  },
  fn(cache) {
    cache.get('name');
  },
});

Run benchmarks in the same environment when comparing performance:

fino bench benchmarks

The benchmark harness prints human, benc.h-style text output. It uses an adaptive minimum-duration measurement loop and supports sync or async measured functions plus setup/teardown outside the timed body. It does not currently provide JSON output, machine-readable result objects, public warmup or fixed-iteration controls, fixed sample counts, variance thresholds, pluggable reporters, or CI regression gates.

Use stable machines for numbers you intend to keep or compare.

Testing AI Behavior

Prompts, tool routing, and agent behavior are tested with evals, which run as ordinary test cases with scorers and reporters. See Evals and OpenTelemetry.