8000
Skip to content

Repository files navigation

Doubter

The Node.js performance testing tool with unit-test-like API.

  • Runs each test in a separate process;
  • Measures execution time using performance;
  • Runs tested functions in multiple batches to reduce garbage collection interference;
  • Warms up tested functions;
  • Measures memory consumption using process.memoryUsage.
npm install toofast --save-dev

🔰 Usage

🚀 Authoring tests

⚙️ Configuration

Usage

Let's write a performance test for a function that computes a factorial.

Create a file factorial.perf.js:

import { describe, test, measure } from 'toofast';

function factorial(x) {
  return x === 0 ? 1 : x * factorial(x - 1);
}

describe('factorial', () => {

  test('of 33', () => {
    measure(() => {
      factorial(33);
    });
  });

  test('of 42', () => {
    measure(() => {
      factorial(42);
    });
  });
});

Call toofast in the same directory with this file:

npx toofast

Authoring tests

TooFast package exports several callbacks that you can use in test file.

test

The minimum setup that you need in a test file is the test callback which encloses a group of measurements. For example, let's say there's a function factorial which performance must be measured. Your whole test could be:

import { test, measure } from 'toofast';

test('factorial of 33', () => {
  measure(() => {
    factorial(33);
  });
});

The measure callback starts the performance measurement. It can be invoked multiple times inside a test block to collect a data population from which an average results are derived.

test('factorial of 33 and 42', measure => {

  measure(() => {
    factorial(33);
  });

  measure(() => {
    factorial(42);
  });
});

The measure callback returns a promise that is resolved as soon as performance measurement is completed.

describe

Creates a block that groups together several related tests.

import { describe, test, measure } from 'toofast';

describe('factorial', () => {

  test('of 42', measure => {
    measure(() => {
      factorial(42);
    });
  });
});

describe blocks can be nested:

describe('Math', () => {

  describe('factorial', () => {
    // Tests go here
  });
});

Lifecycle hooks

Hooks are invoked at different phases of the performance test suite lifecycle: beforeEach, afterEach, afterWarmup, beforeBatch, afterBatch, beforeIteration, and afterIteration.

The chart below demonstrates when they are called.

flowchart TD

describe --> testLifecycle

subgraph testLifecycle [Test lifecycle]
    direction LR

    subgraph warmup [Warmup]
        direction TB
        warmup_beforeWarmup(beforeWarmup) -->
        warmup_beforeBatch(beforeBatch) -->
        warmup_beforeIteration(beforeIteration) -->
        warmup_callback[callback] -->
        warmup_afterIteration(afterIteration) -->
        warmup_afterBatch(afterBatch) -->
        warmup_afterWarmup(afterWarmup)
    end

    subgraph batch [Measure]
        direction TB
        measure_beforeBatch(beforeBatch) -->
        measure_beforeIteration(beforeIteration) -->
        measure_callback[callback] -->
        measure_afterIteration(afterIteration) -->
        measure_afterBatch(afterBatch) -->
        measure_beforeBatch
    end

    beforeEach(beforeEach) -->
    test -->
    warmup -->
    batch -->
    afterEach(afterEach)
end
Loading

Hooks can be registered at root level, or inside a describe or test block. Registered hooks affect all measure calls that are nested in a block.

Hooks are always registered before any measurements are started, so the code below would first register beforeEach and beforeIteration hooks and only after that would run measure.

import { beforeEach, beforeIteration, describe, test, measure } from 'toofast';

describe('factorial', () => {

  beforeEach(() => {
    // Runs before each test
  });

  test('of 42', () => {
    measure(() => {
      factorial(42);
    });

    beforeIteration(() => {
      // Runs before each measure() iteration
    });
  });
});

TypeScript support

TooFast can automatically strip TypeScript types without any additional configuration if *.{ts,mts} files are included:

toofast --include '**/*.perf.ts'

This feature requires Node.js 22.6+.

Configuration

By default, TooFast searches for .toofastrc, toofast.json, or toofast.config.js in the current working directory.

{
  "testOptions": {
    "measureTimeout": 5000
  },
  "include": [
    "**/*.perf.js"
  ]
}
testOptions

The default test options used for all tests.

include

The array of glob patterns of included test files. File paths are resolved relative to the config file. Defaults to **/*.perf.{js,mjs,ts,mts}.

setup

The array of glob patters of files that are evaluated in the test environment before any test suites are run. File paths are resolved relative to the config file.

Test options

measureTimeout

The maximum measure duration in milliseconds. Doesn't include the duration of warmup iterations. Defaults to 10_000.

targetRme

The maximum relative margin of error that must be reached for each measurement [0, 1]. Defaults to 0.05.

warmupIterationCount

The maximum number of warmup iterations that are run before each measurement. Defaults to 1. Set to 0 to disable warmup.

batchIterationCount

The maximum number of iterations in a batch. Unlimited by default.

batchTimeout

The maximum duration of batched measurements in milliseconds. Defaults to 1_000.

batchIntermissionTimeout

The delay between batched measurements in milliseconds. VM is expected to run garbage collector during this delay. Defaults to 200.

Command line arguments

--config

The path to a configuration file.

toofast --config toofast.cfg
--include

The array of glob patterns of included test files.

File paths are resolved relative to the current working directory. Provide this option multiple times to include multiple patterns. include option is specified in a config file is overridden by --include arguments.

toofast --include '**/*.perf.js' --include './perf/**/*.mjs'
--setup

The array of glob patters of files that are evaluated in the test environment before any test suites are run.

File paths are resolved relative to the current working directory. Provide this option multiple times to include multiple patterns. setup option is specified in a config file is overridden by --setup arguments.

toofast --setup setup.perf.js
--measureTimeout

The maximum measure duration in milliseconds. Overrides measureTimeout test option.

--targetRme

The maximum relative margin of error that must be reached for each measurement [0, 1]. Overrides targetRme test option.

toofast --targetRme 0.01
--warmupIterationCount

The maximum number of warmup iterations that are run before each measurement. Overrides warmupIterationCount test option.

--batchIterationCount

The maximum number of iterations in a batch. Overrides batchIterationCount test option.

--batchTimeout

The maximum duration of batched measurements in milliseconds. Overrides batchTimeout test option.

--batchIntermissionTimeout

The delay between batched measurements in milliseconds. Overrides batchIntermissionTimeout test option.

...testPatterns

The array of glob patterns to filter tests that must be executed across all included files.

toofast 'factorial*'

Inline options

Provide option overrides to a particular describe, test and measure blocks. Options provided to a block override both config options and command line arguments. Block-level options are propagate to all underlying blocks as well.

import { describe, test, measure } from 'toofast';

describe('factorial', () => {

  test('of 33', { warmupIterationCount: 1000 }, () => {
    measure(() => {
      factorial(33);
    });
  });
});

About

⚡️ The Node.js performance testing tool with unit-test-like API.

Topics

Resources

Stars

3 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0