How to make your CI 10x faster with e2e?

End-to-end tests are usually the slowest job in CI. They are flaky and unreliable.

We’ve built e2e to solve this problem. It’s a new open-source testing framework for web and mobile apps. It lets you mix deterministic and agentic tests through one API. In the first month, it got to 12k stars on GitHub and 500k+ weekly downloads on npm.

e2e is fast; however, there’s a limit to how much you can speed up the framework itself. The best way to speed up your CI is to run only the tests that matter.

Filters API

To solve this problem, we introduced the Filters API. It lets you filter out tests before running them on a pull request. The built-in filter sends the diff and your test files to a small decision model (Jev) and asks one question per test: can this change affect it?

npm install -D @e2e-dev/decision ai

// e2e.config.ts
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { gateway } from 'ai';
import { decisionFilter } from '@e2e-dev/decision';

export default {
  targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
  filters: [decisionFilter({ model: gateway.decisionModel('typesafe-ai/jev') })],
} satisfies E2EConfig;

In this example, we use the Jev model from Typesafe AI, but you can use any decision-based model.

Filters only run when you pass --changed-since.

steps:
  - uses: actions/checkout@v7
    with:
      fetch-depth: 0
  # install, build, start the app...
  - run: npx e2e run ${{ github.event_name == 'pull_request' && format('--changed-since origin/{0}', github.base_ref) || '' }}
    env:
      AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}

fetch-depth: 0 is important here, as the filter diffs against the merge base with your base branch, and a shallow checkout doesn’t have it.

Here’s an example output on a PR:

decision kept 7 of 77 tests, left out 70 (1 call, 17.6k tokens, $0.0007)

That's 11x fewer tests for less than a tenth of a cent.

Why it's safe to skip tests

How to make sure you’re not skipping tests that are actually important? The built-in filter leans toward running tests that it’s unsure about.

• A test is left out only when the model rates it as under 20% likely to be affected.

• If you changed a test file, its tests run.

• If you changed the e2e config or a lockfile, everything runs.

• If the model can't judge (a diff over 60 kB, a file too big to send, failed calls), the test runs.

The worst case is a PR that runs more tests than it needed.

Tune it to your repo

You know your app better than a model does. A filter is just { name, filter }, so you can wrap the decision filter with your own rules. This one runs everything when infra/ changes and always runs tests tagged smoke:

import type { TestFilter } from 'e2e';
import { gateway } from 'ai';
import { decisionFilter } from '@e2e-dev/decision';

const decision = decisionFilter({ model: gateway.decisionModel('typesafe-ai/jev') });

export const tuned = {
  name: 'decision',
  filter(tests, context) {
    if (context.changes.files.some((file) => file.startsWith('infra/'))) return { exclude: {}, note: 'infra/ changed' };
    const judged = tests.filter((test) => !test.tags.includes('smoke'));
    return decision.filter(judged, context);
  },
} satisfies TestFilter;

You can also write fully custom filters that are fully deterministic or use LLMs.

Make your CI faster with e2e: https://e2e.tester.army/docs/filters

Get started now: ~/ npx e2e init