Shortest

Shortest

Antiwork · Coding

Shortest is an open-source AI testing framework that turns plain-English descriptions into automated end-to-end browser tests. It builds on Playwright and runs on your Anthropic API key.

Interface preview of Shortest

About Shortest

What Is Shortest

Shortest is a free, open-source test automation framework from Antiwork, the company behind Ducky and Kangaroo. Instead of writing step-by-step selectors and assertions, you describe what a user should do in a normal sentence. The AI figures out the clicks, the typing, and the checks that make it happen.

The tool exists to cut the time QA teams and solo developers spend maintaining brittle test scripts. When a button moves or a label changes, a traditional Playwright test breaks and someone has to fix the selector. Shortest leans on an AI model to read the page and adapt, so many small UI changes stop breaking your test suite.

The catch is that Shortest isn't a hosted cloud service. It's a command-line tool you install in your own project, and it needs its own AI model to do the reasoning. That model is Anthropic's Claude, and you pay for those calls yourself.

Getting Started

  1. Run npx @antiwork/shortest init in your project. The command installs the package as a dev dependency, creates a shortest.config.ts file, and adds a .env.local file for your keys.
  2. Add your Anthropic API key to .env.local. Shortest reads it from the ANTHROPIC_API_KEY variable.
  3. Set your baseUrl in the config file so tests know which site to open.
  4. Create a test file matching the pattern in your config, then write a test as one plain-English sentence.
  5. Run pnpm shortest to execute every test, or pass a file name to run just one.

Product Information

A quick look at Shortest's pricing, supported platforms, and performance.

Free PlanYes
Paid PlansFree, plus your own Anthropic API usage
PlatformCLI, macOS, Windows, Linux (Node.js 18+)
DeveloperAntiwork
CategoryCoding
Release DateNov 2024
Latest UpdatedAug 2025
Website Visits1.9K
Website Global Rank7.8M
API AvailabilityNo

Best for

The users, tasks, and scenarios where this tool fits best.

Users

  • Small dev teams without a dedicated QA engineer
  • QA engineers tired of fragile selectors
  • Open-source contributors and tinkerers

Tasks

  • Testing a login or signup flow
  • Checking API responses
  • Running regression checks in CI

Scenarios

  • A startup shipping weekly releases that keeps breaking its one-person test suite
  • A team evaluating AI-driven QA before committing budget

Key features

Tests Written in Plain English

The core idea is a single readable sentence instead of a script. You write something like "Login to the app using email and password" and pass in your credentials, and Shortest works out the rest. That means a new teammate can read a test and understand what it checks without decoding selectors. It also lowers the bar for writing your first end-to-end test.

Built on Playwright

Shortest drives a real Chromium browser through Playwright, the same engine behind many professional test suites. You keep Playwright's browser options and can pass custom context settings in your config. Because it inherits that foundation, the tool fits into existing JavaScript and TypeScript projects rather than forcing a new stack on you.

AI Reasoning with Anthropic Claude

The reasoning layer runs on Anthropic's Claude models through the AI SDK. Shortest reads the page, plans the steps, and decides whether the result matches what you described. You can point it at a different key or provider through the ai.config block, but the default path assumes your own Anthropic account and API key.

Test Chaining and Lifecycle Hooks

You can group related flows into a chain, so a login step feeds a set of actions that follow it. Lifecycle hooks like beforeAll and afterEach let you sign a user in, clear state, or close the page around each test. This makes reusable flows practical, so you write the login sequence once and reuse it across many tests.

API Endpoint Testing

Shortest isn't limited to clicking around a browser. You can describe an API call in natural language, or build a request with a method, URL, and query parameters, and let the AI confirm the response looks right. Teams that test both the interface and the backend get one tool instead of two.

CLI Commands for the Whole Workflow

Beyond running tests, Shortest ships commands to plan, generate, and analyze your tests from the terminal. You can run a single file, a single test by line number, or the whole suite in headless mode for CI. That range means the tool covers setup, authoring, and execution without leaving the command line.

Pros and cons

Pros

  • Free and open source under the MIT license, so there's no subscription to test it out.
  • Plain-English tests are far easier to read and hand off than traditional selector-based scripts.
  • The AI adapts to small UI changes, which cuts the maintenance tax that plagues Playwright suites.
  • Runs on Playwright, so it slots into existing JavaScript and TypeScript projects.
  • Handles both browser flows and API checks in one framework.

Cons

  • You pay for AI usage separately, and every test run spends Anthropic API tokens, so a large suite adds up.
  • It's a command-line tool with no guided setup beyond the config file, so it assumes some developer comfort.
  • Results depend on the AI model reading the page correctly, and an ambiguous sentence can lead to an unexpected step.
  • There's no hosted dashboard or managed cloud option; you own the setup, secrets, and CI wiring.

Frequently asked questions

Shortest is an AI testing framework for writing and running end-to-end tests in plain English. It automates browser flows, like logging in or completing a checkout, and can also check API responses.