We Installed GitHub's Spec Kit. Here's What It Sets Up
GitHub's kit for structured agent development scaffolded real structure in our tests. We stopped before the writing half; our container had no coding agent.
ToolRiot has no affiliate relationship with this tool. This coverage is based on our own testing and the tool's usefulness for the intended audience.
The script's JSON output names each piece of work and points at the spec file it created.
{"BRANCH_NAME":"005-tagging-notes","SPEC_FILE":"/workspace/demo-notes-app/specs/005-tagging-notes/spec.md","FEATURE_NUM":"005"}The spec file it names starts with the template's structure (its first lines are saved as the feature-spec-head artifact); it's yours to fill in from there.
We Installed GitHub's Spec Kit. Here's What It Sets Up
If you use a coding agent, you've probably seen Spec Kit mentioned. It's GitHub's toolkit for "spec-driven development", their name for a simple idea: write the specification, the plan, and the task list before the code. You can see the shape of it in the files the kit installs. The pitch is easy to find. What's harder to find is a plain answer to the first question anyone has: what do you actually get when you install it?
We installed it in a clean container and looked. ToolRiot has no affiliate relationship with GitHub or Spec Kit; this is just the test. Exact commands and versions are in the Technical details at the end.
The install, and what appeared
One pip install, then specify init with the Claude integration selected. It finished without errors and left behind a project skeleton: the file tree lists five workflow templates, a project constitution file, shell scripts, and ten skill files named for the workflow's steps, from specify and plan through tasks and implement.
We opened the specification template, read its first sixty lines closely, and saved a full copy as evidence. It's real, structured content: prioritized user stories, each written to be independently testable, with acceptance scenarios in a given-when-then shape. The plan, task, checklist, and constitution templates we verified by their presence in the file tree, so we'll only vouch in detail for the one we read.
There's also a specify check command. In our container it ended with "Specify CLI is ready to use!" after reporting the terminal coding agents as not found and noting that the tools living inside an IDE can't be checked from a terminal. That matched our empty container.
Starting a feature
In a second run, we used the bundled script that begins each new piece of work, handing it "add tagging to notes" as the feature idea. The saved feature tree from that run lists five numbered spec folders, specs/001-tagging-notes through specs/005-tagging-notes, each holding a spec file. The JSON output for feature 005 is shown below.
That file starts as placeholders, and that's where our test stopped: the file was still a template when we finished. Our container had no connected coding agent, so everything past the scaffold stayed untested in our runs.
So here's our honest boundary: the scaffolding is real and coherent. Whether the full workflow produces better software than prompting without it is a question this test can't answer.
Who it's for
If you're already working with a coding agent on projects with more than one moving part, Spec Kit hands you a written planning structure that would otherwise take real effort to assemble. For quick scripts and single experiments, this much structure may be more than you need.
A simple way to decide: install it, run init in an empty directory, and read the specification template it gives you. That one file will tell you whether this way of working fits how you build.
- ✓ Developers using coding agents on projects with more than one moving part
- ✓ Teams that want agent work grounded in a written, reviewable plan
- ✓ People curious what structured agent workflows look like in practice
- · Quick scripts and single experiments, where this much structure may be more than you need
- · Working without a coding agent, since the workflow is built around one
- · Looking for finished specs out of the box; what init generated in our test was templates
Spec Kit delivered on the half we could test: a clean install, a coherent project skeleton, an environment check whose output was consistent with our container, and a feature script that turned an idea into a numbered spec folder and file. Whether the full agent workflow earns its ceremony is the question for a future test with a connected agent.
Technical details
- Environment
mcr.microsoft.com/playwright/python:v1.61.0-noble · Python 3.12.3 · Ubuntu 24.04 x86_64 · Python environment created with virtualenv- Versions
specify-cli 0.12.17 (first run); the second run did not print a version- Install
virtualenv venv && ./venv/bin/pip install specify-cli && ./venv/bin/specify --help- Project init
./venv/bin/specify init demo-notes-app --integration claude --ignore-agent-tools → skeleton under .specify/ and .claude/skills/ (full tree saved as the project-tree.txt artifact)- Templates
spec-template.md, 131 lines, read closely for its first sixty lines and saved in full as an artifact; plan, tasks, checklist, and constitution templates verified by presence in the saved file tree- Check
./venv/bin/specify check → reported terminal coding agents (Claude Code, Codex CLI, and others) as not found, IDE based tools as not checkable from a terminal, and ended with 'Specify CLI is ready to use!'- Feature scaffold
second run, inside demo-notes-app after git init: bash .specify/scripts/bash/create-new-feature.sh --json 'add tagging to notes' (full compound command in the trial log) → {"BRANCH_NAME":"005-tagging-notes","SPEC_FILE":"/workspace/demo-notes-app/specs/005-tagging-notes/spec.md","FEATURE_NUM":"005"}- Evidence
run 1: environment.txt, project-tree.txt, spec-template.md, specify-check.txt · run 2 (scaffold verification): feature-tree.txt, feature-spec-head.md
We actually tested it.
Everything below comes from our own test: real tasks, real output, no vendor screenshots.
Start with a working project structure
The CLI created a new project with its own templates, scripts, workflows, and configuration files.
✓ We initialized a project with the Claude integration selected successfully and confirmed the generated .specify and .claude directories in the saved file tree.
Best for Developers starting a structured specification workflowScaffold a spec from a feature idea
The generated specification template prompts for prioritized stories, independent tests, and acceptance scenarios.
✓ We inspected the generated template and found structured sections for user stories, priorities, independent tests, and Given/When/Then acceptance scenarios.
Best for Teams that want feature specs built around user journeysPrepare a project for Claude integration
Init completed using the --ignore-agent-tools flag with the Claude integration selected.
✓ The project initialized successfully with Claude integration while agent tool checks were intentionally skipped during setup.
Best for Developers using Claude alongside a local development workflowSee what your environment is ready for
Its check command completed and reported which optional tools were unavailable instead of blocking the project.
✓ The project check finished successfully and ended by confirming that the Specify CLI was ready to use, while listing unavailable optional tools.
Best for Developers checking a new project before they beginHow you'd use it
Set up Spec Kit
Install the CLI in your development environment and review the available initialization options.
Create a project
Initialize a new project and choose the AI integration you plan to use, such as Claude.
Write the specification
Open the generated specification template and use its story, priority, and acceptance scenario sections to shape a feature.
Check your environment
Run the project check to confirm the workspace is ready and see the status of optional tools.
Read how we test AI tools and our editorial policy.