Lab project · free · provider-agnostic
Don't ask the AI if your instructions are better. Make it prove it.
Sharper replays the same repository task with baseline and candidate instructions, runs your existing agent CLI, and gates the result on checks the agent never saw.
An LLM can draft an AGENTS.md file and confidently review its own prose. It cannot establish that the change improved your real repository work without controlled runs, fixed conditions, and independent acceptance checks.
From opinion to a repeatable check.
Current instructions
Export the exact historical task base and overlay the instruction snapshot you use today.
Proposed instructions
Replay the identical task with one changed instruction snapshot and the same agent setup.
Condition order alternates across tasks and attempts. A regression on any task produces hold; a clean improvement produces promote; noisy or tied evidence stays inconclusive.
Run it with the agent you already use.
Clone Sharper beside the repository you want to evaluate. Then invoke it from that target repository:
git clone https://github.com/Gjeesus/sharperthanai.git ../sharperthanai
node ../sharperthanai/scripts/sharper-eval.mjs init
# Add reviewed tasks and your agent command
node ../sharperthanai/scripts/sharper-eval.mjs validate
node ../sharperthanai/scripts/sharper-eval.mjs runThe runner itself needs no package install. Your configured command is an executable plus an argument array, so it is not tied to one model vendor. The agent CLI must already be installed and authenticated.
Evidence the model cannot grade for itself.
Every run gets a fresh repository with no original Git history. Sharper records the agent's patch before applying an optional hidden test patch, rejects edits to protected evaluator paths, and then runs your verifier commands.
| Artifact | What it answers |
|---|---|
| Agent patch | What did the agent actually change? |
| Verifier logs | Did the result satisfy the acceptance checks? |
| Run metadata | Which condition, task, attempt, and duration produced it? |
| Decision report | Did the candidate improve without a task-level regression? |
One command, not a wall of terminals.
The runner owns the task × condition × attempt matrix from one process. It prints compact progress in the initiating terminal and files each setup, agent, verifier, and patch artifact under its run ID. You get one decision report instead of manually matching terminal tabs to outcomes.
Local by design, with an honest security boundary.
Your source, credentials, agent sessions, patches, and results stay in your environment unless your CI uploads them. The current runner isolates Git history and hidden tests, but it is not a container sandbox: the agent inherits the permissions of the account that launches it.
Use least-privilege credentials and a disposable machine or container for untrusted tasks. Review the runner guide and threat boundary before using private code.
Start with a real repository.
Prepare a free instruction inventory and task shortlist, then bring the exported configuration to the local runner.