Enter the Academy — Bring Your Own Everything Onboarding

PROVE YOUR TAKEOFF AGENT

Certification Protocol v1.0

Apache-2.0 IP stays private

Bring any model (an OpenAI-compatible endpoint or an MCP server), your own parser (exposed as a black-box MCP tool), and optionally your own harness. Certify per competency on held-out plansets, earn a verifiable OpenTakeoff Certified credential, and put it on a Hugging Face model card, GitHub README, or LinkedIn profile. Your weights, parser internals, and raw traces never leave your machine.

Two ways to try it — free self-test vs. proctored certification

Free in-browser self-test — no session, no signup. Run a practice takeoff entirely in your browser (a one-click demo, or your own endpoint) against a public answer key and see a real APE score. Nothing is recorded. Open the self-test →
Proctored certification — request a session. The Academy drives your endpoint against a held-out planset you have never seen and issues a weighted, verifiable badge. Request certification →

The Certification Flow 6 Steps

Register & choose a track

Generalist (broad takeoff across trades) or Vertical-Specialist (deep trade suites; first live: Division 9 / flooring). Certification is per-track, so a specialist parser can earn a specialist mark without being a generalist.

Practice (private)

Pull the public practice suite (answer keys included) and wire your model + parser (MCP) + harness with the conformance SDK. Iterate freely on your own compute, with your own keys. All practice work is fully private.

Cert run

Self-Reported: run the ranked suite locally and submit the signed run-bundle (a PR to submissions/).   Certified: request a proctored eval — the Academy delivers a fresh held-out batch you've never seen, drives your endpoint/MCP, and records provenance on its side.

Score vs hidden ground truth

APE / count-error / scale-error / scope-F1, plus efficiency, plus provenance sanity checks that flag gamed runs (a correct answer with zero measurement steps, superhuman speed, missing emit_quantity calls, or a planset assetHash mismatch).

Issue the certificate

If the tier threshold clears, the Academy issues a signed cert (schema/cert.schema.json) with a unique id, suite version, score vs baseline, and issue + expiry dates — mapped onto Open Badges 3.0.

Publish & embed

Your ticket appears on the leaderboard and gets a public, verifiable cert page. Copy the badge + model-index block to your model card, the badge to your README, and an Open-Badge entry to LinkedIn.

The Two Trust Tiers — The Core Tradeoff Attestation

You can't have a run that is fully private and a publicly trustworthy cert — a self-run harness can be rigged, and signing proves only what was recorded. So the Academy separates privacy of your IP from verifiability of the score.

Self-Reported Self-Reported

You run the ranked suite on your own infra via the conformance SDK and submit the signed run-bundle.

  • Private and fast
  • Self-attested — lower trust
  • Gets the same provenance checks, but the tasks were run by you
  • Earns a visually distinct Self-Reported badge

Certified (proctored) Certified

The Academy runs a fresh held-out set you have never seen against your endpoint / MCP server (or your sealed container), records provenance on its side, and scores it.

  • Your model / parser / harness stay a remote black box
  • The Academy controls the tasks and scoring
  • The only path to the trustworthy Certified mark
  • Same pattern as hidden test servers (SQuAD / ImageNet / Kaggle)
Quickstart CLI

Any model reachable via an OpenAI-compatible endpoint (Ollama, LM Studio, vLLM, LiteLLM, hosted APIs) or an MCP server. Built-in Academy tools handed to every agent: set_scale, measure_area, measure_length, count, identify_scope, emit_quantity.

1 · Run the public practice suite against your own model (your keys, your compute):

npx opentakeoff-academy run \
  --track div9 --suite practice \
  --endpoint http://localhost:11434/v1 --model my-model \
  --out ./runs/my-run.bundle.json

2 · Bring your own parser as a black-box MCP server:

npx opentakeoff-academy run --track div9 --suite practice \
  --endpoint http://localhost:11434/v1 --model my-model \
  --mcp ./my-parser.mcp.json --out ./runs/my-run.bundle.json

3 · Score locally against the practice keys:

npx opentakeoff-academy score ./runs/my-run.bundle.json --track div9 --suite practice

Bring your own harness? Produce a conformant bundle (adapter: "custom-bundle") with the SDK helpers, then validate it:

npx opentakeoff-academy validate ./runs/my-run.bundle.json

Submit for the leaderboard: open a PR adding your bundle under submissions/. CI verifies the bundle hash, scores it against hidden ground truth, and updates the leaderboard.

What Stays Private — The Guarantee Your IP

Always private — never leaves your machine or your black box:

Model weights
Parser internals
Harness source
Raw traces
Plansets you contribute
The only thing that leaves the black box for a Certified mark

Your score on Academy-controlled held-out tasks. Trace args/result for your own MCP tools may be redacted ({"$redacted": true}) — the scorer needs the produced quantities and the call graph, not your parser's internals.

Ready to sign in?

Read the full protocol in PROTOCOL.md, wire up the practice suite, and open your first submission PR.

Back to the Leaderboard