Open Reference Stack
I built the open reference stack for getting mid-market AI agents safely into production. It is the method behind every Assessment, and it is public so you can see exactly how the work is done.
Most AI pilots fail on the way to production. The agent works in the demo, then fails silently on a real customer, exposes data it should never have touched, or leaves you unable to answer an auditor asking who approved it. That gap between a pilot and a system in durable production is where the risk lives.
GRADE is a structured way to close that gap. It is not a faster demo. It is the set of checks that turn a promising prototype into something you can put your name on in front of a board.
The Framework
Four pillars. Each one has an answer a board can act on and an implementation a practitioner can build.
You can show a board, an auditor, or a regulator who owns the agent, what it is allowed to do, and how those limits are enforced.
In practice Policy and guardrails at an LLM gateway, scoped access, and a mapping to the OWASP Agentic Top 10, NIST AI RMF, and the EU AI Act deployer timeline.
The agent behaves consistently under real conditions and volume, not just in the one path that looked good in the demo.
In practice A planner, executor, and validator pattern with retries and fallbacks, and cost-per-success tracked as a first-class metric.
The system ships onto infrastructure a mid-market firm can actually run and operate, with no dedicated ML platform team required.
In practice MCP for tool integration and OpenTelemetry tracing wired through every run, so the deployed system is observable rather than a black box.
Quality is measured before and after every change, so you know the agent is improving and can prove it is not quietly regressing.
In practice An evals harness on DeepEval, promptfoo, or Phoenix, with trace-level scoring and CI gating on every change to the system.
The Deliverable
The Scorecard is the board-level output of the framework: a single production-readiness document that grades an AI system across all four pillars in language a non-technical board and an auditor can both read.
It covers where the system stands on governance, reliability, observability, and evaluation, calls out the gaps that carry real production or audit risk, and puts them in priority order. It is the executive translation of the technical work, and it is what anchors the Assessment.
Walkthrough
A short walkthrough of how a pilot moves through GRADE into production, governed and audit-ready.
Walkthrough Video · Coming Soon
A short walkthrough of the GRADE Stack: how a pilot moves into production, governed and audit-ready.
Why It Is Open
A firm cannot open-source its method without giving away the product. A solo senior operator can, and that is the point. The stack is public because the fastest way to prove I can get your agent into production is to show the working code and the checks that get it there.
Read the repo, fork it, or hand it to your own engineers. The open artifact is proof of the work. The Assessment is that same rigor, pointed at your system.
The stack is the finished standard. The experiments and works in progress built around it live inLabs.
The repo shows how GRADE works. The Assessment applies it to your AI and gives your board the Scorecard.