Pillar 3 · How we develop
The Training Matrix.
A controlled training architecture. Lessons from evaluation runs, reasoning systems, security research and reviewed observations enter the learning process deliberately — as curated material, with a record of where it came from — and every change is checked against the evaluation that motivated it.
If a behavior matters, we should be able to test it.
The principle
Training that can be checked.
Most of what shapes a model's behaviour is uncontrolled: large volumes of material nobody has read, whose effect on any particular decision can't be traced. The Training Matrix doesn't pretend to undo that — base models are what they are. It controls what we add on top, and insists that anything we add is tied to a behaviour we can measure before and after.
So a training change is not "more data". It is a hypothesis: this material should change how the model handles this kind of situation. The evaluation that exposed the problem is the test of whether it worked, and a set of evaluations the change was never meant to affect is the test of whether it broke something else.
What it is made of
Controls, and their status.
- Train / eval wall built
- Every catalog entry is marked for training or for evaluation. The mark is one-way and enforced where records are written: evaluation material can be released to training, never the reverse. A test that has been trained on is not a test.
- Teach the move, not the answer built
- Training material is built from a separate statement of the reasoning an entry teaches — never from the scenario text — and checked for verbatim overlap with its source, so a small fine-tune can't simply memorise the record.
- Curated corpus with provenance built
- Material carries where it came from: field-observed, field-grounded, adapted or synthesized. Empirical, testable content is preferred wherever the domain allows it.
- Frozen evaluations built
- An evaluation entry is frozen on its first result, so a before/after comparison measures the model, not an edited test.
- Pre-registered comparisons built
- How a comparison will be scored is written down before the numbers are computed. The same results can point in opposite directions under different scoring conventions; picking the convention afterwards is how a null result gets published as a win.
- Preserved history built
- Evaluation runs are logged per model and version and kept, so repeated runs and version-to-version comparisons are made against a record rather than a recollection.
- Training runs under development
- Repeated training runs against the controlled corpus, with regression testing against the frozen evaluation set, are being built. No results are claimed here.
- Per-discipline assistants under development
- Small, locally run models trained for a specific discipline and evaluated against that discipline's catalog entries. A direction, not a product.
Contributed material
People who contribute scenarios gave them to an evaluation. Training use of contributed scenarios waits on contributor consent terms, which are not in place yet. Until they are, contributed entries test models; they do not train them.
Training engagements
Where the ongoing work lives.
A score tells you where you stand. It doesn't move you. Training slots are multi-turn staged immersion — facilitated, sequenced to an industry, run against material your team has not seen and will not see afterwards.
Split-brain classification
Dual-hemisphere reconciliation: two independent lenses reach a verdict and a collapse probe resolves disagreement. A pattern for building robust security classifiers, taught by running it.
Witness-graded refusal scoring
Refusals and interventions graded against a tamper-evident witness log, so safety behaviour is measured against a defensible record rather than a vibe.
Provenance-anchored evaluation
Every judgement ties back to the detector, version, and verdict that produced it. Reproducible, auditable, and clean under scrutiny.
How an engagement runs
- Scope — your threat model, your target model, and what a pass would actually mean.
- Run — staged sittings against sequences matched to your disciplines, facilitated live.
- Verify — a held-out sitting graded independently, so the result isn't self-reported.
- Report — findings, the failure patterns we saw, and an attestation of what was run.
We sell slots and programmes — a defined engagement, a defined cohort, a defined sequence. Not unlimited access to our material. That is not a product we offer, at any price.