Pillar 2 · Where we test
Raccoon Ridge.
Not a question in a spreadsheet. A working town where an AI takes a job, plays a role, and gets tested against how the work is really done — district by district, scenario by scenario.
The town
- The Flats · river level, the oldest building stock
- Ridge Heights · newer construction on the valley wall
- The Yard · the port and light industrial
- Civic Row · the municipal buildings and the hospital
Why a town
Context and consequences are what a question leaves out.
A benchmark question arrives with everything the model needs, from nobody in particular, with nothing riding on it. A job doesn't. It comes from a customer with a stake in the answer, inside a business with rules, next to other people whose accounts don't agree — and what the model decides early changes what it is faced with later.
The town exists to supply that. It is a testing ground, not entertainment: every building on the map is tall because scenarios exist there, lit because they are live, and the map is generated from the same files the engine runs. Geometry may be authored; claims about the work must be derived.
How a run works
A seat, an incident, and state that carries.
- A seat in a business built
- The AI takes a role — advisor, dispatcher, office contact — at one of the town's businesses, and works a branching incident from that seat under partial information.
- State that carries built
- What gets established early guards what can happen later, so the same final decision can land on a different outcome depending on how the model got there.
- Moving surfaces, fixed judgement built
- Each decision draws its options from a pool of interchangeable wordings. The rubric scores the underlying outcome, never the option text, so a run can't be passed by memorising a transcript.
- Two records of every run built
- The record a customer sees and the full record we keep are built from separate sources rather than by redacting one into the other — a redaction step is a step someone forgets.
- Replays identically built
- A scored run must replay to the same result, whatever else is added to the town later. Town state may schedule a scenario; it never reaches inside one.
- Representative, and says so built
- A business modelled on a real kind of operation cannot load without declaring what it represents and a disclaimer, which is shown before the first decision of every run.
What is not built yet
An evolving research environment.
Raccoon Ridge today is a map of real sites, the businesses and roles they declare, and the scenarios placed inside them. Some districts are drawn dark because no scenarios have been written for them yet; the map says so rather than filling them in.
- The town running on its own in development
- Changing conditions between runs, suppliers, permits, and other agents acting in the same town.
- Long-horizon runs in development
- Running a business for a simulated week: memory over a sustained window, identity drift, reputation that follows an agent from one job to the next. This is the Autonomous City Run on the evaluation page, and it is a waitlist, not a booking.
- Ground on offer experimental
- Some parcels on the map are marked for sale — ground offered for AI work, shown in its own colour so it is never mistaken for work that exists.