We publish the measurements, including the ones we did not want.
Eight reports on what agentic coding does to engineering capability, how to measure it without lying, and where a single beat of friction changes what an engineer retains. Every paper ships with its data, its instrument, and a limits section written before the marketing copy.
Everything else.
Explain-to-merge: where one beat of friction changes what sticks
Friction that teaches and friction that annoys are separated by about ninety seconds and one design decision about timing. We ran four placements of…
Read the paper →
Lines accepted predicts nothing: six productivity proxies against six-month capability
Every AI coding dashboard reports acceptance rate. We correlated six commonly-reported proxies against measured engineer capability six months later,…
Read the paper →
A concept tagger that does not lie: calibration across 41 repositories
The skill graph is only as honest as the classifier that populates it. We measured accuracy and calibration for four tagger designs across 41…
Read the paper →
Spaced retrieval for code: FSRS against 18,000 review events
Spaced repetition was built for vocabulary, where an item is atomic and a lapse is cheap. Code concepts are neither. We fitted FSRS to 18,240 code…
Read the paper →
Model-agnostic by construction: teaching quality across nine models
Bring-your-own-model is a permanent stance, not a launch limitation, which obliges us to show that the teaching layer works on models we do not…
Read the paper →
Bus factor as a leading indicator: 14 organisations, nine months
Concentration of understanding is easy to measure and easy to ignore until someone resigns. We tracked module-level comprehension concentration…
Read the paper →
What an org dashboard should refuse to show: aggregation limits on individual skill data
We built a skill graph accurate enough to be dangerous. This paper sets out the re-identification results that determined which views we ship, which…
Read the paper →What we work on, and what we have not solved.
Each area lists the open problems honestly. If a question is unsolved we would rather say so here than have you discover it in the product.
Skill measurement
Whether an engineer can still explain, reproduce and modify the code they shipped, measured on their own repositories rather than on synthetic exercises.
Skill measurement
Open problems we are working on
- A comprehension instrument validated outside our own organisation
- Separating capability loss from delayed familiarity over windows longer than 90 days
- Measuring understanding of a system, not just of a diff
- Proxies that survive becoming a target for longer than three weeks
Selected work
Learning science
Where to place friction so that it teaches, and how to schedule retrieval for concepts that behave nothing like vocabulary items.
Learning science
Open problems we are working on
- Retention for architectural concepts, currently 26 points behind syntactic ones
- Scheduling that fires on some merges rather than all of them without losing the effect
- Transfer: does retrieving a concept in one system strengthen it in another
- Long-interval behaviour beyond the horizon our event log can observe
Selected work
Agent evaluation
Making the teaching layer work on models we do not control, and making sure the classifier that populates the skill graph knows when it is guessing.
Agent evaluation
Open problems we are working on
- Calibration under genuine distribution shift, not just held-out repositories
- Concept adjacency errors, which are invisible because they look plausible
- Tagging code written through house-specific abstractions
- An evaluation that stays valid as models change every quarter
Selected work
Org signals
Team-level consequences of concentrated understanding, and whether a warning arrives early enough that somebody can still act on it.
Org signals
Open problems we are working on
- Raising precision above 34% without losing the lead time that makes warnings useful
- Why 61% of warnings produce no action at all
- Distinguishing concentration caused by agents from concentration caused by staffing
- Signals for teams too small to aggregate safely
Selected work
Measurement integrity
What a product holding per-engineer comprehension data must refuse to show, and the re-identification work that decides where the line sits.
Measurement integrity
Open problems we are working on
- A formal privacy guarantee for the coverage views, which we do not yet claim
- Composition analysis across views that leak differently
- An answer better than suppression for organisations built out of pairs
- Modelling a patient adversary who accumulates monthly views for a year
Selected work
Who does the work.
One researcher and three external advisors, eight papers. Small enough that every paper has a person you can write to about it. Read more about the team →
Pre-registered, published, and allowed to be wrong.
Three commitments that constrain what we are able to claim.
Pre-registration
Primary outcomes and analysis plans are registered before data collection. It stops us discovering, after the fact, that the result we found was the one we meant to look for.
Publication regardless
No internal review can suppress a result for being commercially inconvenient. Three of our eight papers contain findings that are bad for us, including one that says our best metric collapses within a month of becoming a target.
Corrections in public
Papers carry a version and a changelog. When we get a number wrong we correct it in place and say what changed, rather than quietly replacing the file.