Product Research Pricing About Careers Blog Private beta Talk to us Get the private beta →
Research Publications

Everything we have published, including the corrections.

Thirteen full reports and seven shorter outputs. Papers carry a report number and a version; when a number changes we say so in the paper rather than replacing the file quietly. Correspondence on any of it goes to research@aiworklab.com.

2026

AWL-2026-06 The comprehension gap: 200 engineers, 12 months, and what agentic coding did to recall AI Work Lab Measurement team · AI Work Lab Research We instrumented the workflow of 200 working engineers across 14 companies for a year and measured two things that are usually reported separately: how much they… Paper14 July 2026
AWL-2026-05 Explain-to-merge: where one beat of friction changes what sticks AI Work Lab Learning Science team · AI Work Lab Research Friction that teaches and friction that annoys are separated by about ninety seconds and one design decision about timing. We ran four placements of the same… Paper26 June 2026
AWL-2026-04 Lines accepted predicts nothing: six productivity proxies against six-month capability AI Work Lab Measurement team · AI Work Lab Research Every AI coding dashboard reports acceptance rate. We correlated six commonly-reported proxies against measured engineer capability six months later, on 168… Paper2 June 2026
AWL-2026-03 A concept tagger that does not lie: calibration across 41 repositories AI Work Lab Evaluation team · AI Work Lab Research The skill graph is only as honest as the classifier that populates it. We measured accuracy and calibration for four tagger designs across 41 repositories, and… Paper12 May 2026
AWL-2026-02 Spaced retrieval for code: FSRS against 18,000 review events AI Work Lab Learning Science team · AI Work Lab Research Spaced repetition was built for vocabulary, where an item is atomic and a lapse is cheap. Code concepts are neither. We fitted FSRS to 18,240 code review events and… Paper21 April 2026
AWL-2026-01 Model-agnostic by construction: teaching quality across nine models AI Work Lab Evaluation team · AI Work Lab Research Bring-your-own-model is a permanent stance, not a launch limitation, which obliges us to show that the teaching layer works on models we do not control. We measured… Paper18 March 2026
AWT-2026-04 Gating question generation behind a calibrated tagger AI Work Lab Evaluation team · AI Work Lab Technical Note Why we stopped asking a question for every diff, and what the 0.85 threshold costs. Technical report2 July 2026
· Comprehension checks as desirable difficulty in production developer tools AI Work Lab Learning Science team · Workshop on Learning at Scale for Software Engineering Short paper. Early results from the placement study, reported in full in AWL-2026-05. Workshop paperMay 2026
· concept-taxonomy-v2 and prerequisite graph AI Work Lab Evaluation team · Public release, CC BY 4.0 214 concepts, three levels of granularity, 3,410 curated prerequisite edges. DatasetApril 2026
· What we owe an engineer we are measuring AI Work Lab Security & Privacy team · FOSDEM, Brussels Slides and transcript published. 24 minutes. TalkFebruary 2026

2025

AWL-2025-02 Bus factor as a leading indicator: 14 organisations, nine months AI Work Lab Org Insights team · AI Work Lab Research Concentration of understanding is easy to measure and easy to ignore until someone resigns. We tracked module-level comprehension concentration across 14… Paper14 October 2025
AWL-2025-01 What an org dashboard should refuse to show: aggregation limits on individual skill data AI Work Lab Security & Privacy team · AI Work Lab Research We built a skill graph accurate enough to be dangerous. This paper sets out the re-identification results that determined which views we ship, which we deliberately… Paper20 August 2025
AWT-2025-03 Normalising acceptance telemetry across four vendor definitions AI Work Lab Measurement team · AI Work Lab Technical Note Four tools, four definitions of accepted. The mapping, and what it cannot fix. Technical reportNovember 2025
· Comprehension-weighted ownership as a maintainability signal AI Work Lab Org Insights team · Preprint, under review Extends the bus-factor result in AWL-2025-02 to defect prediction. Comments wanted. PreprintSeptember 2025
· Against per-engineer skill scorecards AI Work Lab Security & Privacy team · AI Work Lab Research Why we will not ship an individual comprehension score to management at any tier. Position paperJuly 2025

2024

AWL-2024-03 Designing a twelve-month cohort study of agentic coding: power, confounding and attrition by simulation M. Bhardwaj · AI Work Lab Research Next month we start following working engineers for a year to measure what agentic tools do to their understanding of the code they ship. Before recruiting anyone,… Paper10 December 2024
AWL-2024-02 One run is an anecdote: variance in repository-level agent evaluations M. Bhardwaj · AI Work Lab Research Coding agents are now compared by a single resolve rate on a few hundred real GitHub issues, from one run. We simulated what repeated runs of the same agent on the… Paper26 September 2024
AWL-2024-01 Reading the review state: a hidden Markov model of how engineers accept AI suggestions M. Bhardwaj · AI Work Lab Research Acceptance rate tells you how often an engineer said yes to a suggestion, not whether they read it. We built a three-state hidden Markov model over the telemetry… Paper23 April 2024

2023

AWL-2023-02 Pass@k is not a ranking: the statistics of code-model leaderboards M. Bhardwaj · AI Work Lab Research We needed to choose which models could sit underneath our teaching layer, and the evidence on offer was a leaderboard of single numbers on 164 problems. We… Paper21 November 2023
AWL-2023-01 An item bank for code comprehension: calibration, adaptive testing and fairness checks with item response theory M. Bhardwaj · AI Work Lab Research Before we can say anything about what AI assistance does to an engineer’s understanding, we need an instrument that measures understanding on a scale, not a… Paper18 April 2023
Citation

BibTeX for the thirteen reports.

If you are citing a technical note, a dataset or a talk instead, write to us and we will send you the entry rather than have you guess at it.

@techreport{bhardwaj2026comprehension,
  author    = {{AI Work Lab Measurement Team}},
  title     = {The comprehension gap: 200 engineers, 12 months, and what agentic coding did to recall},
  year      = {2026},
  number    = {AWL-2026-06},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-1.html}
}

@techreport{bhardwaj2026explain,
  author    = {{AI Work Lab Learning Science Team}},
  title     = {Explain-to-merge: where one beat of friction changes what sticks},
  year      = {2026},
  number    = {AWL-2026-05},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-2.html}
}

@techreport{bhardwaj2026lines,
  author    = {{AI Work Lab Measurement Team}},
  title     = {Lines accepted predicts nothing: six productivity proxies against six-month capability},
  year      = {2026},
  number    = {AWL-2026-04},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-3.html}
}

@techreport{bhardwaj2026concept,
  author    = {{AI Work Lab Evaluation Team}},
  title     = {A concept tagger that does not lie: calibration across 41 repositories},
  year      = {2026},
  number    = {AWL-2026-03},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-4.html}
}

@techreport{bhardwaj2026spaced,
  author    = {{AI Work Lab Learning Science Team}},
  title     = {Spaced retrieval for code: FSRS against 18,000 review events},
  year      = {2026},
  number    = {AWL-2026-02},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-5.html}
}

@techreport{bhardwaj2026model,
  author    = {{AI Work Lab Evaluation Team}},
  title     = {Model-agnostic by construction: teaching quality across nine models},
  year      = {2026},
  number    = {AWL-2026-01},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-6.html}
}

@techreport{bhardwaj2025bus,
  author    = {{AI Work Lab Org Insights Team}},
  title     = {Bus factor as a leading indicator: 14 organisations, nine months},
  year      = {2025},
  number    = {AWL-2025-02},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-7.html}
}

@techreport{bhardwaj2025org,
  author    = {{AI Work Lab Security and Privacy Team}},
  title     = {What an org dashboard should refuse to show: aggregation limits on individual skill data},
  year      = {2025},
  number    = {AWL-2025-01},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-8.html}
}

@techreport{bhardwaj2024designing,
  author    = {Bhardwaj, Mohit},
  title     = {Designing a twelve-month cohort study of agentic coding: power, confounding and attrition by simulation},
  year      = {2024},
  number    = {AWL-2024-03},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-13.html}
}

@techreport{bhardwaj2024one,
  author    = {Bhardwaj, Mohit},
  title     = {One run is an anecdote: variance in repository-level agent evaluations},
  year      = {2024},
  number    = {AWL-2024-02},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-12.html}
}

@techreport{bhardwaj2024reading,
  author    = {Bhardwaj, Mohit},
  title     = {Reading the review state: a hidden Markov model of how engineers accept AI suggestions},
  year      = {2024},
  number    = {AWL-2024-01},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-11.html}
}

@techreport{bhardwaj2023pass,
  author    = {Bhardwaj, Mohit},
  title     = {Pass@k is not a ranking: the statistics of code-model leaderboards},
  year      = {2023},
  number    = {AWL-2023-02},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-10.html}
}

@techreport{bhardwaj2023item,
  author    = {Bhardwaj, Mohit},
  title     = {An item bank for code comprehension: calibration, adaptive testing and fairness checks with item response theory},
  year      = {2023},
  number    = {AWL-2023-01},
  institution = {AI Work Lab},
  url       = {https://aiworklab.com/research-9.html}
}
Access

What you can have, and what we hold back.

Open

Instruments and code

Rubrics, taxonomies, the prerequisite graph, normalisation code and every analysis notebook are published outright. Reproducing our figures should not require our permission.

On request

Labelled corpora

The 12,400-span labelled corpus contains code from private repositories, so it moves under a data agreement rather than a download link.

Never

Individual records

No per-engineer comprehension record leaves the product, to us or to anyone else. The reasoning is set out in AWL-2025-01, along with the requests we declined.