What an org dashboard should refuse to show: aggregation limits on individual skill data

The uncomfortable consequence of measuring comprehension well is that you now hold a per-engineer record of what each person does not understand. There is a version of this product that sells that record to management, and it would probably sell better than the one we built.

This paper is the measurement work behind refusing to. It is a privacy paper with an unusual property: the attacker we model is the paying customer.

Threat model

The adversary is an engineering manager or executive with legitimate dashboard access who wants to determine an individual's comprehension profile from aggregate views. They know their own organisation: who is on which team, who joined when, who works on what. That side knowledge is the whole problem, and no amount of aggregation removes it.

Where aggregation holds and where it fails

On teams of five or more with monthly granularity, maximum attribution confidence was 0.31, which is close to what a well-informed manager could guess without the dashboard. Below five engineers the picture collapses. On teams of three, weekly granularity produced attribution confidence above 0.8 in 44% of attempts.

Time granularity turned out to matter as much as team size, which we did not anticipate. A weekly series lets an observer align a dip in team coverage with the week a specific person was on the specific module. Monthly aggregation blurs the alignment enough to break most attributions.

What we ship, degrade, and refuse

Hard constraints in the product

• Minimum aggregation size of five engineers for any coverage or trend view. Below that, the view is suppressed rather than degraded.
• Concept-level drill-down requires ten or more engineers.
• Monthly granularity only. We do not offer weekly trends at any team size, including the large ones where it would be safe, because a configurable floor becomes an argued-about floor.
• Individual comprehension scores are visible to the individual and to nobody else. There is no administrative override, no enterprise tier that unlocks it, and no export.
• Bus-factor views name modules, never engineers.

What we were asked for and declined

During the alpha we received requests for a per-engineer skill scorecard (eleven times), a ranked list of engineers by comprehension score (four times), and an export of individual scores into an HR system (twice). We declined all of them. Two prospective customers ended the conversation over it, and we have written this paper partly so the next such conversation can start from published reasoning rather than an assertion.

Limits

• Our adversary is modelled as rational and well-informed but not persistent across months. A patient observer accumulating monthly views over a year is a stronger attacker than we simulated.
• Attribution confidence is computed against ground truth we hold. A real adversary has worse information than our simulator in some respects and better in others, notably hallway conversation.
• We did not test collusion between multiple dashboard users in different teams.
• The size-five floor is derived from 14 organisations with a particular team shape. A company organised into pairs would need a different answer than suppression.
• We have not formally analysed the composition of these views with the bus-factor list, which leaks by design. The combination is likely weaker than either view alone.

The simulation harness, attribution model and per-view confidence distributions are published so that customers can check our arithmetic rather than trust our restraint. Organisation-level inputs are not released.

Related work

The re-identification hazard we model is the one Sweeney [1] demonstrated with supposedly anonymous records and Narayanan and Shmatikov [2] generalised to sparse high-dimensional data, which is exactly the shape of a concept-coverage matrix. Dinur and Nissim [3] establish why repeated aggregate queries leak, motivating our monthly-only granularity, and Dwork [4] provides the formal frame we do not yet claim to satisfy. The minimum-aggregation-size approach is k-anonymity in the sense of Sweeney [5], with its known weaknesses against an adversary holding side knowledge, which our threat model assumes. Ajunwa and colleagues [6] survey the workplace monitoring harms that motivate the constraints rather than the arithmetic. The underlying individual measure is documented in AWL-2026-06 [7].

Artefacts

Everything below is published or available on request. A number nobody can reproduce is an advertisement, not a result.

References

1. Sweeney, L. (2002). k-Anonymity: A Model for Protecting Privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(5).
2. Narayanan, A., & Shmatikov, V. (2008). Robust De-anonymization of Large Sparse Datasets. IEEE Symposium on Security and Privacy.
3. Dinur, I., & Nissim, K. (2003). Revealing Information While Preserving Privacy. PODS.
4. Dwork, C. (2006). Differential Privacy. ICALP.
5. Sweeney, L. (2000). Simple Demographics Often Identify People Uniquely. Carnegie Mellon University, Data Privacy Working Paper 3.
6. Ajunwa, I., Crawford, K., & Schultz, J. (2017). Limitless Worker Surveillance. California Law Review, 105(3).
7. Bhardwaj, M. (2026). The comprehension gap: 200 engineers, 12 months, and what agentic coding did to recall. AI Work Lab Research, AWL-2026-06.