Any litigator will tell you that a claim without evidence goes nowhere. The same holds when an engineering team complains that builds crawl, reviews sit idle for days, and nobody can find anything in the tooling. Management hears grievance; nothing moves. DevExOps exists to convert those grievances into a file that survives scrutiny: friction is located, counted with real instruments, assigned to an owner, and re-checked against a bar that was written down before anyone touched the code. The discipline is laid out plainly in the Ops Atlas entry on DevExOps, and this piece turns it into a checklist a team can run in one afternoon.

What belongs in a DevExOps case file

Think of the file the way a chamber thinks of a brief. Every assertion is dated. Every number has a provenance. The core is a repeated cycle: surface the friction, put a survey figure or a system measurement against it, hand the item to the person who can actually fix it, then verify the improvement against a threshold you committed to earlier. A finding that cannot clear that last step was never a finding — it was an opinion with a chart attached.

Two families of evidence do the work. System telemetry counts the waits developers actually sit through: queue minutes before a CI run starts, feedback delay once it does, hours before someone responds to a review request, how long a fresh joiner needs before their first change merges. Survey instruments capture the perception side: whether people say their tools help them, whether they can enter concentration and stay there, whether the cognitive weight of the estate is climbing. Neither family is sufficient alone. When the two disagree, that disagreement is itself evidence worth chasing.

The DevExOps checklist, item by item

1. A named decider. Before any measurement begins, one person must hold the authority to fund a fix. Measuring with nobody positioned to act produces data theatre.
2. A written data-use policy. One page, circulated: the numbers describe teams and work systems, never individuals, and they will never feed performance reviews. Publish the suppression rule at the same time — any group under about five respondents gets folded upward, because with four people the “result” is a name with a percentage.
3. Baseline telemetry. Pull CI feedback and queue figures for the ninetieth percentile, not only the median — a middle value conceals exactly the slow runs that irritate most. Add review response time, local build duration, days from joining to first merged change for new hires, and unplanned work absorbed per engineer-week.
4. Survey discipline. Run the perception instrument at a fixed cadence of every few months, not whenever morale dips. Track the favourable share (the top two boxes on a five-point scale) and the response rate together — a 90% favourable result from a 12% response rate is a rumour.
5. Thresholds written before the fix. Decide what “better” means while the problem is still unsolved. Thresholds set afterwards bend to whatever the numbers happened to do.
6. Small-sample caution. A team of fourteen showing an 18% swing may truly sit anywhere between 6% and 44%. Require the direction to hold across two cycles, pool several teams, or stage the rollout before declaring victory — and remember that extremes drift back toward average on their own.
7. A quarterly closing argument. Re-measure with the same instruments, compare against the pre-committed bar, and record the verdict either way. Findings that verified become precedent; findings that did not become the next case.

Where a legal-adjacent team feels the friction

Readers of this site work with deadlines that courts set, not sprint boards. The waits look different but rhyme: a document portal that takes a minute per page to respond, multiplied across a hundred pages a day; a case-tracking workflow buried three menus deep; a knowledge base where the search box fears hyphens. The same evidence rules apply. Anyone who has worked through a structured guide to checking case status online knows how much difference a well-built path makes versus a badly built one when a filing deadline is close. Teams that internalise the discipline of deciding when a matter needs a specialist now already think this way about legal triage; the checklist above applies the same triage to internal tooling.

There is also a cultural fit worth naming. Legal work already accepts that process protects people — conflict checks, privilege walls, matter numbers. A measurement programme with a published data-use policy and a suppression floor is the same instinct, applied to the people building your systems. Firms that encourage questions, in the spirit of open discussion forums that ordinary people can actually use, will find engineers volunteer better survey answers too.

Closing the loop, quarter after quarter

Run the cycle once and you have a baseline. Run it twice and you have a trajectory. Teams progress along a recognisable path: from anecdote, to baselined numbers, to a working loop where fixes get verified, then to measurement woven into planning, and eventually to a group that re-instruments itself as the estate changes. The jump from anecdote to baseline is the one that needs the checklist most, because it is the step where shortcuts — a vendor’s borrowed target, an invented benchmark — do the lasting damage.

A first quarter looks roughly like this. The opening weeks belong to the decider, the written policy, and the first telemetry pulls — no survey yet, no promises yet. The middle month introduces the perception instrument and its first readout, delivered to the teams being measured before anyone senior sees a slide. The closing month is where credibility is won or lost: three findings, each with a name attached as owner, plus one deliberately small fix shipped early so the group watches the loop complete end to end. Quick wins matter less as proof of competence and more as proof that the machinery actually terminates in a change someone can feel.

Failed verdicts deserve the same filing as successful ones. When a remedy ships and the re-measurement shows the wait barely moved, record that. Six months later, when someone proposes the same remedy with fresh confidence, the file saves the team a second expedition down a documented dead end — which is, in the end, exactly what a well-kept case file is for.

Practitioners of evidence know the temptation of the persuasive number that nobody can defend under questioning. The checklist above is the antidote: decider first, policy in writing, both median and tail, thresholds fixed in advance, small samples treated with respect, and every verdict recorded against the same instruments that opened the case. Teams that hold themselves to that standard stop arguing about whether the tooling hurts and start arguing about which hurt to fix first — which is the argument worth having.