There is a lot going on when developing software, writing requirements, design, architecture, code, tests, integrating it all, testing again, deploying, shipping and handling customer feedback.
This flow involves many people, many professions and many tools. It is sometimes called the Software Factory. Let’s use this metaphor to describe how software creation should be controlled. Read on!
When does a Code Review Escape mean?
Whenever a unit of work leaves a station in the factory, it is examined for its quality and completeness, only then it is transferred to a downstream station for further processing.
When a downstream station detects an issue with the unit of work it received, we call it an escape. It escaped the quality control and completeness check of the previous station.
We can fix the unit of work itself, but that is almost meaningless. What’s important to fix is the quality control and completeness tests of the station from which this defective unit of work escaped. Doing that guarantees the factory gets better over time and produces higher yield.
Putting this to the practical world of software craftsmanship. If you’re not measuring Requirement escape ratios, Design escape ratios and Code review escape ratios, your factory will keep producing sub-par software
Many teams will perform root cause analysis for defects and will fix the defects that were found, that is a good start but completely insufficient. Assume you found a problematic requirement that was implemented. Obviously, you fix the requirement. Does this make your requirement process better? Obviously, no!
An issue with a requirement found in testing is an escape from the requirement station. We need to elevate our thinking to the pattern level. What process did we follow before this unit of work (in this case the aforementioned requirement) left the station? What was wrong with our quality control and completeness check that allowed this requirement to be passed on as a finished unit of work?
The requirement escape must lead to a fix in the factory machinery (process) itself which is the only way to ensure improvement.
The number of escapes (or ratio) needs to be measured for each station and needs to be tracked over time. That allows you to monitor whether your machinery fixes are actually working.
Assume someone proposes that in order to fix the requirement machinery, you need to double the time invested in requirement reviews. You have no way of telling whether this is a good idea or not. What you can do is try it out, change your process and measure the requirement escapes of the next iteration. If they’ve gone down significantly, keep this change. If there’s no change or the number of escapes has grown, revert this process change and try another.
The Bottom Line
To summarize, each thing you do as part of the software development lifecycle, needs to be measured for escapes and you need to know its yield. Every defect must be treated as an indication of a pattern that needs to be modified
Following that, you’ll be shipping world class software in no-time! Good luck
