You’re under pressure to deploy AI in various phases of the SDLC. You want to do it in a structured, secure way where you allow the AI agent to work alongside your current team and phase them out only when the AI performance is better. How to do that? It’s all in this article, read on!
AI Agents vs. Humans
AI writing requirements, AI writing code, AI writing documentation, AI writing tests, there are so many options and so many tools. Where does it make sense to deploy AI and how to actually measure its impact. It is also becoming more and more expensive to use AI and comparing the cost of tokens with the cost of humans is becoming a real challenge.
There are a couple of steps to this process:
- Decide on the SDLC phase you want to deploy AI in, it needs to be a phase where you are currently suffering some pain
- Define a clear, measurable quality gate for this phase (code review escapes, test failures, review comments etc.)
- Deploy AI in this phase giving it roughly half the workload
- Benchmark cost, time to completion and accuracy in the following way:
- Cost – labor hours vs. cost of tokens spent on the same payload
- Time to completion – time it took to achieve the definition of done defined in your quality gate
- Accuracy – similar to yield, how many of the work artifacts were approved by the quality gate and how many rejected
- Optimize on what matters to your organization
You may work in an industry that favors speed over accuracy, where time to market new features which might not be fully stable is key. In this case, optimize for time to completion. If you work in a more traditional industry like Automotive or Healthcare, you may be optimizing for accuracy, the cost of a recall is too big. So optimize for that.
Only when the AI agent performs consistently better on that particular phase, you can gradually reallocate your current team to other tasks, gradually, while keeping the measurement on at any time.
The Bottom Line
Give it a go and let me know how it goes.
If you need advice on how to set the correct KPI or benchmark for the specific SDLC phase, I invite you to an upcoming webinar hosted by the Agile Alliance on June 3rd, featuring our CEO, Mr. Mooly Beeri. Here is a link to the registration page, we would be excited to see you there!
https://agilealliance.org/event/making-continuous-improvement-measurable-in-agile/
Thanks for reading!
