We live in a metrics-fixated world where having more metrics is always thought to be better than having less, and Software Development Life Cycle (SDLC) metrics are no exception. But the truth is that any badly chosen or managed metric will do more harm than good to your organization. To avoid these pitfalls, take ownership for SDLC metrics away from managers and put it into the hands of those who can best manage it: your development teams.
There is no shortage of people who expound the virtues of establishing and tracking SDLC metrics with the mistaken belief that they provide effective “levers of control” for development activities, especially when they are used in combination with reward and punishment. But consider the cautionary tale of Wells Fargo whose use of metrics, punishment, and rewards (which were intended to drive business growth) resulted in massive fines, class action law suits, unhappy customers, and regulation to cap the company’s size (ironically, the opposite of what the metrics were intended to achieve!).
As Atlassian (the maker of Jira) has said “Metrics are a touchy subject. On the one hand, we've all been on a project where no data of any kind was tracked, and it was hard to tell whether we're on track for release or getting more efficient as we go along. On the other hand, many of us have had the misfortune of being on projects where stats were used as a weapon, pitting one team against another or justifying mandatory weekend work. So, it's no surprise that most teams have a love/hate relationship with metrics.”
In his book The Tyranny of Metrics, Jerry Muller shines a light on the dark side of metrics through a series of case studies in a variety of industries. He shows how blind belief in the cult of metrics has led to a culture of gaming and manipulation that results in predictable negative consequences. Understanding this dynamic is critical if you want to avoid the failings of bad metrics use in your organization.
When selecting your organization’s SDLC metrics, consider the U.S. Army’s findings on Counterinsurgency Metrics, which showed that standardized metrics are often deceptive, but metrics developed to fit specific circumstances, especially when selected by practitioners with local experience, could be genuinely informative. The lesson was to abandon fixed metrics and instead determine what is worth counting and what the numbers actually mean in their local context (Muller).
SDLC metrics (like all metrics) are largely misused in industry and can be detrimental to your organization. Having no SDLC metrics at all is better than implementing a bad metric (at best, a bad metric will be a productivity drain, but at its worst, it results in gaming behavior and/or unintended consequences that can actually steer your organization in the wrong direction).
You can avoid falling victim to the pitfalls of bad metrics through careful selection and cultivation of your metrics. Here are some simple rules to follow when defining and implementing SDLC metrics:
(SIDE NOTE: This approach to selecting and managing SDLC metrics will be most effective if your organization has successfully implemented Agile. For those members who have adopted Agile development processes, you will notice that the above approach builds on the Agile tenant of creating self-managing teams and providing them with the skills, tools, and support they need to deliver successfully. As servant leaders, Agile managers entrust their development teams with responsibility to self-organize and self-manage, then let them determine the best practical solutions to the myriad of problems they will encounter. This approach requires instilling both trust and accountability into your development team and generally leads to better results than can be achieved in a command-and-control organization. Why then, would you not also entrust them with figuring out for themselves what the right SDLC metrics are to best perform their responsibilities for the organization? This approach is far more likely to yield effective metrics for your organization than something selected “from on high” and imposed on your development team.
Additionally, Agile development practices (which involve continuous, incremental delivery of working and tested software) is well positioned for gathering metrices that are good measures of the value delivered to your customers.)