Choosing appropriate metrics to measure EA is challenging. We find that most practitioners, and their management, want to see “absolute” metrics, a measure yielding some sort of generally accepted min/max in the form of a quantity or percentage (e.g. 70% of all projects are reviewed).
Instead, we believe metrics are used primarily to tune an EA program. Metrics are relative to some planned target level, and those targets change as the program grows, matures, ages, is accepted by the community, etc. In other words, as it evolves, the types of metrics that are tracked and their targets evolve. Looking forward 6 months, for example, the EA leadership might be trying to establish an EA exception process as part of a stage gate in the development methodology. Early in EA adoption we might see a high percentage of exception requests because the target EA is poorly understood or incomplete. We may establish a goal to reduce exception requests through peer reviews, formal EA reviews, and education. Once the number hits its target floor we set a new goal to not let the percentage rise. If it does it might be an indication that we need to do an EA refresh or provide more project coaching.
In general, EA programs are measured across 3 general categories:
- EA Process Performance – internal measures of EA process activities
- Organizational Coverage – interaction of the EA process and content with people outside of the EA function
- Component Content Utility – the “usefulness” of the EA deliverables the team creates (by influencing project and infrastructure success and evolution)