Document Detail

Title: RS322-1 - Metrics That Matter: Improving Project Progress and Performance Assessment
Publication Date: 11/1/2016
Product Type: Research Summary
Status: Supporting Product
Pages: 34
Summarizes how RT-322 research identified core forecasting and diagnostic metrics that can provide insight into a project's actual progress, performance, and forecast at completion. Also describes how to improve the reliability of these metrics.
Order Now  

Abstract

Assessment of project progress and performance is critically important to the successful delivery of capital facility projects. Project managers are often misinformed in their perceptions of project performance until the project nears its end. Major challenges are related to the lack of consistent, objective, and reliable metrics and indicators, and the lack of appropriate interpretation of these data for establishing suitable corrective action plans. Research conducted by CII Research Team 322 (RT-322) identified several key Forecasting and Diagnostic metrics that can provide insights into a project’s actual progress, performance, and forecast at completion. It also provides information on ways to improve the reliability of these metrics.
 
The RT-322 research methodology included a broad literature review of more than 120 sources to identify progress, performance assessment, and performance forecasting metrics. The review also identified factors to improve metric reliability in several areas, including project scope definition, execution planning, and risk management. Next, the researchers distributed a survey to collect data on the metrics and reliability techniques used on completed projects. In total, 44 survey responses were collected, representing mostly large, industrial projects. Subsequently, 10 case studies provided more in-depth analysis of some of these projects and their use of metrics. This was followed by a Delphi session with 16 subject matter experts with more than 350 years of combined experience, who evaluated and validated the findings from the survey and the case studies. The Delphi session further refined a list of in-flight project control metrics divided into the following categories: Core (“must have”), Validation (metrics that confirm the validity of the Core metrics), Innovative (metrics that are not currently in wide use, but are considered potentially beneficial), and Other Significant (other metrics that fall outside the previous categories, but are perceived to have value).
 
Core metrics were initially selected based on the following project characteristics: large, industrial, reimbursable cost, balanced cost and schedule goals, moderate complexity, and contractor perspective. However, when considering Core metrics for other project characteristics, it was discovered that these metrics tended to be the same across projects—the only differences were the frequency of data collection and the level of effort involved in collecting and analyzing these data. Further statistical analysis of survey responses revealed, that compared to projects using fewer Core metrics, projects that used more Core metrics for project controls experienced higher rates of success at meeting their original budgets. A correlation between the use of more Core metrics and better project cost outcomes was observed at the 95% confidence level using the Spearman’s rank correlation method.
 
The RT-322 research identified in-flight project control metrics for determining project progress measurement, performance assessment, and performance forecasting. Critical factors were identified that individually and collectively influence the reliability of project progress and performance assessment metrics. A useful dictionary of Core metrics, including their definitions, uses, and interpretation information, as well as “metric maps” (demonstrating how each metric is calculated), was produced. Furthermore, the Project Controls Improvement (PCI) Tool was created to provide a more standardized or systematic way of assessing a project’s progression and performance. Using the PCI Tool, project stakeholders can identify the gaps in their project control systems or learn more about Core metrics and steps they can take to improve metric reliability within a dynamic and interactive software environment.