BMM Impacts of Technologies Studies

BMM-ITImpact Topic Summary
BMM ITImpact


These two Benchmarking and Metrics (BMM) reports, published in 2001 and 2006, present then-leading data regarding the extent of implementation and the beneficial impact of information technologies to support project execution and improve performance.

Principal findings are that there is evidence that use of information technologies has benefits on capital project outcomes on multiple metrics, although different methodologies measure different results. There also appears to be evidence of a learning curve for adopters that may impede performance improvements until some maturity using the tools is achieved.

Of note, these two reports utilize two different approaches to measuring the impact of information technology on project performance. The earlier document (BMM2001-10) tracks direct use of specific “design/information technologies” whereas the later document (BMM2006-11) tracks more abstract indices of information technology use for specific task/work functions. These work functions occur at higher and lower levels of definition. These reports track a key evolution in measuring IT use from direct use of specific technologies to more distributed use of technologies on work functions or processes. This change came about as technologies proliferated (and interacted), it become more difficult to assess the impact of a specific technology on performance. Thus a preference in reports to assess level of IT use on project work functions as a way of capturing level of use on common tasks. Both reports also contain methodological approaches of wrapping up level of use into an overall use or index score.

Key Findings and Implementation Tools

1 : Benefits

Increased use of information technologies is positively associated with several project performance outcomes. There is positive association with lower cost, shorter schedules, and lower cost and schedule growth. There are also positive benefits for safety and quality. Percentages for all these categories vary considerable across the reports, from small but statistically significant 1-2% (most benefits range from 1-6%) to highs reports of up to 16-17% for schedule compression. (BMM2006-11, p. 55)
Reference: (BMM2006-11)

2 : Variability

The percentage benefit in performance (or finding of statistically valid benefits) varies considerably across reports and data sets. Broadly, it seems there is evidence of benefits from information technology use, but assessment of the benefits is difficult to measure consistently. IT is not the sole driver of project performance and as such it is difficult to clearly identify the relationship between project performance and IT utilization. (BMM2006-11, p. 55)
Reference: (BMM2006-11)

3 : Learning Curve Effects

BMM2001-10 shows clear evidence of learning curve effects for certain technologies where performance deteriorates with early use but improves with increased use (and associated maturity levels). Figure 2.5 below shows Owner cost growth with use of design/information technology across quartiles. The 4th quartile has the least use of technology, while the 1st quartile has the highest use. As shown in the figure, quartile 3, associated with some use of technology, has the lowest performance and as such is evidence that performance may decrease with early adoption of new technologies. This is consistent with other observations, but was not found in all cases. (BMM2001-10, p. 14)
Reference: (BMM2001-10)

4 : Technology Usage

Both reports show widespread use of technologies across projects and work functions. Use is somewhat higher on larger projects. Both owners and contractors continue to increase the use of technologies and both realize meaningful benefits. (BMM2001-10, p. 25)
Reference: (BMM2001-10)

5 : IT Methodology Measures

Both reports are useful examples of methodologies to measure IT use and compare it to performance. BMM2001-10 reports on direct use of technologies, while BMM2006-11 reports on use of work functions at both the project and firm level. Both reports are detailed enough to use as a resource for those seeking to perform similar measurements or related benchmarking functions. (BMM2006-11, p. 56)
Reference: (BMM2006-11)

Key Performance Indicators

Improved cost, Improved schedule, Improved quality (reduced errors & omissions), Improved safety, Improved predictability, Improved customer satisfaction

Research Publications

Impacts of Automation and Integration Technologies on Project and Company Performance - BMM2006-11

Publication Date: 11/2006 Type: Benchmarking Report Pages: 109 Status: Reference

Impacts of Design/Information Technology on Building and Industrial Projects - BMM2001-10

Publication Date: 10/2001 Type: Benchmarking Report Pages: 39 Status: Reference