BMM Impacts of Technologies Studies

BMM-ITImpact Topic Summary
BMM ITImpact

Overview

These two Benchmarking and Metrics (BMM) reports published in 2001 and 2006, report on leading data at the time of publication regarding extent of implementation and beneficial impact of information technologies to support project execution and improve performance.

Principal findings are that there is evidence that use of information technologies has benefits on capital project outcomes on multiple metrics, although different methodologies measure different results. There also appears to be evidence of a learning curve for adopters that may impede performance improvements until some maturity using the tools is achieved.

Of note, these two reports utilize two different approaches to measuring the impact of information technology on project performance. The earlier document (BMM2001-10) tracks direct use of specific “design/information technologies” whereas the later document (BMM2006-11) tracks more abstract indices of information technology use for specific task/work functions. These work functions occur at higher and lower levels of definition. These reports track a key evolution in measuring IT use from direct use of specific technologies to more distributed use of technologies on work functions or processes. This change came about as technologies proliferated (and interacted), it become more difficult to assess the impact of a specific technology on performance. Thus a preference in reports to assess level of IT use on project work functions as a way of capturing level of use on common tasks. Both reports also contain methodological approaches of wrapping up level of use into an overall use or index score.
 

Key Findings and Implementation Tools

1 : Benefits

Increased use of information technologies is positively associated with several project performance outcomes. There is positive association with lower cost, shorter schedules, and lower cost and schedule growth. There are also positive benefits for safety and quality. Percentages for all these categories vary considerable across the reports, from small but statistically significant 1-2% (most benefits range from 1-6%) to highs reports of up to 16-17% for schedule compression.  (BMM2006-11, page 55)
Reference: (BMM2006-11)

2 : Variability

The percentage benefit in performance (or finding of statistically valid benefits) varies considerably across reports/data sets. Broadly, it seems there is evidence of benefits from information technology use, but assessment of the benefits is difficult to measure consistently. IT is not the sole driver of project performance and as such it is difficult to clearly identify the relationship between project performance and IT utilization.  (BMM2006-11, page 55)
Reference: (BMM2006-11)

3 : Learning Curve Effects

BMM2001-10 shows clear evidence of learning curve effects for certain technologies where performance deteriorates with early use but improves with increased use (and associated maturity levels).  The figure below (Figure 2.5, page 14) shows Owner cost growth with use of design/information technology use across quartiles. The 4th quartile has least use of technology where the 1st quartile has the highest use. As shown in the figure, quartile 3, associated with some use of technology, has the lowest performance and as such is evidence that performance may decrease with early adoption of new technologies. This is consistent with other observations, but was not found in all cases.              (BMM2001-10, page 11)
Reference: (BMM2001-10)

4 : Technology Usage

Both reports show widespread use of technologies across projects and work functions. Use is somewhat higher on larger projects.  Both owners and contractors continue to increase the use of technologies and both realize meaningful benefits.  (BMM2001-10, page 25)
Reference: (BMM2001-10)

5 : IT Methodology Measures

Both reports are useful examples of methodologies to measure IT use and compare it to performance. BMM2001-10 reports on direct use of technologies while BMM2006-11 reports on use of work functions at both the project and firm level. Both reports are detailed enough to use as a resource for those seeking to perform similar measurements or related benchmarking functions. (BMM2006-11, page 56)
Reference: (BMM2006-11)

Key Performance Indicators

Improved cost, Improved schedule, Improved quality (reduced errors & omissions), Improved safety, Improved predictability, Improved customer satisfaction

Research Publications

Impacts of Automation and Integration Technologies on Project and Company Performance - BMM2006-11

Publication Date: 11/2006 Type: Benchmarking Report Pages: 109 Status: Reference

Impacts of Design/Information Technology on Building and Industrial Projects - BMM2001-10

Publication Date: 10/2001 Type: Benchmarking Report Pages: 39 Status: Reference


Tags