Measuring Progress and Defining Productivity Metrics in Model-based Engineering

RT-332 Topic Summary
RT 332

Overview

Have you ever walked into a model review session or looked into a model and wondered about the level of maturity for the deliverables?

Over the past decade, the adoption of a model-driven approach to engineering has dramatically changed the delivery and coordination methods in capital projects. Many firms are already engaged in model-driven engineering processes and they have a positive perception of the value they receive for the time, money, and efforts they have expended on their modeling programs. Nevertheless, there are no established and standardized processes for measuring the progress and productivity of a model-driven engineering process – particularly when the deliverable type, schedule reporting, and their impacts have changed.

In today’s modus operandi, the key underlying question is “How can we accurately measure productivity and progress toward the deliverables in a model-driven approach to engineering without imposing unnecessary work or taking away from actual productivity?” Any process or metric for measurement should provide accurate and timely information about both modeling progression and productivity to the project stakeholders. These processes and metrics should also be easy-to-use, flexible, and extendable for implementation across various project types and lifecycle phases of today’s capital projects.

To fill a gap in current knowledge, the RT-332 research offers a new guideline for measuring progress and defines productivity metrics in a model-driven approach to engineering. As part of this guideline, the team provides a series of standardized definitions to measure the maturity of various modeling disciplines as a function of the maturity of the design components and the quality of the information used in the modeling process. These definitions – categorized into a set of discrete Model Maturity Index (MMI) levels ranging from 100 to 600 levels – provide owners and engineering firms with a clear set of modeling requirements that can be fulfilled per engineering phase.

To successfully implement the MMI definitions and measure progress across both green- and brownfield projects, the RT-332 research also offers a Model Maturity Risk Index (Model MRI) toolkit, together with an addendum to existing model execution plans (ModelXP). Based on the MMI definitions and by accounting for the inter-disciplinary relationships between modeling disciplines, the Model MRI toolkit easily and quickly determines MMI levels for each model discipline, and for each Work Breakdown Structure (WBS) location.

The toolkit also offers insights on the risks associated with the remaining modeling work needed to achieve a certain MMI level, within the same discipline and also across other related modeling disciplines. The Model MRI toolkit can be used as part of any model review session to communicate modeling progress; and also internally to assess the actual modeling progress against client's expectations. The Model Execution Plan (ModelXP) addendum helps project teams to adopt and adapt the MMI definitions and the Model MRI toolkit to their specific project’s needs. The addendum specifically allows project teams to set clear expectations on the maturity of the model deliverables for each model review session and other project milestones.

The provided guideline empowers engineering firms to systematically track productivity in form of engineering-hours in between MMI levels and for each modeling discipline. This level of granularity can also support benchmarking of engineering productivity rates for future projects. Tracking productivity while using the Model MRI toolkit also supports project teams to better measure risk at each stage of the project, and devise control strategies that can keep projects on schedule and on budget.

The MMI definitions, the Model MRI toolkit, and the Model Execution Plan Addendum establish procedures and define metrics by which project stakeholders can reliably measure progress and productivity in a model-driven engineering process. While resources provided in this research are adoptable and adaptable for different types of projects and applications across the process industrial sector, they can be extended for use by other construction industries such as commercial or industrial buildings. The potential for automation of the progress measurement in the modeling environment is demonstrated via a real-world project example using modeling solutions and is being explored as part of ongoing research. During its research, RT-322 explored the automation of the progress tracking and risk assessment in the modeling work by using MMI definitions. Such automation can be achieved by enabling model authoring and the management environment to use MMI levels and provide visual feedback on progress, productivity, and risk.

Key Findings and Implementation Tools

1 : Lack of Standardized Process for Measuring Progress and Productivity in the Industry

An RT-332 survey identified that there are no established, industry-wide standard processes and metrics for measuring the progress and productivity of a model-driven engineering process.

Reference: (FR-332)

2 : Model Maturity Index (MMI) Definitions

MMI definitions, measured in discrete levels of 100, 200, 300, 350, 400, 500, and 600, help measure maturity of the model, by modeling discipline, as a function of what is modeled and the quality of the data used to create the model.

Reference: (FR-332)

3 : Model Maturity Risk Index (Model MRI) Toolkit to Measure Progress and Risk

Through a set of MMI-based questionnaires and by accounting for the relationships between model disciplines, the Model MRI toolkit easily and quickly determines the MMI levels for each model discipline, by its location in the project's Work Breakdown Structure (WBS).

Reference: (FR-332)

4 : Model Execution Plan Addendum

The addendum helps project teams to adopt and adapt the MMI definitions and the Model MRI toolkit to suit their project needs, and to set clear expectations on the maturity of the model deliverables at each project milestone.

Reference: (FR-332)

5 : Units of Measurement

Discipline Unit(s) of Measurement
  Piping   LFT of pipe
  Equipment   Per tag
  Layout   Sq. ft.
  Structural Steel   Tons
  Subsurface foundations   CY, and number of piles
  Electrical   LFT of cable
  HVAC   CFM? BTU? Sq. Ft.
  Architecture   Sq. ft.
  FP   LFT of pipe
  P&IDs   None
  Civil   Acre (earthwork), LFT (drainage)
Reference: (FR-332)

6 : Overall Validation

The applicability of the RT-332 research was extensively validated via several charrettes with industry experts. An example project is provided in Final Report 332, to help project teams adopt and adapt the guidelines based to their projects’ specific needs.

Reference: (FR-332)

Key Performance Indicators


Supporting Resources

Presentations (CII Annual Conference & Workshops)

Session - Measuring Progress and Defining Productivity Metrics in Model-based Engineering

Publication Date: 08/2017 Presenter: Number of Slides: 43 Event Code: AC2017


Tags