Evaluation of Design Effectiveness

This report presents the Objectives Matrix as a method to list, categorize, and weight the factors of design having significant project impact The report discusses the four main components of a matrix: the criteria, the weights, the performance scale, and the performance index. Crucial criteria for design effectiveness are presented, and the use and development of the Design Evaluation Matrix are described. The feasibility of the method is investigated by gathering piping data from seven industry projects, and using these data to construct actual matrices. This final chapter presents conclusions to, and recommendations for, the objectives matrix for design effectiveness evaluation.

Conclusions

1. The seven important criteria measuring design effectiveness immediately after construction and relevant regardless of construction category, construction activity, design user, or project variable are: Accuracy of the Design Documents, Usability of the Design Documents, Cost of the Design Efforts, Constructability of the Design, Economy of the Design, Performance Against Schedule, and Ease of Start-Up.
2. The Design Evaluation Matrix can be used by any design user for any project type or phase.
3. The criteria and weights used in the matrix can be modified by the evaluator to fit any project, and the evaluation call adapt to most needs.
4. The matrix can be as simple (subjective ratings) or sophisticated (sub-matrices) as desired or needed.
5. The Design Evaluation Matrix can track and compare performance over time and measure impacts of various criteria.
6. The Design Evaluation Matrix is feasible. As has been proved by the Design Evaluation Matrix for piping, all data necessary to measure the criteria can be obtained or estimated.
7. The seven design evaluation criteria have proven to be meaningful and can be quantified by various sub-criteria ratios and subjective ratings. The criteria have also proven fairly comprehensive, as no suggestions were made for additional criteria.
8. Not all the data required for the feasibility test, and for the evaluation itself, is readily at hand in some companies. Since no interviewees questioned the significance or relevance of the criteria, or could offer other measurements, the problem is riot inappropriate or inapplicable criteria. Instead, this problem points out the need for a method such as the Design Evaluation Matrix as a means of evaluation.
9. The data from the industry interviews can be inserted into evaluation matrices and used to evaluate piping design.
10. Ratio measures provide a better means of evaluation than do subjective ratings. The subjective ratings are consistently higher than the industry average at score three, and by raising performance indices this may prove misleading. Quantitative measures need to be identified for all sub-criteria.
11. It is possible to track the performance of each of the criteria using the performance index of the Criterion Evaluation Matrix. When compared to past performance indices, the increase or decrease of this performance index indicates improvement or decline in the performance of the criterion.
12. The performance index might also be used to measure the affect on projects of the project variables of schedule, size, and types of contract. Further study of proper use of the matrix may indicate a relationship in movement of the performance index due to changes in the variables.

1. The average values presented in this report are superseded by the more statistically significant numbers future reports will contain.
2. Future studies should also identify measurable criteria and sub-criteria through the use of ratios or other methods for not only just the piping phase of construction projects, but for other construction categories and activities.
3. Future studies should concentrate on the effects of these variables upon the movement of the performance index of a matrix. Potentially, these studies could identify criteria for special consideration when a project variable might have significant impact.
4. Potential uses of the matrix, other than the obvious evaluation purposes, should also be identified. Besides indicating areas for special project consideration (due to project and other variables), other uses should be investigated. These uses might include project cost estimation and control, or unilateral owner rewards based on performance index scores as incentive to designers.