by Michael Tardif, Assoc. AIA
Contributing Editor

Summary: An important part of any successful BIM implementation is measuring your firm’s progress so you can determine whether your investment is yielding beneficial results. Metrics can be difficult to establish for the deployment of a technology that affects business relationships, project delivery methods, staff skill, and workflow as well as the design process, but it is still possible to establish goals and define objective metrics. Not all goals and metrics can be expressed in dollars and cents, but they almost always can be quantified in some way that can be tied—at least indirectly—to the bottom line. The stronger the connection between an organization’s BIM implementation strategy and profitability, the better the results of the BIM implementation are likely to be.

The Capability Maturity Model (CMM) of the National Building Information Model Standard (NBIMS) is a good first step toward establishing BIM implementation benchmarks. The CMM is designed to measure both the “maturity” of a building information model and the processes used to create it. The use of the word “model” is an unfortunate choice of terms here, adding yet another shade of meaning to a word that is already overused in this context. The term is borrowed from the software industry. According to Wikipedia, it was originally developed in 1986 by the Carnegie Mellon Software Engineering Institute (SEI), a federally funded research and development center, as a tool for assessing the ability of government contractors to perform a contracted software project. The CCM concept has since been applied to related disciplines and activities such as software engineering, system engineering, project management, software maintenance, risk management, system acquisition, information technology (IT), and personnel management. Through NBIMS, it, is now being applied to building information modeling.

To minimize confusion, the NBIMS CMM would be more aptly named the Capability Maturity Index, since that is what it truly is: an index, or benchmark, for measuring the maturity of your organization’s BIM capabilities. It identifies 11 categories of maturity, each of which can be scored on a scale of 1 to 10. Version 1 of NBIMS acknowledges that the scale values of the CMM are subjective and in need of further definition and refinement. But the 11 categories appear to encompass all of the relevant categories, and the scale values are useful even in their initial draft state of development, particularly if an organization defines the values more precisely for its own purposes. Information about the Capability Maturity Model can be found in NBIMS v1, which can be downloaded at no cost, but the 11 CMM categories and summary descriptions are explained below.

Data richness. Refers to the degree to which a building information model encompasses the available information about a building. The scale ranges from individual pieces of unrelated data to information that is sufficiently comprehensive and authoritative to be regarded as corporate knowledge.

Lifecycle views. Refers to the degree to which a building information model can be viewed (and used) appropriately by any players throughout the building lifecycle who may have need of the data to execute their responsibilities. The current scale presumes that building data originate in the planning and design phase of a building lifecycle, and measures the number of available views cumulatively from early planning stages through facility management/operations, then beyond “building specific” professionals to real estate portfolio managers, business operations managers, and external users such as emergency first responders. The greater the number of lifecycle views supported by a building information model, the less likely that building information will be redundantly entered into separate information-management or business-process systems. This category of maturity has enormous implications for building owners, as it measures the degree to which building information can be transformed into business information.

Roles or Disciplines. Refers to the number of building-related roles or disciplines that are accommodated in the modeling environment, and thus is a measure of how well information can flow from one role or discipline to another. The scale recognizes that currently available modeling environments are unable to accommodate even one role or discipline fully. The lowest end of the scale is partial accommodation of a single discipline, rising incrementally up to an environment in which all building-related disciplines can rely on the building information model as the sole information resources to perform their jobs. Like the Lifecycle Views scale, this scale presumes that building data originate in the planning and design phase of a building lifecycle and measure the number of roles/disciplines supported cumulatively from early planning stages through facility management/operations.

Business Process. Refers to the degree to which business processes are designed and implemented to capture routinely information in the building information model as an integral part of each business process. This is a key, long-term metric of progress in BIM implementation; one that should be a strategic focus of every BIM software company. Whenever information can be gathered as an integral part of a business process, the compilation of that information is achieved at no additional cost. Whenever data are compiled as a separate process, the cost is greater and resources are diverted from primary business processes, reducing the likelihood that the data compilation task will be completed consistently. Or, to put it another way, any time you have to take time out of your day job to compile data for someone else to use, the chances that you will do it consistently, if at all, are slim. The scale ranges from “business processes undefined and not used to compile data” to “all business processes are designed to collect and maintain information in real time.” The high end of the range is a very high standard of performance to achieve.

Change Management. Refers to the degree to which an organization has developed a documented methodology for changing its business processes. Whenever a business process is found to be flawed or in need of improvement, a formal, documented process is followed that begins with a “root cause analysis” followed by a modification of the business process based on the analysis. The scale ranges from “no evidence of documented change management” to an environment in which business processes are routinely supported by an integrated change management process that includes root cause analysis and feedback loops to assess the effectiveness of the change.

Delivery Method. Refers to the robustness of the IT environment to support data exchange and information assurance. The scale ranges from “BIM is only accessible from a single workstation with no integral information assurance” to “BIM is in a netcentric Web environment, delivered as a service in a service-oriented architecture (SOA), with role-based CAC enabled to enter and access information.”

Timeliness/Response. Measures the degree to which BIM information is sufficiently complete, up-to-date, and accessible to users throughout the lifecycle. The scale ranges from “information is collected when needed to respond to a question” to “information is continually updated from live-feed sensors and accurately reflects real-world conditions; responses to questions are immediate and authoritative.”

Graphical Information. Refers to the degree of sophistication or embodied intelligence of graphical information. The scale ranges from “no graphics in the BIM; text only” to “graphical information stored in the BIM is object-based, parametrically intelligent, and includes information related to time and cost.”

Spatial Capability. Refers to the degree to which the building information model is spatially located in the real world according to Geographic Information Systems (GIS) standards. This metric has implications for users across the building life cycle, including energy design and analysis, authoritative coordination with public infrastructure such as water and utilities, and timely response by emergency first responders. The scale ranges from “the facility is not spatially located” to “information from the BIM is fully recognized in the GIS environment, including support for full metadata interaction.”

Information Accuracy. Measures the degree to which information reflects real-world conditions. The scale ranges from “no ground truth; information is loaded manually, not verified electronically” to “all spaces are calculated automatically and methodologies are in place to ensure that information is accurate.”

Interoperability / IFC Support. Measures the degree to which data can be reliably exchanged among software applications using the open-standard Industry Foundation Classes. The scale ranges from “no interoperability between software applications” to “IFCs are fully supported and used for information exchange.”

Copyright 2008 Michael Tardif. Reprinted with permission.