MA.SP 1.2 Specify Measures

Measurement objectives are refined into precise, quantifiable measures.

Measurement of project and organizational work can typically be traced to one or more measurement information categories. These categories include the following: schedule and progress, effort and cost, size and stability, and quality.

Measures can be either base or derived. Data for base measures are obtained by direct measurement. Data for derived measures come from other data, typically by combining two or more base measures.

Derived measures typically are expressed as ratios, composite indices, or other aggregate summary measures. They are often more quantitatively reliable and meaningfully interpretable than the base measures used to generate them.

Base measures enable the creation of many derived measures or indicators from the same standard data sources. In addition, there are direct relationships among information needs, measurement objectives, measurement categories, base measures, and derived measures. This direct relationship is depicted using some common examples in Table MA.1.

Example Project, Organizational, or Business Objectives	Information Need	Measurement Objective	Measurement Information Categories	Example Base Measures	Example Derived Measures
Shorten time to delivery Be first to market the product	What is the estimated delivery time?	Provide insight into schedule fluctuations and progress	Schedule and progress	Estimated and actual start and end dates by task Estimated and actual start and end dates of acquisition tasks	Milestone performance Percentage of project on time Schedule estimation accuracy
Increase market share by reducing costs of products and services	How accurate are the size and cost estimates?	Provide insight into actual size and costs compared to plan	Size and effort	Estimated and actual effort and size	Productivity
			Effort and cost	Estimated and actual cost	Cost performance Cost variance
Deliver specified functionality	Has scope or project size grown?	Provide insight into actual size compared to plan, identify unplanned growth	Size and stability	Requirements count	Requirements volatility Size estimation accuracy
				Function point count	Estimated vs. actual function points
				Lines of code count	Amount of new, modified, and reused code
Reduce defects in products delivered to the customer by 10% without affecting cost	Where are defects being inserted and detected prior to delivery?	Evaluate the effectiveness of defect detection throughout the product lifecycle	Quality	Number of defects inserted and detected by lifecycle phase Product size	Defect containment by lifecycle phase Defect density
	What is the cost of rework?	Determine the cost of correcting defects	Cost	Number of defects inserted and detected by lifecycle phase Effort hours to correct defects Labor rates	Rework costs
Reduce information system vulnerabilities	What is the magnitude of open vulnerabilities?	Evaluate the effectiveness of mitigating system vulnerabilities	Information assurance	Number of system vulnerabilities identified and number of system vulnerabilities mitigated	Percentage of system vulnerabilities mitigated

As a part of their measurement and analysis activities, projects can also consider the use of Earned Value Management (EVM) for measures related to cost and schedule [EIA 2002b]. EVM is a method for objectively measuring cost and schedule progress and for predicting estimated total costs and target completion dates based on past and current cost and schedule performance trends.

Typical EVM data include the planned cost of accomplishing specific and measurable tasks, the actual cost of completing tasks, and earned value, which is the planned cost of the work actually completed for each task. Using these base measures or similar ones, the project can calculate derived measures such as schedule and cost variance and more complex measures. These derived measures include schedule and cost performance indices. EVM derived measures can assist with estimating the cost for completion and additional resources that may be required.

To manage projects, an acquirer uses supplier data (i.e., base measures) and supplier reported derived measures in addition to measures of acquirer progress and output. Supplier measures required by the acquirer allow the acquirer to comprehensively address measurement objectives and to comprehensively determine the progress of the project. In some cases, these supplier measures will augment acquirer measures (e.g., supplier’s schedule performance index and size estimation accuracy).

In most cases, supplier measures are the primary source of data, especially with regard to the development of the acquired product or service. For instance, measurement and analysis of the product or product components provided by a supplier through technical performance measures is essential for effective management. Technical performance measures are precisely defined measures based on a product requirement, product capability, or some combination of requirements and capabilities.

It is important to use measures to track high-risk items to closure and to help determine risk mitigation and corrective actions. These supplier measures should be defined in the supplier agreement, including a supplier’s measurement collection requirements and measurement reports to be provided to the acquirer.

Example Work Products

1. Specifications of base and derived measures
2. Acceptance criteria for supplier measures

Subpractices

1. Identify candidate measures based on documented measurement objectives.

Measurement objectives are refined into measures. Identified candidate measures are categorized and specified by name and unit of measure.

2. Maintain traceability of measures to measurement objectives.

Interdependencies among candidate measures are identified to enable later data validation and candidate analyses in support of measurement objectives.

3. Identify existing measures that already address measurement objectives.

Specifications for measures may already exist, perhaps established for other purposes earlier or elsewhere in the organization.

4. Specify operational definitions for measures.

Operational definitions are stated in precise and unambiguous terms. They address two important criteria:

• Communication: What has been measured, how was it measured, what are the units of measure, and what has been included or excluded?
• Repeatability: Can the measurement be repeated, given the same definition, to get the same results?

5. Prioritize, review, and update measures.

Proposed specifications of measures are reviewed for their appropriateness with potential end users and other relevant stakeholders. Priorities are set or changed, and specifications of measures are updated as necessary.

6. Specify acceptance criteria based on operational definitions for measures that come from suppliers to the acquirer in a way that enables their intended use.

Measures may be provided by the supplier as detailed measurement data or measurement reports. Measures that come from suppliers should be associated with the acquirer’s acceptance criteria for supplier measures. Acceptance criteria may be captured in measurement specifications or by checklists.

Acceptance criteria should be defined in a way that enables the intended use of supplier measures, such as potential aggregation and analysis. These criteria should include criteria associated with the collection and transfer mechanisms and procedures that are performed by the supplier. Consider all characteristics about supplier measures that can affect their use, such as differences in financial calendars used by different suppliers.