Summary

Select measures and analytic techniques to be used in quantitative management.

Description

Refer to the Measurement and Analysis (MA) (CMMI-SVC) process area for more information about aligning measurement and analysis activities and providing measurement results.


Example Work Products



  1. Definitions of measures and analytic techniques to be used in quantitative management
  2. Traceability of measures back to the quality and process performance objectives
  3. Quality and process performance objectives for selected subprocesses and their attributes
  4. Process performance baselines and models for use by the work group


Subpractices



1. Identify common measures from the organizational process assets that support quantitative management.

Refer to the Organizational Process Definition (OPD) (CMMI-SVC) process area for more information about establishing organizational process assets.


Refer to the Organizational Process Performance (OPP) (CMMI-SVC) process area for more information about establishing performance baselines and models.


Product lines, standard services, and service levels or other stratification criteria can categorize common measures.



2. Identify additional measures that may be needed to cover critical product and process attributes of the selected subprocesses.

In some cases, measures can be research oriented. Such measures should be explicitly identified.



3. Identify the measures to be used in managing subprocesses.

When selecting measures, keep the following considerations in mind:

  • Measures that aggregate data from multiple sources (e.g., different processes, input sources, environments) or over time (e.g., at a phase level) can mask underlying problems, making problem identification and resolution difficult.
  • For short-term work, it may be necessary to aggregate data across similar instances of a process to enable analysis of its process performance while continuing to use the unaggregated data in support of individual work activities.
  • Selection should not be limited to service level or performance measures only. “Analysis measures” (e.g., transaction arrival rates, staff member skill levels, trends in the use of particular service system resources) may provide better insight into process performance.



4. Specify the operational definitions of measures, their collection points in subprocesses, and how the integrity of measures will be determined.

5. Analyze the relationship of identified measures to the quality and process performance objectives for the work and derive subprocess quality and process performance objectives that state targets (e.g., thresholds, ranges) to be met for each measured attribute of each selected subprocess.

 

Examples of derived subprocess quality and process performance objectives include the following:
  • Maintain a code review rate between 75 to 100 lines of code per hour
  • Keep requirements gathering sessions to under three hours
  • Keep test rate over a specified number of test cases per day
  • Maintain rework levels below a specified percent
  • Maintain productivity in generating use cases per day
  • Keep design complexity (fan-out rate) below a specified threshold



6. Identify the statistical and other quantitative techniques to be used in quantitative management.

In quantitative management, the process performance of selected subprocesses is analyzed using statistical and other quantitative techniques that help to characterize subprocess variation, identify when statistically unexpected behavior occurs, recognize when variation is excessive, and investigate why. Examples of statistical techniques that can be used in the analysis of process performance include statistical process control charts, regression analysis, analysis of variance, and time series analysis.

The work can benefit from analyzing the performance of subprocesses not selected for their impact on work performance. Statistical and other quantitative techniques can be identified to address these subprocesses as well.

Statistical and other quantitative techniques sometimes involve the use of graphical displays that help visualize associations among the data and results of analyses. Such graphical displays can help visualize process performance and variation over time (i.e., trends), identify problems or opportunities, and evaluate the effects of particular factors.

 

Examples of graphical displays include the following:
  • Scatterplots
  • Histograms
  • Box and whiskers plots
  • Run charts
  • Ishikawa diagrams


 

Examples of other techniques used to analyze process performance include the following:
  • Tally sheets
  • Classification schemas (e.g., Orthogonal Defect Classification)



7. Determine what process performance baselines and models may be needed to support identified analyses.

In some situations, the set of baselines and models provided as described in Organizational Process Performance may be inadequate to support quantitative work management. This situation can happen when the objectives, processes, stakeholders, skill levels, or environment for the work are different from other projects for which baselines and models were established.

As the work progresses, data from the work can serve as a more representative data set for establishing missing or a work-specific set of process performance baselines and models.

Hypothesis testing comparing work data to prior historical data can confirm the need to establish additional baselines and models specific to the work.



8. Instrument the organizational or work support environment to support collection, derivation, and analysis of measures.

This instrumentation is based on the following:

  • Description of the organization’s set of standard processes
  • Description of the defined process for the work
  • Capabilities of the organizational or work support environment



9. Revise measures and statistical analysis techniques as necessary.