Summary

Select evaluation methods.

Description

Methods for evaluating alternative solutions against established criteria can range from simulations to the use of probabilistic models and decision theory. These methods should be carefully selected. The level of detail of a method should be commensurate with cost, schedule, performance, and risk impacts.

While many problems may require only one evaluation method, some problems may require multiple methods. For example, simulations may augment a trade study to determine which design alternative best meets a given criterion.

Example Work Products



  1. Selected evaluation methods


Subpractices



1. Select methods based on the purpose for analyzing a decision and on the availability of the information used to support the method.

 

For example, the methods used for evaluating a solution when requirements are weakly defined may be different from the methods used when the requirements are well defined.


 

Typical evaluation methods include the following:
  • Testing
  • Modeling and simulation
  • Engineering studies
  • Manufacturing studies
  • Cost studies
  • Business opportunity studies
  • Surveys
  • Extrapolations based on field experience and prototypes
  • End-user review and comment
  • Judgment provided by an expert or group of experts (e.g., Delphi method)



2. Select evaluation methods based on their ability to focus on the issues at hand without being overly influenced by side issues.

Results of simulations can be skewed by random activities in the solution that are not directly related to the issues at hand.



3. Determine the measures needed to support the evaluation method.

Consider the impact on cost, schedule, performance, and risks.