Chapter IV

A. PREFACE

The objective of this chapter is to discuss the basic considerations required to present (to the appropriate decision maker) an orderly, comprehensive and meaningful display of all returns (outputs, benefits, yields, worth) expected for each alternative within the scope of the economic analysis under consideration. The returns of each alternative should be expressed so that the decision maker is able to compare the various alternatives of the economic analysis. (For purposes of this handbook, the term "benefits" is used as the overall term for returns (output, products, services, yields, worth).)

By referring to the Chart, "The Process," you will note the position of our objective in the Economic Analysis Process shown as 4.b., "Determine Benefits." This display assists us in focusing on the broad nature of the benefit determination effort and gives a better understanding of the role that those determinations must play in deciding between alternatives.

General aids which apply to all analyses and which will be dealt with more fully in the "Procedures" section of this chapter are:

1. Use a systematic procedure to establish returns in order to minimize strictly subjective judgment.

2. Discover and record all the benefits, whether or not quantifiable, relevant for each of the alternatives developed in Item 2 of "The Process."

3. Express, if possible, the returns of each alternative in terms of a common denominator or a score.

4. Arrange returns according to some hierarchy of values if a common denominator not available.

The consistency and relevance of the benefits available must be carefully examined. An existing measure with which management is familiar has certain advantages in regard to ready acceptance as a benchmark, but it may not be relevant throughout the entire range of the study. The analyst should be as cautious in accepting a benefit measure just because it's there as he should be in introducing a new untried one that simply accommodates only an aspect of his study. The output information effort under DoD Instruction 7045.11, "Improvement and Use of Output Information in the Department of Defense Programming' Planning, and Budgeting System," fated December 17, 1970, and DoD Directive 5010.15, "Defense Integrated Management Engineering System," dated January 13, 1972, should be studied. Additional literature in this area of analysis is available and is being compiled into a bibliography.

The Process chart indicates that in Items 1, 2 and 3 of any Economic Analysis: (a) the objectives of the study are defined in detail, (b) the alternatives or solutions to meet the objectives are provided, and (c) the assumptions are formulated. We are then ready to determine the costs and benefits related to each of the alternatives as spelled out for the particular economic analysis. This chapter will attempt to outline how we go about determining the benefits of each of the options we feel could meet the objectives of the economic analysis problem. It is best, because of the state of the art, that only an overall methodological approach be used to guide our practitioner.

The following suggested Step-by-Step Procedure will greatly facilitate objective benefit determination.

Step I - Determine, List and Define Relevant Benefits.

Step II - Establish Sources of Information for Benefit Determination.

Step III - Collect and Display Information for Benefit Determination.

Step IV - Summarize, Evaluate and Present Benefit Determination for Alternatives of the Economic Analysis.

B. PROCEDURES

1. Determine and Define the Benefits Relevant for each of the Alternatives of the Economic Analysis.

a. Determine and list the benefits of each alternative -- whether the benefit is thought to be potentially quantifiable or not quantifiable. List all benefits which may possibly shed light on the economic analysis alternatives. It is quite possible that some of the benefits listed in this first attempt will eventually be discarded and others becoming evident further on in the analysis will be added to the list. For instance, if one method causes ten items to be produced and only two are needed, the greater productive capacity of this system may not be a plus factor. Other considerations may come into play such as availability of storage space, cost of storage, obsolescence, etc.

b. Define each benefit in relation to its respective alternative in the economic analysis. Describe each return as well as you can at this step. Remember that at any point in the Benefit Determination Procedure, new or previously unrecognized evidence may cause us to go back and retrace any one or several steps of the procedure.

During this process consideration should be given to the level of decision of the economic analysis. For example: Let us assume that in a five-man warehouse at an installation, we store spare parts on seven shelves. We are considering decreasing the layers of vertical shelving in order to obtain greater warehouse efficiency. Instead of having seven shelves, the items will be stored on five shelves so that all items will be accessible without using ladders (ladder is now used when pulling material from the two top shelves). (Although there may be other alternatives for this problem, such as mechanization, we will restrict ourselves for illustrative purposes.) In this case, the decision could be made by the local operating official and benefits related to economic effects on the community (if any) would not be germane; however, benefits related to customer service, employee morale, safety, etc., could well be considered.

However, if the investment is a large one, such as whether or not to consolidate field activities or buy some special equipment, the decision may be at the Service level (and in some cases, probably, at the OSD level). In such cases, benefit determination related to the economics of the community could be one of the determinants for selecting a particular alternative.

Each situation must be dealt with within the context of the total economic analysis under study.

There is no check list available with which to ascertain that all output returns for an alternative of an economic analysis have been included in the benefit determination, and that all are valid for the particular situation. However, in order to assist the analyst in selecting benefits germane to the study and, hopefully, in excluding spuriously related and nonsignificant information for the decision maker, characteristics such as the following could be reviewed when listing and defining benefits:

Discreteness: Is the benefit clearly and concisely identifiable from all of the other benefits? Does it overlap with any other measure? Is it duplicated? Maintain as separate an entity as is possible.

Quantification: Is the benefit directly/indirectly measurable using valid techniques available from the various disciplines used in analysis? If not, can some method for comparability be used? If quantification is not possible, can other techniques such as ranking, etc., be used decision purposes? Quantification is by no means essential for output information to be useful for analytical purposes, although precision and specificity are needed to the greatest feasible degree.

Discriminative: Is the benefit related to the alternative of the economic analysis? Is it discriminating in relation to the objective of the decision maker? Is it spuriously related to the purposes of the decision and should therefore be excluded?

Also, we will find that the benefits expected of any alternative may fall into various "categories" depending on the kind of program, systems, operation, organization, etc., that has been submitted for economic analysis. Terminology used for these categories is generally descriptive of the benefits included. These are not intended as definitive, but as guides to the analyst in the effort to include all benefits related to an alternative. It should also be cautioned that the list is not intended to be all inclusive; it is only illustrative of some of the types of benefit categories that could be applicable depending on the problem. Some of the categories under which benefits appear are:

(1) Production: Number of commodities or items produced for each alternative. For example: Number of meals served, hours flown, components manufactured. This could be related to comparable time periods of the economic analysis (as in productivity).

(2) Productivity: (related to staffing benefits) number of items per manhour, volume output related to manhours.

(3) Operating Efficiency: At what rate does the system consume resources to achieve its output? For example, miles per gallon, copies per kilowatt hour, mean days per shipment.

(4) Reliability: This describes the system in terms of its probable failure rate. Useful measures may be mean-time-between-failure, the number of service calls per year, percent refusals per warehouse requests.

(5) Accuracy: What is error rate? Measure errors per operating time period. Number of errors per card punched, errors per hundred records, errors per 100 items produced, etc.

(6) Maintainability/Controllability: Has adequate human engineering been performed? Is the system compatible with adequately trained crew members? When the system does fail, is it difficult to repair because of poor accessibility? A useful measure could be based on the average manhours necessary for repairs over a given time period, i.e., downtime, or the crew rate necessary to control and maintain the system.

(7) Manageability: Consider how the workload of the organization will be affected by increased or decreased supervision or inspection time as a result of the system. Man-days could be used as a measure; difference in kind of personnel might be a factor as well as availability of type needed.

(8) Integratability: Consider how the workload and product of the organization will be affected by the changes necessitated in modification of existing facilities or equipment, technical data requirements, initial personnel training, warehouse space for raw goods or parts storage, etc.

(9) Availability: When can each system be delivered/implemented; when is it needed to meet proposed output schedules? What is the lead time for spare parts delivery?

(10) Service Life: Consider how long the proposed system will affect the organization's workload or output. What about obsolescence?

(11) Quality: Will a better quality product/service be obtained? Could quality be graded, thus measurable? If not, a description of improvement could be given. What is the impact of the varied quality?

(12) Acceptability: Consider the alternative in terms of whether it may interfere with the operation of parallel organizations or the operation of prerogatives of higher echelon organizations.

(13) Ecology: Consider the ecological aspects of each alternative. What are the current legislative requirements?

(14) Economic: Consider employment benefits, DoD small business obligations, economically depressed area relationships, legislative requirements.

(15) Morale: Employee morale. This could be measured by an opinion sample survey.

(16) Safety: Number of accidents, hazards involved.

(17) Security: Is security built in? Will more precautions be needed? More guards? Are thefts more likely?

Pertinent benefit categories will become evident as the analysis of the alternatives is performed. The benefits, of course, will be defined/ described in accordance with the requirements of each alternative under review.

2. Determine Sources of Information for Benefits Listed in Step 1.

a. Separate the Benefits defined in Step l into two lists as follows:

List I. Benefits where Back-Up Information is Available.

Benefits for which information in usable form is easily obtainable. Next to each benefit listed, indicate source of information, in what form it is available, and in general terms, next to each benefit, how you propose to gather the needed information and the feasibility of doing so. Should the analyst decide that obtaining the needed information is impractical,for whatever reason, he should be able to support his position. This step applies to benefits which may be quantifiable as well as for those which do not seem quantifiable. It is best to obtain the maximum amount of information in estimating parameters. However, this may not always be feasible.

For example, if in Step 1 you have listed "Production of an Item," check actual data available to see if there are weeks, years, etc., of production records with data which could be used for actual production and estimating purposes when valid statistical or other analytical techniques are used. If the immediate organization does not have such information, is it available for a comparable organization? Is prototype data available, etc. me statistician, mathematician, industrial engineer, etc., will be helpful in determining whether there are techniques available in the relevant disciplines that can be applied to substantive information in order to obtain the benefit determination needed for the economic analysis. (Applying various techniques to data already in the system could preclude the cost and time needed to gather additional data.)

For benefits not quantifiable even by ranking, rating, or related methods, list any appropriate available and reliable sources for narrative detail or use experience judgment sources.

List II. Benefits for Which Back-Up Information is not Available.

For the remaining benefits, or those for which no information sources have been readily identified, the analyst will have to do some research in deciding how to obtain information for his benefit determination. Indicate next to each benefit the method proposed in order to obtain information.

In these instances, information may have to be obtained by conducting a 100 percent collection of relevant data for the benefit in question, a sample survey may be possible for obtaining data, field trips by experts conferring with experts may be needed, specialized libraries may serve as sources for relevant input, and other public agencies or private firms and institutions could prove helpful. The specific circumstance will decide the process.

As examples of what is meant by benefits for which information may not readily be available, we cite benefits such as morale of personnel, safety of an operation, etc. In these instances, a statistical sample could be used to produce the data for the system at hand and could be used as benchmark statistics for the related alternatives and for projection purposes. For a weapons system where data may not be ava of existing systems may serve the same purpose.

With the completion of Step 2 of this procedure, the analyst

a. Identified and defined or described the benefits resulting from each alternative required in the particular economic analysis.

b. Sources of information and/or methods for obtaining the information for each benefit. We can now proceed to Step 3.

3. Collection of Information for Benefit Determination.

a. Organize the method for collecting information for each benefit, collect the applicable data, and record the information for each alternative of the economic analysis.

b. It must again be emphasized that both the subject matter specialist and the individual knowledgeable in the disciplines concerned with formulating quantifiable and nonquantifiable outputs for analysis purposes must cooperate if adequate usable benefit determinations are to be established.

c. At this point, the information collected can be recorded simply by listing the information for each benefit, in tabular form, similar to the following display. (At this point, there is no need to be concerned about scoring, ranking or establishing any hierarchy of values, since this will be the thrust of Step IV.)

After Step 3 has been completed, it would be beneficial for the analyst to review what has been done to see whether benefits should be added/deleted, whether more relevant yardsticks for the associated benefits could be designated, whether with greater imagination and use of analytical techniques available, more adequate benefits and benefit measures could be produced for the decision maker's understanding and consideration.

4. Summarization Evaluation and Presentation of Benefits.

In order for benefit determination to be of value for decision maker, comparative visibility of the benefits of each alternative is necessary. A generalized format, Tables 1 and 2 following, which should, of course, be varied to fit the specific situation, is shown for discussion purposes. The exact method of comparison and the tools and techniques to be used must be left to the analyst in conjunction with the subject matter and professional analytical personnel since proper ''weighing," quantitative and nonquantitative comparisons and over-all scoring of system dimensions will vary with different systems, organizations, programs, etc., being studied in the economic analysis.

Many techniques are available for comparing quantifiable benefits. Some which have been used include graphic analysis, regression analysis, indexing, decision theory, marginal analysis, ratios, linear programming, mathematical and economic statistical modeling. Nonquantifiable benefits may be analyzed by using certain nonparametric statistical techniques. A possible technique for weighing benefits might be a polling technique such as the Delphi method.

In this step, array the benefits and their respective data in order of significance of each benefit to the problem objective. Then, where possible, combine benefits to give a composite score for each alternative. In some problems, it may even be possible to calculate a score for the total alternative directly if data are in the same units. In any event, such consolidation will assist in the decision making process since it reduces some of the detail; however, the detail information for the individual benefits should be accessible. For example, in order to measure the benefit of different warehouse processes, it may be feasible to measure the warehouse's receiving and storing functions. Since receiving is recorded in line items and storing in measurement tons, it is possible to combine the two using a weighted index with respective manhours for each function as weights. Alternative I warehouse benefit would then be equated with base 100 and variation from this could be on par, better, or worse for other alternatives depending on the weighted index calculated from the estimated or actual data.

Another method of composite scoring would be to convert actual output to some common factor such as dollars. In so doing, we implicitly assign to each benefit a portion of the worth of the combined alternative benefits. For instance, we can predict an expected yearly repair cost based on the mean-time-between-failure and average maintenance for each alternative. In converting to dollars, care should be taken in the mathematical relationship between the cost side of each alternative and the cost conversions on the output/benefit side of the equation.

The most significant problem in determining overall technical and logistical competence of a system is deciding upon the proper weights to be given to the various benefits. When objective inherent weights of the system, such as relative manhours, dollars, etc., are not available, the criteria for weighting should be based on how much each contributes to the accomplishment of problem requirements, i.e., the economic analysis problem under consideration.

In situations where it is difficult to project benefits and/or to compute measures, it is desirable to provide as much useful information as possible to enable a decision to be made as to which alternative yields the most benefits.

A composite of total worth or value of a system is not always possible by objective quantitative scoring or weighting. The comparison format, with composites as subtotals of individual benefit statistics, will allow for appraisal by experts and final review by the decision maker.