A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION: Economic-Ecological Evaluation of a Land Reclamation Project

PETER NIJKAMP Free, University, Amsterdam

"In their epic combat with Neptune have the Netherlanders overlooked ecological values in such a way that the quality of human life in the Netherlands has suffered?" --Lynn White, Jr.*

EVALUATION OF PROJECTS A growing interest in project evaluation has developed during the last decade. The increasing need to supply political decision-makers with a set of adequate tools has evoked a vast amount of literature on this subject. Particular attention has been paid to the design of methods of evaluating (alternative) plans, for instance by means of cost-benefit analysis or cost-effectiveness analysis. Contributions in this field were made, among others, by Ben-Shahar et al. [2], Dasgupta and Pearce [1% Eckstein [13; pp. 439-504], English [ 15], Fano [ 16], Feldstein [ 17], Hill [23], Klaassen [28], Layard [3% Lichfield [32], Marglin [33], Mishan [35], Newton [36], Nijkamp [38], Prest and Turvey [44], and Wolfe [51]. The purpose of the present paper is (1) to discuss some alternative methods for project evaluation, (2) to present a multicriteria analysis mainly based on the French school of regional science, (3) to illustrate the latter method by means of a recent land reclamation project in the Netherlands, and (4) to present some new methods for multicriteria decision-making. A systematic way of studying and evaluating the effects of a certain (public) project is to make a distinction between the purely technical and physical effects of the project in question and the economic evaluation of these effects. For instance, the construction of a highway has inter alia the following technical-physical effects: art increase in land occupation by public investments, a rise in the regional accessibility, a decline in the number of accidents, and so forth. All of these effects can be calculated or predicted in a systematic way. Next, however, the question as to the economic evaluation of these effects must be considered in order to calculate the economic impact of the highway on the region in question or to make a choice among alternative plans. Given the previous notions, it is useful to consider a (public) project as a functional and interdependent set of public activities which act as stimuli for a series of * In Science, Vol. 155, No. 3767 (March 10, 1967). 87

88

PAPER5 OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

effects (both intended and unintended). These project stimuli can be integrated in a s t i m u l u s v e c t o r s with typical elements s~ (i = 1, . . . , 1). Such a stimulus vector is composed of all elements of the plan concerned. For example, in the case of constructing new towns the stimulus vector includes infrastructural investments, housing programs, recreation facilities, shopping centers, and the like. All of these stimuli generate a series of effects, some of which are aimed at and some of which are unintended. For example, in the case of a new highway the intended effects are, among others, a rise in the regional accessibility and a rise in the degree of safety, whereas the unintended effects are i n t e r a l i a a rise in air pollution and a rise in gas demand. All these technical-physical effects can be ranked in an e f f e c t v e c t o r r with typical elements r / j = 1 . . . . , J ). This effect vector can be considered as the response vector with respect to the stimulus vector s. Such an impact of stimuli upon effects can be represented in a formal way as:

r =/(s),

(1)

where f is a vector-valued function which transforms stimuli into responses. This function will be called the i m p a c t f u n c t i o n associated with a certain project; it indicates the expected consequence from the implementation of the project in question. In a simplified way this impact function can be written as an impact matrix M (Nijkamp [39]), so that one obtains: r = Ms.

(2)

Furthermore, the effects of a project are frequently not independent with respect to each other, but frequently possess a certain degree of interdependence. For example, an increase in the accessibility will attract new traffic flows and hence increase the number of accidents. Therefore, one may assume: r = h(r) + f ( s ) ,

(3)

where h is a vector-valued function representing the mutual interactions between the effects of a project. In a linear way, equation (3) can be written as: r : Lr + M s ,

(4)

where L is a matrix of mutual interaction coefficients. The latter specification bears a close resemblance to the classical input-output model, and it can easily be written by means of a matrix multiplier as: r - - ( I - - L ) -1 M s ,

(5)

assuming that I -- L is a nonsingular matrix. Once the technical and physical effects of a certain project have been determined, according to equations (I)-(5), the question arises: how to evaluate these effects in

NIJKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

89

economic t e r m s ? Such an economic evaluation is in general required in order to calculate the monetary consequences of a certain project, particularly if a choice among alternative project configurations has to be made. The latter would require a price system related to the effect vector r in order to gauge the social revenues and costs of a certain project. A direct pricing of the elements of the effect vector is impossible, however, owing to the lack of a market system for public investments, owing to external effects, intangibles, and so forth. As a result, an indirect evaluation method is required. One of the classical methods in this field is a c o s t - b e n e f i t a n a l y s i s . This analysis is originally a capital budgeting system primarily concerned with public projects. It represents an attempt to enumerate and to evaluate all relevant (i. e., direct and indirect) costs and benefits of a given project. Denoting the unit benefits and costs of a certain element rj of the effect vector r by bj and cj, respectively, the net benefits B of a project are: J

B= Z(bi-e3r~ j=l.

= t'(b -- 8)r,

(6)

where t is a vector with unit elements, and b and ~7a diagonal matrix with br and ej (j --= 1. . . . , J ) as diagonal elements. In general, cost-benefit analysis is a multiperiod method--an attempt to evaluate the benefits and costs within a certain time horizon T of the project. Therefore, total net benefits of a project are: T

B* =

~

=

Z

B~ e - ~ t

T

t'(bt -- ~)rt e -et ,

(7)

t=l

where p represents the rate of discount of the project in question. On the basis of equation (7), it is in principle possible to make a choice among alternative plans. If each alternative plan is denoted by an index n (n -- 1. . . . . N), the previous selection procedure for an optimal plan configuration can be formalized as the following programming problem: N max

o) =

F, ~.*&,

subject to N

Z&
6, = 0 , 1.

(8)

In addition, one should generally take account of a series of side conditions in plan evaluations, such as limited public resources. The latter would lead to the following constraint associated with equation (8):

90

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE 2;

J

a= Z k.,~r.,~ < &, v t, n=l

j=l

(9)

where r=tj. and k,t~ are theflh element of the nth plan at time t, and the unit public expenditures associated with it, respectively, and where St is the available public budget at time t related to one of the alternative plans. The solution of the previous program gives directly the optimal plan. It is obvious from equations (8) and (9) that each cost-benefit problem is essentially a zero-one problem. The heightened interest among economists and planners in cost-benefit analysis in the last decade is particularly owing to the growth of the public sector and the need of public authorities to evaluate the effects of their actions in a systematic and manageable way. Cost-benefit analysis is increasingly considered as a useful kind of applied welfare economics. Frequently, however, many shortcomings and snags of cost-benefit analysis are overlooked. Cost-benefit analysis can be criticized for several reasons: Intangibles can hardly be assessed in economic terms within a cost-benefit framework since a monetary evaluation of intangibles is generally impossible, or otherwise arbitrary or biased. - - External spill-overs are sometimes difficult to evaluate, as each project gives rise to a large series of indirect multiplier effects which are not solely included in the prices of products or of production factors. The estimation of the length of life of the project in question as well as of the rate of discount is frequently biased, so that a cost-benefit analysis has to be provided at least with a sensitivity analysis. - - The effects of the implementation of the project upon the distribution of welfare are, in general, overlooked, so that the use of a general priority scheme may neglect substantial economic inequalities. - - There is a vast amount of interdependencies among the effects of the project, so that the calculation of the benefits of all these separate effects is very difficult. The same holds true for determining the shadow prices of a certain public investment (Marglin [33]). Cost-benefit analysis is a fragmentary approach, since it attempts to gauge only the economic effects of a project--a very limited scope for practical planning problems which have to take account of many other noneconomic aspects of human well-being. The basic problem inherent in the use of cost-benefit analysis is the fact that the evaluation of a project must be carried out with respect to a single monetary unidimensional criterion. All effects of a project have to be projected into one single monetary dimension. This severe restriction is mainly responsible for all difficulties in the use of cost-benefit analysis pointed out above. Given the previous severe limitations of a cost-benefit analysis, some adaptations have recently been developed. One of the well-known alternatives in this field is the use of cost-effectiveness analysis. -

-

-

-

-

-

N|JKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

91

Cost-effectiveness analysis originates from design processes in systems engineering (Kazanowski [26; pp. 151-165], and [27; pp. 113-150]); it attempts to figure out whether a certain project is worth its cost. This method starts again with the effect vector r. Given the costs of a series of alternative plans (that is, given the separate costs of all effects), one has to determine which alternative is most cost effective in attaining a certain set of goals. By means of a cost-effectiveness chart, more insight is obtained into the relative effectiveness of each alternative plan separately. Therefore, a cost-effectiveness procedure studies the way in which an a priori determined set of goals is attained; the selection criterion is based on the determination of that alternative plan which possesses the highest total effectiveness. One of the basic problems in cost-effectiveness analysis is the specification of the set of goals to be attained and of the marginal rate of substitution between these goals. The specification and evaluation of criteria involves essentially the same problems already met in cost-benefit analysis, albeit that the confrontation of effects and their evaluations are postponed to a later stage. In spite of some refinements with respect to cost-benefit analysis, the essential difficulty of any decision procedure, such as the multiplicity of values, is not completely solved by a cost-effectiveness analysis. An allied method which attempts to bypass some of the aforementioned difficulties is a trade-off analysis (Edmunds and Letey [14]). A trade-off analysis is a method to measure alternative means (or plans) in order to arrive at a prespecified set of goals. In other words, a trade-off analysis attempts to determine whether one alternative project is better than another, given the same set of goals. This method of selecting the best alternative means to achieve the benefit requires, however, again a single dimension (for example, money, time) in order to evaluate probable gains against probable losses. The essential problem here is to turn the trade-off between alternatives into opportunity costs. Next, the goals-achievement method should be mentioned (Hill [23], Hill and Schechter [24; pp. 110-t21]). This method was developed in order to attack the problem of multiple criteria in evaluating alternative plans. In this approach, objectives are expressed in terms of quantitative measures reflecting the degree of achievement of each objective. Each objective associated with a certain plan receives its relevant index, while next for each alternative plan an aggregation of the indices of achievement of each objective is carried out. It is obvious that this aggregation is one of the basic difficulties in a goals-achievement method, since the meaning of an aggregate goals-achievement index is not quite clear. An extension and generalization of a goals-achievement method, such as a multidimensional scalogram analysis, was recently developed by Hill and Tzamir [25]. Finally, the use of correspondence analysis deserves some attention (Benzdcri [3], Lebart and Fdnelon [31], and Spliid [47]). Correspondence analysis focuses in particular on the differences between alternative plans. It is a method of pat~fern recognition of discrepancies between alternative plans, based on a specific variant of

92

PAPERS OF THE REGIONAl. SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

a principal component analysis. At present this method is in a very preliminary stage of evolution. A MULTICRITERIA ANALYSIS FOR ALTERNATIVE PLANS

A multicriteria analysis is a method of dealing with multiple dimensions. The problem of multiple dimensions is primarily present in two fields of regional science: (1) in the case of regional data analysis (classification procedures and reduction procedures of regional data, for example) and (2) in the case of regional planning and decision-making. A survey of several multicriteria methods is contained among others in Benz6cri [3], Bernard and Besson [4], Buffet et al. [7], and Guigou [20]. In this section attention will be paid to the use of multicriteria analysis for regional planning and decision-making. Such a multicriteria analysis bears some resemblance to the aforementioned goals-achievement method as far as the initial stages of analysis are concerned. For the sake of simplicity, only a specific variant of multicriteria analysis, viz., Electre (Elimination and Choice Translating Reality), will be discussed here. A survey of this method is contained among others in Guigou [20], [211, and Roy [45], [46], while an excellent (Dutch) survey is contained in Opschoor and Van der Meer [42]. The method employed in the present paragraph differs to some extent from that adopted by the aforementioned authors. A multicriteria method starts off with the effect vector r (see above). The assumption is made that for each alternative plan n (n = 1. . . . , N), an effect vector r, can be calculated which reflects all relevant outcomes of the project concerned. It should be noted that these outcomes are not necessarily cardinal measures; ordinal measures or even zero-one ]ndicators are equally possible as well, This implies that there is during this stage no need to evaluate the outcomes in monetary dimensions. The elements of the effect vector are formed by welfare indicatorsj(j = 1, . . . , J) which are deemed to be relevant within the decision framework in question. They result from the multiplicity of criteria taken into account by decision-makers. The successive effect vectors r, can be included in a project effect matrix R: R = [r~ . . . .

r,,].

(10)

Matrix R is of order J • N, since there are J criteria to be considered in the evaluation of N alternative projects. This matrix is only a matrix of technical-physical effects of alternative projects without any economic evaluation. Therefore, the next stage is to assign weights to all criteria. In a normal costbenefit analysis, these weights are formed by the net unit benefits of each separate effect originating from each alternative; cf. equation (6). Such a monetary evaluation of each criterion could in principle also be used in a multicriteria analysis, but this is not necessary nor even desired. The purpose of multicriteria analysis is precisely to carry out the evaluation method as long as possible with a multiplicity of dimensions. Therefore, only a set of weights reflecting the relative importance of the criteria is assumed (for example, measured on an ordinal scale).

NIJKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

93

It is obvious that the determination of these weights is overloaded with difficulties, although recently by means of a so-called implicit method (based on a revealed-preference approach) some new attempts were made in order to estimate preference weights of decision-makers based on ex post analysis (Nijkamp [37], and Nijkamp and Somermeyer [41]). For the moment, the assumption will be made that each characteristic element j of a plan n, denoted by r~, can be provided with a relative weight for each criterion separately. It is not neccessary that these weights are linear ones; they may, in principle, depend on the level of each project outcome ri,. These weights represent the relative preferences of the decision-makers for each individual criterion. In this way, it is possible to deal with a multiplicity of viewpoints without aggregating all of these criteria to one single dimension. Instead of matrix (10), one obtains now a weighted project effect matrix V which is obtained by multiplying each element r~,~ (or each row r~ of matrix R) with its corresponding relative weight wj. For a linear weighting system, V is equal to: V = ~ R,

(11)

where k is a diagonal matrix of order aT, with relative weights wi as successive diagonal elements. The next step is to confront the alternative plans pairwise by means of a rating system. Compare, for instance, two alternative plans (n and n'). Next, the successive criteria of plan evaluation can be classified into two distinct sets. The first set contains all criteria which are in accordance with the hypothesis that plan n will be preferred to plan n'. The second set consists of all criteria for which plan n' is preferred to plan n. The first set is called the concordance set C,~; this set is formally defined as:

c~.. = ( j [ r;. > r~.,},

02)

where the symbol > denotes a (weak) preference of plan n to plan n', as far as a certain common criterionj is concerned. The construction of such a concordance set is facilitated when each separate effect is measured by means of an implicit rank criterion, so that (for example) a higher value of a certain effect reflects a more desired state of the plan with respect to the criterion in question. The second set is called the discordance set D,~, and can be represented as: Dr,, = [Jl rs, < rj,,},

(13)

where the symbol -< means: "not preferred to." It is obvious that C~,, will contain more elements as plan n dominates plan n' with respect to more criteria. The relative value of such a concordance set is measured by means of a concordance index. This index is equal to the sum of the weights for those criteria which fall into the concordance set, divided by the sum of ;all weights. Therefore, the concordance index between plan n and n', denoted by C~,, is:

94

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

--

(14)

where t and w are vectors with unit elements and with weights w~., respectively; 6'~"' is a diagonal matrix with 0-1 variables 62"' as diagonal elements, indicating whether or not the corresponding criterion does belong to the concordance set. Formula (14) implies that the concordance index falls between 0 and I. This index represents the relative degree of dominance of plan n with respect to plan n'. If e.~, - 1, there will be a complete (weak) dominance of a// criteria of plan n; if c..~, = 0, there will be no dominance for any criterion, so that for each criterion alternative n is worse than alternative n'. By means of a pairwise comparison of all individual alternative plans, one may construct a concordance matrix C of order N • N containing all elements c.., (n, n' - 1. . . . . N; n 4: n'). It should be noted that this matrix is, in general, not symmetric. Thus far, attention has been paid only to the question of the relative equivalence of alternatives. No attention has been given to the degree to which alternative n is worse than alternative n' for certain criteria. Therefore, in addition to a concordance index, one needs a measure for the relative degree of discordance between alternative plans. Such a measure of discordance should indicate the degree to which plan n' is preferred to plan n for one or more criteria. One should realize that c~, < 1 implies that, for one or more criteria, alternative n is worse than alternative n'; in other words, then there is at least one criterion j for which rj., > r~. How to measure now the relative degree of discordance? For that purpose a new index, a so-called discordance index d..~,, is created. This index, associated with discordance set D~.,, is equal to the ratio of the maximum divergence between the project effects of both alternatives in question with respect to the maximum divergence between the project effects of any two arbitrary alternatives. Therefore, d.., is equal to:

d~., : max { (~I- ~ ' ) l r~ - r~' { } j

m

'

(15)

where m is defined as: m :

max { j r . - r.,I].

(16)

The zero-one diagonal matrix I -- ~""' assures that only the elements from the discordance set D ~ , are taken into account; expression I r. -- r., I denotes the absolute difference between the project effects of plan n and n'. Formula (15) indicates that a maximum discordance between two plans (n and n') implies: d~., = 1, whereas a minimum discordance implies: d.., -----0(which is equivalent to: c.., = 1). There is, however, a serious problem in calculating and using the aforementioned discordance index: this index takes only into account the absolute deviation between

NIJKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

95

the outcomes of alternative plans. The relative evaluation of the different criteria, however, is overlooked. Therefore, it is more logical to calculate the discordance index on the basis of the weighted project effect matrix V (equation (11)) rather than on the basis of the project effect matrix R itself (cf. equation (10)). Therefore, instead of equation (15) one obtains:

d,,, -- max f ( l - - 6~"') l v~ -- v~' l t j

m

(17) '

where v, represents the nth column of V, and where m is adapted in an analogous manner. Before the latter discordance index can be calculated, still another problem should be attacked--the different scales of all criteria. It is evident that the discordance index depends heavily on the scale of each project effect separately. For instance, if the first effect of a plan is measured in numbers of additional employees and the second effect in a percentage increase of regional accessibility, the absolute deviation between the effects of two alternative plans will be dominated by the first effect. Therefore, one has to transform all project outcomes into comparable scales. The most logical way to arrive at equal scales is to carry out a normalization procedure with respect to each criterion separately (see, among others, Nijkamp and Paelinck [40] and Stone [48]). This implies that each criterion vector is divided by its norm. Since such a criterion vector is formed by the successive rows r~ of the project effect matrix, the normalized criterion vector r~ can be written as: t

F5

rY'

-

tl rjll

-

(r;' r;) 1/2"

r]' (18)

It is easily seen that each normalized criterion vector r~' has a unit length. Each separate criterion can be transformed into a normalized vector representation. In this way a meaningful and straightforward comparison of all criteria is allowed. The previous procedure implies that the original project effect matrix R passes into a normalized project effect matrix R*. Analogously, the weighted normalized project effect matrix is denoted by V*. Ultimately, the elements of V* are used in calculating the discordance indicator d~,. In a way similar to the calculation of the concordance matrix C, one may construct a discordance matrix D (of order N x N) including all discordance indices d~,. It is obvious from a comparison between equations (14) and (15) that there exists a certain asymmetry between the concordance index and the discordance index. A drawback of the discordance index is the fact that it is only based on the maximum divergence of one single criterion for alternative plans. Instead of using equation (~ 5), one could employ a slightly different definition of the discordance index. The latter definition takes into account the relative (weighted) discrepancy between two

96

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

alternative plans n and n' with respect to all those criteria, for which plan n does not dominate plan n'. Then the definition of a discordance index would become:

a.., =

b'(I - ~""') I v.*

-

v~, I

(19)

m

In this case m is defined as the maximum of the sum total of the divergences among two alternative plans, so that the condition 0 x< d~, ~< I is again satisfied. Finally, the question arises as to how to use the concordance matrix and the discordance matrix in evaluating alternative plans. It is obvious that a certain plan will receive the highest preference if the concordance index of this plan with respect to all other alternatives will be approximately equal to l, and if its discordance index with respect to the other plans will be approximately equal to 0. Owing to the multiplicity of criteria, a situation where e ~ , = 1 and d~, = 0 rarely occurs, so that, in general, the decision procedure should be based on maximizing the value of c~,, while guaranteeing a certain minimum value of d~,. The latter problem is essentially a programming problem, based on min-max strategies. A unique optimal combination of c ~ , and d ~ , is, however, hard to find, so that one has to consider alternative methods. An adequate method for further analysis appears to be a more careful, pairwise examination of all plans, based on the information included in the concordance matrix and the discordance matrix. A certain plan has a higher chance of being accepted, as its concordance indices with respect to alternative plans are higher and its discordance indices lower. Therefore, it is reasonable to hypothesize that plan n will dominate only plan n' if both its concordance index c.~,~, exceeds at least the average total concordance index and if its discordance index d ~ , falls at least below the average total discordance index. Instead of these average indices one may use in general also any other reasonable threshold value ~ and fi, respectively. A certain project n will be preferred to n' if with respect to a majority of criteria c~, > a and d~,, < ft. Given a certain value of ~ and fi, one has to inspect for each plan n separately as to whether these two threshold conditions are satisfied with respect to plan n' (n' = t, . . . , N; n' 4: n). This gives rise to two sets of dominance relationships. The intersection of these sets represents a smaller set of dominance relationships for alternative plans which satisfy both threshold Conditions. Next, by means of graph-theoretical methods (Guigou [20]) or otherwise, one may attempt to determine a preferable plan which is not dominated by one of the alternatives. The latter method wilt be illustrated later on in more detail by means of a land reclamation project in the Netherlands. The threshold values can, in fact, be employed in a twofold way to arrive at a decision as to the most preferable plan. The first method starts by assigning a very high value to a and a very low value to ft. By means of a successive relaxation

NIJKAMP:

A MWLT|CIIITEIlIA ANALYSIS FOR PROJECT EVALUATION

97

of these threshold values, one can examine which plan satisfies the selection rules imposed a priori. Continuing in this way, the next-optimal plan can be selected, and so on. In general, it will be worthwhile to examine the optimal plan and some nearoptimal plans more thoroughly in order to arrive at a more definite conclusion. The second method is a reverse method: instead of a positive selection of one or more valuable plans, this method attempts to eliminate all nonvaluable plans. This procedure starts by assigning a rather low value to cr and a rather high value to t3, so that most plans satisfy the selection rules. Next, by strengthening the threshold values, an increasing number of plans does not satisfy the selection rules set out above. By means of such_ a successive elimination, one may arrive at the selection o f the optimal plan (or a set of near-optimal plans). It is obvious that each eliminated plan should be dominated by at least one kemaining plan. A LAND RECLAMATION PROJECT

The IJsselmeer is an interior sea in the Netherlands, separated from the North Sea by means of a closing dike. During the last three decades, huge projects h a v e been carried out in order to embank and to reclaim large parts of the IJsselmeer (including the North-East Polder and the Flevo Polders). 1 These reclamation projects were implemented in order to protect the inland areas, to guarantee sufficient water supply, to offer more space to a growing population, and to stimulate agricultural production. One of the last parts of the IJsselmeer, which was originally planned to bediked and reclaimed, is the so-called Markerwaard (see map). At present disagreement is high as to whether this reclamation project should still be carried out. For the sake of clarity, the difficulties inherent to this project will briefly be reviewed on the basis of five facets of this plan: hydraulics, physical planning, environment, recreation, and traffic (Markerwaard [34]). The hydraulic aspect of the land reclamation concerned includes many elements: protection against water floods; drainage of water from polders (from different areas); water supply and water storage; quality of drinking water; navigation; and fishery. The elements of physical planning of the project concerned are, among others: exhaustion of space; degree of urbanization in the Rimcity of Holland; and disappearance of a buffer zone between the central and northern part of the Netherlands. The environmental and ecological aspect includes the following elements: use of an interior sea as a freshwater basin; preservation of Marken as an island in the IJsselmeer; preservation of the typical landscape at the interior coast; preservation of many natural areas in North-Holland; and spatial and infrastructural configuration of the new area; cultivation in the new area (agriculture, forestry, for example); and effect on water pollution in the IJsselmeer. 1 The author is indebted to Hans Vos and Alof Wiechmann, who c ,llected the material for this seclion.

98

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

HOLLAND

k, co

a: k.

Amsterdame~ Marken,~ J

FlevQland

J J

LAND RECLAMATIONPROJECTS IN THE NETHERLANDS

NIJKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

99

The recreational aspects are inter alia related to: water sports; camping and picnic sites; and shore recreation. Finally, the traffic aspects of a reclamation of the area concerned include: volume, spatial distribution, and modal split of the traffic flows; accessibility with respect to the northern part of North-Holland; influence on the national highway system. For some alternative plans (varying from a partial reclamation to a complete reclamation of the Markerwaard), some cost estimates were made--for instance, acquisition of land, drainage, embankment, and the like. Until now, however, no complete and systematic cost-benefit analysis was carried out in order to evaluate the different alternatives for a reclamation of the Markerwaard. An attempt will be made here to illustrate the use of a multicriteria analysis on the basis the Markerwaard project. First, the various alternative plans will briefly be discussed; next, the project effects will be calculated by means of (preliminarily rather crude) estimates of the successive outcomes. Seven alternative plans are successively considered: 1. No further embankment, but only a minimal program in order to satisfy the growing needs for adequate hydraulic provisions (variant 1, Dienst Zuiderzeewerken [11]). 2. A moderate embankment and reclamation program which covers approximately one-tenth of the original area (variant 2 I11]). 3. A completeembankment and reclamation of the Markerwaard (variant 5 [11]). 4. A complete embankment and reclamation of the Markerwaard with wider border lakes than alternative 3 (variant 6 [I 1]). 5. A partial program by digging a canal from Amsterdam to Lelystad and raising the adjacent sea area by spouting the resulting sand on it (cf. Hendrikx [22]). 6. A division of the original plan into two parts: (a) the creation of a new polder near Enkhuizen (called Enkhuizerzand) and of a rather small polder near Marken (called Markermeerpolder) with an interior sea between these polders (cf. Borgstein et al. [5]). 7. A realization of a more adequate spatial layout of the existing areas without affecting the IJsselmeer (apart from a railroad system and other public transportation facilities between North-Holland and the northeastern part of the Netherlands) (cf. Vereniging tot Behoud van het IJsselmeer [50]). In addition to the previous variants, one has to create the project effect matrix R (cf. equation [10]), which includes for each alternative plan its outcomes, divided among a series of criteria. The following criteria will be distinguished in evaluating these alternative plans: (1) The net sacrifice of the present use of the area concerned--that is, the difference between the present use and the alternative new use of the area; this effect will be measured on an ordinal scale varying from 1 (negligible sacrifice) to 10 (huge sacrifice).

I00

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

(2)

The overflow of residential activities from the densely populated Rimcity, measured in numbers (thousands) of expected immigrants. (3) The importance of the project for regional accessibility, measured on an ordinal scale from 1 (very favorite) to l0 (very bad). (4) The importance of the project for protection against water floods, measured on an ordinal scale from I (very reliable) to 10 (very weak). (5) The importance of the project for drainage and other hydraulic works, measured on an ordinal scale from 1 (very important) to I0 (worthless). (6) The influence of the project on new direct job opportunities, measured in thousands of man-years. (7) The costs of all investments associated with the alternative plan concerned, measured in millions Dfl. (8) The contribution to a more balanced distribution of regional welfare, measured on an ordinal scale varying from 1 (substantial contribution) to 10 (negligible contribution). The criteria are measured such that a low value of an effect is preferred to a high value, except criteria 2 and 6, for which a high value is preferred to a low value. Given the aforementioned alternative plans and the previous different criteria, the project effect matrix can in principle be determined. The calculation of the various project effects yielded project matrix (20). The data included in it are not completely accurate, so that the method described here is only illustrative. 1

2

3

4

5

6

7

1

3

4

8

7

3

5

1-

2

0

20

300

270

0

70

0

3

6

6

1

2

5

4

7

4

4

1

1

6

4

7

5

4

3

2

2

4

6

9

6

4

6

32

30

5

20

20

7

450

550

780

795

395

650

365

8

8

7

5

5

8

7

R=4

4 -

(2o)

The previous matrix can be considered as the technical-physical representation of the effects of alternative plans from different points of view. The following paragraph shows how to use such a project effect matrix within the framework of a multicriteria analysis.

A NUMERICAL APPLICATION OF A MULTICRITERIA ANALYSIS

As set out above, the first procedure to be carried out is a normalization procedure; cf. equation (18). This normalization procedure gives rise to the following

NIJKAMt~:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

10|

normalized project effect matrix R* :~ 1

2

3

4

5

6

7

1 -.228

.304

.608

.532

.228

.380

.076

2

.000

.049

.731

.658

.000

.171

.000

3

.464

.464

.077

.155

.387

.309

9542

.344

.344

.086

.086

.516

.344

9602

5

.311

.233

.155

.155

.311

.466

9699

6

.076

.113

.605

.567

.094

.378

.378

7

.287

.351

.498

.507

.252

.415

.233

8

.468

.410

.293

.293

.468

.410

.234

R* = 4

(21)

llt was pointed out earlier that, in addition to the project effect matrix, a set of weights should be specified, which reflect the decision-maker's relative preferences with respect to each criterion separately. Given the fact that frequently these relative political weights are not precisely known, it is reasonable to specify one or more alternative weighing systems. The advantage of the latter procedure is that it provides a (rough) sensitivity analysis of the results with respect to changes in the decision-maker's political weights, and in addition allows the possibility of confronting decision-makers with the consequences of their choice for alternative values of weights. In order to illustrate the previous remarks, two alternative weighing systems will be specified and further analyzed as to their consequences. A first set, denoted by the row vector w;, is assumed to reflect in particular strong preferences with respect to environmental preservation and regional well-being. This set is:

w~ =

1

2

3

4

5

6

7

[.234

.118

.118

.118

.059

.059

.059

8

.234],

(22)

where the weights are chosen such that they add up to 1. A second set of weights, denoted by w~, focuses more on production potential and employment. This set is assumed to be:

w~ =

1

2

3

[.087

.130

.174

4

.044

5

6

7

.044

.217

.217

8

.087].

(23)

It should be noted that the previous sets necessarily contain some arbitrary elements--the reason why this method needs a continuous objectification by means o f ' a frequent confrontation with decision-makers. These weights illustrate once more the difficulties inherent in many evaluation methods. The advantage of dealing explicitly with weights is formed by the fact that the decision-maker's preferences The author is indebted to Wire Kleyn and Ad van Delft for their assistance in the computational work.

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

102

with respect to the criteria and outcomes of a plan are to be revealed in an explicit manner. Even in a normal cost-benefit analysis, many subjective evaluations are undertaken, frequently without an explicit mention. Contrary to a cost-benefit analysis, a multicriteria analysis allows more opportunities for an objectification of political preferences. Furthermore, a cost-benefit analysis can be integrated into a multicriteria analysis by considering the relative net benefit of each effect of the plan in question as a weight associated with the effect concerned. The next step to be undertaken is the calculation of the concordance matrix (C) according to equation (14). For example, given the first set of weights (22), element c1= is calculated as follows: C1~ = .234 q- 0 -l- .118 q- .118 q- 0 -k 0 q- .059 q- 0 = .529 9

(24)

In an analogous manner the complete concordance matrix can be calculated both for the first and the second set of weights. These matrices, denoted by C1 and C2, respectively, are represented in relations (25) and (26). 2

3

4

5

6

.529

.293

.293

.763

.470

.413 -

1 1

--

7

2

.706

--

.293

.293

.588

.704

.413

3

.706

.706

--

.765

.706

.706

.472

C1 ----- 4

.706

.706

.645

--

.765

.706

.472

5

.881

.412

.293

.234

--

.352

.413

6

.529

.647

.294

.293

.647

--

.472

7

.704

.586

.527

.527

.704

.586

--

2

3

4

5

6

7

.522

.304

.304

.392

.392

.392

1 1

--

(25)

2

.696

--

.304

.304

.522

.479

.392

3

.696

.696

--

.913

.696

.696

.609

C~ = 4

.696

.695

.262

--

.696

.696

.609

5

.869

.477

.304

.304

--

.348

.392

6

.608

.652

.304

.304

.652

--

.609

7

.738

.608

.391

.391

.738

.608

--

(26)

As pointed out in the foregoing paragraph, in addition to a concordance matrix a discordance matrix has to be calculated as well. The calculation of such a discordance matrix is based on the weighted normalized project effect matrix V*. This matrix is for the first and second set of weights respectively equal to :

NIJ|[AMP:

103

A MULTICRITER1A ANALYSIS FOR PROJECT EVALUATION

V~=

1

2

3

4

5

6

7

1

.054

.071

.143

.125

.054

.089

.018

2

.000

.006

.086 -.078

.000

.020

.000

3

.055

.055

.009 -.018

.046

.037

.064

4

.041

.041

.010

.010

.061

.041

.071

5

.018

.014

.009

.009

.018

.028

.041

6

.005

.007

.036 - . 0 3 4

.006

.022

.022

7

.017

.021

.029 - . 0 3 0

.015

.025

.014

8

.110

.096

.069

.069

.110

.096

.055

1

2

3

4

5

6

7

.020

.026

.053

.046

.020

.033

.007

(27)

and:

1

2

.000

.006

.095

086

.000

.022

.000

3

.081

.081

.013

027

.067

.054

.094

.015

:015

.004

004

.022

.015

.026

5

.013

.010

.007

007

.013

.020

.030

6

,017

.025

.131

123

.020

.082

.082

7

.062

.076

.108

110

.055

.090

.051

8

.041

.036

.026

.026

.041

.036

.020

V*=4

(28)

On the basis of relations (27) and (28), the discordance matrix associated with the first and second set of weights can be directly calculated. For example, d12 is, according to equation (17), for the first s e t o f weights equal to: d~2=

9110 -- . 096 . 1 4 3 - - .018 = ' 1 1 2 "

(29)

The discordance matrices related to systems (22) and (23) are, respectively: 1

and:

2

3

4

5

6

7

--

.112

.688

.624

.072

.160

.440

2

.136

--

.640

.576

.136

.144

.424

3

.712

.576

--

.144

.712

.432 1.000

D1 = 4

.568

.432

.072

--

.568

.288

.856

5

.160

.160

.688

.624

--

.160

.440

6

.280

.144

.528

.464

.280

--

.568

7

.240

.240

.688

.624

.184

.240

--

(29')

104

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

1

2

3

4

5

6

.930

.123

.570

.570 -

.860

.184

.500

.500

--

.061

.465

.175

.500

.298

.123

--

.482

.175

.518

.06t

.974

.904

--

.544

.544

.307

--

.342

.237

.351

--

t

--

2

.123

.070 1.000 --

.930

3

.404

.281

D2 = 4

.421

5

.06t

6

.246

.123

.640

.561

7

.149

.175

.833

.754

7

-

(30)

Now the question arises: how to use the information from systems (25), (26), (29'), and (30) in order to arrive at definite statements concerning the desirability of one of the alternative projects? The answer can be found by assigning threshold values to the concordance and discordance indices such that given these values one of the alternatives is preferred to all others. Considering the first weights (22), let us assume a threshold value for the concordance index and the discordance index o f . 7 0 0 a n d . 150, respectively. This would imply that each project, which has a concordance index of at least . 700 with respect to a competing project, is preferred to the latter project. Similarly, each project, which has a discordance index of at most . 150 with respect to an alternative one, is acceptable and is preferred to the latter one. Those projects which satisfy simultaneously both requirements should be further examined. An analysis of concordance matrix (25) leads to the conclusion that a threshold value of .700 for the concordance index implies the following successive rank orders for the projects P1 through PT:

P* >- P5 P~ >- P1, P6 P3 >- P*, P~, P4, Ps, P~ P4 >" P,, P2, P~, P~ P5 >-P* P, >- P1, P5

(31)

In a similar manner, one may consider discordance matrix (29'). This gives rise to the following relationships: P, >- e2, P5 P~ >- P,,Ps. P6 P3 >- P4 P4 >- P~ P6 > P~

(32)

|t should be noted that the transivity conditions are not valid for (31) and (32),

NIJ'KAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

105

since each dominance relationship is based on different criteria. This is also the reason why, for example, P1 >- P5 is not inconsistent with P5 >- PI: for certain criteria P1 is preferred to P~, and for others P~ to PI. It can easily be derived that the intersection of (31) and (32) gives rise to the following result:

P~>-P5 P~ > P1, P6 Pa >- P4.

(33)

Nondominated solutions of (33) can be found directly or by means of graph theory. This result indicates that only/'2 and P3, as well as P~, should be further inspected, since the other projects are dominated by one of these three projects. By reducing the threshold value of the concordance index and by increasing the threshold value of the discordance index, one may attempt to arrive at a more definite conclusion concerning projects P2, P3, and PT, but one should be aware of the fact that such an adaptation implies a weaker statement concerning the relative valuation of one of the plans. An alternative method is to count for each remaining plan separately the number of times its concordance index with respect to alternative plans exceeds the threshold value. It appears that P~, P3, and P7 give rise to the following respective outcomes: 2, 5, and 2. In an analogous manner one may count the number of times the discordance index with respect to competing plans falls below the critical threshold value. These outcomes are, respectively, 3, 1, and 0. This suggests that plan 7 could be eliminated and that plan 3 might be slightly preferred to plan 2. Next, the previous results will be compared with the outcomes based on the second set of weights. If one assumes the same threshold values, one finds for the concordance indices:

P~ >- P4 P~ >- P1 P7 >- P1, P~.

(34)

In a similar manner one finds for the discordance indices the following rank orders of projects:

P~ >- P~,P~ P~ > P~ P~ > P4 P4 > P~ Ps > P~, P2 P~ > P2 P7 > P1 The intersection of (34) and (35) gives the following result:

(35)

106

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION, VOLUME THIRTY-FIVE

P~ ~ Pz P7 > P1.

(36)

The latter result leads to the conclusion that P4 and P~ can be eliminated, but that no definite conclusion can be drawn concerning the remaining projects. Therefore, it may be worthwhile to inspect the consequences of a relaxation of the threshold values. Assuming that the threshold values of the concordance index and the discordance index can be set equal to . 600 and . 200, respectively, the following ultimate result is found:

P2 ~ P1 P~ > P~, P~ P~ ~- P~ P5 ~ Pz P6 ~- P~ P7 >- P1, P~.

(37)

The latter result implies that only P~, P~, and P7 require a further examination. By counting the number of times the successive plans satisfy the threshold values, it can easily be derived that plan 3 is most preferable. This leads to the conclusion that in both cases plan 3 would be most worthwhile. It should be noted, however, that the previous results rest on rather weak data. The results obtained so far are more illustrative than reliable, so that a definite conclusion can hardly be drawn. However, research in this field is going on in order to arrive at more accurate material and so at more accurate conclusions concerning the evaluation of the alternative plans. CONCLUSION

The multicriteria analysis set out above and illustrated by means of a numerical example appears to be a useful tool in project evaluation. This method is valuable mainly because it can be used to eliminate a series of less valuable projects and to select one or more good alternatives. The additional advantage of a multicriteda analysis is that it provides the possibility of including qualitative factors without transforming them to a monetary dimension. This implies that less monetary estimates of project effects are to be made. A difficulty of the method is, however, that a s e t o f explicit weights for project outcomes has to be specified, albeit that this method is very well suited to drawing decision-makers into the planning process by asking them to reveal explicitly their relative preferences as to project outcomes and by showing them the consequences of these preferences. A multicriteria analysis does not always provide a unique solution, but sometimes simply a set of "reasonable" solutions. In the latter situation two ways are

NIJKAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

107

open: either to collect more and accurate information (for example, as to the threshold levels), or to use a complementary analysis. The latter situation implies that even a cost-benefit analysis could be used as an approach complementary to a multicriteria analysis. The general conclusion is: (1) a multicriteria analysis is a meaningful tool in guiding choice in multiobjective decision-making; (2) a multicriteria analysis is capable of providing a relative evaluation of projects with respect to each other; and (3) classical evaluation methods (for example, a cost-benefit analysis and a cost-effectiveness analysis) are complementary rather than competitive to a multicriteria analysis. It should be noted that the foregoing static multicriteria analysis can easily be extended toward a multitemporal analysis by including a suitable rate of discount for project outcomes in the future. Uncertainties as to the project outcomes could in pnincipte be attacked by means of a probabilistic approach (cf., for example, Pratt et al. [43]). Finally, some attention will be paid to further research in multicriteria decisionmaking. The previous paragraphs showed that the classical approach to multiple-goal decision-making by means of a single, scalar-valued criterion is frequently a less fruitful simplification. Current decision-making practice is primarily based on multiobjective situations frequently involving conflicting or noncommensurable objective functions. Then the question arises: how to deal with a multiplicity of goals in order to arrive at an optimal or at least a satisfying solution? Traditional decision-making tools (for example, mathematical programming) are only based on a single objective function. Fortunately, recently the attention of operations research and allied fields focused on multiobjective programming; see, among others, Benayoun and Tergny [1], Briskin [6], Charnes et al. [8], DaCunha and Polak [9], Eckenrode [12], Gal and Nedoma [18], Geoffrion [19], Klahr [29], Roy [46], Terry [49], Zeleny [52], and Zeleny and Cochrane [53]. Several paths can be tried to attack the problem of multiobjective decisionmaking. One is to aggregate the different criteria in such a way that a single unique criterion involving a complete ranking of preferences is obtained (for example, by constructing a weighted average of all different criteria). This situation brings us, in fact, back into the realm of traditional programming techniques, but it is frequently a difficult task to tie all criteria of a multiple-goal setting into a single, one-dimensional trade-off function. A second approach adopted in the foregoing multicriteria analysis is to construct a partial ranking of criteria based on a binary rank order. By inspecting the degree of overrating of a criterion for each pair of outcomes, one may try to arrive at an optimal, or at least a reasonable, solution. A more general approach is to formalize a multicriteria decision problem as: max co = f ( x )

(38)

108

PAPERS OF THE REGIONAL SCIENCE A S S O C I A T I O N , V O L U M E THIRTY-FIVE

where f(x) is a vector-valued objective function which can be written as: oil = f l ( x ) , ~o~ = f~(x),

(39)

and where x is a vector of relevant decision variables which should belong to a feasible region K. The crucial point of problem (38) is that the determination of a single optimal solution should be replaced by the determination of a whole set of nondominated solutions. This implies that the solution of multicriteria programming should be based on seeking a result which should adopt the best possible or at least a satisfying value under the given conditions. Thus, instead of one unique solution one has to determine a set of best or admissible solutions. Such a set of solutions is characterized by the fact that for all solutions inside the set it is not possible to affect the individual objective functions oil . . . . . . o~z in such a way that they are all improved, or unchanged and at least one improved (hence the term "nondominated" solution). Therefore, the previous multiobjective programming model can be written as: max oJ =

f(x)

such that there is no x E K with

f(x) > f(x*) and f(x) -~ f(x*)

(40)

Here the essential problem is to find all vectors x* ~ K which are nondominated, so that there does not exist any other vector x ~ K such that f(x) >=f(x*) and f(x)

f(x*). It is clear that this problem of vector function maximization is not easy to solve by means of classical Lagrange-Kuhn-Tucker theory. The solution procedure of the multicriteria analysis set out above can be considered as a very specific algorithm, where the threshold values of the concordance index and of the discordance index determine the area of admissible solutions. The various objective functions are formed by the different criteria of the plan evaluation and they depend, according to equation (1), on the different stimuli, that is, the different plan configurations. In the approach adopted in the multicriteria analysis, the decision variables are not continuous ones, but they are of a discrete nature. This implies that essentially a multicriteria analysis could be conceived of as a discrete multiobjective programming model. Until now, however, no efficient and formal solution algorithm has been developed for these types of problems, so that for the moment the approach adopted in the previous paragraphs seems to be the most efficient one. Even a general solution procedure of continuous multiobjective programming methods is hard to construct, but particularly in the field of linear multiobjective programming considerable progress has been made. In this, procedures like simplex methods and parametric programming techniques are most applicable.

NIJICAMP:

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

109

T h e conclusion o f the a u t h o r is that, in spite o f the p r o m i s i n g results o f the multicriteria analysis set o u t above, in the future m u c h attention should be p a i d by regional scientists to multiobjective p r o g r a m m i n g m e t h o d s in o r d e r to a t t a c k the intriguing p r o b l e m s o f multiple goals in regional p l a n n i n g a n d decision-making. REFERENCES

[1] Benayoun, R., and J. Tergny. "Crit6res multiples en programmation math6matique: Une Solution dans le cas Iin6aire," Revue d'informatique et de rOcherche op~rationelte, Vol. 3, No. 5 (1969), pp. 31-56. [2] Ben Shahar, I. I., A. Mazor, and D. Pines. "Town Planning and Welfare Maximization," Regional Studies, Vol. 3 (1969), pp. 105-113. [3l Benz6cri, J, P. L'Analyse des donndes. I: La Taxonomie ; II: L'Analyse des correspondances. Paris: Dunod, 1973. [4] Bernard, G., and M. L. Besson. "Douze m6thodes d'analyse multicrit6re," Revue informatiqueet de rOcherche opkrationelle, Vol. 5, No. 3 (1971), pp. 19-66. 15] Borgstein, B., T. Miltenburg, and W. Van der Sluis. "Perspectief op de Markerwaard," Tijdschrift der Koninklijke Nederlandse Heidemaatshappij, No. 2 (1971), pp. 51-54. [6] Briskin, L.E. "A Method Unifying Multiple Objective Functions," Management Science, Vol. 13, No. 10 (1966), pp. 406-416. [7] Buffet, P., J. P. Greeny, and M. Mare. "Peut-on choisir en tenant compte de crit6res multiples," Metra, Vol. 6, No. 2 (1967), pp. 283-316. [8] Charnes, A., W. W. Cooper, R. J. Niehaus, and A. Stedry. "Static and Dynamic Assignment Models with Multiple Objectives, and Some Remarks on Organization Design," Management Science, Vol. 15, No. 8 (1969), pp. 365-375. [9] DaCunha, N. O., and E. Polak. "Constrained Minimization under Vector-Valued Criteria in Finite Dimensional Space," Journal of Mathematical Analysis and Applications, Vol. 19 (I967), pp. 103-124. [10] Dasgupta, A . K . , a n d D . W. Pearce. Cost-BenefitAnalysis: Theory and Praetice. London: Macmillan, 1972. [11] Dienst Zuiderzeewerken. Beschouwingen over de Markerwaard. Den Haag: Staatsuitgeverij, 1972. [12] Eckenrode, R.T. "Weighting Multiple Criteria," Management Science, Vol. 12, No. 3 (1965), pp. 180-192. [13] Eckstein, O. "A Survey of the Theory of Public Expenditure Criteria," in J. M. Buchanan, ed., Public Finances: Needs, Sources, and Utilization. Princeton, N.J.: Princeton University Press, 1961. [14] Edmunds, S., and J. Letey. Environmental Administration. New York:. McGraw-Hill, 1973. [15] English, M. J., ed., Cost-Effectiveness. New York: John Wiley, 1968. [16] Fano, P.L. "Evaluating AIternative Plan Configurations: A Cost-Effectiveness Approach," Papers of the Regional Science Association, Vol. 31 (1973), pp. 161-172. [17] Feldstein, M.S. "Net Social Benefit Calculation and the Public Investment Decision," Oxford Economic Papers, Vol. 16 (1964), pp. 139-146. [18]! Gal, T., and J. Nedoma. "Multiparametric Linear Programming," Management Science, Vol. 18, No. 7 (1972), pp. 406-421. [19]j Geoffrion, A.M. "Proper Efficiency and the Theory of Vector Maximization," Journal of Mathematical Analysis and Applications, Vol. 22 (1968), pp. 618-630. [20] Guigou, J.L. "Analyse 6conomique et analyse multidimensionelle," Ph.D. dissertation, University of Lyon, 1972.

110

PAPERS OF THE REGIONAL SCIENCE ASSOCIATION. VOLUME THIRTY-FIVE

[21] Guigou, J . L . "On French Location Models for Production Units," Regional and Urban Economics, Vol. I, No. 2 (1971), pp. 107-138; and Vol. 1, No. 3 (1971), pp. 289-316. [22] Hendrikx, H. J.A. "Is Inpoldering van de Markerwaard nog Nodig en Nuttig?" Hommage h Gerard Lange, University of Amsterdam, 1972. [23] Hill, M. " A Goals-Achievement Matrix for Evaluating Alternative Plans," Journal of the American Institute of Planners, Vol. 34, No. 1 (1968), pp. 19-29. [24] - - , and M. Shechter. "Optimal Goal Achievement in the Development of Outdoor Recreation Facilities," pp. 110-121, in A . G . Wilson, ed., Urban and Regional Planning. London: Pion, 1971. [25] - - , and Y. Tzamir. "Multidimensional Evaluation of Regional Plans Serving Multiple Objectives," Papers of the Regional Science Association, Vol. 29 (1972), pp. 139-165. [26] Kazanowski, A . D . "Cost-Effectiveness Fallacies and Misconceptions Revisited," in J. M. English, ed., Cost-Effectiveness. New York: John WiIey, 1968. [27] " A Standardized Approach to Cost-Effectiveness Evaluations," in English, CostEffectiveness. [28] Klaassen, L . H . "Economic and Social Projects with Environmental Repercussions: A Shadow Project Approach," Regional and Urban Economics, Vol. 3, No. 1 (1973), pp. 83-102. [29] Klahr, C . N . "Multiple Objectives in Mathematical Programming," Operations Research, Vol. 6, No. 6 (1958), pp. 849-855. [30] Layard, R., ed., Cost-Benefit Analysis: Selected Readings. London: Penguin, 1972. [31] Lebart, L., and J. P. F6nelon. Statistique et informatique appliqudes. Paris: Dunod, 1971. [32] Lichfield, N. "Cost-Benefit in Plan Evaluation," The Town Planning Review, Vol. 35 (1964), pp. 160-169. [33] Marglin, S.A. Public Investment Criteria. London: Allen and Unwin, 1967. [34] Markerwaard, special edition of Stedebouw en ValkshMshouding, Vol. 54, No. 4 (1973). [35] Mishan, E.J. Cost-Benefit Aanalysis. London: Allen and Unwin, 1971. [36] Newton, T. Cost-Benefit Analysis in Administration. London: Allen and Unwin, 1972. [37] Nijkamp, P. "Determination of Implicit Social Preference Functions," Report 7010, Econometric Institute, Erasmus University, Rotterdam, 1970. [38] , ed., Environment and Economics: New Contributions to the Economic Analysis of Environmental Problems. Rotterdam: Rotterdam University Press, forthcoming. [39] - "Environmental Research and Spatial Analysis," Research Memorandum No. 3, Faculty of Economics, Free University, Amsterdam, 1974; paper presented at the Third Advanced Institute in Regional Science, 1974, to appear in Karlsruhe Studies in Regional Science, Pion, London. [40] - - , and J. H. P. Paelinck. Operational Theories and Methods in Regional Economics. Farnborough: Saxon House, forthcoming. [41] , and W . H . Somermeyer. "Explicating Implicit Social Preference Functions," The Economics of Planning, Vol. 11, No. 3 (1971), pp. 101-119. [42] Opschoor, J. B., and G . J . Van der Meet. "Multicriteria-analyse: de Electra-methode," Working Paper No. 17, Instituut voor Milieuvraagstukken, Free University, Amsterdam, 1973. [43] Pratt, J. W., H. Raiffa, and R. Schlaifer. "The Foundations of Decision under Uncertainty: An Elementary Exposition," Journal of the American Statistical Association, Vol. 59 (June, 1964), pp. 353-375. [44] Prest, A. R., and R. Turvey. "Cost-Benefit Analysis: A Survey," The Economic Journal, Vol. 75 (1965), pp. 683-735. [45] Roy, B. "Classement et choix en pr6sence de points de vue multiples: La M6thode ELECTRE," Revue d'informatique et de rOcherche operationelle, Vol. 2, No. 8 (1968), pp. 57-75.

NIJKAMP:

[46] [47]

[48] [49] [50] [51] [52] [53]

A MULTICRITERIA ANALYSIS FOR PROJECT EVALUATION

111

"D6cisions avec crit+res multiples: Probl6mes et m6thodes," Metra, Vol. 11, No. 2 (1972), pp. 121-151. Spliid, I. "Use of Correspondence Analysis in a Planning Procedure for Local Governments," paper presented at the Third Advanced Karlsruhe Institute in Regional Science, 1974, to appear in Karlsruhe Studies in Regional Science, Pion, London. Stone, J. R . N . "A Comparison of the Economic Structures of Regions Based on the Concept of Distance," Journal of Regional Science, Vol. 2, No. 2 (1960), pp. 1-20. Terry, H. "Comparative Evaluation of Performance Using Multiple Criteria," Management Science, Vol. 9, No. 3 (1963), pp. 431-442. Vereniging tot Behoud van her IJsselmeer, Plan Waterlely. Alkmaar: Casparie, 1974. Wolfe, J. N., ed., Cost-Benefit and Cost-Effectiveness. London: Allen and Unwin, 1973. Zeleny, M. Linear Multiob]ective Programming. Berlin: Springer-Verlag, 1974. ---, and J.L. Cochrane, eds., Multiple Criteria Decision Making. Columbia, S.C.: University of South Carolina Press, 1973.