All material on this site is (c) copyright to the respective authors.  ISSN - 1481-3440 


First Posted: 1998  Last edited: 23 September 2005

THE EVALUATION OF TECHNICAL ASSISTANCE AS
A CONTRIBUTION TO DEVELOPMENT
by Cyril Belshaw (c)
Originally published in the International Development Review June 1966
Author's Note: The context has of course changed immeasurably. Have the principles changed?
If not, are they being observed? If so, what are the changes?
THE EVALUATION OF TECHNICAL ASSISTANCE has been of concern to international
officials for
over fifteen years [1996]. During that time there have been some attempts, more or less
sporadic, to
develop techniques of evaluation, but very few of the attempts have been coordinated with
others, and
one cannot describe a continuous evolution of thought about the matter. The symposium on
techniques of
evaluation which took place in 1955 under TAB auspices, summarized in the International
Social Science Bulletin, might have taken place today. One is struck by the fact that there is
in existence no bibliography of evaluation reports, and that most evaluation teams or studies
bilateral agencies sometimes lose track of what they have done before. Another impression

Page 2
contact, so that learning and the transfer of experience between them is minimal, and the
growth of technique and conceptualization is slow.
This article is not to make up these deficiencies, but to consider a point which has been lost
sight of because of the considerable duplications of conceptual effort. The nature of
evaluation depends entirely upon the purposes of the evaluation, and so far very few
attempts have been systematically concerned with the effects of technical assistance upon
the development of a country (a polity, a society, an economy) as a working system. Concern
with this overall approach to technical assistance has been growing in recent years, and is
reflected in the debates and resolutions of the Economic and Social Council which have led
to the appointment of a number of evaluation missions. Such missions, however, must work
out their own technique in the short space of time available to them, and must therefore
build very largely upon the past with the minimum of innovation. This paper grows out of
the experience of one such mission and attempts to reflect further upon the possibilities. It is
of course a personal paper, with which the other members of the mission, and the United
Nations, are not associated.
LIMITATIONS ON PROJECT EVALUATION
Most approaches to technical assistance evaluation are concerned with the question, to what
extent did the
project achieve its goals in the most efficient manner? Such a question can often be
answered with a great deal of accuracy, which varies according to whether we can assume
that the goals are specifically defined, and whether the analyst can obtain sufficient data
about results, timing, and the resources used. When the approach is coupled with an
examination of possible technical alternatives, it can lead to an improvement in method, and
often increased results with fewer resources. Evaluations of this kind are necessary to
improve the technical capacity and economy of the agencies providing assistance.
Yet to describe this as an evaluation of the contribution of technical assistance to
development would be to give a false emphasis: the operation is simply an extension of
research into the techniques and methodology of agricul tural, health, educational,
industrial, or other operations, with special emphasis on problems which are of frequent
occurrence in technical assistance.
Even within this conception there is a major limitation to the approach. Frequently the goals
of technical assistance projects are not specified with a sufficient degree of precision to
make such an assessment possible, and the methods used involve skills and intangibles
which weaken attempts at quantification. This is occasionally a matter of careless project
formulation, but very often it is deliberate, since the objective is to probe, to test, to explore,
to stimulate, and since, if the goal were pre-defined, it might dictate an apparent solution
Evaluating Technical Assistance in Development

Page 3
which would otherwise not be appropriate. This is true of open-ended
scientific exploration, of most institutional change, and of projects which involve the
formation of ideas and values.
Linked to this limitation is another, namely that important effects of technical assistance,
both positive and negative, are side-effects. These are not only unforeseen results in closely
allied fields (the use of a school textbook offending mores and creating antipathy to the
school), but ramifying results over many areas (a road creating a demand for markets,
transportation, drainage, organized water supply). To concentrate solely on the specific goal
of the project might mean the setting aside of extremely significant conditions.
Another consideration is the analysis of the goal itself. To accept it as given may be to avoid
asking the question, was it the most appropriate goal under the circumstances, or would
some other uses have been more effective?
And finally sometimes, though not always, the evaluation does not distinguish between the
technical assistance component of a project, and the total project itself. Thus, for example, it
is fairly easy to state that a tuberculosis eradication campaign was associated with a
reduction of disease incidence which in turn was associated with a sharp drop in mortality
attributable to tuberculosis. But it is not always so easy to decide how much of this result is
attributable to the skill of government action itself, to the W.H.O. key assistance, to the
actions of U.S. aid financed teams, or to alterations in the habits of the population in turn
linked with improved housing, water supply, education, nutrition or marketing. Perhaps the
same result would have been achieved without international technical assistance. Perhaps it
would have been achieved more quickly if there had been more of it. Perhaps it was the
crucial catalyst making the whole com-
Because of factors such as these, some approaches to evaluation are more open-ended and
discursive than the tight technical ones which are superficially precise. Thus Professor
Charles Madge, for example, attempted, on behalf of UNESCO, to tap the experience and
knowledge of a limited number of technical assistance experts who had been working in
Thailand in fields where the rural human relations component was highly significant in the
work. The experts were asked to keep diaries indicative of their experience, including
interaction with counterparts and with village people, to answer questions about their
experience and its results, and to add their own analysis of the impact and significance of
the project
in a free-flowing manner. Similarly, Professor Herbert Hyman and his associates at the
United Nations Research Institute for Social Development, have made a detailed statistical
and analytical study of the views of over four hundred experts in ten countries, again
working in areas where rural human relations are of great significance. The most immediate
value of such studies is to cast considerable light upon the attitudes of technical assistance
experts who are, of course, key elements in the chains of interaction which ultimately bring
about a technical assistance result. The views of Resident Representatives, as surveyed by
Evaluating Technical Assistance in Development
]

Page 4
In a limited way the experts are also used to reveal the processes at work during the course
of their project,
and since they are intelligent commentators and observers they can record many events and
interactions which are extremely instructive, and can add an element of depth and humanity
to otherwise mechanical studies. Nevertheless, they cannot do the whole job of analysis.
What one observes depends on what questions one is interested in. Field experts are very
seldom economists or sociologists, and if they make observations of interest to economists
or sociologists, it is quite often because they have absorbed appropriate knowledge as
intelligent men and women stimulated by an overseas environment that has intrigued them.
In the same way, anthropologists made use of missionaries and other field observers in
the early days of the discipline, and now contact other anthropological colleagues for
information of a comparative kind. But anthropologists interested in generalizations are
continuously frustrated because their colleagues have not recorded information which is
essential to the new theory, and in many instances were not even aware of its significance
when they were in the field. The dimension added by the knowledge of the field experts
must eventually be supplemented by more systematic value-free
observation which can go beyond the theories at present currently circulating among the
professional fraternity.
OVERALL CONTRIBUTION OF TECHNICAL AID
These types of investigation are leading into much broader fields than those indicated by
the problem as
to whether a project is being handled in the most efficient or effective way possible. They
are leading in fact towards the question, what is the contribution of technical assistance to
the overall socio-economic development of a given country? At first sight, it would seem
that this question, being even broader, is less capable of being answered.
To some extent this is true, particularly if one expects quantitative precision in one's
answers. It would be
feasible to select particular types of technical assistance where relations between inputs and
outputs are direct, and measure impact on the economy at least. But most technical
assistance, and perhaps the most significant, is not of this order at all. How would one
translate the effect of an adviser on national planning in such terms? How would one put
together the effects of a fellowship concerned with the administration of social welfare and
the advice of an expert who suggested that the time was not ripe for a national standards
laboratory? To add up, for example, the monetary value (in terms of salary and other costs)
of the fellowship and the expert project would not give a comparative indication of
significance, and would not take into account the ramifying effects of decisions which flow
Evaluating Technical Assistance in Development
 

Page 5
from the two enterprises.
One is therefore forced back into a process of analysis which endeavors to place the
technical assistance project in an organic framework, and which endeavors to ascertain what
the implications of the projects are for the economic and social environment in which it is
placed. One must analyze what happens, and what the linkages of events might be. This
implies the selection of information according to judgments about its significance for
organic relationships rather than for its significance for quantitative measurement. (The two
concepts are not necessarily opposed: it is a question of starting point.
One method starts with observations which are measurable now, and builds a pattern of
significance around them; the other starts with patterns of significance, and ultimately tries
to measure what one can within the pattern.)
Thus in the analysis judgment is all important. Judgment in such a field is systematized
around propositions, explicit or not, about the crucial operation of significant relationships.
There is in fact a theory of the operation of a society-economy, and a theory about the way in
which certain factors and relationships operate to bring about development. Technical
assistance is judged according to the manner in which it fits, or contributes to, the operation
of the assumed system.
"Over-all impact" evaluation teams do not normally have the time to elaborate on this matter
systematically and consciously, or to link their observations in more than a passing manner
to current developmental theories, of which the number is legion. Where immediate
knowledge is required, this is all to the good, since to do otherwise would be to carry out a
prior theoretical exer cise without the certainty of agreement, and to lose some of the
exploratory significance of the investigations.
But inherent assumptions are there .
But as time goes on there will be an increased need for greater systematization in the
observations to allow
for comparability, both temporally and cross-culturally, and to enable the validity of the
implied propositions to be tested by further experience. The explicit linkage of over-all
impact evaluation to theories of development will therefore be an advantage. This paper is
an endeavor to push deliberate thinking on this topic a little further.
A POSSIBLE APPROACH
One interesting suggestion, unfortunately in a paper not released for publication, has been
to reduce the whole of the development process to one basic factor, namely the increase in
Evaluating Technical Assistance in Development

Page 6
skills. This dramatic analytical step, cutting through the confusing intricacies of social and
economic theory to look for a prime mover which is not only empirically sound but
eminently subject to policy influence, has a great deal of appeal. Increase in skills brings
together under one heading processes which potentially reflect upon motivation, technical
capacity, efficiency, innovative power, and organizing ability. It is a factor linked not merely
with economic growth. in the sense of a (possibly temporary or fortuitous) alteration in
output, but more fundamentally with development in an organic sense, namely an alteration
in the organized characteristics of a culture, society, economy and polity upon which
enduring and self-sustained growth is based. One cannot envisage such development
without an alteration in skills.
This of course does not end the matter. There is not just one indicator of skills, there are
many. Studies
must range over such matters as technological skill, entrepreneurial ability, and the
wide-ranging questionanswering type of skill usually associated with research or university
education. They must inquire into the degree to which technical assistance has contributed
to the growth of such skills, and pay allention to the problem as to whether the spectrum of
skills is appropriate to the circumstances of the country at the time. There is no doubt that
we have here a basic factor in social development.
However, no single criterion is likely to be good enough. One of the difficulties is that an
increase in skills is not an independent variable, and that it may be supported or
counter-acted by other variables, which ought therefore to be taken into account. It is not
difficult to think of instances in which a dramatic alteration in the availability of physical
resources called forth, to some degree, an increased supply of skills (mining linked with the
supply of engineers, roads with the development of marketing skills). On the other hand,
examples of the over supply of skills are legion, and there are countries where scientists and
engineers are available, but are not being used to capacity because of lack of appropriate
forms of
organization or institutions. A single criterion which hides such factors will reduce the
significance of the
evaluation as a diagnostic tool.
SOME RECOMMENDED CRITERIA
I wish now to set out a series of propositions and questions which in my view are closely
linked with the development process. Since there is no clear agreement among scholars and
administrators as to the theory, or the nature, of development, or as to the weight to be
attached to such factors as are considered to be significant, it is most unlikely that this series
will be generally accepted. Nevertheless, if it has merit as an approximation it will lend
support to the idea that evaluation exercises should endeavor to make explicit the
Evaluating Technical Assistance in Development

Page 7
development theorems on which they are based, and to relate these to the specific technical
assistance programs they are examining. It should be stressed that there is no implication
that projects should, to be effective, contribute in terms of all propositions, or that they are
better or more effective if they contribute to the first rather than the second. At this stage,
evaluation of the impact of technical assistance upon development is not to be achieved
through arithmetical summation but through qualitative analysis.
(1) A technical assistance project contributes to development if the program of which it is a
part perma-
nently alters the effective demand schedule, or consumption pattern, of the country in such a
way that an
increased level of satisfaction is achieved, and the gap between effective demand and the
pre-existing potential demand is narrowed. [Anthropologists note: does the change in culture
increase satisfaction or not?-1998] This is the most difficult proposition to formulate
succinctly, and also by far the most difficult to assess. Note first, the phrase is "pre-existing"
potential demand, because it is very often the case that
the satisfaction of one series of wants at one level opens up a whole range of other
unsatisfied ambitions, or brings them into consciousness in such a way that they can
constitute goals for further action (see proposition two). Note secondly, no attempt has been
made to distinguish between economic, cultural or social wants, or to state which have
prepotence or greater importance. The distinction is unsatisfactory analytically, and in any
case, within the context of this proposition, the choice is for the people to make themselves.
The difficulty of assessment lies in the identification of the effective demand schedule, and
even more in judging potential demand. Work being done at the United Nations Research
Institute for Social Development on levels of living may contribute to the solution of the
first problem, and the second may be solved approximately or temporarily by adopting the
convention that the government's assessment of plans and aspirations is the one to be used.
We cannot, however, rest content with the mere acceptance of government priorities,
particularly since some governments may be out of touch with the aspirations of its citizens,
or may not be able to analyze the implications. It must be admitted that most evaluation
teams will of necessity judiciously and somewhat subjectively combine information gleaned
from government
plans and statements with indications from market behavior, statements of opinion, and
social and economic analyses. [Anthropologists note: an ethnography is a rough estimate of
effective demand in all sectors: a study of the way in which people valuate potential changes
contribvutes to the second sechedule - 1998]
(2) A technical assistance project contributes to development if the program of which it is a
part increases the
satisfaction of wants in such a way that other unsatisfied wants, some of which may be new,
alter their position in the potential demand schedule to such an extent that they become goals
of further action. In other words, an increase in consumption (material or immaterial) spurs
people to try to obtain more, constituting a dynamic force for change. This by no means
Evaluating Technical Assistance in Development
 

Page 8
always happens, particularly in marginal communities, but unless it happens development
associated with self-sustained growth will not be present. It must be admitted that neither
economic theory nor empirical analysis have yet given adequate guide-lines for the
assessment of the multiplying or ramifying effects of particular forms of consumption.
Nevertheless, judgments about this are implicit in the evaluation of the impact of technical
assistance.
(3) A project will contribute to development if it assists a program to increase the range of
indigenous
resources utilized or increase the range of commodities produced (provided this is justified
economically) or to remove bottlenecks in the system of resource exploitation and production,
thus liberating further productive forces.
(4) A similar result will occur, other things being equal, when technical assistance contributes
to an in-
creased division of labor (social concomitant: increased diversification of social role),
provided that this con-
tributes optimally to production or to the direct satisfaction of wants. It should be stressed
here once again that the wants will be material, cultural or in the nature of social welfare
satisfactions, and that a priori judgments of appropriateness or imbalance should, in this
context, be eschewed.
(5) A further proposition linked with (4) is that development reflects innovation, and that the
rate of in-
novation, other things being equal, is associated with the size of the pool of relevant ideas and
information on the one hand, and the ability of the potential innovators to question, observe,
generalize and apply knowledge. Therefore one should ask, does the form in which technical
assistance is given add to the pool of ideas permanently available to the society (the visit of
an expert who took his knowledge away with him would not qualify)? And does it equip
personnel, not merely to apply knowledge statically, but to develop it in the circumstances
of the country? li should be noted here that innovation is not
merely technical in a physical sense, but also implies alterations in modes of organization.
(6) Just as an increase in the velocity of circulation of money may be deemed to increase the
quantity of
money, so an increase in the velocity of circulation of ideas and information may he deemed to
increase the
effective size of the pool of ideas. Thus communications are of vital significance in
development. But communications have a further effect, namely that of assisting individuals
to adapt to one another. They are a vital element in the articulation of a society, a culture, a
polity, an economy. Does technical assistance contribute to increased effectiveness of
communications? Here the evaluation must look for several types of indicators. The use of
two-way transmitter/receivers, the improvement of the stock exchange (or its foundation),
the state of the commodity market, the use of telephones and the mails, the extension of
reliable freight services, the use of computers, may be areas involving technical assistance
Evaluating Technical Assistance in Development
 

Page 9
which have profound social and economic consequences. The problem for assessment may
link equally with boundaries in social relations (for all communication flows along lines
indicated by social structure and social organization) or increasing the effectiveness of
symbol systems (the basis of the means of com-
munication), whether these be conventions such as legal contracts, or language elements
such as literacy, mathematics, or weight symbols. What does technical assistance do, both
directly and indirectly, in these connections ?
(7) In most countries to which technical assistance is given, the spectrum of organized
institutions is either
not as complete as in developed countries, or does not constitute an articulated whole in
such a way that the society operates as a set of inter-acting organizations. Technical
assistance will make a contribution towards the establishment of a modern articulated
society if it meets some of the earlier criteria. But in addition, as an aspect of the division of
labor, and as an extension of the fourth proposition, it will make a contribution if it assists
in the creation of specific institutions. The functional emphasis of these is likely to be the
following, although functions need not be limited in this way:
(a) the creation of institutions which produce skills and knowledge, (b) the organization of
production and services (including cultural and welfare services), (c) the organization of units
of public administration, (d) the organization of institutions to remedy societal ills which are
frictional to the operation of the system.
It should be noted that the emphasis is upon the training of trainers", to use the current
jargon, and
the creation of institutions. If a technical expert simply produces a new textbook, teaches a
number of children, succeeds himself in removing malaria from a village, cures a number of
opium addicts, secures the production of x quantity of a raw material, his impact is limited
to an effect, probably temporary, on growth rather than on development, unless in addition
his action meets the requirements of propositions one and two, Such wants or ills cannot be
dealt with permanently unless permanent institutions are available to assess and act.
Technical assistance thus tends to be geared to institution building, and rightly so, for this
increases the complexity of effective social organization, and the capacity of a society to take
its own responsibility for increasing its output, and its performance from the point of view
of satisfying wants.
(8) Organizations themselves are small socio-economic systems usually capable of improved
performance.
If the operation of organizations is improved so that their contribution to the over-all social
system improves, they will be changed and developed and will be part of a development of the
social system. Technical assistance is likely to make a contribution in several fields, which
would include increasing the efficiency of operation, an improvement in adjustment
mechanisms (both internal and as an aspect of adjustment to external conditions and to
other institutions), and the creation of an orientation toward growth and expansion.
Evaluating Technical Assistance in Development

Page 10
(9) Finally (but this is implied throughout) technical assistance will make a greater
contribution to develop-
ment the more it results in the internalization of the above factors, so that they are not
dependent on external artficial stimulus.
Any technical assistance project can be linked with the above propositions, and analyzed
according to which propositions (if any) it fits. Presumably, if it does not fit any of the
propositions it does not qualify as technical assistance which contributes towards over-all
development. There are indeed technical assistance projects which can be ruled out on these
grounds, or on the grounds that their contribution is minimal. But beyond these extreme
cases, the propositions as they stand do not in themselves give a clear indication of the
relative merits of alternative technical assistance projects or proposals. I do not believe that
any firm agreement on the technique for achieving this next step is at this stage probable;
indeed, at this stage it may be undesirable, since in the present state of our knowledge a
further period of trial and error, of experiment, and of observation, is probably to be
preferred over a too hasty commitment to one line of thought.
A STRATEGY OF DEVELOPMENT
Insofar as there are principles to be discerned or worked out, they will probably crystallize
around the
notion of a strategy of development. This is simply a short way of saying that projects vary
according to the
weight of their impact. Conceptually, such variation is in the degree to which projects have
multiplying or
ramifying effects throughout a socioeconomic system, and it may be argued that the greater the
multiplying or ramifying effects the more strategic the project. This refers to the objective of
the project, and its indirect
consequences. In addition, technical assistance may not be concerned with a total project,
but only part of it, and here again the possible alternatives may be judged according to the
proliferation of their effects.
Unfortunately, this criterion, necessary though it is, is not only a matter of evaluative and
analytic judgment, but is also predicated to some extent on the existence, of a working
socioeconomic system which has a high degree of internal articulation. A project may have
highly ramifying effects in one country, because the effects travel, as it were, through the
social and communicative links of the system, and effects take place as various institutions
adapt to the new conditions. But in another country the same measures may have little or no
effect beyond the immediate implementation of a target, because communication does not
exist or because institutions in the linkage of social interaction are missing, or because they
are non-adaptive. This suggests that a strategy of development may need to place greater or
prior emphasis upon institution building and the communications system in some countries,
Evaluating Technical Assistance in Development

Page 11
or for some sectors.
In any event, the contribution of technical assistance to over-all development cannot be
judged finally without some fairly specific assumptions about the strategy of development
in the circumstances of the country.
Evaluating Technical Assistance in Development