Learning from Experience Evaluating Early Childhood Demonstration Programs (1982) / Chapter Skim
Currently Skimming:
Evaluating Early Childhood Demonstration Programs
Evaluating Early Childhood Demonstration Programs
Pages 3-54
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 3... ...
Formal outcome measurement has gained increasing acceptance as a tool for policy analysis, as a test of accountability, and to some extent as a guide for improving program practices. Programs have been subjected to scrutiny from all sides, as parents, practitioners, and politicians have become increasingly sophisticated about methods and issues that once were the exclusive preserve of the researcher.
|
From page 4... ...
Demonstration programs, the subject of this report, are particularly likely to change as experience accumulates. Consequently, evaluation must address multiple concerns and must shift focus as programs mature or choral Her and == -^w policy issues emerge.
|
From page 5... ...
By the same token, a randomized field experiment, with rigorous control of treatment and subject assignment, is sometimes the most appropriate way to answer questions salient for policy formation or program management. In such situations, government should be encouraged to provide the support necessary to implement experimental designs.
|
From page 6... ...
It is the need to bring experience to bear on practice that distinguishes evaluation from other forms of social scientific inquiry. A Word on Definitions This is a report about the evaluation of demonstration programs for young children and their families.
|
From page 7... ...
We take very seriously the inclusion of families as recipients of services; we emphasize the fact that many contemporary programs attempt to help the child through the family and that outcome measures should reflect this emphasis. Plan of the Report We begin by tracing the historical evolution of demonstration programs from 1960 to the mid-1970s, and of the evaluations undertaken in that period.
|
From page 8... ...
Moreover, Head Start has constantly expert mented with curricula and approaches to service delivery, and it has spawned a vast number of evaluations. For these reasons it dominates our discussion of demonstrations from 1965 until very recently.
|
From page 9... ...
However, for others the results testified only to the narrowness of the study's outcome measures and to other inadequacies of design. Some partisans of Head Start and critics of the Westinghouse-Ohio study, claiming that the program was much more than an attempt at compensatory education or cognitive enrichment, argued that the study bad measured Head Start against a standard more appropriate to its precursors.
|
From page 10... ...
In addition, several offshoot demonstrations, some of them dating from the 1960s, began to get increased attention, notably the Child and Family Resource Program, the Parent-Child Centers, and Parent-Child Development Centers. These projects extend services to children much younger than age three or four, the normal age for Head Start entrants These programs work through the mother or the family
|
From page 11... ...
In addition, federal evaluation research began to concentrate on other children's programs, such as day care, which had existed for many years but had begun to assume new importance for policy in the 1970s. In the next section we attempt to characterize some of the recent program initiatives as well as the policy climate that surrounds programs for young children and their families in the early 1980s.
|
From page 12... ...
To understand the policy context surrounding demonstration programs for children in the 1980s, it is useful to begin by outlining some general considerations that affect the formation of policy. These generic considerations apply to virtually all programs and public issues but shift in emphasis and importance as they are applied to particular programs and issues, at particular times, under particular conditions.
|
From page 13... ...
n The national managers made a conscious, concerted effort to distinguish Head Start from other children's services, notably day care. The latter was seen as controversial -- hence, a politically risky ally.
|
From page 14... ...
Economic pressures, the increased labor force participation of women, and the rise of feminism brought day care into prominence. Federal investment in day care increased under Title XX of the Social Security Act and numerous other federal programs for the working poor, backed by a curious alliance of feminists, liberals, child advocates, and ~workfare conservatives." Although anti-day-care, "pro-family forces remained strong, public subsidy of day care was gradually, if sometimes grudgingly, accepted as a reality.
|
From page 15... ...
Day care is of course not the only type of children's program that underwent major change in the 1970s. Important new initiatives arose in the areas of child health and nutrition.
|
From page 16... ...
Two characteristics of these programs are particularly salient: an emphasis on the family and the community institutions with which it interacts, rather than on the child in isolation, and a stress on localism -- on the diversity, rather than the uniformity, of programs and on their adaptation to local values and conditions. Programs exemplifying these emphases include Head Start's spinoff demonstrations, such as the Parent-Child Development Centers and the Child and Family Resource Program.
|
From page 17... ...
They also focus attention on relations between programs and informal institutions, such as extended families, which in some subcultures have traditionally provided the kind of global support that some demonstration programs aim to provide. They raise basic questions as to whether ecological approaches in general are more effective than interventions aimed at the child alone.
|
From page 18... ...
The need to measure program effects on children has not diminished -- witness the current effort by Head Start to develop a new, comprehensive battery of outcome measures. Concerns about cost, efficiency, and equity have become acute, as the federal government has expanded the scope of its responsibilities Broad entitlements and new initiatives have increased the competition for finite resources in the face of widespread resistance to further taxation and bureaucratic expansion.
|
From page 19... ...
And awareness should make their contribution even greater. IMPLICATIONS FOR OUTCOME MEASUREMENT AND EVALUATION DESIGN The programs and policy issues that have evolved over the past two decades, particularly in the late 1970s, pose serious challenges for evaluators.
|
From page 20... ...
As suggested earlier, experimental designs are ideal for answering certain kinds of evaluation questions, because they provide the most direct means of establishing linkages of cause and effect. Children's academic skills and performance are often important program outcomes, and standardized tests, properly interpreted, measure aspects
|
From page 21... ...
Of course, these children themselves form extremely heterogeneous populations with diverse needs. Accompanying increased public attention to day care has been a concern about the effects of prolonged out-of-home care on children from all social
|
From page 22... ...
In addition, preschool education, once the predominant service for children of low-income families, has been joined by health care and nutrition, referrals to a wide variety of social services, and training and counseling of parents in child care, in dealing with schools and other public institutions, in family relations, and in more peripheral areas such as employment and housing. This breadth of services obviously requires a commensurate breadth of measures-not only better measures of children's physical, intellectual, social, and emotional growth but also measures of the quality of the child's life in the program itself (as programs increasingly become a large part of the child's daily environment)
|
From page 23... ...
An additional, equally important implication of this emphasis on support is that support itself should be measured. There is a need to know whether family-oriented social programs in fact strengthen the family or inadvertently weaken it by creating dependence on government and cutting ties to informal supports such as friends, neighbors, and the extended family.
|
From page 24... ...
Nonuniform treatments challenge evaluation designs in fairly obvious ways. Although it is inappropriate to lump clients into a single treatment group to probe for common outcomes, it is equally unsatisfactory to treat individualized programs simply as a series of case studies.
|
From page 25... ...
Individualization of services also raises a related value issue: how to reconcile legitimate and desirable individual differences with the need to identify a manageable set of outcome measures that are consistent with program goals. Early childhood programs run the risk of attempting to homogenize certain characteristics of their participants.
|
From page 26... ...
Head Start and Title XX day care attempt to provide a wide range of services in a single facility. Demonstrations such as the Child and Family Resource Program try to capitalize on existing services in the community, providing referrals and, if necessary, assist ance and advocacy in securing services to which clients are entitled.
|
From page 27... ...
In this regard the limitations of traditional outcome measures, especially standardized tests of cognitive ability and achievement, have long been recognized. Because the goals of many early childhood programs lie in socialization, rather than cognitive enrichment, calls for better measures of selfconcept, social skills, prosocial behavior, and the like have been frequent and forceful.
|
From page 28... ...
For example, one could speak of a day care program making a child more cooperative with other children, with the presumption that increased cooperativeness will manifest itself in the home or in school, not just in the day care center. Or one could simply speak of a day care center in which a cooperative atmosphere prevails, or in which a particular child behaves cooperatively, with no presumption about crosssituational generality or longitudinal persistence of cooperativeness.
|
From page 29... ...
. However, although many researchers have been concerned with the impact of early day care on mother-infant attachment, few have used
|
From page 30... ...
Instead, even basic researchers working with small samples have used ad hoc modifications of Ainsworth's procedure, with the result that much of the literature on day care and attachment must be viewed as ambiguous (Belsky and Steinberg, 1978)
|
From page 31... ...
as indicators of long-term program effects on individuals. These measures are clearly attractive for their direct social and policy importance.
|
From page 32... ...
One is the National Day Care Study, a large-scale study of center day care, designed to inform federal regulatory policy (Ruopp et al., 1979)
|
From page 33... ...
The study's results had a direct influence on the day care regulations subsequently proposed by the federal government (Federal Register, March 19, 1980) , suggesting that the study's outcome measures had some weight for policy makers.
|
From page 34... ...
, which can also be drawn on to identify parental behaviors likely to be both significant for the child and susceptible to influence by programs. Finally, there is promising new theoretical work on the ecology of human development, which offers both a conceptual framework and specific suggestions about variables and relationsips that might be examined in real-world contexts, such as day care (Bronfenbrenner, 1979)
|
From page 35... ...
Or family support programs may benefit children and parents but simultaneously increase their dependence on government and displace private support systems, such as the extended family. There are no hard-and-fast rules for mapping the universe of potential systemic outcomes.
|
From page 36... ...
. Another example of the utility of such data is provided by demonstrations of service delivery mechanisms, e.g., vouchers for day care, for which head counts of persons served are obviously relevant as outcome measures.
|
From page 37... ...
If the purpose is to determine whether a particular program confers an advantage over existing service systems or agencies, alternative sources of service may provide a useful comparison. In general, experimental designs presume that the treatment/no treatment comparison is the relevant one, but for many policy purposes comparison with the preexisting configuration of services is more relevant.
|
From page 38... ...
Umbrella programs are likely to vary from site to site with respect to such features as scope of services, the role of parents, philosophy or curriculum, nature of the sponsoring agency, links to the school system, etc. In some cases, notably family service programs such as Parent-Child Centers and the Child and Family Resources Program, this diversity is deliberate: Such programs are intended to respond to local needs and to make use of local resources.
|
From page 39... ...
circumstances. Investigation ot site errects can also give the policy maker some indication of which program characteristics can and should be mandated at the federal or state level and which are best left to local initiative.
|
From page 40... ...
An example of the usefulness of findings linking cost to quality of service is provided by the National Day Care Study, mentioned in an earlier section (Ruopp et.
|
From page 41... ...
produced particularly clear-cut evidence of the effectiveness of home-based intervention. Another example is provided by the National Day Care Study (Ruopp et al., 1979)
|
From page 42... ...
We find a great deal of promise in combining both qualitative and quantitative types of studies in the evaluation of early childhood programs. One approach would be to do both and see if they tell the same story Another approach uses qualitative data to enrich and support quantitative findings.
|
From page 43... ...
IMPLICATIONS FOR THE EVALUATION PROCESS Some of our suggestions about design and measurement have indirect implications for the way in which applied research is organized and conducted, for the way in which its results may be presented most effectively, and even for the relationship between applied research and basic social science. Involving Multiple Constituencies in Selecting Outcome Measures Given that demonstration programs affect many constituencies that have a stake or a say in the program's future, ways must be found to involve these groups or at least take account of their concerns in selecting outcome measures.
|
From page 44... ...
Policy makers, government program managers, advocacy groups, practitioners, and parents are among their many audiences. Each group has its own concerns and requires a special form of communication.
|
From page 45... ...
One implication is that both researchers and the people who manage applied research -- particularly government project officers and perhaps even program officers in foundations -- need to develop intimate familiarity with the operations of service programs as well as basic understanding of the policy context surrounding those programs. Technical virtuosity and substantive excellence in an academic discipline do not alone make an effective evaluator.
|
From page 46... ...
Too much intimacy with a program can erode an evaluator's intellectual independence, which is often threatened in any case by his or her financial dependence on the agency sponsoring the Program in question. (Most evaluations are funded and monitored by federal mission agencies or private sponsors that also operate demonstration programs themselves.)
|
From page 47... ...
Drawing on and Contributing to Basic Social Science In some respects, evaluation stands in the same relationship to traditional social science disciplines as do engineering, medicine, and other applied fields to-the physical and biological sciences. Evaluation draws on the theories, findings, and methods of anthropology, economics, history, political science, psychology, sociology, statistics, and kindred basic research fields.
|
From page 48... ...
Because understanding social programs requires a judicious fusion of qualitative and quantitative methods, evaluation may stimulate new methodological work
|
From page 49... ...
(1978) The effects of day care: a critical review Child Development 49:929-949.
|
From page 50... ...
Progress Toward a Free, Appropriate Public Education. A Report to Congress on the Implementation of Public Law 94-142: The Education for All Handicapped Children Act.
|
From page 51... ...
Vol. 3: Child Development and Social Policy.
|
From page 52... ...
Final report of the National Day Care Study, Volume I Cambridge, Mass.: Abt Books.
|
From page 53... ...
(1972) Evaluating Action Programs: Readings in Social Action and Education.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.