Linking Green Schools to Health and Productivity: Research Considerations
One of this committee’s tasks was to identify avenues of research that represent potentially valuable opportunities to leverage existing knowledge into a better understanding of the relationships between green building technologies in schools and the health and performance of students and teachers. In Chapter 1, the committee recommended that
Future green school guidelines should place greater emphasis on building systems, their interrelationships, and overall performance. Where possible, future guidelines should identify potential interactions between building systems, occupants, and operation and maintenance practices and identify conflicts that will necessitate trade-offs among building features to meet differing objectives.
In Chapters 3-7, the committee identified several avenues of research that should be pursued regarding:
The moisture-resistance and durability of materials used in school construction as well as other properties of these materials such as generation of indoor pollutants and the environmental impacts of producing and disposing of these materials.
Documenting a full range of costs and benefits associated with providing ventilation rates that exceed the current ASHRAE standard.
Determining optimum temperature ranges to support student learning, teacher productivity, and occupant comfort in school buildings.
Examining the relationships of exposures from building materials, cleaning products, and cleaning effectiveness to student and teacher health, student learning, or teacher productivity.
The role of light on learning as well as life-long effects on health in children, particularly with regard to the role that lighting in school environments plays in regulating sleep and wakefulness in children.
Determining optimum reverberation times that will ensure adequate speech levels without excessive reverberation in classrooms for children of various ages.
The efficacy, costs, and benefits of alternative ventilation strategies for the dispersion and removal of airborne infectious agents.
The use of ultraviolet germicidal irradiation in supplemental or portable air cleaning devices in school room applications and its effects on human health.
The committee recognizes that additional areas of research could also provide valuable information that could lead to improved indoor environments in schools. However, determining which areas of research may yield the most positive outcomes, and the most valuable information related to school building design, construction, and operations was beyond the resources and scope of this study.
The complexity of evaluating the effects of green schools on building occupants, the difficulty of designing research studies that can control for numerous confounding factors, and the difficulty of detecting significant effects has been discussed. As noted in Chapter 2, much of the research conducted to date has focused on one or two building systems and one or two potential outcomes. Much is still not known about the potential interactions of building systems, materials, operation and maintenance practices and their effects on building occupants, in general, or about school environments in particular. The necessary collaboration between architecture, engineering, science, medicine and social science expertise is a challenge but multidisciplinary research is required to fully study the relationships of indoor environmental quality to human health and performance outcomes.
In the following sections, the committee discusses methodologies that could potentially be used for future interdisciplinary research and identifies issues that should be addressed if the evidence base for the effects of green schools on students’ and teachers’ health, student learning, and teacher productivity is to be improved.
Differing methodologies and evaluation approaches generally apply to the types of outcomes being evaluated: Evaluations of health outcomes for teachers and students require a different model specification than an evaluation of student-level educational outcomes.
As detailed in Chapter 2, outcomes of educational interventions are notoriously difficult to evaluate. The two evaluation methodologies which have been given the greatest credence by the educational evaluation community are randomized experiments and econometric or regression-based techniques.
In randomized experiments students would have the same likelihood of being assigned to green schools as being assigned to conventional ones. This random selection would support an unbiased estimate of effects. Random assignment experiments allow other confounding factors to be ignored, thus greatly reducing the complexity of the evaluation. For various reasons, randomized experiments are relatively rare in educational evaluations (Shadish et al., 2002). Several innovations, such as propensity score matching have been developed to allow unbiased estimates of the effects of educational interventions (Henry et al., 2006), but, to date, these have produced less compelling evidence on the effects of interventions.
Econometric or regression-based techniques to estimate educational effects are often used in large-scale studies and place heavy data requirements on the investigation. The regressions must assess the intervention as a school input and include a full complement of variables to control for other factors. Following Boardman and Murnane (1979), Summers and Wolfe (1977), Hanushek (1997), Zimmer and Toma (2000), and Henry and Rickman (2006), the value-added specification of the school production function would be shown as:
where, Y1j is a measure of educational performance at time 1 for the jth student, B0 is a constant, F1j is a vector of family background influences from time t – 1 to time t for the jth student, S1j is a vector of school inputs from time t – 1 to time t for the jth student, P1j is a vector of peer influences from time t – 1 to time t for the jth student, Yt–1j is a measure of educational performance at time t – 1 for the jth child, and e is the error term.1
To evaluate the effects of green schools on educational performance, a variable indicating student attendance in a green school should be incorporated into the school inputs vector (S1j). A database that includes an indicator of green school attendance would be required. In addition, if selection into green schools is a potential confounding factor, as it is likely to be, an approach that models this selection and uses two-stage, simultaneous estimation may be necessary to produce estimates that are considered unbiased by the education research community.
In similar fashion, analysis of school building characteristics using a fully specified list of components and conditions could produce unbiased estimates of the effects of those characteristics on educational performance. This would require appropriate modeling of selection into schools with particular characteristics. The studies of the effects of building characteristics currently available do not include theoretically important variables and are frequently estimated for groups of students, rather than individuals. As discussed below, these studies are likely to yield biased or inflated estimates of the effects of these building characteristics on student performance.
Other evaluation approaches could be useful in producing evidence concerning the actual effects of educational interventions such as green schools. Extended-term mixed method studies which gather information on implementation and adaptation of the reforms over time could prove useful in understanding and improving the effects of attributes of green schools on student and teacher outcomes (Chatterji, 2004). Evaluations using program theory (Weiss, 1997) or investigating mechanisms by which the program effects are produced (Mark et al., 2000; Henry and Rickman, 2006) could yield important evidence to support the causal attributions of the interventions. However, studies investigating mechanisms do not generally produce estimates of the size of an effect, primarily because they often fail to look at counterfactual data (lack of untreated or non-exposed cases). Case-control studies including crossover intervention investigations have also been used in school settings. The crossover intervention design can account for the effects of external influences (e.g., seasonal variations in respiratory infections, weather, ventilation implications) and has the advantage of having each subject serving as his or her own control. However, case control studies can only be used when the effects of the outcomes of interest are registered quickly and less likely to be useful in studies of educational outcomes where developmental age and exposure periods produce confounding interactions (maturity or treatment-maturity).
Other types of quasi-experimental study designs may be feasible for at least some future studies of the effects of green schools. Interrupted time series designs are recognized as among the strongest quasi-experimental
designs. For a study on the effects of green schools, a time series of student performance before and after the introduction of a new green school could be examined. Comparison schools, where no change occurred, should be available. In addition, data may become available from green schools that are constructed and opened at various points in time. Data on relevant covariates could be examined to assess alternative explanations and strengthen statistical power.
Other quasi-experimental design features may also be possible in certain studies. For example, the Bronzaft studies on noise and learning referred to in Chapter 6 demonstrate a quasi-experimental comparison (between the train and non-train sides of the school) complemented with a removed-treatment design (after the train noise was abated). Combinations of such design features can increase confidence in causal attributions where statistical control alone might not. Similarly, the Hygge et al. (1996) study demonstrates a kind of “switching replication” design, whereby what was initially the treatment group (the schools near the airport) subsequently became the comparison group (after the airport had been moved near other schools) and vice versa.
In designing research studies to evaluate the unbiased effects of green schools on student or teacher outcomes, several additional issues must be addressed.
CONSIDERATIONS IN DESIGNING GREEN SCHOOLS-RELATED RESEARCH
Defining Green Schools for the Purpose of Scientific Inquiry
To date, green schools have been defined by their objectives and multiple design features rather than as an entity possessing a specified set of conditions common to all green schools. Considering green schools as a specific educational “intervention,” and therefore an object to be evaluated, presents significant limitations. For scientific inquiry, including evaluation, an intervention must be defined in a way that is replicable and consistent across studies to produce reliable information about the effects of green schools. Therefore, a definition that accurately and completely describes green schools as an intervention is needed for the purpose of scientific evaluation.
Current green school guidelines, which rely on achieving a minimum number of design features on a checklist, could be a starting point. However, because the checklist approach permits variation in design characteristics, the “green schools intervention” will not be consistent across schools and, therefore, is likely to produce less reliable effects.
As one alternative, specific design features associated with green school designs could be evaluated. This is likely to produce more reliable and less attenuated effect size estimates but would not represent the totality of the effects, including positive effects and negative side effects that may occur within green schools.
Another alternative is to define green schools by a set of performance measures, such as ventilation and lighting. The advantage of this type of definition is to bring the definition closer to the potential learning and health effects than a design-based definition. However, it would require an additional set of measurements to be taken.
Defining Performance and Productivity Outcomes Plausibly Related to Green Schools
Two types of outcome variables were set forth in the committee’s task statement: learning/productivity and health. These outcomes were expected to occur for two groups: students and teachers. Four dimensions of potential variables can be generated by a two-by-two matrix of outcomes: student learning, student health, teacher health, and teacher productivity.
Student learning is typically measured by student achievement on standardized tests. Green schools have the potential to affect student achievement in two distinct ways. First, better lighting and reduction of noise or other building features and characteristics may improve task performance by improving reception of the test stimuli or increasing a student’s ability to concentrate on the tasks. Second, longer term improvements in indoor environmental conditions could result in greater learning and perhaps greater retention of course content.
Laboratory experiments and studies from other environments on adults show affects on task performance. However, to establish effects on learning, carefully controlled studies using randomized experiments (that include baseline measures to verify that random selection had occurred) or econometric models are needed. The period of time between baseline and post-intervention measures must be sufficient for learning effects to have occurred. In addition, the baseline should be measured in a green school (improved lighting and reduced noise environment) to control for task performance effects in the differences. Student learning could be expanded to include other types of measures of educational progress, such as graduation or promotion to higher grades. Across any of these
outcomes, it would be important to use well-established, reliable, and valid measures.
Few school-based studies include direct measures of student health. Randomized experiments with observations of student health as outcomes could be conducted for learning outcomes, with few additional complexities. Understanding how the baseline health status of students affects learning and other performance outcomes is critical to the interpretation of effects attributable to green schools. Selection bias could be important if healthier children are more likely to attend a green school and the assignment to the green school is not done at random from a pool of eligible students.
For econometric studies, a health production function must be developed to fully specify the model. As a start, replacing prior student achievement with prior student health status may be considered, but theoretical work outside the scope of this review will be needed for such studies.
Surrogate measures for student health, such as student attendance, are potentially interesting. Student attendance has the potential for reverse causality with respect to student achievement; that is, poor performance may cause absences rather than better attendance yielding higher achievement. Therefore, attendance should be used as an outcome that is intrinsically important, not as a surrogate for student health.
Direct analogies can be drawn between studies of teacher health and interventions designed to improve worker health in other types of work environments. Studies could be developed to assess the impacts of green schools on teacher health using the aforementioned methodologies. However, since schools have at least an order of magnitude fewer teachers than students, fielding studies with a sufficient number of teachers to achieve an acceptable power to detect effects is likely to be an issue.
The effect of a school building on a teacher’s ability to improve student learning is principally based on the expectation that green schools result in improved teacher health which would, in turn, result in improved teacher attendance. Long-term teacher absences and the use of long-term substitutes have been associated with lower levels of student achievement
in some prior studies. However, given the lack of direct effects and current controversies about measuring teacher productivity, it may be more prudent to focus on the other three outcomes, at least until measures and methods for teacher productivity are developed.
Theory Relating Cause (Green Schools) to Effects
Currently, there is no fully developed theory explaining the links between achieving a green school design and producing health and learning effects. The studies cited as evidence in this report principally address links between exposures and health or student performance. Missing are studies that empirically link green building designs with performance and studies linking building performance changes of the magnitude that can be associated with green schools to learning and health.
There is also ambiguity about the direction of the effects of specific green school design attributes. For example, having and using windows for ventilation could produce: (1) air with lower CO2 levels; (2) higher noise levels; and (3) greater exposure to outdoor allergens. Increasing ventilation may improve the perception of air quality while also increasing background noise in a classroom. Similarly, natural lighting could produce better illumination and higher levels of glare. In each case, the consequences of a green school intervention appear to trigger mechanisms that could counteract each other in terms of the effects on educational outcomes. The potential for counterbalancing effects suggests the need for multidisciplinary evaluation teams to allow the full enumeration of plausible effects from green schools, both positive and negative, and develop intermediate outcome measures that may mediate the overall effects on learning and health.
Omitted Variable Bias
As described in Chapter 8, existing studies of the effects of overall school building condition on student learning are correlational, and do not control for selection of students into old schools or noisy schools. The studies do provide suggestive information for elaborating the models being tested to see if the effects are sustained when additional controls are added to the models. However, without fully specified econometric models or randomized experiments, omitted variable bias may be responsible for producing statistically significant effects. Evaluations using econometric models or randomized experiments would reduce the plausibility of omitted variable bias and yield estimates of the size of effects that could usefully inform decisions about green school design.
Power of the Evaluation to Detect Moderate Effects
Power refers to the ability of a study to find statistically significant effects of a particular size when those effects have in fact occurred. Because both students and teachers are nested within schools and therefore do not represent independent observations, power analysis must include the effects of the nesting, which is often referred to as clustering or cluster effects. Clustering inflates standard errors, which lowers the chances of finding statistical significance for samples of a given size. New power analysis tools (Raudenbush, 2003) indicate that as many as 50 schools could be needed to have a 75 percent probability of detecting a moderate effect size (0.30 standard deviation units), depending on the available pretreatment covariates. Given the cost of new school construction and the complications of obtaining district cooperation to assign students to new schools at random, randomized experiments are unlikely to be both feasible and sufficiently powerful to detect moderate-sized effects on student learning or health. The power issue may motivate use of the econometric approach or more piecemeal studies of the performance features that are likely to be associated with green schools and the building performance outcomes of green schools.
Level of Aggregation
Many studies of educational effects have relied on data that are aggregated or summed to the school level rather than data measured and analyzed at the individual level. Recent research has shown many more effects when micro- or individual-level data are used (e.g., Hanushek et al., 2003). Significant measurement issues are raised by aggregating state achievement tests, which are based on individual state standards, across states. When analyzing aggregate data, it is likely that evaluations can be mounted within a single state and unlikely that in the near future there will be sufficient cases in any state for adequate power to detect effects at the school level. Strong consideration should be given to micro-level econometric studies for overall evaluations of the effects of green schools.
In light of these considerations, it may not be feasible in the near future to fund randomized experiments to evaluate the effects of green schools on health and student learning. Econometric studies with micro-level data may be pursued when there are sufficient schools within a state and data are available on student achievement and other variables to estimate fully specified models. However, it may be necessary to carry out three types of studies using various designs to justify large-scale evaluations:
Studies to assess the building performance characteristics that result from building designs that are constructed to meet green school design standards.
Studies that correlate specific building performance characteristics of the type expected from green school designs with learning and health outcomes across schools.
Carefully controlled efficacy studies on specific schools where baseline data and counter-factual data can be collected and, ideally, students are randomly assigned to the school from a larger population of students.
If findings from these studies are considered sufficiently positive, more costly and exacting methodologies could be justified in pursuit of unbiased estimates of the effects of green schools on health and learning.
Finding 10a: Much is still not known about the potential interactions of building systems, materials, operation and maintenance practices and their effects on building occupants in general, or about school environments in particular. The necessary collaboration between architecture, engineering, physical science, medicine, and social science expertise is a challenge, but multidisciplinary research is required to fully study the potential relationship between a school building and the outcomes of students and teachers.
Finding 10b: In designing research studies to evaluate the unbiased effects of green schools on student learning or student and teacher health, several issues must be addressed. These include defining green schools for the purpose of scientific inquiry, defining performance and productivity outcomes plausibly related to green schools, and fully developing a theory explaining the links between green school design and health and learning effects. Finally, the hypotheses from these theories should be tested in ways that reduce systematic biases and provide compelling evidence about these linkages.
Finding 10c: Currently, the theory and evidence connecting green schools or characteristics associated with green schools to teacher or student outcomes is not sufficient to justify large-scale evaluations. However, the committee does consider it useful to carry out studies that assess the positive and negative consequences of the design and construction features as well as building performance characteristics that are associated with green schools using more rigorous study designs.
Finding 10d: Large-scale evaluations using randomized experiments and econometric or regression-based techniques should be conducted if they are justified from the results of smaller and less expensive studies, such as those outlined in Finding 10c. Finally, it is possible that improvements to the large-scale data sets that contain student achievement data will allow for relatively low cost studies of the effects of the school building environment on student achievement, which the committee considers to be an important side benefit.