IMPLEMENTATION OF SCHOOL STANDARDS: EVALUATING PROGRESS AND IMPACT
The recommended nutrition standards are one of several elements of a school nutrition policy that could significantly improve the nutritional quality of the foods available, promoted, and consumed in schools. To be effective, these standards must be implemented by a wide range of organizations and individuals: they must understand what is expected of them, and will need resources and support. A system is needed to track progress, resolve bottlenecks in implementation, and evaluate both the implementation and documentation and the resulting changes. While proposing a complete implementation and evaluation plan is beyond the scope of this committee, this chapter provides a framework and a set of benchmarks on which such a plan can be developed.
A Framework for Implementation
One way to identify the implementation requirements and the elements that should be tracked over time is to focus on the major steps needed from the time of the report release until changes in the diets of children during the school day have been established. Box 6-1 identifies four steps: awareness and understanding by diverse organizations; stakeholders decisions to implement the recommended nutrition standards by these organizations; changes in food and beverage availability in schools; and changes in the
Key Elements for Success
dietary intake of children during the school day. Each of these is elaborated upon below, including specific suggestions for implementation and tracking of progress over time.
Awareness and Understanding of Standards
Awareness and understanding of the standards by personnel in schools, school boards, school district administrators, state agencies, state boards of education and legislatures, Congress, the U.S. Department of Agriculture (USDA), the U.S. Department of Health and Human Services (DHHS), food and beverage industry and vendors, as well as health professionals, child advocates, parents, and children are essential to implementation.
In recent years, many schools and local, state, and federal legislatures, as well as other agencies, have implemented a variety of model nutrition standards. The first step, therefore, is for the committee’s recommendations to be disseminated and compared to current standards. In many cases, this
may be a complex process because of the level of detail involved and investments already made in the current standards. In order for dissemination to be effectively carried out, states and school districts must harmonize their current standards with the new standards, to the greatest extent possible. Doing so will simplify many aspects of the on-going implementation, enforcement, and evaluation of nutrition standards and encourage industry to develop healthful products for the school setting. For example, making the recommendations (and simplified tools/resources based on them) available to the full range of school personnel will facilitate discussion of the recommendations at federal, state, and local levels, as well as school- and community-level promotional activities and training of local school staff.
Decisions and Actions
The second key element for success comprises positive decisions and actions taken to implement the standards, including
supportive legislation at federal, state, and/or local levels;
supportive regulations issued by federal, state, and/or local agencies;
incorporation of standards into school wellness policies; and
development of food and beverage products that meet the standards.
Until recently, foods and beverages served outside the federally reimbursable school nutrition programs have been largely unregulated in school settings. To be effective, the committee’s recommendations will require a wide range of actions by legislatures, school boards, schools, government agencies, nonprofit organizations, and the food industry.
Taking steps to ensure that model nutrition standards and model school policies, as produced by many organizations in recent years, harmonize with the committee’s recommended standards to the greatest extent possible will facilitate implementation and avoid confusion. By collaborating with the food and beverage industry, federal agencies will formalize criteria and labels for products for the school setting; develop similar criteria for whole-grain products; develop consistent criteria for combination products; and identify the amount of added sugars on the nutrition facts panel on labels as recommended in Chapter 5.
Changes in Food Availability
Positive changes in food and beverage availability and practices in schools include
products offered à la carte and via vending machines, stores, and snack bars are consistent with the recommended standards; and
products used in celebrations, fund-raising, and after-school activities are consistent with the standards.
While many of the foregoing identify decisions and actions needed at the state and federal levels, the success with which the recommended standards are translated into improved diets for school-age children depends on the ability and willingness of schools and school districts to implement them. Actions that facilitate this process are discussed below.
A responsible party such as the school wellness policy administrator may assume the task of ensuring compliance with nutrition standards. This work may be facilitated by the development of federal or state programs such as a Nutrition-Friendly Schools Initiative to encourage compliance and certify those schools that have fully complied with the recommended nutrition standards. Compliance can be verified and progress assessed by health educators, local health departments, or other outside stakeholders with similar expertise. In addition, schools might seek assistance from relevant state agencies, Centers for Disease Control and Prevention (CDC) staff, USDA regional offices, and qualified local organizations and agencies. Such assistance and collaboration would be greatly facilitated if state and federal agencies specifically earmarked funds for this purpose.
Changes in Food Sources
Positive changes in the food and beverage sources during the extended school day, including reformulating products to comply with the recommended standards and improving the nutrient composition of school-age children’s diets, are the expected outcomes from implementation of the recommended nutrition standards. However, to ensure the success of the intended outcomes, it is important that benchmarking become a part of the implementation process.
One of the committee’s assigned tasks was to develop benchmarks to guide future evaluation studies of the application of the recommended standards. In this report, the term “benchmarks” refers to the key elements of standards implementation and impact that would be most useful to evaluate.
The committee anticipates that the potential impact of the standards will be carefully considered by school districts, researchers from a number of related disciplines, and government agencies at the state and federal lev-
els. Evaluation at the local level may enable school districts to document successes and to identify potential problems and devise solutions. Broader evaluation at the state and federal levels, and evaluations that may be more scientifically rigorous, will help to inform local, state, and national policy action on the recommended nutrition standards for schools. The evaluation process can be as simple as the development of a descriptive assessment checklist for key stakeholders in local school districts, or as complex as a meticulous pre- and post-intervention appraisal that is conducted as part of university research or as part of a broader evaluation at the state or federal level.
School districts, states, stakeholder organizations, and researchers may wish to discuss the potential for these evaluation options at the time that they plan the implementation of the new policies. It is advisable to examine the impact of the standards after they have been in place for at least one school year.
Ease and Extent of Nutrition Standards Adoption
In order for the recommended nutrition standards to be adopted, key decision-makers must be informed of the policy and understand its contents and likely outcomes. These decision-makers include some or all of the following individuals and groups: school personnel, especially those responsible for food acquisition and preparation; parents and parent organizations; school boards; school district administrators; state agencies and legislatures; key members of Congress and their staffs; federal agencies, especially USDA and CDC; and food and beverage producers and vendors. Important benchmarks to consider are the breadth and depth of the awareness and understanding of the nutrition standards by stakeholders and policymakers, and the extent to which the full policy is adopted and accepted. Key issues that might be examined include the use of the specific standards for foods and beverages; place-and-time rules; determination of whether the standards were used according to age group; observations on the practicality/impracticality of various aspects of the policy; and views on policy implementation barriers at the district or school level.
Other related questions may include, How well accepted is the policy among students (at different ages), parents, teachers, and the broader community? How are policies monitored and by whom? How well enforced is the policy in the day-to-day world of school? What are the specific enforcement issues, if any? What school-related factors explain the ease or difficulty of change?
Other benchmarks may include cataloging decisions at all levels of government that implement the recommended nutrition standards. Appropriate questions to consider are the following:
Were new regulations or policies to implement part or all of these standards issued by local, state, or federal agencies?
Was legislation requiring part or all of these nutrition standards adopted at the local, state, or federal levels?
The federally mandated local wellness policies and CDC school health program promotion initiatives both provide an excellent opportunity for school districts to set nutrition standards that only allow the offering of healthy foods and beverages. Were these standards incorporated in part or in full in local or state school wellness policies, or as part of the comprehensive coordinated school health programs recommended by CDC?
Response of Food Producers, Manufacturers, and Vendors
In order for the recommended nutrition standards to be successful, food producers, manufacturers, and vendors must supply the amounts, kinds, and forms of the fruits, vegetables, whole grains, and nonfat and low-fat dairy products needed. Manufacturers and vendors are the source of fruits, vegetables, and whole-grain and combination products that meet the recommended standards. Therefore, in some cases, foods and beverages may need to be reformulated or uniformly portion-packaged to comply with these standards. Entirely new products may also be developed. Examining whether those manufacturers that supply foods and beverages to schools are willing to match product offerings to schools’ needs under the new standards is the role of evaluators of the implementation of the standards.
Moreover, it will be important to assess and recognize whether food and beverage providers of all types engage in any innovative marketing practices in schools to promote the healthful foods and beverages offered under the new nutrition standards. A related issue of interest will be whether local producers involved in “farm-to-school” efforts are able to take advantage of the changes in the nutrition standards to incorporate local produce and other products into the foods and beverages offered in the school setting.
Foods and Beverages Available in Schools
The standards cover foods and beverages offered à la carte, in vending machines, in student stores, and in snack bars, as well as those used in celebrations, in fund-raising, and during after-school activities. An examination of the actual foods and beverages available to students of different ages, in different school venues, and at different times of day will be an essential benchmark to determine if the standards change food availability in a positive way if new products are introduced that violate the spirit of
the standards, or if “black marketing” of foods and beverages by students becomes a common practice.
Impact on Children’s Diets
The ultimate goal of the recommended nutrition standards is to contribute positively to the nutritional quality and healthfulness of diets of school-age children and to the food habits they develop. Benchmarks in this area will include whether changes in the foods and beverages consumed during the school day and improvements in nutritional intake at school occurred.
Are children more likely to consume federally reimbursable school breakfasts and lunches and after school-snacks?
Are there changes in à la carte or vending sales or in the amounts and kinds of food purchased off campus or brought from home?
Do children consume increased numbers of servings of fruits, vegetables, nonfat and low-fat dairy products, and whole grains as a result of the implementation of these standards?
Do children consume less total, saturated, and trans fat; sodium and added sugar; and more fiber? Do they consume more water and fewer soft drinks?
Is excess caloric intake reduced as a result of the implementation of the standards?
Is there a significant increase in intakes of nutrients of concern?
In addition to the above, it will be useful for studies to examine whether the overall dietary intakes of children and/or the distributions of body mass indexes are affected positively by these changes. The tracking of these outcomes at local and state levels may also be useful for overall monitoring and community awareness purposes. However, users of such data must bear in mind that such outcomes reflect a much wider set of behavioral and environmental factors at home and in the community, and are not to be expected to change merely in response to improvements in the school food environment. Positive changes in dietary intake have been shown to improve health. However, in evaluating the impact of the standards, the general focus should be on whether dietary intake has changed as a result of differences in the kinds of foods and beverages made available during the school day.
Finally, it would be helpful to know whether there is an overall impact on those students who were most likely to consume competitive foods and beverages before the implementation of the recommended nutrition stan-
dards. This information will help in determining if changes in the kinds of offerings lead to greater improvements in their nutritional intake at school and if there was a spillover effect on their food attitudes or preferences beyond the school day.
Impact on School Operations and Finances
Major changes in one aspect of school operations can have related effects on other aspects of the school—positive and negative, expected and unexpected. It will be useful to examine what these effects may be. One is the potential impact on the overall school budget and on school food service revenues. Are revenues lost or gained, and in which areas of the budget? What are the impacts of these changes, and, in the case of lost revenue, if any, what changes have been made to reverse or adjust to the loss? Implementation of the recommended standards probably will influence other aspects of school operations, and these changes also could be examined and reported as part of the benchmarking process.
Programs That Track Progress
Examples of programs that track the progress of evaluation activities once they are put into practice include the CDC Youth Risk Behavior Surveillance System (YRBSS) and School Physical Activity and Nutrition (SPAN) survey. The YRBSS gathers data on six categories of priority health-risk behaviors among children and young adults, including unhealthy dietary behaviors, and physical inactivity. The YRBSS also monitors general health status and the prevalence of overweight and asthma in children and adolescents. The program includes a national school-based survey conducted by CDC and state and local school-based surveys conducted by state and local education and health agencies. Examples of ways that the national YRBSS data are used by CDC and other federal agencies include assessing trends in priority health-risk behaviors among high school students, and monitoring progress toward achieving fifteen Healthy People 2010 health objectives. State and local agencies and nongovernmental organizations use YRBSS data to set school health and health promotion program goals, such as wellness policies; support modification of school health curricula programs; support new legislation and policies that promote health; and seek funding for new initiatives.
The SPAN survey was developed to assess nutrition behaviors, attitudes and knowledge, and physical activity behaviors among 4th, 8th, and 11th grade students. Example applications of the SPAN survey include SPAN 2000–2002 and 2004–2005 studies, the Houston-Harris County STEPS Consortium, the Travis County CATCH program, and Robert Wood John-
son Foundation projects. These programs illustrate how the benchmarks identified in this report could be incorporated into federal, state, and local systems to track the progress or outcomes from implementation of the recommended standards and actions.
SOME FINAL THOUGHTS
Although there are uncertainties about the optimal implementation process of the recommended nutrition standards, and their ultimate impact on the potential outcomes described in this chapter, the committee is confident that implementation will contribute greatly to an overall healthful eating environment in U.S. schools. Already, there are many success stories from schools, school districts, and states that have implemented standards similar to the ones proposed by the committee. Enhanced collaboration among the many affected groups, with sharing of different implementation strategies and outcome data, will provide guidance for ongoing and future efforts to improve the dietary intake and food habits of U.S. children.