National Academies Press: OpenBook

Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide (2023)

Chapter: Chapter 5 - Guidance on Ways to Grow Evaluative Thinking

« Previous: Chapter 4 - Guidance on Identifying, Selecting, and Adapting Countermeasures and Strategies
Page 23
Suggested Citation:"Chapter 5 - Guidance on Ways to Grow Evaluative Thinking." National Academies of Sciences, Engineering, and Medicine. 2023. Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide. Washington, DC: The National Academies Press. doi: 10.17226/27197.
×
Page 23
Page 24
Suggested Citation:"Chapter 5 - Guidance on Ways to Grow Evaluative Thinking." National Academies of Sciences, Engineering, and Medicine. 2023. Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide. Washington, DC: The National Academies Press. doi: 10.17226/27197.
×
Page 24
Page 25
Suggested Citation:"Chapter 5 - Guidance on Ways to Grow Evaluative Thinking." National Academies of Sciences, Engineering, and Medicine. 2023. Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide. Washington, DC: The National Academies Press. doi: 10.17226/27197.
×
Page 25

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

23   Guidance on Ways to Grow Evaluative Thinking You and those working with you will invest significant time and resources in planning and implementing countermeasures and strategies to improve roadway safety. Taking even small evaluation steps can reveal ways that you can be more effective. According to the American Evaluation Association, “evaluation involves assessing the strengths and weaknesses of strategies to improve their effectiveness” (American Evaluation Association 2019). By understanding the strengths and weaknesses of a countermeasure or strategy, you can make changes in the future to make the countermeasure or strategy more effective. There are several reasons for evaluating countermeasures and strategies. Arguably, the most important are to ensure systematic learning and to provide needed information in order to refine and improve countermeasures and strategies. Other reasons for evaluation include • Assessing effectiveness and informing good management practices by – Comparing actual implementation with implementation plans and understanding reasons for deviations, – Comparing actual outcomes with intended outcomes, – Comparing outcomes with those of previous years, and – Establishing realistic intended outcomes for future performance. • Fostering sustained improvements in roadway safety by – Focusing attention on issues important to the effectiveness of the countermeasure or strategy, – Promoting a countermeasure or strategy by documenting and sharing its effectiveness, – Recruiting new partners who want to join in contributing to effective countermeasures and strategies, – Enhancing the image of the countermeasure or strategy, – Sustaining or increasing funding, – Providing direction and informing training for staff and partners to implement the counter- measure or strategy effectively in the future, – Informing what training and technical assistance is needed to improve effectiveness, – Informing long-range planning, and – Justifying the investment of resources by legislators or other stakeholders by showing the countermeasure or strategy is effective. A good first step is to create a logic model for the countermeasure or strategy (see Chapter 2). You can then identify things that you want to monitor and measure. Two forms of evaluation could be particularly helpful: • Process evaluations: Process evaluations examine the way the countermeasure or strategy was implemented. Using the logic model, process evaluations ask questions about the outputs. Were the activities conducted as planned and with fidelity to any available guidance? Was the C H A P T E R   5

24 Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide appropriate audience reached, and to what extent? Following are some specific examples of process evaluation questions: – If the countermeasure or strategy called for 5 days of enhanced visible enforcement, did that actually happen? How much enforcement occurred each day? If there was a program in the school, did the teacher complete all four modules? – Did the countermeasure or strategy reach a sufficient portion of the population to make a difference? For example, did most people in the community know about the enhanced enforcement? Did the messages sent out actually reach people? With regard to the school program, what percentage of students received all four modules? • Outcome evaluations: Outcome evaluations measure the effect of the countermeasure or strategy on creating change. In the logic model, outcome evaluations ask questions related to the short-term, intermediate, and long-term outcomes. An outcome evaluation could assess the degree to which the countermeasure or strategy changed beliefs. An outcome evaluation could also assess the degree to which there was a change in behavior. Following are some specific examples of outcome evaluation questions: – Did people’s perception of getting caught for not using a seat belt increase? – Did students’ knowledge change as a result of the school program? – Did seat belt use change among the focus population? No matter what type of evaluation you pursue, evaluation data and what was learned should be shared with the team and used to explore ways of improving future implementation. Suggestions for process and outcome evaluations can be categorized by general approach: laws, policies, and rules; enforcement and adjudication; education; and intervention. Laws, policies, and rules: • Keep track of how many workplaces or organizations adopt policies about safe driving. Ask how many employees the organizations have and estimate what proportion of the community now has a safe driving policy at their place of work. • Ask organization leadership what, if any, other activities they conduced to promote new policy or train employees. Add a question or two to workplace surveys to ask about awareness of the policy and driving behavior. Monitor changes in awareness and behavior over time. • Ask families if they have family rules about safe driving practices. Work with the local school to add a question or two to an existent survey or create a short survey to include with back- to-school paperwork. Monitor the percentage of families with rules over time. Enforcement and adjudication: • Track the number of officers working on a high-visibility campaign and the number of shifts or hours spent on high-visibility enforcement. • Survey law enforcement officers before and after training (pre- and postdesign) to assess their knowledge and confidence about detecting impaired drivers. Education: • Track the quantity and quality of specific content (whatever the topic of interest is—seat belts, speeding, sharing the road with agricultural equipment) in driver’s education classes. Monitor for changes over time. • Conduct community surveys at selected locations (e.g., Post Office, Department of Motor Vehicles, grocery store) to ask people whether they have seen the media campaign. Vary days, times, and locations to try to get a wide cross section of community members. Do a few surveys before the campaign starts to determine the baseline and then monitor for changes over time. • Survey students who participate in the Seat Belt Convincer before and after the experience to determine whether it led to any changes in attitudes or planned behavior. Do a follow-up survey to see if changes persist.

Guidance on Ways to Grow Evaluative Thinking 25   Intervention: • Track the number and type of cases in impaired driving court. • Survey community members before, during, and after activities that promote bystander engagement to assess their comfort in asking others not to drive after drinking and whether they have asked anyone to not drive in the past month. Monitor for changes over time. There may be skilled evaluators in your community. Evaluators often work across a variety of subjects, and community-oriented evaluators may be eager to lend their expertise to local efforts to improve roadway safety. You might consider reaching out to such individuals and invit- ing them to join the coalition. The American Evaluation Association has a directory resource (https://my.eval.org/find-an-evaluator) that may be helpful if you are looking for an evaluator partner. Box 7 lists additional resources for learning more about evaluation and evaluative thinking. Box 7. Resources for Learning More About Evaluation and Evaluative Thinking Guidance for Evaluating Traffic Safety Culture Strategies https://www.mdt.mt.gov/research/projects/trafficsafety-strategies.aspx This site includes a comprehensive report, a guidance document, and a poster providing more in-depth information about evaluating roadway safety culture strategies. Evaluation and Enhancement of Community Outreach Programs https://injury.research.chop.edu/teen-driving-safety/evaluation-community -outreach-programs-teen-driver-safety This site offers resources from the Children’s Hospital of Philadelphia Center for Injury Research & Prevention, in partnership with the Pennsylvania DOT, including a guide, videos, and additional resources. The information can be applied to total roadway safety and includes examples related to teen driver safety. Community Toolbox Evaluating the Initiative Toolkit https://ctb.ku.edu/en/evaluating-initiative The Community Toolbox is a thorough resource on community development and covers everything from coalition development and community assessment to program implementation and evaluation. Some of the resource material is intended for health improvement efforts, and since health improvement efforts often focus on human behavior, the information can be readily applied to evaluation of behavioral countermeasures and strategies to improve roadway safety. This evaluation toolkit is just one of many resources included in the Community Toolbox.

Next: Acronyms and Abbreviations »
Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide Get This Book
×
 Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide
Buy Paperback | $68.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Roadway fatalities and serious injuries are a significant public health concern in rural and tribal settings. Creating a coalition of interested individuals is part of the Safe System Approach that addresses the high rates of these fatalities and serious injuries.

BTSCRP Research Report 8: Highway Safety Behavioral Strategies for Rural and Tribal Areas: A Guide, from TRB's Behavioral Transportation Safety Cooperative Research Program, details this approach, which includes strategies for safer people, safer roads, safer vehicles, safer speeds, and post-crash care.

Supplemental to the report are BTSCRP Web-Only Document 4: Highway Safety Behavioral Strategies for Rural Areas and a video that explains how to create a logic model.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!