Approaches to the Development of Character Proceedings of a Workshop (2017) / Chapter Skim
Currently Skimming:
6 Measuring Character
6 Measuring Character
Pages 63-76
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
Noel Card of the University of Connecticut reviewed methodological challenges in the measurement of character and proposed suggestions for addressing them. Discussants Clark McKown of Rush University Medical Center and Nancy Deutsch of the University of Virginia offered two additional perspectives.
|
From page 64... ...
For example, he commented, a survey or interview might include multiple questions about an aspect of character to "get at it in different ways." Measures of internal consistency indicate how "repeatable" these different questions are -- the degree to which they are measuring the same thing, based on how similar the scores are across the group of questions. Psychometricians tend to "turn up their noses" at this type of reliability, Card explained, because it is based on assumptions about how parallel the questions truly are.
|
From page 65... ...
The arrows across the bottom circle represent the way a carefully chosen set of items will "triangulate around the bullseye," Card explained. The distribution of the dots in Figure 6-1 represents a situation in which there is relatively low internal consistency reliability.
|
From page 66... ...
That is, an item performs differentially if it yields different results for individuals who in reality have comparable knowledge or skills only because something about the way it is designed favors one individual over the other. Items that are equivalent across time allow researchers to accurately assess changes that result from an intervention, for example.
|
From page 67... ...
Implications for Measuring Character A clear definition of what is to be measured is important for each of these basic psychometric properties, Card commented. In the arena of character, he noted, there are multiple definitions of many constructs, and operational definitions often have "fuzzy boundaries." This is not a problem in itself but it means that measures or scales might more closely match one definition than another in ways that are difficult to trace.
|
From page 68... ...
• Intervention leads to higher prosocial behavior and impacts the measurement so intervention effect is hidden. • Intervention reduces prosocial behavior but heightens reporting, obscuring the harmful impact of the intervention.
|
From page 69... ...
Some researchers may develop a measure without adequately testing its psychometric properties, Card added, which may be one reason why so many researchers in this area have chosen not to rely on the previous literature. 1 There are several categories of validity; construct validity is the type Card had described, a measure of the degree to which an assessment measures what it is intended to measure.
|
From page 70... ...
Studies that include detailed psychometric analysis tend to be less interesting to publishers, he noted, but it is important that this information be available, perhaps as a component of a larger study. "We need to shift perceptions," Card suggested, so that people in the field value psychometric analysis and "don't just view it as a preliminary to more interesting results." Meta-analysis of a body of research results is an important tool for identifying findings that reflect a much larger sample than any single study could, as well as a more diverse set of study settings and types of measures.
|
From page 71... ...
They are also measuring mental and behavioral components of self-control. McKown and his colleagues made firm decisions about constructs that would and would not be included in their work, which, he argued, allowed them to develop a measurement system that meets the psychometric standards Card described.
|
From page 72... ...
in their accountability system for schools -- an effort McKown said was "bold and important." However, a public controversy developed over the concern that using measures that are based on self-reports for this sort of high-stakes purpose is not good measurement practice. Reasonable people can disagree about whether the indicators that CORE Districts used provide evidence that is valid basis for accountability decisions, McKown commented, but the situation highlights the importance of consequential validity for all measures of character and social and emotional learning.
|
From page 73... ...
Social science researchers focus much of their energy on defining, undertanding, and measuring constructs and how interventions might s influence them, Deutsch continued, yet they seldom question some of the assumptions that underlie these descriptions of human behaviors and thinking. In particular, researchers rarely "acknowledge that all constructs are social constructions." Ideas about character that underlie the constructs researchers measure in this field differ across contexts, and this is important to the study and measurement of character, Deutsch commented, Psychometric principles such as validity and reliability are formal acknowledgments that measurement tools are subject to error, Deutsch continued, yet they also reflect an assumption that "something called character exists as a singular, objective fact." The measurement challenge is generally understood to be one of getting as close as possible to assessing this true construct, without, in Deutsch's view, sufficient regard for the social, historical, and political influences on that construct.
|
From page 74... ...
For example, some research has shown that approaches to social and emotional learning designed to be neutral may inadvertently reflect misunderstanding of the diverse ways students from marginalized backgrounds express social and emotional skills. Redefining the relevant social and emotional constructs in ways that reflect awareness of conscious and unconscious biases and the role of power and privilege, Deutsch explained, leads to the use of different measures and altered ideas about students' behavior and need for intervention.
|
From page 75... ...
In the social sciences, qualitative analysis examines the components of a social setting: the context. Qualitative research can help explain the meaning of outliers and sources of variation that are apparent in quantitative research.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.