Skip to main content

Currently Skimming:

2 Assessing Cognitive Skills
Pages 15-38

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 15...
... . It includes creativity to generate new and innovative solutions, integrat ing seemingly unrelated information, and entertaining possibilities that others may miss (Houston, 2007)
From page 16...
... DEFINING THE CONSTRUCT One of the first steps in developing an assessment is to define the construct and operationalize it in a way that supports the development of assessment tasks. Defining some of the constructs included within the scope of 21st century skills is significantly more challenging than defining more traditional constructs, such as reading comprehension or mathematics computational skills.
From page 17...
... examined the body of research on the relationships between measures of general cognitive ability (historically referred to as IQ) and job outcomes, 2 Respectively, the Graduate Record Exam, Medical College Admission Test, Law School Admission Test, Graduate Management Admission Test, Miller Analogies Test, and Pharmacy College Admission Test.
From page 18...
... All of the correlations are above .30, which Kuncel characterized as demonstrating a strong relationship between general cognitive ability and job performance across a vari ety of performance measures. Together, Kuncel said, these two reviews present a body of evidence documenting that verbal and quantitative skills along with general cognitive ability are predictive of college and career performance.
From page 19...
... For instance, a domain-general concept of the construct of "critical thinking" is often indistinguishable from general cognitive ability or general reasoning and learning skills. To demonstrate, Kuncel presented three definitions of critical thinking that commonly appear in the literature: 1.
From page 20...
... . With regard to discriminant validity, Kuncel conducted a literature search that yielded 19 correlations between critical-thinking skills and traditional measures of cognitive abilities, such as the Miller Analogies Test and the SAT (Adams et al., 1999; Bauer and Liang, 2003; Bondy et al., 2001; Cano and Martinez, 1991; Edwards, 1950; Facione et al., 1995, 1998; Spector et al., 2000; Watson and Glaser, 1994)
From page 21...
... Given this relatively weak evidence of convergent and discriminant validity, Kuncel argued, it is important to determine if critical thinking is correlated differently than cognitive ability with important outcome variables like grades or job performance. That is, do measures of critical-thinking skills show incremental validity beyond the information provided by measures of general cognitive ability?
From page 22...
... He said this conception is not useful, and he summarized his meta-analysis findings as demonstrating little evidence that critical thinking exists as a domain-general construct distinct from general cognitive ability. He highlighted the fact that some may view critical thinking as a specific skill that, once learned, can be applied in many situations.
From page 23...
... Kuncel said it is possible to train specific skills that aid in making good judgments in some situations, but the literature does not demonstrate that it is possible to train universally effective critical thinking skills. He noted, "I think you can give people a nice toolbox with all sorts of tools they can apply to a variety of tasks, problems, issues, decisions, citizenship questions, and learning those things will be very valuable, but I dissent on them being global and trainable as a global skill." Transfer from One Context to Another There is a commonplace assumption, Eric Anderman noted in his presentation, that learners readily transfer the skills they have learned in one course or context to situations and problems that arise in another.
From page 24...
... Second, the use of general practices intended to improve students' thinking are not usually successful as a means of improving their overall cognitive abilities. The research suggests students may become more adept in the specific skill taught, but this does not transfer to an overall increase in cognitive ability.
From page 25...
... . Further, the PISA 2012 assessment of problem-solving competency will not test simple reproduction of domain-based knowledge, but will focus on the cognitive skills required to solve unfamiliar problems encountered in life and lying outside traditional curricular domains.
From page 26...
... The first involves information retrieval. This process requires the test taker to quickly explore a given system to find out how the relevant variables are related to each other.
From page 27...
... Scoring of the items is based on the processes that the test taker uses to solve the problem and involves awarding points for the use of certain processes. For information retrieval, the focus is on identifying the need to collect baseline data (referred to in PISA terminology as identifying the "zero round")
From page 28...
... Partial credit is given if some progress toward the target goals can be registered, and no credit is given if there is no progress toward target goals at all. PISA items are classified as static versus interactive.
From page 29...
... The Bar Examination for Lawyers12 The Bar examination is administered by each jurisdiction in the United States as one step in the process to license lawyers. The National Council of Bar Examiners (NCBE)
From page 30...
... Examinees pay to take the test, and the costs are $54 for the MBE, $20 for the MEE, and $20 for the MPT. Susan Case, who has spent her career working on licensing exams -- first the medical licensing exam for physicians and then the bar exam for lawyers -- noted the Bar examination is like other tests used to award professional licensure.
From page 31...
... demonstrate an understanding of the fundamental legal principles relevant to the probable resolution of the issues raised by the factual situation. The MEE lasts for 6 hours and consists of nine 30-minute questions.
From page 32...
... At this time, they review sample papers and receive training on how to apply the scoring guidelines in a consistent fashion. Each component of the Bar examination (MBE, MEE, MPT)
From page 33...
... (Acquiring Research Investigative and Evaluative Skills) The summary below is based on materials provided by Art Graesser, including his presentation13 and two background papers he supplied to the committee (Graesser et al., 2010; Millis et al., in press)
From page 34...
... The student is engaged through the Auto Tutor, which involves a "tutor agent" that serves as a narrator, and a "student agent" that serves in different roles, depending on the skill level of the student. The system makes use of three kinds of modules -- interactive training, case studies, and interrogations.
From page 35...
... Packet Tracer The summary below is based on materials provided by John Behrens, including his presentation14 and a background paper he forwarded in preparation for the workshop (Behrens et al., in press)
From page 36...
... In the Networking Academy, students work with an online instructor, and they proceed through an established curriculum that incorporates numerous interactive activities. Behrens talked specifically about a new program Cisco has developed called "Packet Tracer," a computer package that uses simulations to provide instruction and includes an interactive and adaptable assess ment component.
From page 37...
... The instructors can receive and view the log in order to evaluate how well the student understands the tasks and what needs to be done. Packet Tracer can simulate a broad range of devices and networking protocols, including a wide range of PC facilities covering communication cards, power functionality, web browsers, and operating system configurations.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.