Skip to main content

Currently Skimming:

Summary
Pages 1-8

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 1...
... The State Department uses the results of the FSI assessment to make decisions related to certification, job placement, promotion, retention, and pay. To help FSI keep pace with current developments in language assessment, the agency asked the National Academies of Sciences, Engineering, and Medicine to conduct a review of the strengths and weaknesses of some key assessment1 approaches that are available for assessing language proficiency2 that FSI could apply in its context.
From page 2...
... Specific choices for individual assessment methods and task types have to be understood and justified in the context of the specific ways that test scores are interpreted and used, rather than in the abstract: more is required than a simple choice for an oral interview or a computer-adaptive reading test. The desirable technical characteristics of an assessment result from an iterative process that shapes key design and implementation decisions while considering evidence about how the decisions fit with the specific context in which they will be used.
From page 3...
... Research in applied linguistics over the past few decades has led to a nuanced understanding of second and foreign language proficiency that goes well beyond a traditional focus on grammar and vocabulary. This newer perspective highlights the value of the expression of meanings implied in a given context, multiple varieties of any given language, the increasing use of multiple languages in a single conversation or context, and the recognition that communication in real-world settings typically uses multiple language skills in combination, frequently together with nonlinguistic modalities, such as graphics and new technologies.
From page 4...
... POSSIBLE CHANGES TO THE FSI TEST The committee considered possible changes to the FSI test that might be motivated in response to particular goals for improving the test. Such goals might arise from an evaluation of the validity of the interpretations and uses of the test, guided by a principled approach, which suggests particular ways the current test should be strengthened.
From page 5...
... SUMMARY 5 TABLE S-1  Possible Changes to the FSI Test to Meet Potential Goals Potential Test Construct, Reliability and Fairness Potential Instructional and Possible Change Considerations Practical Considerations Using Multiple Measures • Better coverage of Foreign • Additional cost for Service language uses test development and • Greater reliability and administration fairness Scoring Listening on the • More systematic use of • Potential for positive Speaking Test listening information effect on instruction already generated by the • Additional complexity to test the scoring process • Possibility of increased measurement error Adding Target-Language • Coverage of Foreign • Potential for positive Writing as a Response Service language uses that effect on instruction Mode for Some Reading or involve writing • Extra cost for test Listening Tasks development and administration Adding Paired or Group • Better coverage of Foreign • Potential for positive Oral Tests Service language uses effect on instruction related to interactional • Cost and practical competence challenges of coordinating • Possibility of increased tests measurement error due to partner variability Using Recorded Listening • Potential for better • Potential for positive Tasks That Use a Range generalization of listening effect on instruction of Language Varieties and assessment to typical • Increased cost for Unscripted Texts range of Foreign Service test development and contexts administration Incorporating Language • Better coverage of Foreign • Minor modifications to Supports (such as dictionary Service language uses current test and translation apps) Adding a Scenario-Based • Better coverage of • Potential for positive Assessment complex Foreign Service effect on instruction language uses • Increased cost for test development and administration continued
From page 6...
... In addition, the committee noted potential instructional and practical considerations related to these possible changes. CONSIDERATIONS IN EVALUATING VALIDITY Evaluating the validity of the interpretation and use of test scores is central to a principled approach to test development and use.
From page 7...
... One key aspect of professional standards is the importance of careful and systematic documentation of the design, administration, and scoring of a test as a good practice to help ensure the validity, reliability, and fairness of the interpretations and decisions supported by a testing program. BALANCING EVALUATION AND THE IMPLEMENTATION OF NEW APPROACHES At the heart of the FSI's choice about how to strengthen its testing program lies a decision about the balance between (1)
From page 8...
... In requesting this report, FSI has clearly chosen a forward-looking strategy. Using this report as a starting point and thinking deliberatively about these complicated choices, FSI could enhance its assessment practices by improving its understanding of the test construct and how it is assessed; the reliability of the test scores and the fairness of their use; the potential beneficial influence of the test on instruction; and the understanding, usefulness, and acceptance of the assessment across the State Department community.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.