A Principled Approach to Language Assessment Considerations for the U.S. Foreign Service Institute (2020) / Chapter Skim
Currently Skimming:
3 Language Constructs and Language Use
3 Language Constructs and Language Use
Pages 27-40
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 27... ...
The goal of defining the construct is to provide a basis not only for the development of a given assessment but also for the interpretation and use of its results. For FSI, the construct will relate to descriptions of the language proficiency of Foreign Service officers who need to use a given language at a foreign post.
|
From page 28... ...
To address FSI's desire to keep pace with developments in language assessment, this section summarizes four key approaches to defining language proficiency and their implications for the design and operationalization of test constructs and for the meaningful interpretation of performance on a test: trait-based, interactionist, meaning-oriented, and task-based. This summary illustrates the expansion of the construct of language proficiency over time, but the committee is not suggesting that all assessments should use the broadest measure possible.
|
From page 29... ...
It has been used, for example, to guide the development of the Canadian Language Benchmarks Standards for adults learning English as a second language.3 Alongside language knowledge, this model also specifies the role that strategic processing plays in the ability to complete language-related tasks, which underlies the examinee's ability to consider context, content, language, and dispositions while generating responses during a test, all considerations in the skill-level descriptions used by FSI. Interactionist Approach Despite its strengths, the trait-based approach does not fully specify how language ability is affected by the context of language use.
|
From page 30... ...
The FSI test is a form of language test for specific purposes in that many of the tasks reflect characteristics of the Foreign Service context and require test takers to engage language and content knowledge specific to Foreign Service work. Meaning-Oriented Approach Extending the interactionist approach, a meaning-oriented approach to the construct definition of language proficiency added "the ability to effectively express, understand, dynamically co-construct, negotiate, and repair variegated meanings in a wide range of language contexts" (Purpura, 2017, p.
|
From page 31... ...
, it is called a task-based approach to construct definition.4 The task-based approach seeks to create assessment conditions that approximate real-life contexts in which the tasks "replicate typical task procedures, content, situations, interlocutors, and other factors, in order to provide trustworthy indications of the extent to which learners can handle the real-world requirements of task performance" (Norris, 2016, p.
|
From page 32... ...
Important dimensions of the socio ultural perspective include the value of multiple varieties of any given c language use, the increasingly multilingual nature of language use, and the recognition that communication is multimodal. The idea of the value of multiple varieties5 of any given language reflects an important shift in assessment away from the notion of a "native speaker" as the gold standard of language proficiency (Davies, 2003; Dewaele, 2018)
|
From page 33... ...
For example, the TOEFL iBT now uses varieties of English from North America, the United Kingdom, New Zealand, and Australia as test inputs. Some language testing researchers also are beginning to design language tests that include translanguaging components in the test input and that allow for translanguaging in the response, thus "enabling test takers to draw on their entire repertoires as multilingual persons, and more authentically representing and valuing the translanguaged reality of current workplace language practice" (Baker and Hope, 2019, p.
|
From page 34... ...
In two sections of the test, test takers are required to use two languages: in the interview section of the speaking test, they interview the tester in the tested language and report what they learn to the examiner in English; in the reading in depth section of the reading test, they read a passage on a specialized topic in the target language and then summarize it orally in English. These tasks likely also occur in a similarly multilingual way in the daily work of Foreign Service officers.6 LANGUAGE USE IN PROFESSIONAL SETTINGS Moving from considering language use in a broad sense to its use in a specific work-related or professional context -- in FSI's case, the use of language in Foreign Service tasks -- raises a separate set of assessment issues.
|
From page 35... ...
However, a few notable methods that could be used in a foreign language assessment context to infer specific language demands from known job demands include
|
From page 36... ...
Job Analysis and Language Assessment for Professional Purposes Target language use analysis is an examination of tasks and knowledge, skills, abilities, and other characteristics that are relevant to the development of language tests (Bachman and Palmer, 1996, 2010)
|
From page 37... ...
could be built based on information combined from job analyses and target language use analyses. In a work setting, developers can identify subskills and stimulus situations directly from job analysis -- using tasks and knowledge, skills, abilities, and other characteristics -- and weight these elements according to their importance to overall job functioning, creating linkages that support validity argumentation.
|
From page 38... ...
Sampling from these different language varieties to develop a language assessment for professional purposes involves careful analysis of the professional context (the target language use domain) and the purpose of the assessment (Knoch and Macqueen, 2020)
|
From page 39... ...
Traditional assessment design frameworks often focus on the context, elicitation, and proficiency dimensions of assessments. However, many fail to explicitly address these other factors in the design stage, even though they can affect performance.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.