APEC logo, HRWG

Human Resources
Development
Working Group

Personal tools
Views

Language Assessment

From APEC HRDWG Wiki

Jump to: navigation, search

Contents

Overview

Language assessment is the way to measure student learning of languages. It is a gauge of what a student knows and/or can do and, more indirectly, of how well instruction is proceeding. Language assessment is part of a system for establishing what an economy, province, or state will establish as the framework for instruction. Language assessment should be based on Language Policies and Language Content Standards, which make clear what students will be expected to know and be able to do as a result of having participating in language learning experience.  Language Performance Standards are important because they determine which areas and levels of performance by the students will be examined in order to assess their learning. The assessment may cover listening, speaking, reading, writing or cultural understanding. It may place equal weight on knowledge (understanding how the language works theoretically) and proficiency (ability to use the language practically), or may give greater weight to one or the other.

Language Assessment in the APEC Economies


Language Assessment as practiced in the APEC economies has recently been reviewed by Chen et al. in a research paper developed as background for the Seminar on Standards for English and Other Foreign Languages in APEC Economies held in Chinese Taipei December 3-5, 2007. That work was designed to discuss how assessment and the use of standards applies to the phenomenon of language learning in the various APEC economies. Learning English in APEC economies has been growing for many years. Interest has also spread to include other languages more recently because of increased migration and the development of regional partnerships and joint economic projects. Economies have become interested in language instruction as a form of investment.That interest has led to a desire for reliable, accurate assessments. 

APEC’s formal interest in language standards and assessments began with the 2004 Education Network Summit in Beijing, China. At that time, research on instruction and curriculum was presented from both Western and Eastern perspectives. Later that year, at the 3rd APEC Education Ministerial Meeting in Santiago, Chile, the Strategic Plan for English Language/Foreign Language Learning was adopted. The strategic plan emphasized quality teaching, policy setting, and the value of testing and evaluation. Since standards were understood to be critical to testing, a survey of their use by the APEC economies was conducted prior to the Beijing Summit. The results of the survey led to a recommendation that clear standards and adequate assessment systems for both teachers and students be jointly developed.

Chinese Taipei and Chile took the lead in researching standards and assessment and their prospective benefits to the APEC economies. The focus of their work has been learning English since “…the reality of APEC [is that] English is the primary foreign language of focus in 80% of the economies...” [1]

As the demand for assessment has grown, in recent years, three sets of assessment scales have emerged as major points of reference: The American Council on the Teaching of Foreign Languages (ACTFL) Speaking Proficiency Guidelines, the Common European Framework of Reference (CEFR), and the Canadian Language Benchmarks. While these were created for somewhat different purposes (the ACTFL Guidelines primarily for use in higher education to assess students’ foreign language abilities; the Common European Framework to describe the range of skills of European students learning a new language in a foreign country within Europe; and the Canadian Benchmarks to assess the language skills of prospective immigrant employees), each has been highly influential. In addition, the International English Language Testing System (IELTS) and the International Second Language Proficiency Ratings (ISLPR) are very widely used in Australia, New Zealand and other APEC economies. The ISLPR has influenced other scales since its development, including the ACTFL Scales.

Criticisms of the scales have focused on unwarranted assumptions about normative communication in a language and on the use of the scales to serve political and social ends. While many of the scales has no empirical basis, the ISLPR developers, for example, have engaged in a vast amount of empirical research. In addition, the question of how to use each as the basis for test development is of considerable concern. However, that is not to say that they cannot be nor that they can’t make a considerable contribution to test development. The question is one of understanding the nature of scales and how they contribute. The scales provide simple and explicit descriptions of the levels they espouse using so that teachers and learners alike can comprehend what they describe. They are focused on proficiency instead of achievement and are based on the practical learning of language educators and testers over decades of practice[2]  Another criticism, according to Duff, is the lack of adequate coordination internationally in the assessment realm.[3]

As to tests, the other major set of tools on hand, major developers have been the Educational Testing Service (ETS) of the USA with its Test of English as a Foreign Language (TOEFL), the Test of English for International Communication (TOEIC), and the University of Cambridge Local Examinations Syndicate (UCLES), who is the managing agent for the International English Language Testing System (IELTS), and has its Cambridge Certificate of Proficiency (CPE), and the Cambridge Certificate in Advanced English (CAE), among others. As with the assessment scales, these assessment devices have shifted from an emphasis on achievement and individual “points of language” as pertain to skills in listening, speaking, reading, and writing, to an emphasis on context, communication ability, and proficiency in language use. The purposes and validation of these tests, referred to by many as high-stakes tests, are covered in an APEC research paper written by Stoynoff (2008). In that paper, Stoynoff notes, “In APEC economies where English is the priority foreign language, English tests frequently perform a gate-keeping function that significantly affects test-takers’ educational, employment, and career advancement opportunities . . . When the scores on tests are used to make decisions that have serious consequences, they are considered high-stakes tests . . .”[4]

Chen et al. also reviewed the English language assessment instruments for APEC's English-dominant and non-English-dominant economies.[5]


Table: Sample Language Assessments Used by APEC Economies

The following table contains samples of language assessments from various APEC members. Of the APEC Economies, only Australia, Canada, and USA have decentralized systems where decisions of assessments are made at the provincial or state level.

Economy

Test Name

Description

Brunei Darussalam

Cambridge International Examinations

List of skills assessed by the University of Cambridge International Examinations, updated annually. Brunei Darussalam utilizes the Brunei Cambridge General Certificate of Education (BCGCE) ‘O’ Levels to assess students in grades 10-11 and the Brunei Cambridge General Certificate of Education (BCGCE) ‘A’ Levels for grades 12–13. These sites provide syllabi and practice exams.

Canada


British Columbia Provincial Exams



Proof of Language Proficiency




Pan-Canadian Assessment Program

Sample exams and answer keys administered to high school students to certify the performance of graduating seniors.  Grade 12 assessments in English include English, Mandarin, and other pertinent languages.


Explains methods for becoming certified proficient in language by the Ontario College of Teachers. Lists acceptable countries where study has been completed and acceptable language tests to indicate sufficient knowledge when successfully completed.


Describes of national examination given to students aged 13 and 15 at regular intervals. Skills tested are reading, mathematics, and science, and the examination is administered in both French and English. Results are broken down nationally and by province.

Chile

ALTE Levels 

Table describing the Association of Language Testers of Europe proficiency scale, including benchmarks to be made at grades 8 and 12 and teaching level. The Ministry advocates no single test for assessment. Available in Spanish from Chile's Ministry of Education website.

China

College English Test (CET)

National Matriculation
English Test
(NMET)

CET and NMET are two significant EFL tests developed and used in China, both are aligned with China’s national English curriculum. The CET battery is designed to assess undergraduates’ achievement of the requirements specified in the national English syllabus for non-English majors, while the NMET is a large-scale, high stakes test designed to assess test takers’ English language ability and scores are used to make university admission decisions.

Hong Kong-China

Basic Competency Assessment

Describes of the Online Student assessment tool that measures individual performance in three key learning areas: English, Chinese, and mathematics. Intended to complement territory-wide system Assessment already in place that measures progress of the entire educational system.

New Zealand

Assessment



National Certificate of Educational Achievement (NCEA)

Describes the assessment process from training, to gathering information, to adapting instruction. Includes descriptions of assessment tools.

Provides access to searches for Achievement Standards and Unit Standards for multiple foreign and native languages. Credits from these standards contribute to the New Zealand Government's secondary school qualification, the NCEA.

Singapore

Singapore Examinations and Assessment Board

Provides information on national assessments in Singapore including the Primary School Leaving Examination for 12-year-olds, the Singapore-Cambridge General Certificate of Education (Ordinary Level) Examination for 16-year-olds, and Singapore-Cambridge General Certificate of Education (Advanced Level) Examination for 18-year-olds.  These sites provide syllabi and practice exams.  Subject syllabi outline the assessment objectives for various languages, including English, Chinese, Malay, Tamil, as well as other languages offered in the school curriculum.

Chinese Taipei

General English Proficiency Test

Describes standardized test commissioned by the Ministry of Education to test English proficiency at five levels, from elementary to superior. Recognized by governmental, educational, and private institutions.

United States

Proficiency Guidelines




Test of English as a Foreign Language

Describes the assessment scale aimed at measuring a foreign language learner's performance in key areas of language acquisition. Developed by ACTFL and the U.S. Foreign Service Institute. In early 2006, ACTFL launched the ACTFL OPIc, a computer-delivered version of the ACTFL Oral Proficiency Interview accessed via the Internet. In 2009, an expanded version of the OPIc will be launched, and it will assess the full range of ACTFL proficiency levels from Novice through Superior.


Describes the test that measures the ability of foreign language speakers to utilize English, primarily in an academic setting. Developed by Educational Testing Services and administered on paper and online in many locations around the world.

Cross-border

Programme for International Student Assessment

Describes the assessment administered by the Organization for Economic Cooperation and Development that tests a sample of 15-year-old students from numerous nations in reading, mathematics, and science.  There were 13 APEC Member Economies involved in the last test (2006) and 16 are scheduled to participate in the next (2009).


Table: Other Foreign Language Assessment Tools

 The following table is adapted from a "Background Research Paper" written by Chen, et al[6], for the APEC Seminar on Language Standards and their Assessment.  It describes several assessment tools for learners of languages related to the APEC Economies.

Language Test Name Details

Chinese

Hanyu Shuiping Kaoshi (HSK)

The Hanyu Shuiping Kaoshi (Chinese Proficiency Test) is China's national standardized test designed and developed by the HSK Center of Beijing Language and Culture University to assess the Chinese language proficiency of non-native speakers (including foreigners, overseas Chinese and students from Chinese national minorities).

Japanese

Japanese Language Proficiency Test (JLPT)

The test is divided into four levels and is used by people planning to study in Japan or to obtain certification of their proficiency in Japanese.

Japanese Examination for Japanese University Admission for International Students (EJU)

The EJU is used to evaluate the Japanese language proficiency and the basic academic abilities of international students who wish to study at Japanese universities at the undergraduate level.

Computer-adaptive test in Japanese (LTRC, Australia)

This computer-adaptive placement test (CAT) in Japanese, developed for undergraduate students at the University of Melbourne, contains 225 multiple choice grammar items tested extensively in Australia, China and Japan.

The Japanese Language Test for Tour Guides (LTRC, Australia)

This test provides a practical assessment of Japanese-speaking guides' linguistic ability when interacting with tourists in a variety of situations.

Korean

Korean Language Proficiency Test (KLPT)

The Korean Language Proficiency Test Association is a nonprofit organization approved by the Ministry of Labour and the Ministry of Justice as the sole agency managing Korean language proficiency testing for non-native workers seeking employment in Korea. Two different exams are available: the KLPT and the Basic KLPT. Each marks the standard skill requirement for command of the language in carrying out specific operations for those who want to work for Korean companies.

Spanish

Los Diplomas de Español como Lengua Extranjera (DELE)

The DELEs (Diplomas de Español como Lengua Extranjera), issued by the Ministry of Educacion and Science of Spain, offer an official accreditation on the degree of mastery of the Spanish language for citizens of countries where Spanish is not the official language. The exams for obtaining the DELE (Inicial, Básico and Superior) consists of five tests: reading comprehension, writing expression, listening comprehension, grammar and vocabulary, and oral expression. To obtain the DELE, the grade "apto" (satisfactory) is required on all the tests within the same examination period.

The Diplomas are recognized not only by official institutions of Spanish-speaking countries, but also, increasingly by corporations, chambers of commerce and educational institutions of the United States and Canada.

Arizona's Spanish proficiency test (USA)

Leslie Grant. 1997. "Testing the language proficiency of bilingual teachers: Arizona's Spanish proficiency test." Language Testing 14, 1, pp. 23 – 46.

Thai

Thai Language Certification in Japan

Chulalongkorn University Test of Thai Writing

The Association for Thai Language Certification in Japan (日本泰語検定試験) was one of the few hits found when searching for Thai language proficiency tests.

The Test of Thai Writing was developed in consultation with Professor David E. Ingram of Australia.

Multiple Languages

Foreign Language Proficiency Test (FLPT)

The FLPT is developed for use by personnel in government and public institutions to select personnel for overseas placement or study. The test is provided in five languages, English, Japanese, French, German, and Spanish. It includes listening, usage, vocabulary, reading, and speaking components.

International Second Language Proficiency Ratings (ISLPR) While its dominant use nowadays in Australia is for English Second Language proficiency, it was specifically designed to be applicable to any language with versions in several other languages including French, Italian, Japanese, Chinese, and others. There are also exemplar versions in English and other languages for specific purposes including versions for teachers of languages other than English, engineering, business etc.

Teacher Proficiency Tests: Japanese, Italian and Indonesian (LTRC, Australia)

These Languages Other Than English (LOTE) teacher proficiency tests were developed to identify appropriate standards of language proficiency for the teaching of Indonesian, Italian and Japanese in schools.

Occupational Foreign Language Tests: Japanese and Korean for Tourism and Hospitality Industry (LTRC, Australia)


Bilingual Health Language Proficiency Test (LTRC, Australia)

These audiotape-based tests, available at beginners and intermediate levels, assess how well candidates are able to interact verbally in service situations within the tourism and hospitality industry.


This telephone-based test, available in Arabic, Cantonese, Greek, Italian, Mandarin, Spanish and Vietnamese, assesses how well bilingual health professionals are able to interact with patients with limited English skills.

Language Testing International (LTI)

LTI arranges American Council on the Teaching of Foreign Languages (ACTFL) language proficiency assessments in 50+ languages for corporations, government agencies, academic institutions and individuals. LTI was founded in 1992 in response to the growing need for standardized, valid language proficiency assessments conducted by certified testers.


In addition, the Center for Applied Linguistics (CAL) has developed a Foreign Language Assessment Directory that compiles information for nearly 200 assessment tools of numerous languages and is searchable by language, grade level, proficiency level, intended use, and skill tested.
 

Recent Developments in Language Assessment

Standardization and Coordination

The section on standardization and coordination that follows is excerpted from Foreign Language Policies, Research, and Educational Possibilities"[7]. The Common European Framework for Languages (CEFL) referred to in this excerpt is identical to the Common European Framework of Reference (CEFR) elsewhere.

Many have claimed that there is little coordination internationally in assessment or testing initiatives. Lambert (2001) observed, and many would agree, that the widespread adoption of the American Council on the Teaching of Foreign Languages (ACTFL) proficiency standards, testing protocols, and rating scales in the United States has helped standardize and modernize language teaching methods in that economy, particularly at the postsecondary level.[8] The emphasis on oral proficiency and on the ability to speak about a range of topics accurately and fluently on the Oral Proficiency Interview has motivated teachers and students to pay more attention to oral skills and less to textual translation than was previously the case. The Teachers of English to Speakers of Other Languages, a global network of educators, develops standardized practices for their profession that are widely accepted in the United States and elsewhere. However, according to Lambert, better coordination between Foreign Language (FL) curriculum development and evaluation in the United States and in Europe is needed, For example, under the direction of the Council of Europe (2001), an impressive functional approach to task-based teaching and assessment has been developed in Europe for at least 20 foreign languages across a wide range of proficiency levels.[9] The Common European Framework for Languages (CEFL) is now guiding language teaching policies and assessment in most countries in the European Union, even though this effort is not well known in North American FL education or policy circles. However, a section of the Modern Language Journal provides a good introduction to CEFL[10] and a number of commentaries and critiques by European experts regarding its implementation and its connection with pedagogy and curriculum. The action-oriented “can do” ideology underlining the CEFL, the use of language portfolios prepared by students, and a hands-on approach to assessment are highly consistent with a 21st-century orientation to and appreciation of practical language competencies.

Quality and Fairness

The section on quality and fairness that follows is excerpted from “Developments in English Language Assessment."

Kunnan (2008) emphasizes that the most important challenge in large-scale assessment is the issue of fairness. He defines fairness in terms of the use of fair content and test methods in assessing language ability and the fair use of the scores obtained from the test. Whether test users rely on international or locally developed tests, they have a responsibility to ensure adequate evidence exists to support the interpretations and use of the scores from the test. In cases where there is a lack of evidence available in the public domain for a high-stakes EFL measure, test score users should be cautious about the inferences they make on the basis of the scores.
Among APEC member economies, ETS and Cambridge-ESOL are two major players with regard to English language test development, and they have detailed protocols in place to monitor the quality and fairness of their tests. ETS has aligned its test development practices with those advocated in the Code of Fair Testing Practices (Joint Committee on Testing Practices, 2004), and Cambridge-ESOL's practices conform to the standards for test quality and fairness advocated in the ALTE Code of Practice (2001). As a result, the English proficiency tests and supporting documentation produced by these leading test development centers not only meet current international standards, but they also represent exemplars for the global language testing community.
Among current trends in assessing English language ability, four issues have implications for APEC economies: (1) adoption of professional standards to the design and use of high-stakes assessments, (2) determination of the standard (norms) of English to be applied to assessment of EFL ability, (3) representation of L2 ability, and (4) inclusion of performance-based tasks of speaking and writing ability in high-stakes tests. When APEC economy members decide that a locally developed EFL test is preferable to an international test, a theoretical conceptualization of L2 ability can assist test designers in their work.[11]

References

  1. Chen, Howard et al. "Background Research Paper for APEC EDNET Seminar on Language Standards and their Assessment." 2007.
  2. Chen, Howard et al. "Background Research Paper for APEC EDNET Seminar on Language Standards and their Assessment." 2007.
  3. Duff, Patricia A.: Foreign Language Policies, Research, and Educational Possibilities. APEC Education Symposium, held in Xi'an, China, 2008.1.14-7
  4. Stoynoff, Stephen.: Developments in English Language Assessment. Prepared as part of APEC Strategic Plan for English and Other Languages, 2008.
  5. Chen, Howard et al. "Background Research Paper for APEC EDNET Seminar on Language Standards and their Assessment." 2007.
  6. Chen, Howard et al. "Background Research Paper for APEC EDNET Seminar on Language Standards and their Assessment." 2007.
  7. Duff, Patricia A.: Foreign Language Policies, Research, and Educational Possibilities. APEC Education Symposium, held in Xi'an, China, 2008.1.14-7
  8. Lambert, R. D. (2001). Updating the foreign language agenda. Modern Language Journal, 85, 347-362.
  9. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Retrieved from the World-Wide Web 25 August 2010 at http://www.coe.int/t/dg4/linguistic/cadre_en.asp.
  10. Little, D. (2007). The Common European Framework of Reference for Languages: Perspectives on the making of supranational language education policy. Modern Language Journal, 91: 645-655.
  11. Stoynoff, Stephen.: Developments in English Language Assessment. Prepared as part of APEC Strategic Plan for English and Other Languages, 2008.
 
Asia-Pacific Economic Cooperation (APEC)