In Quebec colleges, ESL is competency-based. Competencies are expressed as level-specific statements of what a student will demonstrate by the end of the semester. Students must take two English courses at one of four proficiency levels: 100 (A2), 101 (B1), 102 (B2), or 103 (C1). To earn a college diploma, every student must demonstrate the competency in two courses at one of these levels.

ESL courses usually have only one competency, which is divided into four elements (reading, listening, writing, and speaking). Each element of the competency has performance criteria that help teachers decide if the element of the competency has been achieved. Since we score student performance on 100 and the passing score is 60, minimally achieving the competency means passing the course with score of 60%.

Teachers routinely test each of these elements separately. This means that a student who achieves a score of 80% in writing, 80% in reading, 40% in speaking, and 40% in listening could still pass the course with a 60%. That’s a problem.

Triple threshold to pass

To ensure that each student leaves college with the ability to hold a short conversation in English, some colleges insist that students must pass multiple elements of the competency to pass the course. Ahunstic College is one such college. We have a departmental policy that requires the students in our 100-level (A2) to pass listening with a 60% or more, speaking with a 60% or more, and achieve a course total of 60% or greater. We call this the triple threshold. Here’s how it is explained in our course plans:

Pour réussir le cours 604-100-MQ, l’étudiant doit obtenir la note minimale de 60% pour la note globale, de 60% pour le volet de compréhension orale, ainsi que de 60% pour le volet de production orale. Puisque la compétence visée pour ce cours est de « comprendre et exprimer des messages simples an anglais », l’évaluation de ces deux éléments de compétence doit attester d’un seuil minimal et incontournable de réussite dans un contexte représentatif d’une situation réelle de communication en anglais. Ceci ne s’applique que dans ces trois volets, car la présence de spontanéité nécessite un minimum de connaissance pour pouvoir communiquer avec succès, ce qui n’est pas le cas dans des situations de production et de compréhension écrite. Lorsque les évaluations démontrent qu’un de ces trois seuils de réussite n’est pas atteint et que la note globale est supérieure ou égale à 60%, une note maximale de 55% sera inscrite au bulletin.

This is an imperfect policy. There is no provision made for deaf students or students with a severe stutter. Furthermore, the justification given is that aural-oral skills are required for situations requiring spontaneous interaction, but teachers are not required to test listening within the context of spontaneous interactive conversation. We don’t have to and probably shouldn’t. The ministry performance criterion for listening implies that for high beginners, the listening evaluation will involve a recording played twice. If that doesn’t sound very social or realistic, check out this alternative to ordinary listening quizzes here.

Putting all that aside, how can we insure that students reach a minimal level of listening comprehension so that they can pass their 100 (A2) level English Second Language course? Should we be explicitly teaching vocabulary and grammar, or rely on comprehensible input?

My answer is a little long, but it has to be.

Inferring meaning from context

There is a relationship between the percentage of the vocabulary in a text that a student knows and the student’s comprehension of that text. An early study into reading comprehension (Laufer, 1989) estimated the minimum percentage of explicit knowledge of vocabulary necessary for second language learners to understand the essential message of an unfamiliar texts to be 95%. Eleven years later, that number was revised to 98% (Hu & Nation, 2000). Another eleven years after that, Schmitt and his colleagues asked the question again and found the 98% threshold to be a more realistic estimate of coverage that a student must master to infer the meaning of the remaining 2% of the vocabulary in a reading text (Schmitt et al., 2011).

It is rather worrying that even if the student possesses an explicit knowledge of 100% of the words in a text, he or she will only understand 68-75% of the information contained in the text (Schmitt et al., 2011). The remaining 25%-32% shortfall in comprehension is probably due to missing grammar. However, if the student knows less than 90% of vocabulary in a text, that same student will understand less than 50% of the information it contains. Vocabulary knowledge seems to be the most important part of comprehension.

So do teachers and their students spend much time explicitly focusing on the vocabulary that students will encounter in their final exams? It seems not. One study (Folse, 2010) found that students hesitate to ask about the vocabulary in a text, asking questions to the teacher on average fewer than two times per lesson. This may give teachers a false sense that the students don’t need explicit vocabulary instruction.

Indeed, teachers as a group are certainly not very eager to teach vocabulary. Folse reports that teachers initiate very few explicit vocabulary teaching episodes, and there are striking differences between individual teachers—some teachers never initiated an explicit focus on vocabulary throughout the five lessons Folse observed them.

Teachers’ instructional strategies were also found to be suboptimal. Even the teachers who took individual students’ questions on the meaning of new vocabulary didn’t write the words on the board and didn’t rebroadcast students’ questions and the answers to the rest of the class. Explanations were sometimes overly complicated with very few aural repetitions of the new vocabulary provided. Choral rehearsals were nearly nonexistent .

Rehearsals and choral repetitions help with retrieval. An earlier study (Folse, 2006) found that increasing the number of times students were required to retrieve a new word had the greatest effect on their vocabulary learning. In short, teachers could do a better job of teaching vocabulary with more focus on explicit learning and more focused repetition.  

Listening comprehension challenges

Listening comprehension, like reading comprehension, also depends on vocabulary knowledge, but there are additional challenges. On listening tests, students need to know the same 98%-100% of the vocabulary, but students also need to perceive those words and process them for meaning at 140-160 words a minute, set in their phonological contexts, with attendant vowel reductions, liaison, accent, and music overdubs, and recording signal noise. If two or more speakers speak over each other, students will have to infer the inaudible words from context.  A significant number of our 100-level students will have to do all that that while coping with generalized anxiety, attention deficits, and dyslexia. Listening in your second language is indeed challenging.

Here’s is what our weakest students have to demonstrate by the end of the semester (4SA0).

• Reconnaissance du sens général et des idées essentielles d’un message d’au moins trois minutes exprimé à un débit normal et comportant un vocabulaire d’usage courant, après deux écoutes.

Students have to recognize the essential meaning of the listening text. The listening should 3 minutes long at minimum. It should be spoken at a normal rate. It should comprise commonly used vocabulary, and students should only have to listen to it twice to understand it. Many of our students can do that. But some can’t.  We can offer struggling students tutoring services, but how can the tutor help students if he doesn’t know what specifically is on the final listening test?

What does it mean to teach and test competency-based ESL?

Competency-based education does not mean that teachers should select a listening test at random or suppose that introducing the theme the lesson before the test is likely to be adequate preparation for our weakest students. Competency-based education does not mean proficiency testing.

Richards and Rogers (2001) in their book “Approaches and Methods in Language Teaching” explain that competency-based ESL means all of the following:

1. explicit learning goals that students can work toward step-by-step
2. by practicing the vocabulary and structures that teachers have accurately predicted will be encountered
3. and evaluated in the specific social context in which the skill will likely be used 

Demonstrating a ministerial competency is achieved by performing a predictable complex task after step-by-step instruction with the help of a teacher who has predicted what specifically the student needs to know beforehand to comprehend ~75% of the meaning of the message. That will require 100% of the words in the listening to be explicitly taught and efficiently practiced.

Competency-based education does not mean that students will change proficiency levels within 13 weeks, entering the classroom as a Mise-a-Niveau students (CEFR A1, IELTS 2.0-3.0, TOEFL <25) and rising to a next level of proficiency (CEFR A2, IELTS 3.5, TOEFL 25-30). That’s never going to happen with 45 hours of instruction.

Rather, in terms of listening skills, competency-based education means learning enough vocabulary, grammar, and pronunciation (for improved perception) in a classroom to decipher the meaning of a specific listening text that students have been prepared step-by-step to understand. Students should be able to walk into an exam, recognize the testing context as being like the social context that they might normally encounter a 3-minute recorded message in English, and they should be well-prepared to recognize 98%-100% of the vocabulary used in that listening because they came to class and did the homework. A test like that measures achievement of the targeted competency. It does not test proficiency.

Sound pedagogy  

How can we ensure achievement of the competency? The clearest, most concise, theory-based, and research-supported answer to this question comes from Gatbonton and Segalowitz (2005), who posit that languages are best taught by maximizing the repeated exchange of meaningful messages with a focus on target structures and accuracy.  The keywords in this definition are “meaningful,” “repeated,” and “focused.” Classroom practices and teaching strategies should be reviewed with these notions in mind.

What concrete practice strategies should we be employing in the classroom?

In a study (Laufer & Hulstijn, 1998) comparing comprehensible input, cloze texts, and comprehensible output, it was found that output led to the best retention of new vocabulary. In the study, students either read a letter with target vocabulary in bold, filled in blanks of the letter with target vocabulary, or wrote their own letters with a focus on target vocabulary. The researchers found that the third condition—meaningful output—accelerated learning 75% over fill-in-the-blanks and 160% over comprehensible input.

Clearly, exposure and cloze exercises are not as good as focused meaningful production.  Folse (2006) offers two explanations for the enhanced effectiveness of composition: drafting is inherently repetitive since L2 writers draft sentences which they revise and rewrite multiple times, and writing involves more time on task than reading for meaning.

Processing depth may be another reason for better retention of target vocabulary, but processing depth remains impossible to define or measure.

Teachers have options in the classroom. In short, filling in the blanks and enhanced input are not our optimal choices. A better choice could be automatically scored writing assignments or automatically scored speaking assignments and automatically scored webcam orals. Why? They offer students more opportunities for focused formative feedback, revision, and meaningful repetition.

Listening exam preparation

How do you apply our definition of sound pedagogy (repeated, meaningful, and focused) to the listening element of the 100-level competency? Backward design. We should start with the final evaluation task planned for the end of the semester and work backwards, making sure that students meaningfully produce 100% of the words in it multiple times over the course of the 13-14 weeks. 100-level teachers could individually or collectively choose a 3-minute video, extract the subtitles, and create meaningful listening and speaking exercises online and in person.

How can a teacher know which words need to be explicitly taught? I have created a tool for you here. Enter a text, click “Check level.” The vocabulary will be coded for part of speech and separated into proficiency levels.

Good pedagogy is time-consuming. There are a number of things we can do to manage our workload. We can lighten our load by automating corrections. College teachers in Quebec can schedule preparation time between January 2^nd and January 21^st , June 1^st and June 15^th, and August 15^th and August 21^st. If that is not enough, we can collaborate with colleagues, or we can write a grant proposal to get a project funded.

Let’s put good pedagogy to work for our weakest students and ensure everyone achieves all elements of the competency in our ESL courses.

   

References

Folse, K. F., (2006). The effect of type of written exercise on L2 vocabulary retention. TESOL Quarterly, 40 (2), 273-293.

Folse, K. F., (2010). Is explicit vocabulary focus the reading teacher’s job?  Reading in a Foreign Language, 22, 139-160.

Gatbonton, E., & Segalowitz, N. (2005). Rethinking Communicative Language Teaching: A Focus on Access to Fluency. The Canadian Modern Language Review, 61(3), 325-353.

Hu, M., & Nation, I. S. P. (2000). Vocabulary density and reading comprehension. Reading in a Foreign Language, 23, 403–430.

Laufer, B., & Hulstijn, J. (1998, March). What leads to better incidental vocabulary learning: Comprehensible input or comprehensible output? Paper presented at the Pacific Second Language Research Forum, Tokyo.

Laufer, B., & Kimmel, M. (1997). The bilingualised dictionaries: How learners really use them. System.

Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26–43. https://doi.org/10.1111/j.1540‑4781.2011.01146.x.

Richards, J., & Rodgers, T. (2001). Approaches and Methods in Language Teaching (p. 204). New York: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511667305.021