T.C.
GAZİ UNIVERSITY
INSTITUTE OF EDUCATIONAL SCIENCES
DEPARTMENT OF ENGLISH LANGUAGE TEACHING
WASHBACK EFFECTS OF HIGH STAKES EXAMS, KPDS and
ÜDS (YDS), ON LANGUAGE LEARNING OF ACADEMIC
PERSONNEL
(Nevşehir Case Study)
MA THESIS
By
Bekir ÇAKILDERE
Ankara May, 2013
T.C.
GAZİ UNIVERSITY
INSTITUTE OF EDUCATIONAL SCIENCES
DEPARTMENT OF ENGLISH LANGUAGE TEACHING
WASHBACK EFFECTS OF HIGH STAKES EXAMS, KPDS and
ÜDS (YDS), ON LANGUAGE LEARNING OF ACADEMIC
PERSONNEL
(Nevşehir Case Study)
MA THESIS
By
Bekir ÇAKILDERE
Supervisor: Assistant Professor Dr. Kadriye Dilek AKPINAR
Ankara May, 2013
ÖZET
Bir testin eğitim ve öğrenim üzerindeki etkisi “washback” etkisi olarak tanımlanır. “Washback” etkisi üç kategoriye ayrılabilir. Eğer bir test öğrenmeyi olumlu bir şekilde etkiliyorsa buna olumlu washback; olumsuz etkiliyorsa olumsuz washback; herhangi bir şekilde etkilemiyorsa, etkisiz washback olarak adlandırılır. Bu çalışmanın amacı KPDS ve ÜDS’nin akademisyenlerin dil gelişimi üzerindeki washback etkilerini ortaya çıkarmaktır.
Bu çalışma Nevşehir Üniversitesinde yapılmıştır. Verileri elde etmek için anket ve mülakat uygulaması yapılmıştır. Anket uygulaması toplam 103 akademik personel üzerinde yapılmıştır. Mülakat uygulaması ise iki Yardımcı Doçent Doktor ve iki Araştırma Görevlisi üzerinde yüz yüze uygulanmıştır. Verileri analiz etmek için SPSS 19.0 programı kullanılmıştır. Verilerin daha iyi anlaşılması için ortanca değer, standart sapma, frekans ve yüzdelik gibi betimsel analizler yapılmıştır.
Sonuçlara gelince, KPDS ve ÜDS’nin farklı beceriler üzerinde olumlu ve olumsuz etkileri olduğu analiz edilmiştir. Okuma becerisinin olumlu etkilenmiştir. Katılımcıların çoğu KPDS ve ÜDS’de sıkça sorulduğu için okuma becerilerini geliştirmeye çalıştıklarını ifade etmişlerdir. KPDS ve ÜDS’nin kelime gelişimi üzerinde de olumlu etkisi vardır. Son olarak, gramer de olumlu bir şekilde etkilenmiştir.
Dinleme, konuşma ve yazma becerilerine gelince, KPDS ve ÜDS adayları bu becerilerden test etmediği için bu sınavların bu beceriler üzerinde olumsuz etkisi olduğu bulunmuştur. Ayrıca, katılımcıların çoğu bu beceriler KPDS ve ÜDS’de test edilseydi bu becerilere çalışacaklarını belirtmişlerdir.
ABSTRACT
The influence that a test has on learning and teaching is defined as washback effect. Washback effect can be divided into three categories; positive, negative, or neutral. If a test influences learning in a positive way, it is called as positive washback. However, if it influences the learning process in a negative way, then it is called
negative washback. If a test does not affect education in any way, then it is called neutral washback. The purpose of this study is to find out the washback effects of two
high-stakes tests, KPDS and ÜDS, on the language development of academic personnel. The study was conducted at Nevsehir University. A questionnaire and interviews with academicians were used to collect the data. The questionnaire part focused on 103 academic personnel. For the interview part, two assistant professors and two research assistants were interviewed face to face. To analyze the data a program called SPSS version 19.0 was used. In order to better understand the data obtained from the surveys, descriptive statistics (mean, SD, frequency and percentage) of the data were presented.
As for the results, it has been found out that KPDS and ÜDS had some positive and negative washback effects on different skills. Reading is the only skill on which these exams had positive washback effect. Most of the participants agreed that they studied to improve their reading comprehension since it was widely tested in KPDS and ÜDS. In addition, KPDS and ÜDS had positive washback effect on vocabulary and grammar
As to listening, speaking, and writing, since KPDS and ÜDS do not test these skills, these exams had strong negative effects on these skills. Furthermore, the participants also suggested that if KPDS and ÜDS tested listening, speaking, and writing, they would study to improve these skills.
EĞİTİM BİLİMLERİ ENSTİTÜSÜ MÜDÜRLÜĞÜ’NE
Bekir ÇAKILDERE’nin “Washback Effects of High Stakes Exams, KPDS and ÜDS (YDS), on Language Learning of Academic Personnel” başlıklı tezi ……… tarihinde jürimiz tarafından İngilizce Öğretmenliği Ana Bilim Dalında Yüksek Lisans Tezi olarak kabul edilmiştir.
İsim Soyisim İmza
Başkan:Doç. Dr. Arif Sarıçoban ... Üye (Tez Danışmanı): Yrd. Doç. Dr. Kadriye Dilek Akpınar …... Üye: Yrd. Doç. Dr. Z. Müge Tavil ...
ACKNOWLEDGEMENTS
I owe special thanks to my supervisor Asst. Prof. Dr. Kadriye Dilek Akpınar without whose support and patience it would be impossible for me to finish this thesis. She was always there when I needed any kind of assistance and guidance. I would like to thank her for answering my calls even on Sunday evenings. She gave me invaluable feedback and support.
I would like to take this opportunity to thank my jury members, Associate Prof. Dr. Arif Sarıçoban and Asst. Prof. Dr. Zekiye Müge Tavil for the invaluable feedback they have given.
I would like to thank Dr. Sinan Özmen for the support he gave me. He was a great help to me in preparing the questionnaire of the study. His articles also enlightened me and gave me ideas about my study.
I would like to take this opportunity to thank my instructors Asst. Prof. Dr. Cemal Çakır, Asst. Prof. Dr. Neslihan Özkan, Asst. Prof. Dr. Gülşen Demir, Korkut Uluç İşisağ, Asst. Prof. Dr. Aslı Özlem Tarakçıoğlu, Asst. Prof. Dr. Semra Saraçoğlu at the MA ELT program for sharing their profound knowledge through the courses they have given.
Although all these special people helped me within the process; if Prof. Dr. Abdulvahid Çakır, the head of Gazi University English Language Teaching Department, had not given me the support to do my MA degree at Gazi University and conduct my study at Gazi University ELT department, it would not have been possible to study this program.
I am also thankful to my volunteer participants at Nevşehir University. My colleagues at Nevsehir University always supported me and I would like thank to all of them for being such a great support. I also would like to thank to my colleague, Selman Saygı, and Mustafa Kara who helped me with editing the study.
I am grateful to our head of department at Nevşehir University, Asst. Prof. Dr. Hamide Çakır, for letting me study at Gazi University. I am also grateful to our vice principals Erçin Özzade and Yasin Günden for preparing my course schedule accordingly. I thank all my MA ELT classmates, especially Ayşegül Yeşil, for their support.
Last but not least, I thank my lifelong friend, my wife Merve Çakıldere for her continual support. Although my son, Ahmet Enes, made it harder for me to write my thesis in that he always wanted to play with me whenever I started to work on my thesis, I had more strength and willingness to write my thesis after playing with him. He gave me positive energy. I would like to give him my love.
TABLE OF CONTENTS
ÖZET………...……...i
ABSTRACT………...ii
APPROVAL OF THE JURY...iii
ACKNOWLEDGEMENTS………...iv TABLE OF CONTENTS………...vi LIST OF TABLES………...…….x LIST OF FIGURES………...xii LIST OF ABBREVIATIONS………...xiv CHAPTER 1 INTRODUCTION………...1
1.1. Background of the Study.………...1
1.2. Statement of the Problem………...2
1.3. Purpose of the Study.………...3
1.4. Significance of the Study……….………...4
1.5. Assumptions of the Study……….………...5
1.6. Limitations of the Study……….………...6
1.7. Descriptions……….………...6
CHAPTER 2 REVIEW OF LITERATURE………...7
2.1. Introduction...7 2.2. Definitions of Washback...7 2.3. 15 Hypotheses of Washback...9 2.4. Washback Models...11 2.5. Washback Types...15 2.5.1. Positive Washback...16 2.5.2.Negative Washback...18
2.5.3. Neutral Washback………20
2.6. Review of Washback Studies...20
2.7. Washback Studies in Turkey...29
2.8. Structures of KPDS and ÜDS...32
2.8.1. Vocabulary Part...33
2.8.2. Grammar Part...34
2.8.3. Sentence Completion Part...35
2.8.4. Reading Comprehension Part...36
2.8.5. Translation Part...38
2.8.6. Dialogue Part...38
2.8.7. Paragraph Completion Part...39
2.8.8. Irrelevant Sentence Part...40
CHAPTER 3 METHODOLOGY...41
3.1. Introduction...41
3.2. Setting and the Participants...41
3.3. Instruments and Procedures...43
3.4. Piloting...44
3.5. Data Collection...45
3.6. Data Analysis...45
CHAPTER 4 RESULTS AND DISCUSSION...47
4.1. Introduction...47
4.2. Demographic Characteristics of the Survey Takers...47
4.2.1. Age and Gender...47
4.2.2. Academic Title and Field of Study...48
4.3.1. Item Number One (Reading) ...49
4.3.2. Item Number Two (Listening) ...51
4.3.3. Item Number Three (Speaking) ...53
4.3.4. Item Number Four (Writing) ...54
4.4. Research Question Number 2: Strategies That Academicians Think Will Help Them Succeed in KPDS and ÜDS...56
4.4.1. Item Number Five (Going Abroad for Language Learning)...56
4.4.2. Item Number Six (Getting Extra Help) ...58
4.4.3 Item Number Seven (Learning the Strategies about the Questions)...59
4.4.4. Item Number Eight (Question Solving) ...60
4.5.Research Question3: Motivation, Anxiety and Attitudes towards KPDS and ÜDS 61 4.5.1. Item Number Nine and Ten (Motivation or Demotivation)...61
4.5.2. Item Number Eleven and Twelve (Stress and Anxiety)...63
4.5.3. Item Number Thirteen (Effect of Anxiety on Learning) ...64
4.6. Washback effects of KPDS and ÜDS on Language Development...65
4.6.1. Positive Washback of KPDS and ÜDS...66
4.6.1.1. Washback on Grammar...66
4.6.1.2. Washback on Vocabulary ...67
4.6.1.3. Washback on Reading ...67
4.6.2. Negative Washback of KPDS and ÜDS...68
4.6.2.1. Washback on Pronunciation...69
4.6.2.2. Washback on Speaking...70
4.6.2.3. Washback on Listening...72
4.6.2.4. Washback on Writing...73
4.8. Analysis of the Interviews...76 4.9. Discussion...79 4.9.1. Research Question #1 ...79 4.9.2. Research Question #2 ...81 4.9.3. Research Question #3 ...83 4.9.4. Research Question #4 ...84 4.9.5. Research Question #5 ...84
4.9.5.1. Research Question #5a ...85
4.9.5.2. Research Question #5b ...85
4.9.5.3. Research Question #5c ...85
4.9.5.4. Research Question #5d ...86
4.9.5.5. Research Question #5e ...87
CHAPTER 5 CONCLUSIONS AND SUGGESTIONS ...88
5.1. Introduction ...88
5.2. Brief Summary of the Study...88
5.3. Pedagogical Implications of the Study...90
5.4. Suggestions for Further Research...91
REFERENCES...92
LIST OF TABLES
Table 1 Research conducted on candidates taking a test...21
Table 2 Research conducted on Language Teachers...21
Table 3 Research conducted on Test developers...22
Table 4 Research conducted on Teacher trainers and curriculum planners...22
Table 5 Research conducted on Teacher advisors...22
Table 6 Research conducted on Head teachers and other school administrators...22
Table 7 Research conducted on Inspectors...22
Table 8 Research conducted on Score users...22
Table 9 Research conducted on Parents...23
Table 10 Research conducted on Material developers...23
Table 11 Research conducted on Teaching Processes...23
Table 12 Age of Participants...42
Table 13 Gender of Participants...42
Table 14 Academic Titles of Participants...42
Table 15 Field of Studies of Participants...42
Table 16 Time Participants spent with learning English...43
Table 17 Case Processing Summary of the Reliability Analysis...44
Table 18 Reliability analysis of the survey...44
Table 19 Participants’ Age and Gender...48
Table 20 Participants’ Field of Study and Title ...49
Table 21 Mean value of the item number one...49
Table 23 Crosstabulation of Participants’’ fields and their answers for the item number
two...52
Table 24 Mean value of the item number three...53
Table 25 Mean value of the item number four...54
Table 26 Answers of the academicians for the item number four on the basis of academic title. ...55
Table 27 Mean value of the item number five...56
Table 28 Mean value of the item number six...58
Table 29 Mean value of the item number seven...59
Table 30 Mean value of the item number eight...60
Table 31 KPDS and ÜDS motivate me to learn English (mean value)...61
Table 32 KPDS and ÜDS demotivate me to learn English (mean value)...62
Table 33 Mean value of the item number eleven...63
Table 34 Frequency analysis of the item number eleven...63
Table 35 Mean value of the item number thirteen...65
Table 36 Mean value of the item number fourteen...66
Table 37 Mean value of the item number fifteen...67
Table 38 Frequency analysis of the item number fifteen...67
Table 39 Mean value of the item number sixteen...68
Table 40 Mean value of the item number seventeen...69
Table 41 Mean value of the item number nineteen...70
Table 42 Mean value of the item number twenty one...72
LIST OF FIGURES
Figure 1 Participants Processes Products...12
Figure 2 Green's Washback Model...14
Figure 3 Facets of washback (Gates, 1995)...16
Figure 4 Percent of participants answering the item number one...50
Figure 5 I do reading activities (on the basis of academic title)...50
Figure 6 Percentages of participants answering the item number two...51
Figure 7 Answers of the academicians for the item number two on the basis of academic title………...52
Figure 8 Percentages of the participants’ answers for the item number three...53
Figure 9 Answers of the academicians for the item number three on the basis of Academic title...54
Figure 10 Percentages of the participants’ answers for the item number four...55
Figure 11 Percentages of responds of the participants for the item number five...57
Figure 12 Responds of the academicians for the item number four on the basis of study………..…57
Figure 13 Percentages of responds of the participants for the item number six...58
Figure 14 Answers of the academicians for the item number four on the basis of their ages...59
Figure 15 Percentages of responds of the participants for the item number seven...60
Figure 16 Percentages of responds of the participants for the item number eight…….61
Figure 17 Percentages of responds of the participants for the item number nine...62
Figure 18 Percentages of responds of the participants for the item number ten...62
Figure 20 Percentages of responds of the participants for the item number twelve...64 Figure 21 Percentages of the answers of the participants for the item number thirteen..65 Figure 22 Percentages of the answers of the participants for fourteenth item…………65 Figure 23 Percentages of the answers of the participants for the item number sixteen..68 Figure 24 Percentages of the answers of the participants for seventeenth item…...69 Figure 25 Percentages of the answers of the participants for eighteenth item………...70 Figure 26 Percentages of the answers of the participants for the item number nineteen71 Figure 27 Percentages of the answers of the participants for the item number twenty..71 Figure 28 Percentages of the answers of the participants for twenty first item…….….72 Figure 29 Percentages of the answers of the participants for twenty second item….…73 Figure 30 Percentages of the answers of the participants for the item number twenty three...74 Figure 31 Percentages of the answers of the participants for the item number twenty four...74 Figure 32 Percentages of the responds of the participants for the item number twenty five...75 Figure 33 Percentages of the responds of the participants for the item number
LIST OF ABBREVIATIONS
KPDS: State Personnel Language Test
ÜDS: Inter-University Foreign Language Examination YDS: Foreign Language Examination
ÖSYM: Assessment, Selection and Placement Center YÖK: Higher Education Institute
SBS : Placement Test (for high school admission) LYS: University Admission Examination
ALES: Academic Personnel and Master Education Exam TOEFL: Test of English as a Foreign Language
CHAPTER 1
INTRODUCTION
1.1. Background to the Study
At the end of nearly all education processes, every learner is generally required to take an exam by which their learning process will be evaluated. So candidates are required to take an exam and this exam will have some kind of effect on their leaning. Negative or positive, strong or weak, the influence of tests on learning has been termed as ‘washback’ or ‘backwash’ (Biggs, 1995), and used as a synonym for ‘impact’ in the field of language testing. According to Alderson and Wall (1993), the notion that testing influences teaching is referred to as ‘backwash’ in general education circles. Frederiksen (1984) suggests that any test is likely to have an impact on the behavior of students and teachers, provided that they know about the test in advance. When students know from which subjects they will be tested, they spend more time on the skills which are tested and very little or no time on the skills that are excluded in the target test. In addition, he claims that if tests fail to assess the abilities that are desired to be fostered, they may cause test bias against teaching crucial skills that are not tested.
Every test has either a positive washback, negative washback, or neutral washback on teaching. If a test influences learning in a positive way, it is called as
positive washback. However, if it influences the learning process and product in a
negative way, then it is called negative washback. There are also some tests that affect the learning neither positively nor negatively. These kinds of tests are defined as tests which have neutral washback effects on learning.
In Turkey, there are a lot of exams which students need to go through starting with secondary school to be admitted to Science High Schools. So, we can say that the Turkish educational system is an exam-driven one. Among these exams we can count: High School Entrance Exam (HSEE; which is called as SBS in Turkey), Proficiency Test for Academic and Graduate Studies (PTAGS; which is called as ALES in Turkish). State Personnel Language Examination (Kamu Personeli Dil Sınavı; hereafter KPDS) and Inter-University Foreign Language Examination (Üniversiteler Arası Dil Sınavı;
hereafter ÜDS) which are high-stakes tests administered in Turkey are also examples of these kinds of exams. The name of KPDS and ÜDS has changed in April 2013. These two exams are combined and have been replaced by YDS (Foreign Language Examination). The structure of YDS is the same as KPDS and ÜDS. The only thing that changed is the name of these exams. As a result, findings of this study can said to be valid for the new test “YDS”.
1.2. Statement of the Problem
KPDS and ÜDS do not seem to have an officially intended washback effect on the curriculum or the teaching/learning process in EFL classrooms or private language courses. However, nearly every academician takes one of these foreign language exams accepted by Higher Education Institution (Yüksek Öğretim Kurumu; hereafter YÖK). Of all the foreign language exams, KPDS and ÜDS are the most preferred ones in Turkey. Most common use of these two examinations for academic personnel can be divided into three categories; first of which is; using the scores of these exams to attend a PhD program. Though it might differ from university to university, most universities in Turkey require students to get at least 55 points to be admitted as a PhD candidate. In other words, students have to get at least 55 points from a high stakes language test accepted by YÖK in order to be a PhD student in Turkish state universities. Secondly these exams are used as a prerequisite for the academic promotion. Assistant professors in Turkey are supposed to get at least 65 points to be promoted, which is one of the several requirements of being promoted to associate professorship. Thirdly, when applying to a master’s degree program, applicants are also asked to provide their scores of these exams. Although there is no minimum limit, the higher scores the students have from these exams the more chance they will have for admission. Thus, we can conclude that these exams are of high level importance for the academicians in Turkish higher education context.
Every year tens of thousands of people in Turkey are taking language tests for many reasons. Some take these tests to be promoted such as academic personnel or others take these exams for language compensation such as people working in a state institution. There are also different kinds of tests assessing language skills of the attendees which are administered in Turkey such as TOEFL (Test of English as a Foreign Language) and IELTS (International English Language Testing System). TOEFL and IELTS are worldwide accepted exams whereas KPDS and ÜDS have
limited acceptance. Nevertheless, KPDS and ÜDS are more popular than TOEFL and IELTS in Turkey. According to the statistics of Assessment, Selection and Placement Center (called as ÖSYM in Turkish), in 2011 fall, 90104 attendees took KPDS and 60688 attendees took ÜDS. This number is increasing every year.
A language test should definitely be multi-faceted. As KPDS and ÜDS test only grammar and reading comprehension skills rather than including writing, speaking and listening skills, too; we can say that they are not multi-faceted tests.
If people are learning a language with the aim of passing an exam at the end of the learning process, they tend to study the subjects or improve the skills that they will be tested. For example; if an English test does not involve listening skills, most of the attendees will not have a tendency to improve that specific skill. Thus, most of the academic personnel who take KPDS and ÜDS as an academic requirement of YÖK will mostly focus on passing the exam. As a result, they will not give any effort to improve the skills such as listening, speaking and writing which are not included in these tests. In this case, though the primary purpose of these exams should push the learners to learn a language with its all dimensions, a complete language learning process is hardly managed due to the structure and content of these exams. Yet, in the literature of the washback effect of high stakes exams, tests such as TOEFL and IELTS take the greater place (Alderson and Hamp-Lyons, 1996; Reynolds, 2010) when compared with the limited body of research about the washback effects of KPDS and ÜDS. KPDS and ÜDS as the most frequently preferred high stakes exams especially in Turkish academic context have been neglected in terms of their washback effect on the attendees. Thus, the present study focuses on the washback effect of these high stakes exams on language learning process of academic personnel at Nevşehir University, Turkey.
1.3. Purpose of the Study
There have been some studies conducted on washback effect in various contexts of teaching and learning (Shohamy, 1993; Watanabe, 1996;Cheng, 1997). These studies generally aim to create a more effective learning and teaching environment for language learners with the help of examining the effects of the tests on learning process. Broadly speaking, this study aims to examine the effects of high stakes exams (which refers to
language. The study also aims to answer following research and sub-research question questions.
1- Do KPDS and ÜDS have washback impact on the language development of academic personnel?
1a- What is the washback effect of KPDS and ÜDS on grammar?
1b- What is the washback effect of KPDS and ÜDS on reading and vocabulary?
1c- What is the washback effect of KPDS and ÜDS on pronunciation?
1d- What is the washback effect of KPDS and ÜDS on listening and speaking?
1e- What is the washback effect of KPDS and ÜDS on writing?
2- Which language skills do the Turkish academic personnel focus on while getting prepared for KPDS and ÜDS?
3- What kind of test strategies do academic personnel make use of while getting prepared for KPDS and ÜDS?
4- How does Turkish academic staff feel about KPDS and ÜDS tests? (motivated, stressed, anxious)
5- Does the academic staff think that KPDS and ÜDS affect their English language ability positively or negatively?
1.4. Significance of the Study
Foreign language score of KPDS or ÜDS is one of the most important admission requirements of Turkish universities while they are selecting their academic personnel. To be an assistant professor in any Turkish university, individuals are required to submit their language score which must be at least 65 points. Moreover, to be admitted to universities as a lecturer or a research assistant the higher foreign language points you have the more chance you will have since there is not any minimum score. The main reason why YÖK urges academicians to take KPDS and ÜDS is to encourage the academicians learn a foreign language which is needed for a better communication in academic world. If an academician does not learn a foreign language, s/he will not be able to keep up with academic world. Among the language tests that academicians can take, KPDS and ÜDS are the ones most academicians prefer to take. However, the
question whether KPDS and ÜDS really foster a complete language education including all skills such as writing, listening and speaking have not been analyzed in a detailed way. Can it be said that these exams fulfill the requirements which are necessary for learning and using a foreign language properly? Without acquiring the four skills of language learning, academicians will not feel sufficient neither to keep up with the academic world nor to identify themselves in the global world as a worldwide academician. There are so many assistant professors, associate professors and even professors in Turkey who, though providing the foreign language requirement of YÖK through KPDS or ÜDS, cannot use the language communicatively. The results of this study will reveal the effects of KPDS and ÜDS on language learning, and the content and structure of these exams might be adapted according to these results for more effective language learning.
There is limited research about the effects of KPDS and ÜDS on language learning of academic personnel. There are only two MA theses about washback effect of the exams on YOK’s thesis search website, and neither of them is related to KPDS and ÜDS. The first study is titled as Teachers’ and Students’ Perceptions about
Classroom-Based Speaking Tests and Their Washback (Duran, 2011), which is related
with washback effects of speaking tests. The second study which is titled as The
Washback Effects of Foreign Language Component of the University Entrance Examination on the Teaching and Learning Context of English Language Groups in Secondary Education is conducted by Sevimli (2007). The current study is assumed to
shed light to the existing body of the field since it would be the first study on washback effect of KPDS and ÜDS on Turkish Academic Personnel. Since testing has an impact on learning process of learners, it is important to be aware of the washback effects of these tests to create the tests accordingly.
1.5. Assumptions of the Study
Three assumptions can be made for the study. It is assumed that:
1. All the subjects’ responses to the questionnaires and interviews are honest and sincere.
2. The instrument (questionnaires and interview questions) utilized in the data collection process are valid and reliable.
3. The items used in questionnaires yield consistent results regarding the effects of KPDS and ÜDS.
1.6. Limitations of the Study
There are several limitations of the present study, first of which is that the study is limited with only Nevşehir University. The second limitation is the number of the samples that is acquired from 103 Academic Personnel. The study is also limited to research assistants, lecturers and assistant professors. Associate professors and professors are not asked to take the questionnaire as they have already fulfilled the language requirement of YÖK. So, only the academic personnel who took less than 65 points from KPDS or ÜDS will be surveyed to minimize the variables as the answers of those who took more than 65 and those who took less than 65 might be different.
The questionnaire is limited with 26 items, which is another limitation. Finally, the assistant professors are from different major study fields which might be considered as a variable that cannot be controlled.
1.7. Descriptions
Washback: unpleasant after-effects of an event or situation (Collins Cobuild Dictionary).
Washback (its meaning in the study): the influence of testing on teaching and learning (Alderson and Wall, 1993).
High- stakes test: A high-stakes test is a test which has important consequences for the test taker. If the examinee passes the test, then the examinee may receive significant benefits, such as a high school diploma, a scholarship, or a license to practice law. If the examinee fails the test, then the examinee may receive significant disadvantages (http://en.wikipedia.org).
KPDS: State Personnel Language Test
ÜDS: Inter-University Foreign Language Examination YDS: Foreign Language Examination
ÖSYM: Assessment, Selection and Placement Center YÖK: Higher Education Institute
SBS : Placement Test (for high school admission) LYS: University Admission Examination
ALES: Academic Personnel and Master Education Exam TOEFL: Test of English as a Foreign Language
CHAPTER 2
REVIEW OF LITERATURE
2.1. Introduction
This chapter contextualizes the current study with literature review of the definitions of washback and positive and/or negative washback, and provides background information about the educational context in Turkey. Finally, washback studies conducted in different parts of the world and Turkey are summarized and some issues related to the intended or unintended consequences of external tests are highlighted. 2.2. Definitions of Washback
There are so many researchers defined the term “washback” in various ways emphasizing the same meaning. As Spratt (2005) states, the term ‘washback’ is used in the literature in various contexts, which reveal differences in scope, actor and intentionality.
Buck (1988) defines washback as the influence of the test on the classroom. With a close explanation of what Buck says; Bailey (1996) defines the term as the influence of testing on teaching and learning. He also claims that it is widely held to exist and to be important; but relatively little empirical research has been done to document its exact nature or mechanisms by which it works.
Messick (1996) views the term from a different perspective by stating “the extent to which the test influences language teachers and learners to do things that they would not necessarily otherwise do.” Some other researchers such as Alderson and Wall (1993), Spolsky (1994), Alderson and Hamp Lyons (1996), Bachman and Palmer (1996), Shohamy, Donitsa, and Irit (1996),Watanabe (1996), McNamara (2000) and Hughes (2003) stress that, the effect which testing has on learning and teaching is called as washback.
researchers, although they have used different terms for the same concept. Hughes (2003) calls the term as backwash whereas Bachman and Palmer (1996) call it as test
impact. The way standardized tests affect teaching and learning is usually called
backwash in educational arena and washback in Applied Linguistics (Karabulut, 2007). Hamp-Lyons (1997) states that the term “impact” is preferred in the general education and educational measurement literature instead of washback. In fact, the main difference between “impact” and “washback” is that the former can affect wider contexts. Broadly speaking, if the effects of tests are mainly within the classroom and related to teaching and learning, it mostly affects the curriculum of education. As a result, these effects are related to washback. However, if the tests influence individuals, policies or practices, education system, and society, it is accepted as test impact (Bahman and Palmer, 1996; McNamara, 2000; Taylor, 2005; Wall). Alderson and Wall (1993) also limit the use of the term ‘washback’ to classroom attitudes of teachers and students and explain that tests are powerful determiners of what goes on in classrooms. Frederiksen and Collins (1989) state that “systematic validity” appears when “a systematically valid test triggers changes in instructions and curriculum and reinforce the improvement of cognitive skills that the test aims to measure, even over a period of time” (p. 27).
Bachman and Palmer (1996) suggest that application of a test has both micro and macro level impacts. By micro level impacts, they refer students and teachers who are directly affected by the testing process. However, macro level impacts refer to the effects of tests on society and educational system in a broader sense. Buck (1992) states ‘washback’ as the effect of a test over what teachers and students do in classrooms, that is on micro level. Furthermore, Pearson (1988) examines the micro view of teaching and learning that might be influenced by examinations. He points out that the public examinations affect the attitudes, behaviors, and motivation of teachers, students, and parents. Pierce (1992), on the other hand, uses the term “washback” on the macro level to indicate “the impact of a test on classroom pedagogy, curriculum development, and educational policy”. Cohen (1994: 41) also views the macro aspects of washback with regard to “how assessment instruments affect educational practices and beliefs”.
Pearson (1988) points out that public examinations influence the attitudes, behaviors, and motivation of teachers, learners and parents, and because examinations
often come at the end of a course, this influence is seen working a backward direction, hence the term washback shows up. These exams, most of which are high-stakes exams have crucial outcomes. As Qi (2005) claims, high stakes tests produce washback, in the sense that they can have strong effects on teaching and learning. High-stakes testing refers to tests whose results are seen correctly or incorrectly by students, teachers, administrators, or the general public as the basis upon which important decisions are made that immediately and directly affect the student.
2.3. 15 Hypotheses of Washback
In their study, Alderson and Wall (1993) created 15 hypotheses regarding washback to determine washback effects in teaching and learning. These hypotheses were of great help and inspiration for the researcher of the current study.
1- A test will influence teaching: either positively or negatively, every test has an effect on teaching. Teachers tend to teach their subject either on the basis of the test or prepare their tests on the basis of the subject covered in the classroom. 2- A test will influence learning: Whenever teaching is affected by anything, so is
the learning as students learn what they are taught.
3- A test will influence what teachers teach. When all of the same grade students take the same test in a school, city or country and when the subjects that will be asked in the test are known beforehand, teachers plan their lessons accordingly, that is, they try to cover whatever the test will demand from the students.
4- A test will influence how teachers teach: Not only what subjects will be covered, but also how they will be covered will be affected by a test. To illustrate; if students are asked to use their communicative skills in a test, the teacher will adopt a more communicative way of teaching in the classroom. 5- A test will influence what learners learn: If what teachers teach changes, what
students learn changes, too.
6- A test will influence how learners learn. If a teacher changes how s/he teaches, students will have to keep up with the teacher accordingly.
7- A test will influence the rate and the sequence of teaching: The subjects which are more frequently asked in tests have priority over less frequently asked ones while being taught by teachers. A teacher will give more importance to those
8- A test will influence the rate and the sequence of learning: When teaching rate and sequence are affected, learning rate and sequence will be affected too. Teaching and learning are inter-related terms. When either of them is affected by any factor, the other will be affected too. Namely, reflection of any effect on teaching will be seen on learning or vice versa.
9- A test will influence the degree and depth of teaching: While planning the lesson, teachers plan about how much to teach by taking the test into account. 10- A test will influence the degree and depth of learning: A test will influence how
much the students will learn.
11- A test will influence attitudes to the content, method, etc. of teaching and learning: The content and method of teaching and as a result, learning will be shaped by a test.
12- Tests that have important consequences will have washback: The tests which have the most washback effects are high stakes tests as they have crucial consequences at both micro and macro level. We can say that the more important the test is for students the more washback effects it will have on learning and teaching.
13- Tests that do not have important consequences will have no washback: If there are no consequences at the end of a test, that is to say, if students do not attain anything with their exam results, the test will not have washback. To exemplify; if a teacher says that s/he will not consider quiz results in general assessment before the quiz, students will have a tendency of ignoring and not studying the quiz. Hence, the quiz will not have washback.
14-Tests will have washback on all learners and teachers: As some tests also influence the method, materials, and curriculum in general, they will affect all learners and teachers in a classroom.
15-Tests will have washback effects for some learners and some teachers, but not for others: Some tests while affecting some of the students may not affect the others.
In conclusion; there are some exam-driven countries like Turkey where curriculum is planned and designed according to the subjects covered by the high stakes test which all of the students take like University Placement Exam (LYS). As they want their students to be successful in high stakes exams, teachers plan their curriculum
taking high stakes exams into consideration. The 15 hypotheses mentioned above are valid especially in these kinds of educational contexts.
2.4. Washback Models
Hughes (2003) created a model for washback. In his model, there are three aspects of washback; participants, process, and product. The nature of a test may first affect the perceptions and attitudes of the participants. These perceptions and attitudes, in turn, may influence what the participants do to carry out their work (process), including practicing the kind of items that exist in the test, which will affect the learning outcomes, and the product of work. Using on Hughes’s model and Alderson and Wall’s (1993) washback hypotheses, Bailey (1996) created a figure (taken from Bailey 1996: 264 - figure 1) to examine how washback works in context. Figure 1 clearly indicates that a test may influence participants, who are students, teachers and materials writers and curriculum designers, and products which are learning, teaching, and new materials and new curricula. When all of these are affected, researchers and as a result research results are affected too.
Figure 1 Participants, Processes, and Products
Bailey started his research by asking the following questions: 1- What is washback?
2- How does it work?
3- How can we investigate washback?
4- How can we promote beneficial washback?
Trying to find the answers to these questions, Bailey came up with the first model of washback which is shown on figure 1.
According to Bailey’s study, a test affects two different categories, the first of which consists of teachers, students, materials writers, curriculum designers and TEST Students Learning Teachers Teaching Materials writers and curriculum designers New materials and new curricula Researchers Research results
researchers and the second of which consists of learning, teaching, new materials, new curricula and research findings.
As to influence of a test, Bailey talks about the distinction between learner washback and programme washback. She suggests that learner washback happens when a test affect the learner himself/herself whereas programme washback involves language teaching personnel affected by a test. Programme washback includes teachers, administrators, and curriculum and material designers. These people tend to organize their system of education on the basis of the test. On the last part of her study, Bailey gives clues about how beneficial washback (positive effects of tests) could be promoted. She claims that beneficial washback of testing process can be attained by:
1-promoting of language learning goals, 2-building in authenticity,
3-introducing learner autonomy and self-assessment, 4-including detailed score reporting.
She finishes her paper with the view that as long as a test is based on reasonable
theoretical principles, it uses authentic tasks and tests, and the test takers buy into the
assessment process. (1996: 275-7).
In another model developed by Watanabe (2004) whose studies are mostly based on Japan, washback is conceptualized in terms of five dimensions which are:
1-intensity, 2-specifity 3-length, 4-intentionalty, 5-value.
He claims that the aspects of learning and teaching which are affected by a test and the factors facilitating washback are test-related, status-related, and stakeholder-related. He differentiated the features of a test which are directly linked with behaviors in classrooms. Thus, he claims that washback appears under two conditions:
1-if teaching is different in exam preparation and non-exam preparation classes taught by the same teacher;
The last but not the least, the model developed by Green (2007) was a great value for researchers studying washback. The model is based on his PhD studies and compiled in his volume Washback in Context (2007). Though adding new features, Green built his model on Bailey’s studies and focused on process aspects. However, unlike Bailey, Green focuses on the test (International English Language Testing System-IELTS) in a more detailed way. His model as shown in figure 2 (taken from Green, 2007: 24) covers washback direction,
variability and intensity. Green’s models start with test design characteristics and related
validity issues of construct representation which is identified with washback by Messick (1996).
Figure 2 Green's Washback Model
Potential for Potential for positive washback negative washback
Washback Variability Other stakeholders Course providers Participant Characteristics and values Materials writers Knowledge/understanding of test demands Publishers Resource to meet test demands Teachers
Acceptance of test demands Learners Washback Intensity
Perception of Perception of Backwash to test importance test difficulty participant Important Easy No backwash
Unimportant Unachievable Intense washback
Focal Construct Overlap Test design Characteristics item format content complexity etc. challenging
He claims that test design issues are mostly identified with the direction of washback. They have either beneficial (positive) or damaging (negative) effects on learning and teaching. The figure 2 shows us that the more important the test is for learners, the more washback effects it has. On the other hand, if the test is unimportant there seems to exist no washback. If the difficulty level of the test is unachievable, again, no washback exists. Surprisingly enough, even when the level of the test is easy, washback does not exist. So, perception of test difficulty can be claimed to have nothing to do with intense washback.
2.5. Washback Types
Washback can be divided into three categories; positive, negative, or neutral. Every test has one of these effects. Every test affects learning/teaching in a positive, negative, or neutral way. If the test affects learning/teaching positively, it has positive washback. On the other hand, if it affects learning/teaching negatively, it has negative washback. Some tests have neither positive nor negative effects on learning/teaching. These kinds of tests are referred as tests having neutral washback effects. Bailey (1996) argues that processes involved in washback differ at a large extent, depending on the constituency of participants. Students may participate any of the following processes when they are faced with a vital test. Selection among these processes would lead to either beneficial or negative washback, depending on whether or not their use promoted the learners’ actual language development or only their test-taking skills. The processes Bailey (1996) are as follows:
-Listening to non-interactive language (radio, television,etc.) -Applying test-taking strategies.
-Skipping language classes to study for the test. -Reading widely in the target language.
-Enrolling in, requesting or demanding additional test preparation classes or tutorials. -Studying vocabulary and grammar rules.
-Enrolling in test preparation courses
-Requesting guidance in their studying and feedback on their performance -Practicing items similar in format to those on the test.
2.5.1. Positive Washback
According to Fournier-Kowaleski (2005), positive washback can be defined as activities incorporated into classroom instruction as a result of the test that promotes the development of language acquisition, and/or the development of language skills (reading, writing, speaking, and listening). Alderson and Wall (1993) associate positive washback with tests that influence teaching and learning in a constructive way. If a test has positive consequences on learning/teaching, it can be said that it has a positive effect on learning/teaching. For example, let us say that a teacher wants to teach past tense in English. After going through the rules of the past tense, if the teacher makes an exam including past tense, students will study to improve their knowledge of past tense since they will be tested about it. When they study past tense to pass the test, it will result in their learning past tense. Hence, the test will have a positive effect on the learning of students. Bachman (1990) confirms that positive washback occurs when the exam reflects the skills and content taught in the classroom. Gates (1995) defines washback as the influence of testing on teaching and learning and he created a grid (Figure 3) that shows the facets of washback. According to him, if a test is crucial for the test takers, it is likely to have a strong washback effect. However, if it is not a significant test with little or no impact on what happens in the language classroom, a weak washback effect appears. Positive washback occurs if the classroom objectives and test content complement each other. If not, negative washback is expected.
Figure 3 Facets of washback (Gates, 1995) Positive Negative
Strong
Weak
KPDS and ÜDS are in the upper right cell in this grid, because they are substantial for test takers, and they are hypothesized to have negative washback because the content of it is limited to multiple choice vocabulary, grammar, and reading items which do not reflect the target language use; it does not test listening, speaking, and writing skills of the test takers.
Crooks (1988) discusses that evaluation activities in class can have a positive effect on students in that teachers stress the need for ‘deep learning’ rather than ‘surface
learning’, use evaluation to assist students rather than to judge them, use feedback to focus students’ attention on their progress set high but attainable standards, and select evaluation tasks to suit the goals being assessed (cited in Wall, 1997: 292).
As to the question of how to achieve positive washback, Hughes (2003, p.53-56) lists seven ways of having positive washback:
1. Test the abilities whose development you want to encourage. 2. Sample widely and unpredictably.
3. Use direct testing.
4. Make testing criterion-referenced. 5. Base achievement tests on objectives.
6. Ensure the test is known and understood by students and teachers. 7. Where necessary provide assistance to teachers.
Turner (2005, 2009) explains washback effect from a different perspective. In her definition, the washback effect of a high-stakes external exam is positive if it represents the curriculum well in terms of its content and procedures. Her point is that positive washback effects can be achieved when the impact of a test can help a teacher change or align some instruction with general concepts represented in the test. She also suggests that a test’s washback effect will be negative if its content and procedures do not represent the curriculum well.
Brown and Hudson (1998) state that washback effects can be either negative or positive. A positive washback effect occurs when the assessment procedures are positive. A clear example they used to illustrate their point is that if a program sets a series of communicative performance objectives and tests the students using performance assessments (e.g., role plays, interviews) and personal-response assessments (e.g., self-assessments, conferences), positive washback effect can be produced in favour of the communicative performance objectives. However, if the program ends up assessing the students with multiple-choice structure tests, a negative washback effect will exist.
2.5.2. Negative Washback
Negative washback is the undesirable effect on teaching and learning. If a test has negative consequences on learning/teaching, it can be said that it has negative impact on learning/teaching. To illustrate, let’s say a teacher wants to teach “if clauses “in English. After going through the subject, if the teacher makes an exam including this topic, students will study it. However, if the teacher says that “if clauses” will not be asked in the test, students will not tend to study. Students’ not studying the subject will have a negative impact on learning. Hence, the test will have a negative effect on the learning of students.
About the negative effects of tests on teaching, Buck (1992) says that “it seems likely that translation tests could have very negative washback indeed, and lead to activities which would not be beneficial to second language learners” (as cited in Watanabe, 1996: 319). However, his opinion has been criticized since it depends on just self-reports of the learners rather than a systematic empirical research (see Watanabe, 1996).
Messick (1996) states that construct under-representation and construct-irrelevant variance are the sources of negative washback. The tests may fail to reflect the learning principles and/or course objectives to which they are supposedly related. In Turkish educational context, most of the exams have negative washback as test takers study only the subjects that the tests demand. Noble and Smith (1994) and Smith (1991) talk about negative consequences of teaching to the test that teachers have a tendency to ignore subjects and activities that do not directly contribute to passing the exam, and prefer to drill on multiple-choice worksheets which are likely to boost the scores but unlikely to promote general understanding. This is the very case in Turkey. In terms of influences of high-stakes tests, Smith et al. (1990) found that teachers neglect the materials that are not tested because of the pressure to improve students’ test scores. This is especially true for KPDS and ÜDS in which teachers have a tendency to ignore subjects and activities that do not directly contribute to passing these exams. Even the sequence of the curriculum is changed according to the content of the test based on past exam papers (Madaus, 1988). Alderson and Wall (1993: 115) also cite Vernon’s (1956: 166) quote that “teachers tend to ignore subjects and activities which are not directly related to passing the exam so that examinations distort the curriculum”. In none of KPDS and
ÜDS private courses, do the teachers teach the skills of speaking, listening and writing due to the structure of these tests.
Madaus (1988) argues that the long term negative effects of using measurement as the primary motivating factor of the educational process on curriculum, teaching, and learning, outweigh positive benefits attributed to it. According to Hamp-Lyons (1997) there are two essential problems with standardized tests. First, they fail to adequately measure student learning. Even more important, their use has encouraged, or at least helped perpetuate classroom practices that fail to provide high-quality education, particularly for children from low-income families. The reasons for this include:
1- The multiple choice format 2- Norm-referencing
3- Making decisions using one test
4 -The use of these tests for accountability (cited in Hamp-Lyons,1997)
According to Shohamy (1992) some conditions may cause negative washback: 1-When the writing of tests does not involve those who are expected to carry out the changes from the test.
2-When emphasis is mostly on proficiency and less means that lead to it
3- When the information that a test provides is not detailed and specific and does not contain meaningful feedback, it is difficult to expect that tests will lead meaningful improvement in learning.
4- When reliance is on tests to create change
5-When tests are introduced as authoritative tools, are judgmental, are prescriptive, and dictated from above.
In conclusion, Cheng and Curtis (2004) suggest that the educational context in which washback appears may have a role on determining the type of washback. They talk about this educational context with five wh questions. The meanings of these five wh questions are as follows:
1-the people who teach and manage the program, 2-the school where the teaching and testing take place,
3-when the program takes place (including the length of the program and a
5-How stands for different teaching and testing methods applied by the people in
that context.
According to Pearson, (1988) positive or negative washback is dependent to the quality of the test. He suggests that a test’s washback effect will be negative if it fails to reflect the learning principles and/or course objectives to which it supposedly relates, and it will be positive if the effects are useful and “encourage the whole range of desired changes.” However, Alderson and Wall (1993) assert that the quality of washback effect might be independent to the quality of a test. So, any test, good or bad, can be said to have useful or detrimental washback depending on the educational context. Davies (1985) suggests that a good test should be “an obedient servant of teaching” so that it may result in positive washback.
The teachers and the administration of a school that make important decisions on the methods of teaching, length of teaching and curriculum, can lead to washback to be positive or negative. As an example, some high school administers and teachers in Turkey may adjust curriculum on the basis of LYS and YGS. Though school curriculum is not test based, it may turn out to be so and have negative or positive washback in return.
2.5.3. Neutral Washback
If a test does not affect education in any way, then it is called neutral washback. These kinds of tests have neither positive nor negative effects on learning of the test-takers.
2.6. Review of Washback Studies
So many researchers have investigated the effects of washback on learning/teaching so far. Through this part, these researches will be summed. The results of these studies are very important in that they will let us develop our education and plan the curricula accordingly. Whereas some studies investigated the impact of world-wide-known high stakes test like TOEFL, TOEIC, and IELTS (Alderson and Hamp-Lyons, 1996; Newsfields, 2005; Reynolds, 2010; Rhami and Nazland, 2010) the others (Cheng, 1997; Hsu, 2009; Manjarres, 2005; Mohammadi, n.d; Shih, 2009; Watanabe, 1996) investigated nationwide exams. Saville (2009) tabulated (table 1) washback studies of researchers, methods of research and where the research is conducted. Some of these studies include test takers, language teachers, test developers, teacher trainers,
curriculum planners, teacher advisors, head teachers and administrators, inspectors, and material designers as participant. The summary of these studies is tabulated in Table 1: Table 1 Research conducted on candidates taking a test.
Researchers Research Methods Geographical Context Fullive (1992)
Observation
and interview Hong Kong Cohen (1984) Questionnaire United States Wall and
Alderson (1993) Observation Sri Lanka Ingulsrud (1994) Observation Japan Sturman (1996)
Questionnaire and
Ethnographic Japan Shohamy, Donitsa
and Ferman (1996) Questionnaire Israel
Cheng (1997) Questionnaire Hong Kong
Alderson and
Hamp-Lyons (1996) Group discussion United States Hughes(1988) Criterion-related Turkey
Table 2 Research conducted on Language Teachers
Researchers Research Methods Geographical Context
Hughes(1988) Observation Turkey
Cheng (1997)
Interview
and observation Hong Kong
Lam (1994) Questionnaire Hong Kong
Andrews (1994a) Questionnaire Hong Kong Wall and
Alderson (1993) Observation Sri Lanka Shohamy et al.
(1996) Observation Israel
Watanabe (1996) Observation Japan
Alderson and Hamp-Lyons (1996)
Interview
and observation United States Wall (1999)
Interview
Table 3 Research conducted on Test developers
Researchers Research Methods Geographical Context
Andrews(1994) Interview Hong Kong
Andrews
and Fullilove(1994) Interview Hong Kong
Table 4 Research conducted on Teacher trainers and curriculum planners Researchers Research Methods Geographical Context Andrews
and Fullilove(1994) Interview Hong Kong
Table 5 Research conducted on Teacher advisors
Researchers Research Methods Geographical Context Wall and Alderson
(1994) Interview Sri Lanka
Table 6 Research conducted on Head teachers and other school administrators Researchers Research Methods Geographical Context
Fullilove(1992) Interview Hong Kong
Hughes(1997) Observation Turkey
Shohamy (1993)
Interview
and questionnaire Israel Shohamy et al.
(1996)
Interview
and questionnaire Israel
Table 7 Research conducted on Inspectors
Researchers Research Methods Geographical Context Shohamy et al.
(1996) Interview Israel
Table 8 Research conducted on Score users
Researchers Research Methods Geographical Context Andrews
Table 9 Research conducted on Parents
Researchers Research Methods Geographical Context Andrews (1994b) Observation Hong Kong
Cheng (1997)
Student questionnaire Eliciting parents’
behaviour Hong Kong
Fullilove (1992) Hong Kong
Ingulsrud
(1994) Indirect observation Japan
Shohamy et al. (1996)
Student questionnaire Eliciting parent’s
opinions Israel
Table 10 Research conducted on Material developers
Researchers Research Methods Geographical Context Cheng (1997)
Interview
and observation Hong Kong
Table 11 Research conducted on Teaching Processes
Researchers Research Methods Geographical Context Alderson and Wall
(1993) Observation Sri Lanka
Turner (2011)
Observation
and interview Canada
While going through the history of washback studies, it would be appropriate to start with the study of Wall and Alderson (1993) which was conducted in Sri Lanka, where a new language examination was employed for the first time. A series of textbooks introduced new ideas to teach ESL in terms of content and methodology. These textbooks included communicative tasks. The target of the new examination was reinforcing the ideas introduced in the textbooks. Wall and Alderson observed fourteen schools to see how teachers used these text books and how they viewed their own teaching and probable effects upon it. The observations continued after the test was introduced for the first time, and later on, follow-up interviews were conducted. According to findings, some of the teachers used the textbook, but it was hard to tell whether they were using the text books to get students be prepared for the exam or to
textbook in order to practice for the test. Another result was that since the test did not involve communicative skills, less time and attention was paid to practice oral skills than writing skills. Another interesting finding was that towards the third term of the academic year, as the time of the exam got closer, teachers abandoned the textbooks and focused more on past exam papers and commercial exam preparation publications. This was a clear sign of the effect of the test on education (Wall and Alderson, 1993, p.61-62).
One of the most interesting studies conducted on washback effect is the one which deals with TOEFL. Alderson and Hamp-Lyons (1996) investigated TOEFL and its washback effects on instruction. Through the study, the researchers were not trying to prove or falsify any of the Washback Hypotheses; they were simply trying to understand how everything was going on in TOEFL preparation classrooms (Alderson and Hamp-Lyons, 1996, p. 282). The research project focused on teachers who teach TOEFL preparation classes. The researchers used classroom observations, teacher interviews, and exploratory student interviews to gather data. They were trying to find out how TOEFL examination affected classroom instruction, teacher attitudes, and student attitudes, (Alderson and Hamp-Lyons, 1996). It was in this study that the
Washback Hypotheses were created in an attempt to better understand the complexities
associated with washback. They compared their findings with non TOEFL preparation classes. The results of the study revealed that TOEFL does affect teaching even when instructors are the same for a TOEFL focused classroom and non-TOELF focused classroom. Other than the test itself, there might be other variables affecting the teaching.
Watanabe (1996) observed two different English exam-preparation classes taught by two different teachers. Whereas one of the classrooms was grammar-translation oriented, the other was not. From the classroom observations, it was found that translation oriented university entrance examinations do not influence the two teachers in the same way, that is, the examinations induce washback on one teacher, but not on the other. Watanabe, however, thinks that teacher factors, such as teachers’ educational background, personal beliefs, and teaching experience might have some roles in the results. He concluded that such factors might outweigh the effect of the entrance examinations. Interviews with the teachers prior to classroom observations and
also pre- and post-observation discussions were conducted with each teacher. From the study it could be concluded that the entrance examination caused only some types of negative washback to only some aspects of some teachers’ lessons.
Another famous high stakes test research project is Liying Cheng’s (2000) The
Washback Effect on Classroom Teaching of Changes in Public Examinations. The study
focused on the changes introduced in 1993 to the Hong Kong Certificate of Education Examination in English (HKCEE). Changes in the test resulted in changes in the role of the student as a language learner, demanding that learners meet expectations that are necessary for the development of skills, and the completion of the tasks. The goal of the study was to determine the extent of the changes in the test. These actually resulted in classroom changes for teaching and learning English (Cheng, 2000; Cheng, 2004). The study showed that “the change of the HKCEE toward an integrated and task-based approach showed teachers the possibility of something new, but it did not necessarily enable teachers to teach something new” (Cheng, 2004, p. 164). In other words, when a change is forced through a test, it does not necessarily mean that the intended changes or improvements will occur. So, we can say that the test did not have any intended washback effects.
In Israel, another washback study was conducted by Shohamy (1992) when three new language tests were first introduced. The first one of these was a reading test which administration did not continue to employ as the quality of the test was criticized by testing researchers; the questions were insensitive to cultural differences and very biased; and the test was badly administered. The second test was an oral EFL test which resulted in practicing oral language but limited to the activities and tasks that are similar to the ones in the test. The last test was an Arabic as a Second Language (ASL) test which showed an impact on teaching and learning activities in preparation of the test in terms of stopping teaching new material and turning to reviewing material; replacing class textbooks with worksheets that were identical to previous years’ tests; adding review sessions. Shohamy, Donitsa-Schmidt, and Ferman (1996) wanted to reveal the impacts of the last two tests; Arabic as a Second Language (ASL) and EFL. They collected data from three types of instruments. First, they gave questionnaires to students about the awareness of the test, allotted time to teaching activities and practicing for the test and so on. Then, they had some interviews with teachers and
