• Sonuç bulunamadı

An experimental study of the multiple-choice test technique for measuring reading comprehension of EFL students

N/A
N/A
Protected

Academic year: 2021

Share "An experimental study of the multiple-choice test technique for measuring reading comprehension of EFL students"

Copied!
81
0
0

Yükleniyor.... (view fulltext now)

Tam metin

(1)

V.

MULTIPLE - CHOICE TEST

TECHNIQUE FOR MEASURING

READiNG COMPREHENSION OP EFL STUDEIC’

r n r - .·

r·,». X ri P Qw u.,/ L w

:z ;L e i m t i t u z t r - - T\ ^ <'* y··.,'·' . j V' ; c le n c e s o f B ill:erus U mI 1 ecu it y N 1 P a IfC ix £ ^ -:t C/f ·vi-d C'iCC c i C'5r‘T···"·'"·

th e D e g r e e o f MS:¿cer o t A r ie

(2)

A B S T R A C T S i n c e r e a d i n g is an i n t e g r a t i v e a n d d i f f i c u l t s k i l l to m e a s u r e no s i n g l e t e s t a d e q u a t e l y m e a s u r e s r e a d i n g a b il it y. T e s t i n g r e a d i n g c o m p r e h e n s i o n is p r o b l e m a t i c , e s p e c i a l l y in a f o r e i g n or s e c o n d l a n g u a g e s e t t i n g , b e c a u s e t e s t r e s u l t s m a y be invalid; a r e a d i n g c o m p r e h e n s i o n te st s c o r e m a y be m e a s u r i n g s o m e t h i n g o t h e r t h a n the r e a d i n g c o m p r e h e n s i o n a b i l i t y of the s t u d e n t . T h e p u r p o s e of th is s t u d y is to t e s t two r e a d i n g c o m p r e h e n s i o n t e s t i n g t e c h n i q u e s , m u l t i p l e - c h o i c e a n d r e t e l l i n g by c o n d u c t i n g a n e x p e r i m e n t u s i n g i n t e r m e d i a t e level E F L s t u d e n t s f r o m B i l k e n t U n i v e r s i t y S c h o o l o f E n g l i s h L a n g u a g e ( B U S E D as s u b j e c t s . T h e goal of the e x p e r i m e n t is to c o r r e l a t e t h e s e two t e s t i n g t e c h n i q u e s w i t h two o t h e r m e a s u r e s of r e a d i n g c o m p r e h e n s i o n a b i l i t y , t e a c h e r r a t i n g s a n d t h e r e a d i n g s e c t i o n of a BU S E L a c h i e v e m e n t exam. The c o r r e l a t i o n s of t h e s e f o u r m e a s u r e s w e r e done u s i n g the P e a r s o n P r o d u c t M o m e n t C o r r e l a t i o n a n d the T e t r a c h o r i c C o r r e l a t i o n . T h e r e s u l t s s h o w e d th at the m u l t i p l e - c h o i c e r e a d i n g t e s t di d not c o r r e l a t e s i g n i f i c a n t l y w i t h the o t h e r t h r e e m e a s u r e s , i m p l y i n g t h a t the m u l t i p l e - c h o i c e t e s t did not s u c c e s s f u l l y p r e d i c t the r e a d i n g a b i l i t i e s of s t u d e n t s . O n the o t h e r hand, the c o r r e l a t i o n of the r e t e l l i n g p r o c e d u r e w i t h the o t h e r m e a s u r e s s h o w e d s i g n i f i c a n c e ,

(3)

r e a d i n g c o m p r e h e n s i o n . B e c a u s e the total n u m b e r o-f s u b j e c t s in t h i s s t u d y w a s small, o n l y 18, we s h o u l d h e s i t a t e be-Fore g e n e r a l i z i n g a n d a p p l y i n g t h e s e r e s u l t s to o t h e r E F L s i t u a t i o n s . H o w e v e r , the -findings o f the e x p e r i m e n t s h o u l d be o f i n t e r e s t to all t h o s e i n v o l v e d in t e s t i n g the r e a d i n g c o m p r e h e n s i o n a b i l i t y o f f o r e i g n or s e c o n d l a n g u a g e s t u d e n t s .

(4)

AN E X P E R I M E N T A L S T U D Y OF T H E M U L T I P L E - C H O I C E T E ST T E C H N I Q U E F O R M E A S U R I N G R E A D I N G C O M P R E H E N S I O N OF E F L S T U D E N T S A T H E S I S S U B M I T T E D TO T H E I N S T I T U T E OF E C O N O M I C S A N D S O C I A L S C I E N C E S OF B I L K E N T U N I V E R S I T Y IN P A R T I A L F U L F I L L M E N T OF THE R E Q U I R E M E N T S F O R THE D E G R E E OF M A S T E R OF A R T S IN T H E T E A C H I N G OF E N G L I S H AS A F O R E I G N L A N G U A G E BY D E M E T c e l e b i A U G U S T 1991 Q j£ 1 e i a L ' 1'.

(5)

■cus

ізві

(6)

1 1 B I L K E N T U N I V E R S I T Y I N S T I T U T E OF E C O N O M I C S A N D S O C I A L S C I E N C E S MA T H E S I S E X A M I N A T I O N R E S U L T F O R M J u l y 31, 1971 Th e e x a m i n i n g c o m m i t t e e a p p o i n t e d by th e I n s t i t u t e o-F E c o n o m i c s a n d S o c i a l S c i e n c e s -For th e t h e s i s e x a m i n a t i o n o-F the MA T E F L s t u d e n t D e m e t C e le bi

has r e a d t h e t h e s i s o-F the s t u d e n t . The c o m m i t t e e has d e c i d e d th at t h e t h e s i s of the s t u d e n t is s a t i s f a c t o r y . T h e s i s t i t l e A n E x p e r i m e n t a l S t u d y of the M u l t i p l e - C h o i c e Test T e c h n i q u e for M e a s u r i n g R e a d i n g C o m p r e h e n s i o n o f E F L S t u d e n t s . T h e s i s A d v i s o r : Mr. W i l l i a m A n c k e r B i l k e n t U n i v e r s i t y , MA T E F L P r o g r a m C o m m i t t e e M e m b e r s : Dr. J a m e s C. S t a l k e r B i l k e n t U n i v e r s i t y , MA T E F L P r o g r a m Dr. L i o n e l K a u f m a n B i l k e n t U n i v e r s i t y , MA T E F L P r o g r a m

(7)
(8)

I l l W e c e r t i f y t h a t we h a v e r e a d c o m b i n e d o p i n i o n it is f u l l y q u a l i t y as a t h e s i s f o r t h e th is t h e s i s a n d t h a t in our a d e q u a t e , in s c o p e a n d in d e g r e e o f M a s t e r o f Arts. iy\ W i l l i a m A n c k e r ( A d v i s o r ) lames C. S t a l k e r ( C o m m i t t e e M e m b e r ) L i o n e l K a u f m a n ( C o m m i t t e M e m b e r ) A p p r o v e d for the I n s t i t u t e o f E c o n o m i c s a n d S o c i a l S c i e n c e s I n s t i t u t e o f E c o n o m i c s a n d S o c i a l S c i e n c e s

(9)

TABLE OF CONTENTS

SECTIONS

PAGES

1.0 I N T R O D U C T I O N 1 . 1 B a c k g r o u n d o-F th e S t u d y 1.2 S t a t e m e n t of t h e R e s e a r c h Q u e s t i o n 1.3 H y p o t h e s e s 1.3.1 Null H y p o t h e s i s 1 . 3. 2 E x p e r i m e n t a l H y p o t h e s i s 1.4 P u r p o s e a n d M e t h o d o l o g y of the S t u d y 1.5 O r g a n i z a t i o n o f t h e S t u d y 1 2 3 3 3 3 5 2 . 0 R E V I E W OF L I T E R A T U R E 2.1 R e a d i n g C o m p r e h e n s i o n 2. 2 T e s t i n g R e a d i n g C o m p r e h e n s i o n 2 . 3 M u l t i p l e - C h o i c e T e s t s 2 . 4 S o m e W e a k n e s s e s s a n d S t r e n g t h s of M u l t i p l e - C h o i c e T e s t s 2 . 5 S u g g e s t i o n s for P r e p a r i n g B e t t e r M u l t i p l e - C h o i c e R e a d i n g T e s t s 2. 5.1 S e l e c t i o n of M u l t i p l e - C h o i c e D i s t r a c t o r s 2 . 5 . 2 H o w to I m p r o v e M u l t i p l e - C h o i c e R e a d i n g T e s t s 6 9 10 12 15 15 16

(10)

2 . 5 . 3 An A l t e r n a t i v e A p p r o a c h to M u l t i p l e - C h o i c e R e a d i n g Items 2 . 6 R e t e l l i n g - A M o r e D i r e c t W a y o-F T e s t i n g R e a d i n g 2 . 7 C o n c l u s i o n s 17 18 19 3 . 0 M E T H O D O L O G Y 3.1 I n t r o d u c t i o n 20 3 . 2 P a r t i c i p a n t s a n d S e t t i n g 21 3 . 2. 1 S u b j e c t s 21 3 . 2 . 2 E x p e r i m e n t e r s 22 3 . 3 M a t e r i a l s 23 3 . 4 Two O u t s i d e M e a s u r e s U s e d as the C r i t e r i a for M e a s u r i n g R e a d i n g A b i l i t y 24 3 . 5 P r o c e d u r e s o f D a t a C o l l e c t i o n 26 4 . 0 D A T A A N A L Y S I S 4.1 I n t r o d u c t i o n 4 . 2 D a t a A n a l y s i s 4. 2.1 P e a r s o n P r o d u c t M o m e n t C o r r e l a t i o n 4 . 2 . 2 T e t r a c h o r i c C o r r e l a t i o n 4 . 3 R e s u l t s 28 29 31 33 40 5 . 0 C O N C L U S I O N S 5.1 S u m m a r y of the S t u d y 4 2

(11)

5 . 2 A s s e s s m e n t o f t h e S t u d y 44 5 . 3 P e d a g o g i c a l I m p l i c a t i o n s 46 5. 4 I m p l i c a t i o n s for F u t u r e R e s e a r c h 47 B I B L I O G R A P H Y 50 A P P E N D I X A 52 A P P E N D I X B 5 3 A P P E N D I X C 54 A P P E N D I X D 56 A P P E N D I X E 58 A P P E N D I X F 59 A P P E N D I X 6 . 1 , 2 , 3 , 4 , 5 , 6 60 A P P E N D I X H . 1,2,3 66

(12)

LIST OF TABLES

V I 1

TABLES

PAGES

T a b l e 4.1 P P M C C a l c u l a t i o n s B e t w e e n th e F o u r T e s t s T a b l e 4. 2 T e t r a c h o r i c C o r r e l a t i o n B e t w e e n the M u l t i p l e - C h o i c e T e s t a n d th e T e a c h e r s ’ R a t i n g 31 37 T a b l e 4 . 3 T e t r a c h o r i c C o r r e l a t i o n B e t w e e n R e t e l l i n g a n d the T e a c h e r s ’ R a t i n g 38 T a b l e 4. 4 T e t r a c h o r i c C o r r e l a t i o n B e t w e e n the B U S E L R e a d i n g E x a m a n d the T e a c h e r s ’ Rati ng 4 0

(13)

A C K N O W L E D G E M E N T S I am i n d e p t e d to my a d v i s o r Mr. W i l l i a m A n c k e r for his g u i d a n c e , f e e d b a c k a n d e n c o u r a g e m e n t w h i l e w r i t i n g th is th esis. I w o u l d like to e x p r e s s my t h a n k s to Dr. J a m e s C. S t a l k e r a n d Dr. L i o n e l K a u f m a n for th eir v a l u a b l e c o m m e n t s a n d p r o f e s s i o n a l a s s i s t a n c e . I o w e s p e c i a l t h a n k s to my c o l l è g u e s at B U S E L a n d my c l a s s m a t e s in the M A T E F L P r o g r a m . W i t h o u t t h e i r he lp it w o u l d h a v e b e e n i m p o s s i b l e to c a r r y ou t t h is s t u d y . My s i n c e r e t h a n k s go to my c l a s s m a t e Aysel S u t c u for her e n d l e s s m o ra l s u p p o r t a n d c o o p e r a t i o n . I w o u l d a l s o like to e x p r e s s my deep a p p r e c i a t i o n to my f a t h e r a n d my b r o t h e r - i n - l a w for t h e i r a s s i s t a n c e in u s i n g t h e c o m p u t e r .

(14)

C H A P T E R I I N T R O D U C T I O N 1 . 1 B a c k g r o u n d of the S t u d y In E n g l i s h - m e d i u m u n i v e r s i t i e s r e a d i n g c o m p r e h e n s i o n is c o n s i d e r e d to be o n e o-F the m o s t i m p o r t a n t s k i l l s as s t u d e n t s a r e r e q u i r e d to do all t h e i r a c a d e m i c r e a d i n g in E n g l i s h . R e a d i n g c o m p r e h e n s i o n t e s t s a r e u s e d in a s s e s s i n g the c o m p r e h e n s i o n of r e a d e r s in g e n e r a l a n d E E L s t u d e n t s . T h e r e are a v a r i e t y of t e s t s t h a t a r e u s e d in m e a s u r i n g d i f f e r e n t r e a d i n g s k i l l s f o r e x a m p l e , p i c t u r e s a n d s e n t e n c e m a t c h i n g , t r u e - f a l s e tests, c l o z e tests, m u l t i p l e - c h o i c e tests, a n d r e t e l l i n g . A m o n g t h e s e t e s t s m u l t i p l e - c h o i c e t e s t s a r e w i d e l y u s e d as o b j e c t i v e tests, e s p e c i a l l y to m e a s u r e r e a d i n g c o m p r e h e n s i o n , but w h e t h e r or not t h e s e t e s t s a s s e s s c o m p r e h e n s i o n a d e q u a t e l y is a q u e s t i o n . A c c o r d i n g to A s l a n i a n (1985), m u l t i p l e - c h o i c e t e s t s a r e an i n a d e q u a t e m e a n s of a s s e s s i n g the c o m p r e h e n s i o n o f r e a d e r s in g e n e r a l a n d of E S L / E F L s t u d e n t s in p a r t i c u l a r . A s l a n i a n s t a t e s that c o m p r e h e n s i o n is a c o m p l e x p h e n o m e n o n a n d c a n n o t be m e a s u r e d by o b j e c t i v e t e s t s al one; d i s c u s s i n g the w e a k n e s s e s o f m u l t i p l e - c h o i c e r e a d i n g tests, sh e says: A m o n g th e d r a w b a c k s i n h e r e n t in m u l t i p l e - c h o i c e r e a d i n g t e s t s is the f a c t t h a t m o s t p r o v i d e s a m p l e p a s s a g e s t h a t a r e too t r u n c a t e d or i n c o m p l e t e to r e s e m b l e the no rmal p r i n t e d

(15)

T h e s e p a s s a g e s o f t e n lack a p r o p e r i n t r o d u c t i o n , s u f f i c i e n t c o n t e n t d e v e l o p m e n t , a n d a c o h e r e n t c o n c l u s i o n . To e x p e c t a reader, e s p e c i a l l y an ES L reader, to u n d e r s t a n d the o m i s s i o n s t h a t ha v e be en m a d e to r e n d e r the t e s t e c o n o m i c a l is to e x p e c t c o n f u s i o n , not c o m p r e h e n s i o n . (1985, p. 1)

Heat on , o n the o t h e r hand, a r g u e s t h a t a " m u l t i p l e -c h o i -c e t e s t o f f e r s a u s e f u l w a y of t e s t i n g r e a d i n g c o m p r e h e n s i o n , a n d the e x t e n t to w h i c h a t e s t is s u c c e s s f u l in m e a s u r i n g w h a t it s e t s o u t to m e a s u r e d e p e n d s l a r g e l y o n the e f f e c t i v e n e s s o f e a c h of the items u s ed " (1988, p. 117).

U n l i k e m u l t i p l e - c h o i c e , w h i c h is a r e c o g n i t i o n t y pe of te st a n d the r e s u l t s of w h i c h do not n e c e s s a r i l y s h o w c o m p r e h e n s i o n , the r e t e l l i n g p r o c e d u r e is a m o r e d i r e c t a n d v a l i d w a y of t e s t i n g r e a d i n g c o m p r e h e n s i o n as h e re the t e s t e e h a s to r e p e a t the t e x t in h i s / h e r o w n words. In the r e t e l l i n g p r o c e d u r e the t e s t e e will e i t h e r be a b l e to i n t e r p r e t the te xt c o r r e c t l y or not, w h i c h will s h o w w h e t h e r c o m p r e h e n s i o n took place. 1.2 S t a t e m e n t o f R e s e a r c h Q u e s t i o n T h e p u r p o s e of th is s t u d y w a s to f i n d o u t how well m u l t i p l e - c h o i c e t e s t i n g a n d r e t e l l i n g t e s t i n g m e a s u r e s t u d e n t s ’ r e a d i n g c o m p r e h e n s i o n . To f i n d ou t the v a l i d i t y of t h e s e two r e a d i n g t e s t s t h e y w e r e c o m p a r e d a n d c o r r e l a t e d w i t h t e a c h e r r a t i n g s a n d B U S E L r e a d i n g

(16)

e x a m re su lt s. 1.3 H y p o t h e s e s 1.3.1 Null H y p o t h e s i s a. T h e r e is no c o r r e l a t i o n b e t w e e n t h e m u l t i p l e - c h o i c e r e a d i n g t e s t a n d the t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g exam. b. S i m i l a r l y , t h e r e is no c o r r e l a t i o n b e t w e e n the r e t e l l i n g p r o c e d u r e a n d t h e t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g exam. 1.3.2 E x p e r i m e n ta l H y p o t h e s i s a. T h e r e is a p o s i t i y e c o r r e l a t i o n b e t w e e n the m u l t i p l e - c h o i c e r e a d i n g t e s t a n d the t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g exam. b. T h e r e is a p o s i t i y e c o r r e l a t i o n b e t w e e n the r e t e l l i n g p r o c e d u r e a n d t h e t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g exam. 1.4 P u r p o s e a n d M e t h o d o l o g y of the S t u d y M u l t i p l e - c h o i c e t e s t s a r e w i d e l y u s e d in the p r e p a r a t o r y s c h o o l s o-F E n g l i s h - m e d i u m u n i y e r s i ties, i n c l u d i n g B i l k e n t U n i y e r s i t y S c h o o l o-f E n g l i s h L a n g u a g e (BUSEL), To r a s s e s s i n g r e a d i n g c o m p r e h e n s i o n w i t h o u t r e a l l y q u e s t i o n i n g the y a l i d i t y of t h e s e t e s t s as m e a s u r e s oT c o m p r e h e n s i o n . T h i s s t u d y a i m s at f i n d i n g how well the m u l t i p l e - c h o i c e r e a d i n g t e s t a n d the

(17)

a r e a d i n g p a s s a g e . In o r d e r to -find o u t w h e t h e r the m u l t i p l e - c h o i c e r e a d i n g t e s t or the r e t e l l i n g p r o c e d u r e (or both tests) a r e a p p r o p r i a t e m e a n s o-F a s s e s s i n g r e a d i n g c o m p r e h e n s i o n , t h e y w e r e c o m p a r e d a n d c o r r e l a t e d w i t h the t e a c h e r r a t i n g s (see C h a p t e r IV, s e c t i o n 4.2.3) a n d the r e s u l t s o-F the B U S E L r e a d i n g e x a m (see A p p e n d i x A). So t e s t i n g the c o m p r e h e n s i o n o-F a r e a d i n g t e xt t h r o u g h a m u l t i p l e - c h o i c e t e s t a n d t e s t i n g the s a m e t e xt t h r o u g h r e t e l l i n g w e r e c o r r e l a t e d w i t h t h e two o u t s i d e m e a s u r e s , t e a c h e r r a t i n g s a n d B U S E L r e a d i n g exam, to f i n d o u t w h i c h o f the two r e a d i n g t e s t i n g t e c h n i q u e s best m e a s u r e the r e a d i n g a b i l i t i e s of s t u d e n t s . The s t u d y w a s c o n d u c t e d w i t h 18 i n t e r m e d i a t e level B i l k e n t U n i v e r s i t y S c h o o l o f E n g l i s h L a n g u a g e s t u d e n t s . The s t u d e n t s ’ two c l a s s t e a c h e r s w e r e a s k e d to r a t e e a c h of the s t u d e n t s ’ r e a d i n g a b i l i t i e s on a r e a d i n g r a t i n g scale. In a d d i t i o n , the m o s t r e c e n t B U S E L r e a d i n g e x a m r e s u l t s w e r e used. T h e s e two r a t i n g s w e r e t a k e n as the c r i t e r i a f o r d e t e r m i n i n g w h e t h e r t h e tw o t e s t s - m u l t i p l e - c h o i c e a n d r e t e l l i n g - m e a s u r e s t u d e n t s ’ r e a d i n g a b i l i t i e s . S u b j e c t s w e r e f i r s t g i v e n a r e a d i n g p a s s a g e w i t h 8 m u l t i p l e - c h o i c e it ems a n d w e r e a s k e d to c o m p l e t e it (see A p p e n d i x B ) . A f t e r the f i r s t ta sk w a s c o m p l e t e d s u b j e c t s w e r e a s k e d to r e te ll the r e a d i n g p a s s a g e by r e p e a t i n g t h e i r u n d e r s t a n d i n g of the p a s s a g e in t h e i r o w n

(18)

words. S u b j e c t s w e r e t h e n g r a d e d o n t h e s e two dif-Ferent t y p e s of t e s t s a n d the s c o r e s o b t a i n e d w e r e c o m p a r e d . A c o r r e l a t i o n a l s t u d y w a s do ne b e t w e e n t h e s e two t e s t s a n d the t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g e x a m as to w h i c h t e s t w a s a b e t t e r m e a n s of a s s e s s i n g r e a d i n g c o m p r e h e n s i o n . 1.5 O r g a n i z a t i o n of t h e S t u d y C h a p t e r 11 - T h i s c h a p t e r g i v e s a r e v i e w of l i t e r a t u r e a b o u t r e a d i n g c o m p r e h e n s i o n , m u l t i p l e - c h o i c e tests, a n d r e t e l l i n g . C h a p t e r III - T h i s c h a p t e r g i v e s a d e t a i l e d e x p l a n a t i o n of the m e t h o d o l o g y u s e d for c o l l e c t i n g a n d a n a l y s i n g th is data. C h a p t e r IV - In t h i s c h a p t e r data is p r e s e n t e d a n d the p r o c e d u r e c a r r i e d o u t to a n a l y s e data is e x p l a i n e d s t ep by step. T h i s c h a p t e r a l s o g i v e s a n idea a b o u t the r e s u l t s o f the study.

C h a p t e r V - T h i s fi nal c h a p t e r g i v e s an o v e r a l l s u m m a r y of the s t u d y f o c u s i n g o n the resu lt s. In t h i s c h a p t e r the h y p o t h e s i s of t h e s t u d y a n d the o b t a i n e d r e s u l t a n d the i m p l i c a t i o n s for f u r t h e r s t u d y a r e d i s c u s s e d .

(19)

R E V I E W OF L I T E R A T U R E

2.1 R e a d i n g C o m p r e h e n s i o n

R e a d i n g is o n e o f t h e m o s t i m p o r t a n t s k i l l s t h a t is t a u g h t at E n g l i s h - m e d i u m u n i v e r s i t i e s . T e a c h i n g t h is sk ill is g i v e n g r e a t e m p h a s i s but f i r s t of all it is i m p o r t a n t to u n d e r s t a n d w h a t r e a d i n g is in o r d e r to t e a c h it s u c c e s s f u l l y . T h e r e a r e s e v e r a l d e f i n i t i o n s of re ad in g. A c c o r d i n g to R i l e y (1977) r e a d i n g is a r e c e p t i v e skill, in t h e s e n s e that, the r e a d e r is e x p e c t e d to d e c o d e t h e v i s u a l f o r m of the te xt into c o m p r e h e n s i b l e m e s s a g e s in the brain. T h e r e a d e r has to i n t e g r a t e his r e a d i n g a b i l i t i e s in o r d e r to d e c o d e the m e s s a g e c o r r e c t l y . R i l e y (1977, p. 3) q u o t i n g f r o m G o o d m a n f u r t h e r e x p l a i n s the c o n c e p t as: The r e a d e r a s s o c i a t e s t h e s e m a n t i c p r o p e r t i e s of l a n g u a g e to f o r m w h i c h a p p e a r s o n t h e p a p e r as a n i n t e g r a t i o n of all g r a m m a t i c a l u n i t s of a la ng ua ge , s i n c e the w r i t t e n f o r m a n d the s p o k e n f o r m of E n g l i s h is r a t h e r d i f f e r e n t , a p h o n e m i c c o r r e s p o n d e n c e is well n e c e s s a r y to c o m p r e h e n d the w r i t t e n form. As p o i n t e d o u t above, to d e c o d e the m e s s a g e a n d to r e a d for c o m p r e h e n s i o n , t h e r e a d e r has to u s e a n u m b e r of s u b s k i l l s d u r i n g the r e a d i n g pr oc e s s . A c c o r d i n g to R i l e y (1977, p. 4), a n i n t e l l i g e n t r e a d e r has to be a b l e to us e the f o l l o w i n g s u b s k i l l s w h i l e r e a d i n g fo r c o m p r e h e n s i o n : 1) S e l e c t m a i n idea f r o m the p a s s a g e 2) S e l e c t r e l e v a n t d e t a i l s to s u p p o r t the m a i n

(20)

i dea 3) R e c o g n i z e i r r e l e v a n c i e s , c o n t r a d i c t i o n s a n d non s e q u i t u r s 4) U s e logical c o n n e c t o r s a n d s e q u e n c e s i g n a l s 5) D r a w c o n c l u s i o n s 6) M a k e g e n e r a l i z a t i o n s 7) A p p l y p r i n c i p l e s to o t h e r i n s t an ce s. All t h e s e s u b s k i l l s c o n s t i t u t e a w h o l e in c o m p r e h e n d i n g a w r i t t e n t e x t a n d the t e s t e e is e x p o s e d to a s i m i l a r p r o c e s s in a r e a d i n g c o m p r e h e n s i o n test. G o o d m a n (qtd .i n S i n g e r a n d R u d d e l l , 1985) o n the o t h e r hand, de-Fines c o m p r e h e n s i o n as the e n d p r o d u c t o-F a n y act o-F r e ad in g, a n d s h o w s how c o m p r e h e n s i o n m a y c h a n g e in t h e c o u r s e of t e s t i n g for it; S i n c e c o m p r e h e n s i o n is a c o n s t r u c t i v e p r o c e s s in w h i c h r e a d e r s m a k e s e n s e of t h e text, it g o e s o n d u r i n g r e a d i n g a n d e v e n long a f t e r w a r d s as t h e r e a d e r r e c o n s i d e r s a n d r e c o n s t r u c t s w h a t has b e e n c o m p r e h e n d e d ; thus, c o m p r e h e n s i o n m a y be c h a n g e d in the c o u r s e of t e s t i n g it. T h e r e a d e r m a y c h a n g e w h a t he or she u n d e r s t o o d o n the b a s i s o f the t e st q u e s t i o n s w h i c h s e e m to r e q u i r e p a r t i c u l a r r e s p o n s e s a n d v i ew s, (p. 831)

G a l l o w a y (1988) l o ok s at c o m p r e h e n s i o n in t e r m s of how well the p u r p o s e of th e r e a d e r in a p p r o a c h i n g the t e x t c o r r e s p o n d s w i t h the p u r p o s e o f the w r i t e r in p r e p a r i n g it. V a l e t t e (1977) has a s i m i l a r p o i n t o f v i e w a n d t h i n k s t h a t the m a i n o b j e c t i v e o f r e a d i n g fo r c o m p r e h e n s i o n is to u n d e r s t a n d the w r i t t e n m e s s a g e . If r e a d i n g is c o n s i d e r e d as a c o m m u n i c a t i v e a c t i v i t y , then, t h e r e a l i z a t i o n of c o m m u n i c a t i o n in r e a d i n g is the m o m e n t w h e n the r e a d e r u n d e r s t a n d s the m e s s a g e the w r i t e r w a n t s

(21)

to co nvey. J o h n s t o n (1983) e x a m i n e s r e a d i n g -from a n o t h e r p e r s p e c t i v e a n d s e e s it as, "any r e a d e r i n t e r a c t i o n w i t h the text" (p. 2). He s t a t e s t h a t a r e a d e r c o m p r e h e n d s a t e x t a f t e r i n t e r a c t i n g w i t h it. C o m p r e h e n s i o n is s t o r e d in the r e a d e r ’s m e m o r y a n d c a n be e x a m i n e d by g e t t i n g the r e a d e r to e x p r e s s s e g m e n t s of t h e s t o r e d m a t e r i a l . He f u r t h e r p o i n t s o u t t h a t t h e p r o c e s s e s i n v o l v e d in g e t t i n g t h e r e ar e g i v e n less e m p h a s i s t h a n the final p r o d u c t (i.e. the c o n t e n t s of m e m o r y ) . The p r o d u c t i m p l i e s t h at l o n g - t e r m m e m o r y has a n i m p o r t a n t r o l e in c o m p r e h e n s i o n a n d d e t e r m i n e s how s u c c e s s f u l the r e a d e r is at c o m p r e h e n d i n g . A c c o r d i n g to J o h n s t o n , (1983) c o m p r e h e n s i o n c a n be m e a s u r e d by s t a n d a r d i z e d t e s t s a n d f r e e recall m e a s u r e s . In c o n t r a s t , C a r r o l l (1971, ctd. in J o h n s t o n , 1983, p. 6) a r g u e s t h a t c o m p r e h e n s i o n is a p r o c e s s t h a t t a k e s p l a c e on r e c e p t i o n of i n f o r m a t i o n a n d t h a t o n l y s h o r t t e r m m e m o r y is invo lv ed . He stat es , "As s o o n as l o ng er i n t e r v a l s a r e i n v o l v e d in t h e t e s t i n g o f c o m p r e h e n s i o n , t h e r e is t h e p o s s i b i l i t y t h a t we a r e s t u d y i n g m e m o r y p r o c e s s e s a l o n g with, or in p l a c e of, c o m p r e h e n s i o n p r o c e s s e s " . R o y e r a n d C u n n i n g h a m (1978, in J o h n s t o n , 1983) ta k e i s s u e w i t h bo th the p r o c e s s a n d p r o d u c t a p p r o a c h e s a n d c o n t e n d t h a t "... c o m p r e h e n s i o n p r o c e s s e s a n d m e m o r y

(22)

p r o c e s s e s a r e i n e x t r i c a b l y i n t e r t w i n e d . ..We a s s u m e t h at a c o m p r e h e n d e d m e s s a g e wi ll be r e t a i n e d in m e m o r y b e tt er t h a n an u n c o m p r e h e n d e d m e s s a g e " (p. 2). P e a r s o n a n d J o h n s o n (1978, qtd. in J o h n s t o n , 1983) b e l i e v e t h a t : C o m p r e h e n s i o n is b u i l d i n g b r i d g e s b e t w e e n the new a n d the k n o w n . ..c o m p r e h e n s i o n is a c t i v e not pa ss i v e ; t h a t is, the r e a d e r c a n n o t he lp but i n t e r p r e t a n d a l t e r w h a t he r e a d s in a c c o r d a n c e w i t h p r i o r k n o w l e d g e a b o u t th e t o p i c u n d e r d i s c u s s i o n . C o m p r e h e n s i o n is not s i m p l y a m a t t e r o-f r e c o r d i n g a n d r e p o r t i n g v e r b a t i m w h a t has b e e n read, c o m p r e h e n s i o n

i n v o l v e s a g r e a t deal o-F in-ference making. (p. 24) 2 . 2 T e s t i n g R e a d i n g C o m p r e h e n s i o n B e c a u s e r e a d i n g a n d c o m p r e h e n d i n g w h a t is r e a d ar e in te rn al p r o c e s s e s a n d t a k e s p l a c e in the brain, t h e y ar e d i f f i c u l t to o b s e r v e a n d m e a s u r e . R e s e a r c h does not a l l o w us to say th at t h e r e is a s i n g l e t e s t i n g t e c h n i q u e th at is the m o s t a p p r o p r i a t e a n d r e l i a b l e wa y of t e s t i n g r e a d i n g c o m p r e h e n s i o n . M a d s e n (1983) a g r e e s w i t h the idea th at r e a d i n g c o m p r e h e n s i o n t e s t s a r e d i f f i c u l t a n d says; " T e s t i n g r e a d i n g c o m p r e h e n s i o n is o n e of the m o s t i n t e g r a t i v e a n d c h a l l e n g i n g k i n d s of r e a d i n g te st type s" (p . 87).

O i l e r (1979) a l s o c o n t r i b u t e s to the idea that r e a d i n g t e s t s ar e i n t e g r a t i v e a n d c h a l l e n g i n g a n d s a y s t h a t t h e s e t e s t s r e q u i r e the t e s t e e to r e a d a n d a n s w e r q u e s t i o n s - but t h e y u s u a l l y l e av e o p e n the q u e s t i o n of

(23)

w h a t the t e s t is a t e s t of. A r e a d i n g t e s t m a y be a g o o d t e s t o f o v e r a l l l a n g u a g e p r o f i c i e n c y or a te st of l i s t e n i n g c o m p r e h e n s i o n r a t h e r t h a n b e i n g a g o o d r e a d i n g c o m p r e h e n s i o n test. T h e r e a r e m a n y w a y s to t e s t r e a d i n g c o m p r e h e n s i o n . O n e o f the m o s t c o m m o n a n d o b j e c t i v e w a y s is a r e a d i n g p a s s a g e f o l l o w e d by m u l t i p l e - c h o i c e q u e s t i o n s . As m e n t i o n e d above, c o m p r e h e n s i o n is v i e w e d as a b r i d g e b e t w e e n t h e new a n d the known, but W o o d (1987) does not t h in k t h a t t h i s is the c a s e e s p e c i a l l y w i t h m u l t i p l e -c h o i -c e r e a d i n g tests. S h e s t a t e s her v i e w as f o l l ow s: In d o i n g a m u l t i p l e c h o i c e t e s t ho we ve r, the r e a d e r does not n e c e s s a r i l y b u i l d a b r i d g e b e t w e e n his p a s t e x p e r i e n c e a n d b a c k g r o u n d i n f o r m a t i o n a n d the new. R e a d e r s m u s t s e e w h a t th e t e s t e r w a n t s t h e m to see, w h i c h is o f t e n literal f a c t u a l i n f o r m a t i o n , e a s y to l o c a t e in the t e x t w i t h o u t c a r e f u l reading. T h e r e f o r e , t h e s e t e x t s a r e o f t e n c r i t i c i z e d for t e s t i n g l i t t l e m o r e t h a n t h e r e a d e r ’s a b i l i t y to a n s w e r m u l t i p l e c h o i c e q u e s t i o n s . (p. 4) 2 . 3 M u l t i p l e - C h o i c e T e s t s

M u l t i p l e - c h o i c e it ems t a k e m a n y forms, but the initial p a r t of e a c h m u l t i p l e - c h o i c e item is k n o w n as the stem; the c h o i c e s f r o m w h i c h the s t u d e n t s s e l e c t th ei r a n s w e r s a r e r e f e r r e d to as o p t i o n s / r e s p o n s e s / a l t e r n a t i v e s . O n e o p t i o n is the a n s w e r / c o r r e c t o p t i o n / key, w h i l e t h e o t h e r o p t i o n s a r e d i s t r a c t o r s . Th e task

(24)

o-F a d i s t r a c t o r is to d i s t r a c t the m a j o r i t y of poor s t u d e n t s (i.e t h o s e w h o do not k n o w the answer) f r o m the c o r r e c t o p t i o n (Heaton, 1988, p. 28).

H e a t o n (1988) s u g g e s t s s o m e g u i d e l i n e s on p r e p a r i n g m u l t i p l e - c h o i c e t e s t s of re ad in g. T h e r e a d i n g p a s s a g e u s e d in the t e s t m i g h t v a r y f r o m 50 to 100 w o r d s at the e l e m e n t a r y level, 20 0 to 3 0 0 w o r d s at th e i n t e r m e d i a t e level, a n d 4 0 0 to 6 0 0 w o r d s at the a d v a n c e d level. He g o e s on to s a y th at the p a s s a g e m u s t be a b l e to p r o v i d e a s u f f i c i e n t n u m b e r of m u l t i p l e - c h o i c e c o m p r e h e n s i o n items a n d th e l e n g t h of th e p a s s a g e s h o u l d be r e l a t e d to its level of d i f f i c u l t y . D e c i d i n g o n t h e nu m b e r of items d e p e n d s o n the l e n g t h a n d c o m p l e x i t y of the text. T e s t e e s n e e d m u c h m o r e t i m e to c o m p l e t e a r e a d i n g c o m p r e h e n s i o n m u l t i p l e - c h o i c e t e st t h a n a m u l t i p l e - c h o i c e v o c a b u l a r y t e s t as t h e y f i r s t h a v e to r e a d the p a s s a g e a n d t h e n a n s w e r the q u e s t i o n s . The n u m b e r of a l t e r n a t i v e s , or o p t i o n s , for e a c h m u l t i p l e - c h o i c e item is f i v e in m o s t p u b l i c t e s t s but as it is e x t r e m e l y d i f f i c u l t to c o n s t r u c t f o u r e q u a l l y g o o d d i s t r a c t o r s , f o u r o p t i o n s a r e r e c o m m e n d e d for m o s t c l a s s r o o m tests, t h r e e b e in g the d i s t r a c t o r s a n d on e b e in g the c o r r e c t a n s w er . If a m u l t i p l e - c h o i c e r e a d i n g c o m p r e h e n s i o n t e s t has b e e n c a r e f u l l y p r e p a r e d , then, the c h o i c e of the c o r r e c t o p t i o n m u s t d e p e n d o n a t e s t e e ' s c o m p r e h e n s i o n o f the r e a d i n g t e x t r a t h e r t h a n on ge ne ra l

(25)

k n o w l e d g e or i n t e l l i g e n c e .

E x t e n s i v e r e s e a r c h has be en do ne in th is -field o-f t e s t i n g r e ad in g, r e v e a l i n g bo th w e a k n e s s e s a n d s t r e n g t h s

in the u s e o-f m u l t i p l e - c h o i c e tests.

2.4 S o m e W e a k n e s s e s a n d S t r e n g t h s o-f M u l t i p l e - C h o i c e T e s t s Th e c o n s t r u c t i o n o-f m u l t i p l e - c h o i c e t e s t s is g e n e r a l l y c o m p l i c a t e d a n d di-f-ficult. B e n s o u s s a n (1984) a r g u e s t h a t m u l t i p l e - c h o i c e t e s t s a r e dif-ficult to c o n s t r u c t a n d t h a t t h e y do not r e q u i r e the s t u d e n t to w r i t e in o r d e r to p r o v i d e e v i d e n c e of his r e a d i n g c o m p r e h e n s i o n . O i l e r (1979) does not f a v o r m u l t i p l e -c h o i -c e t e s t s a n d s e e s a lot o f p r o b l e m s in -c o n s t r u -c t i n g a n d a d m i n i s t e r i n g t h e s e tests. He l i st s a n u m b e r of p r i n c i p l e s t h a t s h o u l d be t a k e n into c o n s i d e r a t i o n w h i l e p r e p a r i n g m u l t i p l e - c h o i c e tests. For e x a m p l e , ar e the items in t h e t e s t r e l a t e d to the skill, c o n s t r u c t , or c u r r i c u l u m t h a t the t e s t is s u p p o s e d to a s s e s s ? A r e the items a p p r o p r i a t e l y c h o s e n a n d the d i s t r a c t o r s a t t r a c t i v e e n o u g h to d i s t i n g u i s h b e t w e e n the g o o d a n d the p o or l e a r n e r s ? Is the te st a s k i n g the t e s t e e s to g u e s s f a c t s t h a t ar e not s t a t e d or i m p l i e d ? A r e t h e r e c l u e s a b o u t the c o r r e c t a n s w e r in the o p t i o n s g i v e n ? Th e o n l y v a l u e s t h a t O i l e r (1979) s e e s in u s i n g m u l t i p l e - c h o i c e t e s t s are th at t h e y a r e o b j e c t i v e a n d th at t h e y ar e e a s y a n d

(26)

r e l i a b l e to score. A l s o w h e n large n u m b e r s o-f p e o p l e ar e to be tested, m u l t i p l e - c h o i c e t e st s a r e v e r y e c o n o m i c a l . O i l e r b e l i e v e s in t h e p r a c t i c a l i t y of m u l t i p l e - c h o i c e tests, but do es not b e l i e v e in t h e i r r e l i a b i l i t y or v a l i d i t y a n d does not r e c o m m e n d the i n s t r u c t i o n a l u s e of m u l t i p l e - c h o i c e tests. H u g h e s (1987, p. 59) s h a r e s t h e s a m e o p i n i o n as O i l e r a n d s a y s that, "the m o s t o b v i o u s a d v a n t a g e of m u l t i p l e - c h o i c e is t h a t s c o r i n g c a n be p e r f e c t l y reli ab le , r a p i d a n d e c o n o m i c a l " . He a l s o sees l i m i t a t i o n s a n d d i f f i c u l t i e s w i t h m u l t i p l e - c h o i c e tests:

If t h e r e is a lack of fit b e t w e e n at least s o m e c a n d i d a t e s ’p r o d u c t ! v e a n d r e c e p t i v e skil ls , t h e n p e r f o r m a n c e on a m u l t i p l e - c h o i c e test m a y g i v e a q u i t e i n a c c u r a t e p i c t u r e of t h o s e c a n d i d a t e s ’ a b i l i t y ... The c h a n c e of g u e s s i n g t h e c o r r e c t a n s w e r in a t h r e e - o p t i o n m u l t i p l e - c h o i c e item is o n e in three, or r o u g h l y t h i r t y - t h r e e per cent. O n a v e r a g e we w o u l d e x p e c t s o m e o n e to s c o r e 33 o n a 1 0 0 — item te st p u r e l y by g u e s s w o r k . We w o u l d e x p e c t s o m e p e o p l e to s c o r e f e w e r t h a n t h a t by g u e s s i n g , o t h e r s to s c o r e more. T h e t r o u b l e is t h a t we c a n n e ve r kn ow w h a t p a r t of any p a r t i c u l a r i n d i v i d u a l ’s s c o r e has c o m e t h r o u g h g u e s s i n g . <p. 61)

In m u l t i p l e - c h o i c e tests, item w r i t i n g is o n e o f the m o s t d i f f i c u l t a s p e c t s . H u g h e s (1989) does not r e c o m m e n d m u l t i p l e - c h o i c e t e s t s for r e g u l a r a c h i e v e m e n t t e s t i n g w i t h i n i n s t i t u t i o n s b e c a u s e he t h i n k s t h a t the a m o u n t of wo rk a n d e x p e r t i s e n e e d e d to p r e p a r e g o o d m u l t i p l e — c h o i c e t e s t s a r e so g r e a t t h a t it is not w o r t h it. M a d s e n (1903) list s t h r e e o b j e c t i o n s to m u l t i p l e -13

(27)

c h o i c e tests. First, he b e l i e v e s t h a t m u l t i p l e - c h o i c e t e s t s are p a s s i v e a n d r e l y o n r e c o g n i t i o n r a t h e r t h a n p r o d u c t i o n . S e c o n d l y , t h e y e x p o s e s t u d e n t s to er rors, a n d -finally t h e y m a y i n d i c a t e th at a s t u d e n t w h o has just g u e s s e d on th e t e s t is s u c c e s s f u l . A s l a n i a n <1985) a l s o does not f a v o u r m u l t i p l e - c h o i c e t e s t s a n d b e l i e v e s t h a t t h e s e t e s t s ar e an i n a d e q u a t e m e a n s of a s s e s s i n g the c o m p r e h e n s i o n of r e a d e r s in g e n e r a l a n d E S L s t u d e n t s in p a r t i c u l a r . S h e s t a t e s that c o m p r e h e n s i o n is a c o m p l e x p h e n o m e n o n a n d c a n n o t be m e a s u r e d by o b j e c t i v e t e s t s alone. A s l a n i a n a r g u e s th at m u l t i p l e - c h o i c e t e s t s o f t e n m e a s u r e w h a t the t e s t - m a k e r s b e l i e v e is i m p o r t a n t , not w h a t the s t u d e n t u n d e r s t a n d s or t h i n k s is i m p o r t a n t in the r e a d i n g p a s s a g e . S h e a l s o b e l i e v e s t h a t in m u l t i p l e - c h o i c e c o m p r e h e n s i o n t e s t s the c o r r e c t r e s p o n s e is o f t e n not s e l e c t e d by s e a r c h i n g for the m e a n i n g in the p a s s a g e but by e l i m i n a t i n g the o b v i o u s l y u n a c c e p t a b l e or a b s u r d d i s t r a c t o r s .

A l t h o u g h t h e r e is c o n t r o v e r s y a b o u t m u l t i p l e - c h o i c e r e a d i n g tests, m a n y t e a c h e r s like m u l t i p l e - c h o i c e r e a d i n g t e s t s a n d t h i n k t h a t t h e y r e f l e c t the r e c e p t i v e n a t u r e of reading. T h e s e t e a c h e r s do not b e l i e v e that it is p o s s i b l e to p a s s an o b j e c t i v e te st just by gu es s i n g .

(28)

2. 5 S u g g e s t i o n s for P r e p a ri ng B e t t e r M u l t i p l e - C h o i c e R e a d i n g T e s t s

M u l t i p l e - c h o i c e t e s t s a r e o b j e c t i v e , e a s y to score, t h e r e f o r e e c o n o m i c a l . T h e s e t e s t s ar e w i d e l y u s e d a n d s e e m to be h e r e to stay. I n s t e a d o-f m e r e l y a c c e p t i n g the s h o r t c o m i n g s o-F t h e s e tests, however, we s h o u l d try to deal w i t h t h e m a n d f i n d s o l u t i o n s to m a k e t h e m m o r e valid, t h a t is, m a k e t h e m r e a l l y m e a s u r e w h a t t h e y are s a i d to m e a s u r e . 15 2.5.1 S e l e c t i o n of M u l t i p l e - C h o i c e D i s t r a c t o r s As m e n t i o n e d before, the m o s t d i f f i c u l t a r e a in c o n s t r u c t i n g m u l t i p l e - c h o i c e t e s t s is t h e s e l e c t i o n of d i s t r a c t o r s . It is o f t e n the t e a c h e r s w h o c h o o s e the d i s t r a c t o r s a n d t h e y u s u a l l y u s e t h e i r i n t u i t i o n w h i l e p r e p a r i n g them. C o h e n (1980) t h i n k s t h a t p r o b l e m s m i g h t a r i s e in t h a t a t e a c h e r ’s i n t u i t i o n s a b o u t w h a t will " d i s t r a c t " s t u d e n t s a w a y f r o m the c o r r e c t a n s w e r m i g h t not a l w a y s be a c c u r a t e . D o b s o n (1974, in Cohen, 1980) r e s e a r c h e d the p o s s i b i l i t y o f o b t a i n i n g d i s t r a c t o r s for m u l t i p l e - c h o i c e items f r o m a m o n g h i g h f r e q u e n c y i n c o r r e c t a n s w e r s o n c o m p l e t i o n it ems g i v e n to s t u d e n t s . S h e s h o w e d th at d i s t r a c t o r s p r e p a r e d f r o m i n c o r r e c t s t u d e n t a n s w e r s for d i s t i n g u i s h i n g g o o d f r o m bad r e a d e r s w e r e m o r e r e li ab le . In her study, a m u l t i p l e - c h o i c e t e s t w i t h d i s t r a c t o r s o b t a i n e d f r o m a n a l y s i s o f a n s w e r s to c o m p l e t i o n e x e r c i s e s

(29)

w a s -Found to be m o r e r e l i a b l e , a n d m o r e d i f f i c u l t , w h e n a d m i n i s t e r e d to 60 E S L s t u d e n t s t h a n w a s a t e s t w i t h d i s t r a c t o r s w r i t t e n by two p r o f e s s i o n a l item w r i t e r s u s i n g s t a n d a r d r u l e s of item w r i t i n g . 2 . 5 . 2 H ow to Im p r o v e M u l t i p l e - C h o i c e R e a d i n g T e s t s A s l a n i a n (1985) p r o v i d e s s e v e n s u g g e s t i o n s o n how to i m p r o v e w r i t i n g m u l t i p l e - c h o i c e r e a d i n g c o m p r e h e n s i o n t e s t s : (1) M o r e a u t h e n t i c pa s s a g e s , w i t h p r o p e r i n t r o d u c t i o n s , r e a s o n a b l e d e v e l o p m e n t s , a n d c o h e r e n t c o n c l u s i o n s o u g h t to r e p l a c e the choppy, i n c o n s i s t e n t p a r a g r a p h s c u r r e n t l y used. (2) M o r e d e v e l o p e d , i n f e r e n t i a l , a n d p a r a p h r a s e d c h o i c e s s h o u l d r e p l a c e th e literal, o b v i o u s , a n d s o m e t i m e s a b s u r d c h o i c e s that s e r v e as d i s t r a c t o r s . (3) M o r e v a r i e t y of c o n t e n t a n d g e n r e w o u l d m a k e the t e s t less c u l t u r a l l y biased.

(4) M o r e of th e q u e s t i o n s s h o u l d o n l y be a n s w e r a b l e a f t e r the s t u d e n t has r e a d a n d c o m p r e h e n d e d the p a ss ag e. (5) V e r y f r e q u e n t l y , m u l t i p l e - c h o i c e q u e s t i o n s f o l l o w the o r d e r o f the p a ss ag e, t h e r e b y s e n d i n g s t u d e n t s to a l i m i t e d pa rt of the t e xt for t h e i r a n s w e r . C h r o n o l o g i c a l o r d e r e n c o u r a g e s the i s o l a t i o n of bits of t e xt r a t h e r t h a n t h e i r m e a n i n g f u l s y n t h e s i s ... If the q u e s t i o n s w e r e jumbled, a n e l e m e n t of s e a r c h w o u l d be i n c o r p o r a t e d into the p r o c es s. S t u d e n t s w o u l d n e e d s o m e g r a s p of th e te xt in its t o t a l i t y to be a b l e to go to the r e l e v a n t part. (6) C o m p r e h e n s i o n t e s t s s h o u l d c o n t a i n q u e s t i o n s o n the f i g u r a t i v e a s p e c t s of th e p a s s a g e s a n d o n e v a l u a t i v e or c r i t i c a l a n d a f f e c t i v e c o m p r e h e n s i o n . In o t h e r words, w h a t the w r i t e r s a y s s h o u l d be as i m p o r t a n t as how a n d to w h a t e n d he s a y s it. (7) M o s t im po r t a n t , t e s t s c o r e s s h o u l d not be the o n l y c r i t e r i o n by w h i c h to d e t e r m i n e a s t u d e n t ’s

(30)

p l a c e m e n t . T h e y c o u l d be u s e d in c o n j u n c t i o n w i t h s u c h r e a d i n g a c t i v i t i e s as oral r e a d i n g a n d r e t e l l i n g , l i s t e n i n g c o m p r e h e n s i o n , s u m m a r i z i n g a n d p a r a p h r a s i n g , a n d -Following w r i t t e n i n s t r u c t i o n s , (p. 40-41) 2 . 5 . 3 A n A l t e r na t i v e A p p r o a c h to M u l t i p l e - C h o i c e R e a d i n g I te rns M u n b y <in Cohen, 1980) s u g g e s t s a n o t h e r a p p r o a c h a b o u t m u l t i p l e - c h o i c e r e a d i n g c o m p r e h e n s i o n it ems w h i c h

i n v o l v e s oral d i s c u s s i o n in small groups. In t h is ap p r o a c h , the d i s t r a c t o r s a r e p a r t i a l l y c o rr ec t, but t h e r e is o n l y o n e c o m p l e t e l y c o r r e c t answ er . In o r d e r to s e l e c t the best a n s w e r s t u d e n t s n e e d to u s e a p a r t i c u l a r " m i c r o - s k i l l " or a c o m b i n a t i o n of m i c r o - s k i l l s . M u n b y s e e s th is a p p r o a c h as t r a i n i n g s t u d e n t s in p r o b l e m s o l v i n g t h r o u g h m u l t i p l e - c h o i c e r e a d i n g c o m p r e h e n s i o n q u e s t i o n s . Fi rst, the s t u d e n t s r e a d a p a s s a g e s i l e n t l y a n d a n s w e r a s e r i e s of m u l t i p l e - c h o i c e q u e s t i o n s . T h e n th ey get into g r o u p s o f f i v e or six to d i s c u s s m u l t i p l e - c h o i c e q u e s t i o n s a n d to c o m e up w i t h c o r r e c t a n s w e r s t h r o u g h m a j o r i t y vote. T h e n e a c h g r o u p g i v e s the r e a s o n s for r e j e c t i n g the d i s t r a c t o r s in e a c h q u e s t i o n . F i n a l l y t h e r e is a c l a s s - l e v e l d i s c u s s i o n of th e items. Munby, in th is w a y b e l i e v e s t h a t s t u d e n t s u n d e r s t a n d a n d c o me to c o n c l u s i o n s by r e a s o n i n g , r a t h e r t h a n just g u e s s i n g or r e c o g n i z i n g the c o r r e c t an swer. 17

(31)

2 . 6 R e t e l l i n g - a m o r e d i r e c t w a y o-F t e s t i n g r e a d i n g R e t e l l i n g , as i n d i c a t e d e a r 1 ier (Aslanian, 1985), is a n o t h e r m e t h o d of t e s t i n g r e ad in g. In the r e t e l l i n g p r o c e d u r e , a f t e r r e a d i n g a p a s s a g e th e s t u d e n t is a s k e d to r e p e a t or re tell as m u c h as p o s s i b l e o n w h a t he has r e a d a n d u n d e r s t o o d . W h i l e the s t u d e n t do es the r e t e l l i n g , the t e a c h e r c a n c h e c k w h a t the s t u d e n t r e c a l l s a g a i n s t a c h e c k l i s t . If th e s t u d e n t has u n d e r s t o o d w h a t he has read, he s h o u l d be a b l e to r e t r i e v e the i n f o r m a t i o n . In r e t e l l i n g t h e r e is i n d i v i d u a l i z e d c o m p r e h e n s i o n , th at is, th e s t u d e n t g i v e s his o w n u n d e r s t a n d i n g of the p a s s a g e . R e t e l l i n g g i v e s the r e a d e r ’s i n t e r p r e t a t i o n o f the r e a d i n g p a s s a g e a n d f r o m t h i s i n t e r p r e t a t i o n it is e a s y to u n d e r s t a n d w h e t h e r the s t u d e n t c o m p r e h e n d e d th e p a s s a g e or not. J o h n s t o n (1983) s a y s t h a t the r e t e l l i n g p r o c e d u r e is g u i t e t i m e c o n s u m i n g as it r e g u i r e s i n d i v i d u a l test in g, but w i t h t h i s t e s t i n g t e c h n i g u e we c a n l e ar n s o m e t h i n g a b o u t how i n f o r m a t i o n is s t o r e d a n d o r g a n i z e d in the memo ry , a n d w h a t s t r a t e g i e s a r e u s e d in r e t r i e v i n g the s t o r e d memo ry . J o h n s t o n (1983, p. 55) b e l i e v e s that d u r i n g the recall p r o c e s s " P a t t e r n s of i n tr us io ns , d i s t o r t i o n s a n d o m i s s i o n s m a y p r o v i d e v a l u a b l e i n f o r m a t i o n on s p e c i f i c i n f l u e n c e s o f t h e i n d i v i d u a l ' s b a c k g r o u n d k n o w l e d g e " . Re call a n d r e t e l l i n g i n v o l v e p r o d u c t i v e s k i l l s

(32)

r a t h e r t h a n r e c e p t i v e s k i l l s . So -Failure to p r o d u c e i n f o r m a t i o n c o u l d be due to p r o d u c t i o n de fi c i t s . A 1 t e r n a t i v e l y , " s c h e m a s e l e c t i o n or a c c e s s m a y be b l o c k e d or i n a p p r o p r i a t e or it m a y be t h a t t h e t e s t e r has r e a d the pa s s a g e , g i v e s o n l y a c u r s o r y p r o t o c o l " (J oh nston, 1983, p. 55). 19 2 . 6 C o n c l u s i o n s R e s e a r c h has s h o w n t h a t m u l t i p l e - c h o i c e t e s t s ar e not f a v o u r e d as t e s t s o f r e a d i n g c o m p r e h e n s i o n b e c a u s e of n u m e r o u s d r a w b a c k s . T h e s e t e s t s m o s t l y d e p e n d o n r e c o g n i t i o n a n d r e q u i r e the s t u d e n t to u s e h i s / h e r r e c e p t i v e s k i l l s , th us r a i s i n g d o u b t s as to w h a t a b i l i t y t h e y a r e r e a l l y m e a s u r i n g . R e t e l l i n g , on the o t h e r hand, is s o l e l y b a s e d o n p r o d u c t i v e s k il ls , a n d r e q u i r e s the s t u d e n t to put h i s / h e r real l a n g u a g e into u s e w h i l e c o m p l e t i n g the task. F r o m t h i s r e s e a r c h e v i d e n c e it c a n be a r g u e d t h at the r e t e l l i n g p r o c e d u r e is a m o r e a u t h e n t i c a n d r e l i a b l e w a y o f m e a s u r i n g r e a d i n g c o m p r e h e n s i o n as it is a m o r e d i r e c t a n d c o m m u n i c a t i v e w a y of t e s t i n g the r e a d i n g a b i l i t i e s o f s t u d e n t s .

(33)

M E T H 0 D 0 L 0 5 Y 3.1 I n t r o d u c t i o n M u l t i p l e - c h o i c e t e s t s h a v e a lot of a d v a n t a g e s ; in p a r t i c u l a r , t h e y a r e o b j e c t i v e a n d e c o n o m i c a l , t h a t is, e a s y to s c o r e a n d t i m e savi ng . T h e r e f o r e , t h e s e t e s t s h a v e b e e n w i d e l y u s e d in E F L / E S L s i t u a t i o n s in the a s s e s s m e n t o f all l a n g u a g e e l e m e n t s f r o m p h o n o l o g i c a l f e a t u r e s like s t r e s s a n d i n t o n a t i o n p a t t e r n s to v o c a b u l a r y , g r a m m a r a n d r e a d i n g c o m p r e h e n s i o n . H o w e v e r , in r e c e n t years, w i t h the d e v e l o p m e n t of c o m m u n i c a t i v e tests, the v a l i d i t y of m u l t i p l e - c h o i c e t e s t s has be en q u e s t i o n e d , w i t h m u l t i p l e - c h o i c e b e in g c o n s i d e r e d an i n d i r e c t w a y of a s s e s s i n g l a n g u a g e use. A c c o r d i n g to r e s e a r c h e r s like O i l e r (1979), H u g h e s (1989), M a d s e n (1983) a n d A s l a n i a n (1985), the v a l i d i t y of m u l t i p l e - c h o i c e t e s t s as a t e s t i n g t e c h n i q u e for m e a s u r i n g r e a d i n g c o m p r e h e n s i o n is still d e b a t a b l e .

The c o n c e r n of t h i s s t u d y is to f i n d ou t how well m u l t i p l e - c h o i c e r e a d i n g t e s t s m e a s u r e r e a d i n g a b i l i t i e s o f s t u d e n t s . Th is is do ne by c o m p a r i n g this t e s t i n g t e c h n i q u e w i t h a n o t h e r r e a d i n g c o m p r e h e n s i o n te st t e c h n i q u e " r e t e l l i n g " , in w h i c h s u b j e c t s are a s k e d to g i v e t h e i r i n d i v i d u a l oral a c c o u n t of the r e a d i n g p a s s a g e . T h e r e s u l t s o b t a i n e d f r o m the m u l t i p l e - c h o i c e r e a d i n g t e s t a n d t h e r e t e l l i n g p r o c e d u r e a r e t h e n

(34)

c o r r e l a t e d w i t h e a c h o t h e r a n d w i t h the t e a c h e r r a t i n g s a n d the B U S E L r e a d i n g e x a m to d e t e r m i n e w h i c h o-F t h e s e t w o r e a d i n g t e s t s h i g h l y c o r r e l a t e w i t h the two o u t s i d e m e a s u r e s . 2 1 3 . 2 P a r t i c i p a n t s a n d S e t t i n g 3 . 2. 1 S u b j e ct s The s u b j e c t s of t h i s s t u d y w e r e B i l k e n t U n i v e r s i t y S c h o o l o f E n g l i s h L a n g u a g e <BUSEL) f i r s t y e ar i n t e r m e d i a t e level s c i e n c e s t u d e n t s . S t u d e n t s w h o e n t e r B i l k e n t U n i v e r s i t y f i r s t t a k e a n E n g l i s h p r o f i c i e n c y e x a m - the B U S E L E x e m p t i o n Exam, a n d t h o s e w h o p a s s th is e x a m b e c o m e f i r s t y e a r s t u d e n t s . T h o s e w h o a r e not s u c c e s s f u l in t h i s e x a m t a k e a P l a c e m e n t T e s t a n d f r o m the s c o r e s t h e y o b t a i n in t h i s t e s t t h e i r p r o f i c i e n c y l e v e l s ar e d e t e r m i n e d , a n d a c c o r d i n g to t h e i r l e v e l s t h e y a r e t h e n p l a c e d into b e g i nn er , e l e m e n t a r y , i n t e r m e d i a t e or u p p e r - i n t e r m e d i a t e cl a s s e s . T h e r e f o r e , all the B U S E L i n t e r m e d i a t e level s u b j e c t s w h o to ok p a r t in th is s t u d y w e r e c o n s i d e r e d to h a v e s i m i l a r p r o f i c i e n c y l e v e l s of E n g l i s h . A total o f 18 s u b j e c t s f r o m a n i n t e r m e d i a t e level s c i e n c e c l a s s p a r t i c i p a t e d in the study. S u b j e c t s w e r e s e l e c t e d a f t e r a p i l o t s t u d y w a s do ne w i t h o n e i n t e r m e d i a t e a n d two e l e m e n t a r y c l a s s e s . F r o m the r e s u l t s o f t h e s e 50 s t u d e n t s a n item a n a l y s i s wa s do ne

(35)

(see A p p e n d i x B) a n d s o m e c h a n g e s w e r e m a d e in s o m e of the d i s t r a c t o r s . F r o m t h e r e s u l t s o f t h i s p i l o t study, the level of the s u b j e c t s ’ w a s d e t e r m i n e d to be i n t e r m e d i a t e . S u b j e c t s w e r e not i n f o r m e d a b o u t the n a t u r e o f the study. T h e s t u d y wa s c o n d u c t e d at B U S E L in the s u b j e c t s ’ usua l c l a s s r o o m s at t h e i r u s ua l c l a s s h o u r s w i t h the i n t e n t i o n o f l o w e r i n g s t r e s s a n d a n x i e t y . 3 . 2 . 2 E x p e r i m e n t e r s S i n c e the r e t e l l i n g p r o c e d u r e r e q u i r e s in d i v i d u a l i n t e r v i e w s , it wa s c o n d u c t e d by t h e r e s e a r c h e r a n d s e v e n o t h e r T u r k i s h t e a c h e r s of E n g l i s h o n e of w h o m wa s the t e a c h e r of t h a t class. T e a c h e r s w e r e g i v e n i n f o r m a t i o n o n how to c o n d u c t the r e t e l l i n g p r o c e d u r e . T h e f i r s t t h i n g the t e a c h e r s w e r e a s k e d to do w a s to c r e a t e a n o n - t h r e a t e n i n g a t m o s p h e r e f o r the s t u d e n t s like by a s k i n g t h e i r names. T h e ne xt t h i n g the t e a c h e r s w e r e a s k e d to do was to tell the s t u d e n t s t h a t t h e y c o u l d do the r e t e l l i n g in e i t h e r l a n g u a g e - E n g l i s h or T u r k i s h . T h e n t h e y w e r e a s k e d to to tell the s t u d e n t s to t h in k a b o u t t h e p a s s a g e t h e y had just read, a n d ask t h e m to recall a n d re tell all the d e t a i l s t h e y c o u l d r e m e m b e r a b o u t th e pa s s a g e . Th e t e a c h e r s w e r e a l s o a s k e d to i n f o r m the s t u d e n t s t h a t t h ey w e r e not g o i n g to be i n t e r r u p t e d a n d t h e y w o u l d be g i v e n

(36)

a s e c o n d c h a n c e to t h in k a b o u t the p a s s a g e o n c e a g a i n a n d a d d a n y a d d i t i o n a l p o i n t s t h a t c a m e to t h e i r minds. For a c c u r a c y a n d e-f-Ficiency, the t e a c h e r s w e r e g i v e n c h e c k l i s t s to c a r r y o u t t h e r e t e l l i n g p r o c e d u r e . 2 3 3. 3 M a t e r i a 1s M a t e r i a l s u s e d in t h i s s t u d y i n c l u d e d two r e a d i n g c o m p r e h e n s i o n tests: a m u l t i p l e - c h o i c e r e a d i n g c o m p r e h e n s i o n te st a n d a r e t e l l i n g c h e c k l i s t . I n s t r u m e n t I A r e a d i n g p a s s a g e of 469 w o r d s wa s c h o s e n to be a p p r o p r i a t e fo r i n t e r m e d i a t e level s t u d e n t s (see A p p e n d i x C ) . T h i s p a s s a g e w a s u s e d by C h a v e z - 0 1 l e r et a l . ( 1 9 8 5 ) as a c l o z e task, a n d t h e o r i g i n a l s o u r c e w a s P r a n i n s k a s (1959) (both c i t e d in J o n z 1990). Th e p a s s a g e wa s f o l l o w e d by e i g h t m u l t i p l e - c h o i c e c o m p r e h e n s i o n q u e s t i o n s (see A p p e n d i x D ) . The d i s t r a c t o r s of the m u l t i p l e - c h o i c e te st w e r e p r e p a r e d by t a k i n g D o b s o n ’s (1974) s u g g e s t i o n into a c c o u n t for p r e p a r i n g b e t t e r d i s t r a c t o r s as m e n t i o n e d in c h a p t e r II, t h a t is, by c h o o s i n g d i s t r a c t o r s f r o m " h i g h - f r e q u e n c y " i n c o r r e c t s t u d e n t a n s w e r s in c o m p l e t i o n items. To o b t a i n i n c o r r e c t s t u d e n t a n s w e r s first, o p e n - e n d e d c o m p r e h e n s i o n q u e s t i o n s w e r e p r e p a r e d o n the s a m e r e a d i n g p a s s a g e a n d w e r e g i v e n to the s t u d e n t s o f a n o t h e r i n t e r m e d i a t e level class. Then, f r o m the u s a b l e w r o n g a n s w e r s g i v e n to t h e s e q u e s t i o n s , s o m e

(37)

o-F the d i s t r a c t o r s -For th e m u l t i p l e - c h o i c e q u e s t i o n s w e r e w r i t t e n .

I n s t r u m e n t 11

A c h e c k l i s t w a s p r e p a r e d for th e r e t e l l i n g p r o c e d u r e c o v e r i n g n e a r l y all the i m p o r t a n t p o i n t s of the r e a d i n g p a ss ag e. T h i s w a s do ne by p u t t i n g s l a s h e s in b e t w e e n e a c h i m p o r t a n t p i e c e o f i n f o r m a t i o n in the p a ss ag e. T h e r e w e r e 40 total s l a s h e s a n d so 40 p i e c e s of

i n f o r m a t i o n to be given, but s u b j e c t s w e r e not e x p e c t e d to re tell e a c h a n d e v e r y on e of t h e s e p i e c e s as it was i m p o s s i b l e to r e m e m b e r the w h o l e text. The p u r p o s e for g i v i n g the c h e c k l i s t in a d e t a i l e d f o r m w a s to he lp the e x p e r i m e n t e r s c o n d u c t i n g the r e t e l l i n g p r o c e d u r e a n d a l s o to g i v e to tal c r e d i t to t h o s e s u b j e c t s w h o r e m e m b e r e d the p a s s a g e w i t h m o r e d e t a i l s t h a n the o t h e r s (see A p p e n d i x E) . 3. 4 T w o O u t s i d e M e a s u r e s u s e d as the C r i t e r i a for M e a s u r i n g R e a d i n g A b i l i t y T e a c h e r R a t i n g s E a c h c l a s s at B U S E L r e c e i v e s 20 h o u r s of E n g l i s h i n s t r u c t i o n a w e e k w h i c h is s h a r e d by tw o t e a c he rs . On e of the t e a c h e r s - the m a i n t e a c h e r - t e a c h e s the c l a s s 12 h o u r s a week, w h i l e t h e o t h e r m e e t s t h e c l a s s for the r e m a i n i n g 8 hours. To i n c r e a s e the r e l i a b i l i t y of the

Şekil

TABLE OF  CONTENTS SECTIONS PAGES 1.0  I N T R O D U C T I O N 1 . 1  B a c k g r o u n d   o-F  th e   S t u d y 1.2  S t a t e m e n t   of  t h e  R e s e a r c h   Q u e s t i o n 1.3  H y p o t h e s e s 1.3.1  Null  H y p o t h e s i s 1

Referanslar

Benzer Belgeler

Life was made possible by the assembly of monomers such as nucleic acids, fatty acids, amino acids, which came together to form larger molecules, namely proteins, car- bohydrates

ġekil 2 Salihli Sağ Sahil Sulama Birliği Alanının Temel Toprak Haritası (Usul ve Bayramin, 2004)... ġekil 3 ÇalıĢma alanı drenaj sınıfı dağılım haritası

Ülkemizde dağılım gösteren Mığrı balıklarının morfolojik ve biyolojik özelliklerinin tanımlandığı bu derleme çalışmasında konu hakkında gerek ülkemizde gerekse

An Egyptian scholar further argued that, by establishing the Middle East Defense Organisation, the US aimed to change Britain’s position in Egypt “/rom that o f an

Although anti- American sentiment, different ideologies and motivations for foreign policy have influence in base politics, they do not necessarily transform into policies against

There are several works on finite time observer design for cases where the measurements are continuous instead of being piecewise constant; see, e.g., Engel and Kreisselmeier

Türk imalat sanayinde sermaye sahipliği açısından farklı statüde faaliyet gösteren kamu, özel ve yabancı sermaye türlerinin satış gelirleri trendlerini Gri

Seyahat işletmelerinin organizasyon yapısı içinde yer alan orta ve üst kademe yöneticiler literatüre göre, seyahat acentası sorumlu müdürü, pazarlama müdürü, operasyon