The influence of readibility of examination questions on achievement in senior secondary school mathematics : a study on verbal problems with special reference to second language readers

This study investigates the influence of readability of mathematics examination questions on achievement. The aim of any mathematics examination is to assess whether the aims of a specific mathematics programme have been realized. Readability factors that unnecessarily prevent a clear understanding of questions could jeopardize this aim. The important issue is, therefore, whether there are indeed readability factors in mathematics examination questions that cause comprehension problems for students and, if there are, do they hav~ any effect on test scores? The issue of readability is of even greater importance for second language readers. In the South Mrican context, the reading problems of second language readers are of particular importance as most students at school are second language learners. An important question would therefore be: What readability factors cause comprehension difficulties for second language students, especially those whose mother tongue is not kindred to English? Furthermore, what is the influence of cultural factors on readability? This study provides answers to these and other related questions for mathematics text at senior secondary school level. Protocol analysis was used to ascertain what readability problems are experienced by students when reading examination questions in mathematics. Three different language groups, comprising 17 -18-year-old students, were used in the study: English First Language students and two groups who had English as a second language. One second language group had Mrikaans as first language while the other group comprised Mrican students whose mother tongue is unrelated to English. A framework was developed to analyse the protocols and it comprised five categories: unfamiliar vocabulary structural problems obscure information visualization difficulties non-verbal factors Mter the protocol study, students were asked to adapt the examination questions to a more comprehensible form. Students' adaptations addressed lexical, syntactical, discourse and non-verbal factors. Most of the readability problems identified in the literature study were verified in the empirical study. However, the empirical study generated additional readability problems that are mainly restricted to mathematics text and relate to nonverbal factors like mathematical expressions. During the last phase of the empirical study a composite test was used to test the hypothesis that improved readability of the common language used in mathematics examination questions' will improve achievement. Nine socalled "word problems" from previous examination papers were set in three different versions: original, adapted and non-verbal. The hypothesis was confirmed in a number of important cases. A significant finding of the study was, therefore, that readability factors not only influence the comprehension of mathematics examination questions, but also have a marked influence on students' achievement levels. The results of the empirical study are reported quantitatively as well as qualitatively. Other results include the following: Not only second language students, but also first language students experienced a variety of readability problems. All three language groups demonstrated the same level of competency on the non-verbal versions. When comparing test scores of the verbal versions, differences in achievement levels between the different language groups were often caused by linguistic and cultural factors. Cultural thought patterns, typical of a mother tongue but absent in a second language, were often a source of comprehension difficulties for second language readers. This study has led to certain conclusions for teaching and examination practice. For example, factors influencing the readability of ordinary English should be considered with other factors when writing mathematics examination questions. Furthermore, the distinctly different reading needs of second language students suggest that examination papers be set, so that the language needs of second language learners are accommodated. Guidelines for writing more readable examination questions were developed and are presented as a readability checklist. Suggestions for further research include the investigation of the influence of readability on achievement in authentic examination conditions.