Abstract

This research aims to study tertiary English language teachers’ experience and perspective of ESL assessment in a public sector university in Lahore. Being a core responsibility and of primary importance in the teaching and learning process, language assessment employs a variety of methods. This research gives an insight into the various difficulties and issues faced by teachers during an assessment. The objective of this research is to explore the hindrances faced by language teachers during the process of assessment. Following the interpretive approach, data were collected from twelve teachers teaching English at the undergraduate level through in-depth interviews. Teachers from a public sector university in Lahore were interviewed till a point of saturation was reached. The interviews of these twelve participants were later transcribed and coded and themes were extracted from them thereafter. The data analysis reveals that the obstacles teachers have to face during assessment can be removed from the system through the adoption of systematic policies. In this way, the best assessment practices according to teachers can be emphasized for the purpose of ESL assessment at the tertiary level. Many improvements can be brought into the whole system to make it better for all stakeholders.

Key Words

Language Assessment, ESL, Reliability, Washback

Introduction

Assessment carries vital importance in learning and teaching for both teachers and students (Tosuncuoglu, I., 2018). Teachers are informed about their instruction, and students' progress, whereas students get to know how much they learnt or grasped the content being taught through assessment. Different researchers have defined assessment according to their own perspectives. Most of them define it as a systematic process which gives the required data to arrive at some decisions. Assessment is considered a tool to find out and report on learning and its relation to classroom activities (Amua-Sekyi, E. T., 2016). There are some researchers who have given a broad definition of assessment and have made it possible to include various instruments for the purpose of assessment e.g. informal observations, self-assessments, quizzes, interviews, tests etc (Ari Huhta as cited in Spolsky & Hult, 2008, p. 469). 

This research aims to investigate Problems faced by teachers during the process of teaching. Once the problems are identified, it is hoped that the policy planners can look for solutions and implement them to improve the system.




Assessment

Assessment means a process of collecting information about the learning of an individual, for the purpose of gauging the learning that has taken place (Yambi, T., 2018). It can be both graded and non-graded. Thus it includes all the activities arranged in the classroom (Amua-Sekyi, E. T., 2016). 

There are different issues which are faced by teachers while conducting assessments in class. Following is a review of the important issues.




Reliability

It is important to check the reliability of assessment practices in ESL classrooms (Chiedu, R. E., & Omenogor, H. D., 2014; Li, J., Huang, J., & Cheng, S., 2022). Reliability has been considered the most important characteristic of language testing specifically (Chiedu, R. E., & Omenogor, H. D., 2014). 

The reliability of assessment based on classroom can be made difficult by two common sources (Hughes, A., 2003 as cited in Chiedu, R. E., & Omenogor, H. D., 2014): 

? Variation in people who are scoring and collecting the information 

? Methods and instruments being used for the collection of data

According to Chiedu, R. E. and Omenogor, H. D. (2014) there is a possibility of homogeneous learners in the same class. They would perceive and attempt the test similarly, decreasing its reliability. A class of mixed-ability students performing similar increases the reliability of the test. 

Environmental factors play an important role in determining the reliability of a test. If the weather is too hot, or too cold, the performance of test takers would get affected.  Factors like fatigue, stress, motivation, noise and sickness can also influence the reliability of the test (Chiedu, R. E. and Omenogor, H. D., 2014). 




Assessing Language Literacy

It has been observed in language classes that while testing, language literacy can become an issue in itself. It is possible from both points of view: the teacher is unable to use language efficiently considering the multi-lingual classroom, or the learners are unable to understand the questions being asked. The teachers can find it difficult to apply the knowledge they have about English Language and assessment due to a varied class (Bachman, L. F., 2013), and the learners might find it difficult to understand the vocabulary and syntax used by the teachers (Haynes, J., 2005). Due to this language barrier, at times teachers are unable to convey the questions to the students. On the other hand, students are also unable to answer questions properly as they are unable to understand the questions. Thus, language assessment literacy is very important for both the stakeholders in the process of assessment: teachers as well as students (Giraldo, F., 2018).




Rote-Learning

Rote learning is a commonly used method of learning a language. It is the repetition of certain phrases or sentences till they are memorized fully by the learner. Some teachers and parents strictly criticize it (Ali, D. et al., 2016). Teachers emphasize concept-based learning. They advocate extensive reading for the said purpose (HR, N. H., 2020). When concepts are given more importance as compared to memorization of concepts, they believe the knowledge is long-lasting (Medwell, J., & Wray, D., 2020)

Washback

The term washback means the effect testing/assessment has on the teaching and learning in a classroom (Lodhi, M. A. et al., 2018). Washback is a facet of consequential validity or an aspect of impact and has become an important area of study in applied linguistics and is even more important in the area of language testing and assessment. "Washback occurs when it is the testing instrument rather than the statement of desired learner outcomes that determines the nature of the curriculum and the course of instruction" (Valette, 1994: 10). Washback effects are put-forth by various types of tests including TOEFL, IELTS, Oral Proficiency Interview etc. Washback has been recognized as one of the ethical issues of high-stakes assessments since the 1990s following related issues presented by Messick's (1989) extended model of validity (Bachman, 2000). 

Researchers have mostly focused on the ‘importance’ of washback occurrence. It has been noticed as positive or negative, resulting in a beneficial or harmful effect on the process of teaching-learning (Lodhi, M. A. et al., 2018). It is assumed to be negative when the teaching is done for a test which is providing the students with strategies for taking tests, or passive acquisition of knowledge which is not providing enough scope to the students for participation, and it is considered to have a positive effect when students are being assigned the activities that are aimed to develop creative skills based on knowledge (Fulcher & Davidson, 2007; Lodhi, M. A. et al., 2018).




Lack of Resources

There is an issue of lack of resources for conducting assessment properly too. Important resources for conducting assessments include time, space, stationery and other material. It has been generally observed that time is short and content is more, so teachers have to rush to finish all the syllabus (Blecher, H., & Hörlin, M., 2021;  Metin, M., 2013; Mohammedseid, A., 2018). Another problem is the very large number of students and small classrooms. The students have to be accommodated in small classrooms, where extreme weather conditions not only affect the reliability of assessment, but make it difficult for the teacher to give equal attention to all students (Blecher, H., & Hörlin, M., 2021). At times, teachers are not free to decide the modes of assessment due to interference of authorities, then too it can become problematic for them (Metin, M., 2013). On the other hand, when teachers are not provided with any common guidelines and they are free to do what they want to do in their classes, then too lack of uniformity can become an issue (Mohammedseid, A. 2018).




Research Question:

This research is based on the following research question:

1. What are the major difficulties faced by teachers while conducting assessments?

Methodology

This research is inductive in nature. Interpretivism aims to explore through an inductive method. Different methodologies can be used by interpretive researchers and case study is one of them. Interpretivism believes that the "social world can only be understood from the standpoint of the individuals who are part of the ongoing action being investigated" (Cohen et al, 2003: 19). For this research, an attempt would be made to start from individuals and later understand the phenomenon broadly. The theory would emerge from the data (Cresswell, 2003) and the researcher would “see themselves as participants in the situation they investigate” (Edge and Richards, 1998: 336).

Population and Sample

This research focuses on English language university teachers as its population, therefore the sampling was purposive. The objective was to continue to conduct interviews till saturation was reached. Saturation is taken as a point in research when information from participants starts getting repeated and no new data is obtained (Saunders et al., 2018). For interviews, saturation is mostly reached between the 9th and 17th interviews (Hennink and Kaiser, 2022). After interviewing 12 teachers, it was felt by the researcher that no new information was being gathered. Thus no more interviews were conducted. 




Data Collection Procedure

The English Language teachers were interviewed using a semi-structured interview. The rationale was to give liberty to the teachers to express their views, observations and recommendations. The selected teachers were asked for their consent through the signing of a consent form and were contacted at their convenience in the time slots given by them. The interviews were conducted on Zoom considering Covid-19. Four of the teachers asked for a rescheduling of their interviews due to sudden changes in their plans. Certain technical and connectivity issues were also faced during some of the interviews, but overall the experience was smooth, and the interviews were conducted successfully. Considering the ethical principles, the teachers ensured the anonymity and confidentiality of the data (Bulmer, 2008). 




Data Collection Tool

Interviews have been used as the main data collection tool for this research. Interviews have been considered “the most prominent data collection tool in qualitative research” Punch (2009:144). Interviews give flexibility to the researcher. The structure of interviews was kept flexible (Jamshed, S. 2014). The nature of the interviews was semi-structured. The rationale was to modify it according to to need to collect as relevant data as possible (Campbell, S., 2015; Robson, 2002). Keeping the research questions in consideration, an interview protocol was made. It helped both the interviewer and interviewee during the process of the interview (Jamshed, S. 2014).

The interview protocol was validated through a two-step procedure. In the first step, validation was sought through expert opinion. For this purpose, 3 experts have requested their review of the interview protocol. Once their recommendations had been incorporated, the interview protocol was shown to them again. They approved it the second time. The second step in the validation process was to pilot-test it. Three teachers were requested to give interviews for this purpose. After their interviews, they were asked for suggestions to improve the questions. The suggestions were incorporated, and they were shown the questions again. After their approval, the interview protocol was considered suitable for the research. 




Analysis of the Collected Data 

After the collection of data, it was analysed using a systematic approach involving six steps (Braun, V., & Clarke, V. 2006 as cited in Kiger, M. E., & Varpio, L., 2020). They were familiarization, coding, generating themes, reviewing themes, defining and naming themes and writing up. 

Findings

The participants of the study had different designations. The collected data shows that 84% of the sample were Lecturers and 8% were Assistant professors. Associate Professors were also 8% of the sample. Thus the majority of the teachers were lecturers. The demographic information also shows that 41% of the sample had 1-10 years of experience, 50% of the sample had 11-20 years of experience, and 9% had experience of 21-30 years. The demographic information for qualification shows that 91% of the sample had done MS/MPhil and 9% were PhD. Thus majority had the basic qualification for University Lecturer which is MS/M.Phil. The collected data shows that 58% of the sample had received training, while 42% of the sample had not received training. Thus majority had received training in the field of assessment. The demographic information of the participants has been presented in the table below:




 

Table 1. Sample Demographic Information of Designation, Years of Experience, Qualification and Training Received

Sr. No	Demographics	Details	Frequency	Percentage
1	Designation	Lecturer Assistant Professor Associate Professor	10 1 1	84% 8% 8%
2	Years of Experience	1-10 years 11-20 years 21-30 years	5 6 1	42% 50% 8%
3	Qualification	MS/M. Phil PhD	11 1	92% 8%
4	Training received	Yes No	7 5	58% 42%

 




The findings of this research show that teachers face various kinds of difficulties during assessments in classes. The difficulties can be divided into 4 broad areas, which are assessment literary, lack of resources, learning strategies of students and certain administrative difficulties the teachers have to face during the process of assessment. 

Figure 1

Difficulties faced by University ESL Teachers



The findings show that the assessment of literacy is one of the main problems faced by teachers during the assessment. The students are unable to understand the questions as they have been stated in the question papers, and when they are unable to under the questions, they cannot answer them appropriately either. This has been expressed by teacher B as, “biggest obstacle is not understanding what the teacher wants them to answer, so deciphering the question is the real problem that is experienced by the students. So using language in a question that students can understand is the real issue”.

The other problem is the lack of resources. The resources include time, space, technological facilities, stationery items etc. The lack of resources affects the reliability of the whole assessment process as well. The content to be taught and assessed is also quite large, which even affects the teaching. Teacher E confessed, "The course is very lengthy and it takes a lot of time in covering it. Therefore, teachers should get enough time to assess students properly. The courses should be revised." Certain technological facilities are required for assessment especially if it is presented, but they are either not available, or available after a long procedure, as teacher D shared, "there is a lack of infrastructure, we are bound with a board and a marker. There is no concept of smart boards. The listening skills of students don't develop. The exposure is quite limited.”

 Most of the students still resort to learning strategies like rote learning. Language teachers believe there should be more conceptual teaching and extensive reading. It will make concept-based examinations possible, reducing the effectiveness of rote learning. This will discourage the students from learning by memorization. They would start focusing more on content. Teacher D stressed, "What bothers me a lot is rote learning, memorizing, and since we cannot forget the vicious cycle when we are setting the paper, as it is again based on it.” 

Discussion

In light of the above presentation of data, it can be seen that English language teachers have identified the difficulties they face during an assessment. One difficulty that is faced by the teachers is the lack of assessment of language literacy on the part of students. It has been discovered that in line with the findings of Haynes, J. (2005), the students are unable to understand the language used by the teachers to assess them, and the result is they do not write the required answers. Thus as per Giraldo, F. (2018), both stakeholders should have assessment literacy. Another reason for the inability to understand teachers' instruction can be increased anxiety it can be due to the washback effect. Consistent with the findings of Lodhi, M. A. et al. (2018), washback affects learning and, there are times when students' anxiety level increases and they are unable to understand the teachers' instructions. It also hampers their understanding of the questions in the question paper and increases their anxiety. Due to increased anxiety levels, their comprehension gets affected. They cannot even recall the skills they had learned previously in class. This is the negative effect of washback as stated by Fulcher and Davidson (2007).

Another difficulty faced by the teachers is the lack of resources. In consistence with the findings of other researchers, it has been found that time, lack of appropriate rooms, number of students in class, technological facilities and other infrastructure is not available for assessment most of the time (Blecher, H., & Hörlin, M., 2021;  Metin, M., 2013; Mohammedseid, A., 2018). Due to this, not only assessment process gets affected, but its reliability also gets affected. Properly ventilated rooms are a requirement of assessment. Consistent with the opinion of Chiedu, R. E. and Omenogor, H. D., (2014), it has been found that at times the weather conditions are extreme. It is either too cold or too hot and humid. The students cannot sit comfortably for the long duration of the assessment. The result is that their performance suffers.

Traditional learning strategies play their part too. There are many students who resort to rote learning as their basic language learning strategy. The teachers have to face difficulties as a result of it. The teachers wish to have more concept-based learning as compared to learning by memorization. In line with the findings of Ali, D. et al. (2016), the teachers expressed their dislike for rote learning. The teachers preferred learning through concepts. In consistence with HR, N.H. (2020), emphasized the importance of extensive reading. They said that conceptual knowledge should be given more importance, because, similar to the findings of Medwell, J., & Wray, D. (2020), they expressed their belief that knowledge is more long-lasting. 

Conclusion

This research studied the problems faced by teachers during assessments, and the best assessment practices that teachers supported to be emphasized. It also paid attention to teachers' suggestions for improvement. It can be concluded that as per teachers, one of the main issues was found to be with the comprehension of students in understanding the language used by teachers in examinations, as they do not write relevant answers. Secondly, lack of resources is another major issue. In the absence of proper resources, the assessment cannot be executed efficiently, and it affects its reliability. Thirdly, students’ language learning strategies, especially rote-learning causes problems. Instead of focusing on concepts, they focus on memorization. The teachers want the system to be changed in such a way that concepts-based learning and assessments get more centralized. If the system can be changed to incorporate more presentations and application-based learning utilizing the facilities available, the whole concept of learning and teaching can benefit. Thus, the need is to increase the assessment of language literacy, emphasize the application of knowledge, and use proper resources. This would improve the assessment system for both the teachers and the students. 

References

• Alderson, J. C. (2005). Diagnosing Foreign Language Proficiency: The Interface Between Learning and Assessment. London: Continuum.
• Amirian, M. R., Ghonsooli, B., & Amirian, K. (2020). Investigating fairness of reading comprehension section of INUEE: Learner’s attitudes towards DIF sources. International Journal of Language Testing, 10(2), 88-100.
• Amua-Sekyi, E. T. (2016). Assessment, Student Learning and Classroom Practice: A Review. Journal of Education and Practice, 7(21), 1-6.
• Andrews, S. (2004). Washback and curriculum innovation. In L. Cheng, Y. Watanabe, & A. Curtis, (eds.), Washback in language testing: Research contexts and methods, 37-50. Lawrence Erlbaum, Mahwah, NJ.
• Bachman, L. (2004). Statistical Analysis for Language Assessment. Cambridge: Cambridge University Press.
• Bachman, L., & Palmer, A. (2010). Language Assessment in Practice. Oxford: Oxford University Press.
• Brown, H. (2004). Language Assessment: Principles and Classroom Practices. Newyork: Pearson Education.
• Brown, J. D., & Hudson, T. (1998). The Alternatives in Language Assessment. TESOL Quarterly, 32(4), 653.
• Cheng, L., Rogers, T., & Hu, H. (2004). ESL/EFL instructors’ classroom assessment practices: purposes, methods, and procedures. Language Testing, 21(3), 360–389.
• Chiedu, R. E., & Omenogor, H. D. (2014). The concept of reliability in language testing: issues and solutions. Journal of Resourcefulness and Distinction, 8(1), 1- 9.
• Cohen, A. D. (1996). Second Language Assessment. In Celce-Murcia, M. (Ed.). (2001). Teaching English as a second or foreign language (3rd ed.). New York: Heinle & Heinle.
• Cohen, L., Manion, L., & Morrison, K. (2003). Research Methods in Education. London: Routledge.
• Crotty, M. (1998). The Foundations of Social Research. London: Sage.
• Etikan, I., & Bala, K. (2017). Sampling and Sampling Methods. Biometrics & Biostatistics International Journal, 5(6), 215–217.
• Genesee, F., & Upshur, J. A. (1996). Classroom-based evaluation in second language education. Cambridge: Cambridge University Press.
• Giraldo, F. (2018). Language Assessment Literacy: Implications for Language Teachers. Profile: Issues in Teachers ́ Professional Development, 20(1), 179– 195.
• Grix, J. (2004). The Foundations of Research. London: Palgrave Macmillan.
• Harris, M., & McCann, P. (1994). Assessment (Handbook for the English classroom). Oxford: Heinemann Publishers.
• Haynes, J. (2005). Challenges for ELLs in content area learning. In TESOL annual convention, Baltimore, MD.
• Hennink, M., & Kaiser, B. N. (2021). Sample Sizes for Saturation in Qualitative research: a Systematic Review of Empirical Tests. Social Science & Medicine, 292, 114523.
• Khoii, R., & Shamsi, N. (2012). A fairness issue: Test method facet and the validity of grammar subtests of high-stakes admissions tests. Literacy Information and Computer Education Journal (LICEJ), 1(1), 892-900.
• Kunnan, A. (2005). Language assessment from a wider context. In E. Hinkel, (ed.), Handbook of research in second language teaching and learning. 779- 794. Mahwah, NJ: Erlbaum.
• Lambert, D., & Lines, D. (2000). Understanding assessment: Purposes, perceptions, practice. Routledge. Falmer.
• Lewy, A. (1990). Formative and Summative Evaluation. In H. Walberg, & G. Haertel, The International Encyclopedia of Educational Evaluation. 26-28.
• Lodhi, M. A., Robab, I., Mukhtar, S., Farman, H., & Farrukh, S. (2018). Impact of Washback on ESL Students’ Performance at Secondary Level. International Journal of English Linguistics, 8(6), 227.
• McNamara, M. V. (1977). Applying Norm- Referenced and Criterion-Referenced Measurement in Education. Boston: Allyun & Bacon.
• Medwell, J., & Wray, D. (2020). Concept- Based Teaching and Learning: A Review of The Research Literature. ICERI2020 Proceedings, 486-496.
• Muñoz, R. E. (2017). The Effect of Washback on EFL Learners’ Attitudes toward Tests. Studies in English Language Teaching, 5(3), 516.
• Messick, S. (1989). Validity. In R. Linn (ed.), Educational measurement, Washington DC: Oryx and the American Council on Education.
• Nimehchisalem, V., Kai, E. F. S., & Nowrouzi, S. (2019). English as a Second Language Learners and Teachers’ Conceptions of Language Assessment. The Journal of AsiaTEFL, 16(4), 1348–1359.
• Sankary, D. (2013). Role of rote learning as an ESL learning strategy. Language in India, 13(7), 441.
• Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2017). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & Quantity, 52(4), 1893–1907.
• Schissel, J. L. (2018). Test Impact and Washback. The TESOL Encyclopedia of English Language Teaching, 1–6.
• Tosuncuoglu, I. (2018). Importance of Assessment in ELT. Journal of Education and Training Studies, 6(9), 163-167.
• Valette, R. M. (1994). Teaching, testing, and assessment: conceptualizing the relationship. In C. Hancock, (ed.), Teaching, testing, and assessment: making the connection. 1-42. Northeast Conference on the Teaching of Foreign Languages.
• Wellington, J. (2000). Educational Research: Contemporary Issues and Practical Approaches. London: Continuum.
• Wiggins, G. (1989a). Teaching to the (authentic) test. Educational Leadership, 48(7), 41-47.
• Wiggins, G. (1991). Standards, not standardization: Evoking quality student work. Educational Leadership, 48(5), 18-25.
• Wiles, R., Crow, G., Heath, S., & Charles, V. (2008). Anonymity and confidentiality.
• Wilson, R. J. (2000). A model of assessment- in-practice. In Cheng, L., Rogers, T., & Hu, H., ESL/EFL instructors’ classroom assessment practices: purposes, methods and procedures 360-389.