Research Insights in Teaching and Learning

Editors: Faridah Abdul Malik Liza Abdullah Engku Haliza Engku Ibrahim Suhaila Abdullah RESEARCH INSIGHTS IN TEACHING AND LEARNING CENTRE FOR LANGUAGES AND PRE-UNIVERSITY ACADEMIC DEVELOPMENT (CELPAD) INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA

First Edition, 2023 © CENTRE FOR LANGUAGES AND PRE-UNIVERSITY ACADEMIC DEVELOPMENT (CELPAD) UIAM (online) All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without any prior written permission from the publisher. Editors: Faridah Abdul Malik Liza Abdullah Engku Haliza Engku Ibrahim Suhaila Abdullah Published by: CELPAD Research Unit, CELPAD, International Islamic University Malaysia 53100 Gombak, Kuala Lumpur [email protected] 03-64213517

TABLE OF CONTENTS PREFACE............................................................................................................................. v CHAPTER 1...........................................................................................................................1 Teachers’ Knowledge and Perception of CEFR-aligned Curriculum Introduction...................................................................................................... 2 Literature Review.............................................................................................. 4 Materials and Methods...................................................................................... 6 Results and Discussion...................................................................................... 7 Conclusion........................................................................................................15 References.........................................................................................................16 CHAPTER 2.........................................................................................................................17 Examining Achievability of CLOs for CEFR-Mapped English for Academic Writing Course Introduction....................................................................................................19 Literature Review............................................................................................19 Materials and Methods....................................................................................21 Results.............................................................................................................23 Discussion........................................................................................................25 Conclusion.......................................................................................................26 References........................................................................................................27 CHAPTER 3.........................................................................................................................28 Examining the Attainment of Learning Outcome for Spoken Production in a CEFR-Mapped English Course at CELPAD, IIUM: A case study Introduction.....................................................................................................30 Literature Review.............................................................................................32 Materials and Methods.....................................................................................33 Results..............................................................................................................35 Discussion.........................................................................................................41 Conclusion........................................................................................................43 References.........................................................................................................44 CHAPTER 4.........................................................................................................................45 The impact of CEFR implementation on language education in Malaysia Introduction.......................................................................................................46 Literature Review...............................................................................................47 Methods.............................................................................................................50 Findings/Results.................................................................................................51 Discussion..........................................................................................................54 Limitations/Recommendations/Future Studies..................................................55 i

Conclusion........................................................................................................57 References.........................................................................................................58 CHAPTER 5.........................................................................................................................59 Designing a Localised CEFR-Based Writing Test Introduction.....................................................................................................60 Literature Review.............................................................................................61 Methods...........................................................................................................63 Findings and Results........................................................................................64 Discussion........................................................................................................68 Limitations/Recommendations/Future Studies ...............................................69 Conclusion.......................................................................................................69 References........................................................................................................70 CHAPTER 6.........................................................................................................................71 Unveiling Tertiary Level Students’ Confidence in English Oral Comprehension: Insights from a CEFR-Informed Course Introduction......................................................................................................72 Literature Review..............................................................................................75 Methodology.....................................................................................................78 Findings.............................................................................................................81 Discussion..........................................................................................................88 Conclusion and Recommendations....................................................................89 References..........................................................................................................91 CHAPTER 7.........................................................................................................................95 The impact of CEFR implementation on language education in Malaysia Introduction......................................................................................................96 Literature Review..............................................................................................98 Conclusion.......................................................................................................103 References........................................................................................................104 CHAPTER 8.......................................................................................................................107 Writing Performance in the English Proficiency Test of New-intake University Students Introduction/Literature Review.......................................................................108 Methodology...................................................................................................110 Findings and Discussion..................................................................................110 Conclusion.......................................................................................................115 References........................................................................................................116 CHAPTER 9.......................................................................................................................117 Developing indicators for intangible performance in values-based assessment for courses in CELPAD Introduction......................................................................................................120 ii

Literature Review............................................................................................121 Materials and Methods....................................................................................123 Results and Discussion................................................................................... 124 Conclusion......................................................................................................129 Limitations/Recommendations/Future Studies ............................................. 130 References...................................................................................................... 132 CHAPTER 10.....................................................................................................................134 Exploring Learners’ Perceptions of Project-Based Learning (PBL) in Improving Productive Language Skills Introduction....................................................................................................135 Literature Review............................................................................................139 Materials and Methods....................................................................................138 Results.............................................................................................................140 Discussion........................................................................................................143 Conclusion.......................................................................................................145 Limitations/Recommendations/FutureStudies.................................................145 References........................................................................................................146 CHAPTER 11.....................................................................................................................159 Utilising Flipped Classrooms to Enhance the Reading of Short Stories in English as a Second Language (ESL) Context: A Literature Review Introduction.....................................................................................................151 Literature Review.............................................................................................152 Methodology....................................................................................................165 Conclusion.......................................................................................................166 References....................................................................................................... 167 CHAPTER 12.....................................................................................................................172 Perubahan Dan Pengesahan Modul Penilaian Bacaan Al-Quran TPT Pelajar UIAM Pendahuluan....................................................................................................173 Tinjauan literatur............................................................................................ 174 Bahan dan Metod........................................................................................... 175 Dapatan kajian................................................................................................180 Perbincangan...................................................................................................181 Kesimpulan......................................................................................................182 Limitasi, cadangan dan kajian masa hadapan..................................................182 Bibliografi........................................................................................................183 iii

CHAPTER 13.....................................................................................................................185 Analisis Keperluan Pembelajaran Berasaskan Projek (PBP) Bahasa Melayu Kerjaya Terhadap Kecekapan Berkomunikasi : Satu Kajian Kualitatif Pengenalan.....................................................................................................186 Sorotan Kajian...............................................................................................189 Metodologi Kajian.........................................................................................191 Dapatan Kajian dan Perbincangan................................................................192 Limitasi dan Cadangan Kajian Lanjut...........................................................200 Bibliografi..................................................................................................... 202 CHAPTER 14........................................................................................................................ 1 Kesahan dan Kebolehpercayaan Instrumen Bahasa Ibunda & Identiti Budaya terhadap Prasetia Keusahawanan: Kajian dalam kalangan Pelajar IIUM Pendahuluan.................................................................................................. 206 Sorotan literatur.............................................................................................208 Kaedah Penyelidikan......................................................................................211 Kajian Rintis..................................................................................................212 Kesimpulan....................................................................................................215 Rujukan.........................................................................................................217 CHAPTER 15.............................................................................................................................220 ‘مبادئ ومحددات تصميم البرنامج التعليمي الترفيهي بالمحمول في تعليم التعبيرات االصطالحية العربية وتعّلمها املقدمة.....................................................................................................................221 منهجية الدراسة...........................................................................................................223 نتائج الدراسة...............................................................................................................225 االستنتاجات و اخالمتة..................................................................................................232 أوال: أهم النتائج............................................................................................................232 ثانًيا: أهم معوقات تصميم البرنامج التعليمي الترفيهي بالمحمول........................................232 ثالًثا: الخالصة...............................................................................................................233 املراجع.......................................................................................................................234 CHAPTER 16............................................................................................................................237 توظيف التكنولوجيا في الفصول التي ال تدعمها بشكل كامل: ما الذي يجعل مدرسي اللغات األجنبية يستمرون في توظيفها؟ املقدمة........................................................................................................................239 طريقة البحث................................................................................................................241 النتائج والمناقشات........................................................................................................242 الخاتمة.........................................................................................................................250 قائمة المراجع................................................................................................................251 CONTRIBUTORS ....................................................................................................................253 iv

PREFACE ِه ٱلَّرْحَم ٰـِن ٱلَّرِحيِم َّل ِبْسِم ٱل In the name of Allah, the Most Gracious and The Most Merciful! Praises to the Almighty! Commencing from the International Language Symposium 2022 (ILS 2022) that was conducted in September 2022, the idea of creating an e-book that contained the proceedings of the symposium was born. In the process of producing this book in its virtual form, many interested academics who presented in the symposium as well as those who wished to contribute and share their articles were willing to have their articles published in this edition. Language teachers and researchers hold a responsibility to continue improving their teaching and teaching activities in the classroom. Since the learning environment changes following the times, language teachers should also adapt to the current trends and share their journeys and discoveries to help each other in accommodating the differences. When sharing ideas is conducted through research, this will assist others to learn and adopt new strategies in the classrooms. Hence, it is highly encouraged for language teachers and researchers to publish papers in promoting more effective teaching and learning in adapting to the current academic environment. This book compiles papers by authors who presented in the abovementioned symposium as well as papers by staff of the Centre of Languages and Pre-university Academic Development, mainly from the English Language Division, Bahase Melayu Division, Quranic Language Division and Tilawah Division. The articles which are written in English, Arabic and Malay offer particular insights into the teaching and learning of languages in an English medium institution of higher learning. It focuses on current research pertaining to Common European Framework of Reference for Languages (CEFR), areas in improving language skills, problem-based learning, project-based learning, the use of flipped classroom, Qur’anic reading strategies and technological applications in language learning. The objective of this publication is to spur more research activities among advanced researchers and teachers who are beginners in the field of research. The editors would like to express much gratitude to Allah the Almighty, for granting strength and willingness to overcome the trials of publishing this e-book. The editors also thank all parties involved namely fellow colleagues, the team of reviewers, the graphic designer-typesetter and the higher authorities for making this publication a reality. Wassalam and thank you. Editors: Faridah Abdul Malik Liza Abdullah Engku Haliza Engku Ibrahim Suhaila Abdullah v

Teachers’ Knowledge and Perception of CEFR-aligned Curriculum Faridah Abdul Malik , Lily Azlina Ahmad , Nellia Lizrina Salleh English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) Abstract The launching of Malaysia English Language Roadmap (2015-2025) by Ministry of Education reflects the government’s commitment in improving students’ proficiency in the English Language. The roadmap aims at ensuring the implementation of a fully aligned CEFR English Language curriculum is achieved by 2025. Not only schools were instructed to align their curriculum to CEFR, but tertiary institutions were also urged to do the same. Accordingly, the Centre for Languages and Pre-University Academic Development (CELPAD) of IIUM formed a committee to review and align its English Language courses to CEFR. The implementation of CEFR aligned curriculum came about in Semester 1, 2021/22. It has now been three semesters since CEFR-aligned curriculum was implemented on the presessional English language programme at CELPAD. As part of an academic review exercise, a questionnaire consisting of 22 questions was distributed to CELPAD English Language teachers to determine their knowledge and perception of the CEFR aligned curriculum. The findings have shed light on areas that require more attention to help teachers understand CEFR better, which, in turn, ensures that teaching and learning using CEFR aligned curriculum is successful. Keywords: CEFR, tertiary institution, pre-sessional programme Introduction The Ministry of Education recognises the critical role of English proficiency in enhancing the global competitiveness of the younger generation. In pursuit of this goal, the English Language Teaching Center (ELTC) initiated the English Language Roadmap, which was launched in 2013 and is set to conclude by 2025. The primary objective of this roadmap is to enhance English language proficiency among students across all levels of education in Malaysia. The English Language Roadmap in Malaysia is designed to elevate English language proficiency and usage among Malaysians. It offers a structured framework for the development and implementation of English language education, encompassing educational institutions from primary schools to higher education. The core focus is on improving English language teaching and learning techniques, along with assessment practices, to cultivate individuals with strong English language skills capable of global competition. Moreover, the roadmap aims to ensure the effective integration of English into various sectors, including academia, business, and technology, to support Malaysia's economic and social advancement (Ministry of Education Malaysia, 2015). 2 1 2 3 3 2 1

The roadmap encompasses three phases. Phase 1, spanning from 2013 to 2015, concentrated on enhancing English language proficiency among school teachers. They underwent various training programmes to acquire CEFR knowledge. Phase 2, commencing in 2016, unfolded in two parts. The first segment involved aligning School Based Assessment (SBA) syllabi and curricula with CEFR descriptors. This phase also saw the selection of CEFR-aligned textbooks and support materials. The second part of Phase 2 delved into validating CEFR levels for each educational stage, paving the way for the implementation of the new CEFR-aligned curricula from 2017 to 2020 (Ministry of Education Malaysia, 2013). Phase 3 of the English Language Roadmap in Malaysia encompasses the evaluation, review, and revision of the previous phases. This stage involves a comprehensive assessment of the implementation of the CEFR, including a thorough review of descriptors, textbooks, and teaching materials. The critical outcome of this phase is the development of CEFR-M, which is a localised adaptation of the CEFR tailored to Malaysia's specific linguistic and educational context. This phase plays a crucial role in refining and customising the CEFR framework to suit the needs and requirements of English language education in Malaysia. In accordance with the framework established in the English Language Roadmap, the year 2022 marked the entry of the inaugural cohort of students who had undergone a CEFRaligned English language curriculum in schools into higher education institutions. In preparation for their admission, Malaysian tertiary institutions were required to align their curriculum with the CEFR by the Ministry of Higher Education. CEFR-aligned Curriculum at CELPAD In 2016, the English Language Division (ELD), CELPAD initiated a series of CEFR familiarisation workshops for its teachers. These workshops, led by CEFR master trainers, aimed to provide teachers with valuable insights into the CEFR framework. In 2018, the ELD conducted a comprehensive review of its English Language courses, collecting feedback from both internal and external reviewers. The academic review process commenced in 2019, coinciding with instructions from the Ministry of Higher Education to align English Language courses with CEFR standards. All public universities, overseen by the Jawatankuasa Pemantauan Perlaksanaan Roadmap Bahasa Inggeris di Universiti Awam (Taskforce of the Implementation of English Language Roadmap in Public Universities), were mandated to implement CEFR-aligned English Language courses by 2020. However, the initial plan to introduce these aligned courses in Semester 1, 2020/2021 was delayed due to the challenges posed by the COVID-19 pandemic and the enforcement of the Movement Control Order (MCO). Following approval from the university senate, the newly aligned CEFR English Language courses were introduced in Semester 1, 2021/2022 at CELPAD. After a year of implementation, it is crucial for CELPAD to gather feedback on teachers' perceptions of the CEFR-aligned curriculum. Understanding teachers' perspectives will provide valuable insights for further refinement and improvement of the curriculum. This is also important considering the limited number of studies conducted at the tertiary level up to this point. 3

This research therefore aims at answering these three questions: 1.How much knowledge do CELPAD English teachers have on CEFR? 2.What are the teachers’ perceptions of CEFR? 3.What are the challenges faced by the teachers in implementing the CEFR-aligned curriculum? Literature Review There have been numerous studies done to determine teachers’ knowledge and perception of CEFR in Malaysia. Uri and Aziz (2018) conducted a study to find out teachers and Ministry of Education officials views on the adoption of CEFR onto the Form 5 English syllabus and assessment. The data collected from 331 English secondary school teachers revealed that though the teachers were very optimistic that the framework would help to improve the proficiency of Malaysians, their level of knowledge and awareness about CEFR was low and they reported to have only a minimum exposure to the CEFR. Alih et al. (2020) carried out a study to gauge teachers’ knowledge and beliefs on the CEFR implementation among secondary school teachers in Johor. The semi-structured interviews with seven teachers suggested that their knowledge of CEFR was still inadequate. Some were still not clear of the objectives of the education reform, what each CEFR level entails and had different understandings of the recommended teaching methods to be used. In addition, most of the teachers admitted that they were still confused on how to assess the students according to the CEFR standards. However, despite reporting having insufficient knowledge, all of the teachers believed that the education reform was indeed a good initiative by the Ministry to uplift the quality of English language education in Malaysia. A study by Chong and Yamat (2021) reported a survey finding among twenty inservice English teachers in Malaysia on their level of readiness in the implementation of the CEFR-aligned curriculum. With regards to knowledge on CEFR, only 40% of respondents claimed that they understand both the CEFR-aligned curriculum content and the proficiency scale of CEFR and its descriptors. This seems to reflect that the teachers’ level of readiness is not high enough even though the CEFR-aligned curriculum had started to be implemented at their respective school. They believed more training should be conducted to help them to better understand the usage of the CEFR framework. In order to examine the perceptions of secondary school teachers from East Malaysia, a survey was carried out among 105 teachers in Sarawak (Nii & Yunus, 2022). The findings indicated that the teachers were fully onboard with the implementation of CEFR as it would help produce quality students that can perform well in any situations that requires English and would be accepted globally. However, most of them felt that they were not fully prepared for it as they needed more time to familiarise themselves with the new syllabus and its marking scheme. They also needed readjustment in terms of shifting from a teacher-centered to student-centered approach as propagated by CEFR. 4

There have also been numerous studies carried out to investigate ESL teachers’ perceptions on the implementation of CEFR in Malaysian Primary schools and challenges it poses. One such study was conducted by Ngu and Aziz (2019) on teachers in Subang Jaya. Using a mixed-method design, they collected data from 65 questionnaire respondents and semi-structured interviews with three teachers. With regards to knowledge of the CEFR, the findings revealed that many of the teachers had little knowledge and exposure about the CEFR. Some of the challenges identified were lack of training, non-availability of local based textbooks and inadequate ICT support and resources. Despite these issues, the teachers were very optimistic about the CEFR implementation as they believed it is vital to improve the level of English language proficiency among Malaysians. In another study, Khair and Shah (2021) carried out a survey among 136 primary school teachers. The results of the survey indicated positive acceptance of the CEFF-aligned curriculum by the schoolteachers as they believe the new curriculum can improve students’ competency in English. Nevertheless, they felt that more workshops and training are needed as they still lack indepth understanding of the framework. A more recent study looking at primary school teachers’ perceptions on CEFR was conducted by Kaur and Mah (2022). The five hundred primary school teachers who responded to their survey seemed to report similar sentiments in that though the curriculum reform was well received, the teachers generally felt the actual implementation would be challenging and they need more support and guidance from the ministry. Many studies on determining teachers’ knowledge and perceptions of the CEFR have also been conducted overseas. Sulu and Kir (2014) conducted survey research in Turkey to find out what language teachers know about CEFR and their views on the applicability of the CEFR documents. It was found that more than half of the participants stated that they have read the documents but have not attended any in-service training to enhance their knowledge. The teachers also seemed to be indifferent towards the issues raised in other studies by the European Union. Another Turkish study was carried out by Yakışık and Gürocak (2018) comparing state and private schools EFL teachers’ perceptions on the CEFR. In general, these teachers revealed that they have some insights into the CEFR and its components despite not attending any in-service training on the subject. All the teachers reported having positive views about the CEFR as they believed that the CEFR helped them in designing curriculum, testing and assessments and developing materials for their students. Nevertheless, the teachers from the private schools seemed to be more in favour of the CEFR in all aspects of it compared to those from the state schools. A study on Uzbekistan EFL teachers’ knowledge and beliefs on the usefulness of the CEFR was carried out by Musoeva (2019). 250 EFL teachers from all levels of education participated in the survey. The findings showed that more than half of the respondents claimed that they are familiar with the CEFR, have read the CEFR documents and know how to apply it in their teachings. Nevertheless, the teachers do have some misunderstandings with regards to aspects of assessment according to the CEFR. Kanchai (2019) reported findings of a qualitative study on Thai EFL university lecturers. One of the research objectives is to measure to what extent they understand the concept of the CEFR. The interview with 33 lecturers revealed that they had a reasonably good understanding of the CEFR within the domains of assessment, the CEFR levels and language teaching and learning applications. However, they seemed to have little insight into the action-oriented approach underlying the CEFR. 5

Le and Pham (2019) examined how English teachers in a Vietnamese university perceive the CEFR's top-down implementation for non-English major students. Their finding showed that teachers understood the CEFR's importance and were generally positive about its readiness and implementation. However, they expressed concerns about workload, including time constraints, access to materials, and addressing proficiency gaps. From several studies discussed above, it can be concluded that in Malaysia, there has been cautious optimism among educators regarding the CEFR's potential benefits, despite limited actual knowledge and awareness of its intricacies. The need for additional training and resources is evident, with many educators acknowledging the positive impact the CEFR could have on English language proficiency. While there is an overarching belief in the value of the CEFR initiative, educators highlight the necessity for ongoing professional development, sufficient resources, and collaborative efforts between educators and the Ministry to ensure its effective implementation. Likewise, outside Malaysia, educators demonstrate varying levels of familiarity with the CEFR, often engaging with its documents independently. There is a notable trend of limited participation in in-service training, indicating a potential gap in structured professional development. This is particularly relevant given that some teachers express indifference towards issues raised by European Union studies. In different contexts, such as Turkey, Uzbekistan, Thailand, and Central Vietnam, educators generally hold positive views towards the CEFR. However, there are unique challenges in each context, including disparities between state and private school teachers in Turkey, misconceptions in assessment in Uzbekistan, and a need for a deeper understanding of the action-oriented approach underlying the CEFR in Thailand and Central Vietnam. In summary, both in Malaysia and outside it, there is a consensus on the need for structured professional development and tailored in-service training to enhance educators' understanding and application of the CEFR framework. Adequate resources and continued research are also vital to address specific misconceptions and gaps in knowledge. Materials and methods To address the research questions, a survey was administered to all English Language Department (ELD) staff members at CELPAD. The questionnaire was adapted from Ngu and Aziz (2019). It consists of three parts: Part 1 collected demographic information with 3 items.Part 2 gauged teachers’ knowledge of CEFR, featuring 6 items. The first 5 items employed a multipleanswer format, allowing respondents to select all applicable options. Item number 6 is an openended question whereby the respondents have to list down three aspects of CEFR that they know, apart from the reference level. Part 3 is on the teachers’ perception of CEFR which has 11 five-point Likert scale items ranging from Strongly Agree, Agree, Neither Agree nor Disagree, Disagree and Strongly Disagree. 7 questions addressed the benefits of CEFR-aligned curriculum to teachers while the remaining 4 questions dealt with challenges faced by teachers in implementing the curriculum. 6

Participant Characteristics Sub Profile Percentage Gender Male Female 7% 93% Year of Teaching 1 – 5 years 6 – 10 years 11 – 20 years More than 20 years 10% 5% 49% 36% Position in the department Key personnel Ordinary teachers 32% 68% The findings from the study were analysed through percentage calculations and visually represented using tables, charts and graphs. Results and Discussion Table 1 displays the demographic characteristics of the participants. A total of 61 respondents participated in the survey, comprising 93% female and 7% male staff members. These respondents have varying lengths of teaching experience at CELPAD. The largest proportion of respondents (49%) reported teaching for a period ranging from 11 to 20 years, followed by 36% with over 20 years of experience, 10% with 1 to 5 years of experience, and 5% with 6 to 10 years of experience. Additionally, 68% of the respondents held the position of ordinary teacher, while the remaining respondents were comprised of key departmental personnel. Table 1: Demographic Profile of Respondents The rest of the findings will be presented in accordance with the three Research Questions posed for this study. RQ 1: How much knowledge do CELPAD English teachers have on CEFR? Figure 1 shows the respondents’ familiarity with CEFR. A significant proportion of the respondents (46%) indicated that they were either familiar or very familiar with the CEFR. This suggests a noteworthy level of awareness and knowledge among the participants. Furthermore, 41% reported that they knew the main ideas of the CEFR, indicating a substantial understanding of its foundational concepts. However, it is noteworthy that 13% of participants mentioned having only a vague idea about the CEFR, highlighting a potential area for further education and clarification. Remarkably, none of the respondents claimed to be acquainted solely with the name 7

of the CEFR without a deeper comprehension of its contents. This indicates a commendable level of substance in the participants' understanding of the framework. Figure 1: Familiarity to CEFR The findings in Figure 2 show the respondents’ level of engagement with documents related to the CEFR. A substantial 43.3% reported having read the CEFR itself or its companion materials, demonstrating a proactive approach to familiarising themselves with the framework. Moreover, 45% of participants reported engaging with documents related to the CEFR, including resources like the European Language Portfolio (ELP) and user guides. This indicates an active effort among the participants to seek out supplementary materials that provide a more comprehensive understanding of the CEFR. A notable 63.3% mentioned reading documents that presented the CEFR in a summarised manner. This demonstrates a practical approach to digesting the framework's core concepts in a condensed format. Figure 2: Engagement with CEFR documents 8

It can be seen in Figure 3 that a significant majority of the respondents (68.3%) indicated exposure to the CEFR-related documents during their in-service teacher training. This underscores the importance of ongoing professional development in ensuring educators are well-acquainted with the CEFR. Conversely, none of the respondents reported being introduced to the CEFR during their pre-service teacher training. Additionally, 45% of participants noted encountering the CEFR at conferences, seminars, and related events. Furthermore, 31.7% mentioned other contexts where they were introduced to the CEFR. This included workshops, online courses, and departmental initiatives, indicating a diverse range of educational settings where CEFR exposure occurred. Figure 3: Introduction to CEFR in Educational Contexts Figure 4 shows the participants’ encounters with specific references to the CEFR in various documents. Notably, a considerable 78.7% reported encountering references in textbooks and teaching materials. Moreover, 72.1% mentioned encountering CEFR references in official documents issued by their institutions, including instructions, curricula, syllabuses, and examination guidelines. Additionally, 45.9% identified CEFR references in official national documents, such as ministerial decrees, national curricula, and examination guidelines. In scholarly contexts, 31.1% reported encountering specific references to the CEFR in articles or reviews in professional journals. 9

Figure 4: Specific references related to CEFR Figure 5 illustrates the responses of lecturers on the purposes of CEFR. In the survey, all respondents (100%) recognised the primary purpose of the CEFR in establishing global proficiency standards. Moreover, over half (57.4%) acknowledged the CEFR's crucial role in guiding curriculum development by indicating what aspects of language and language use should be included in teaching programmes. Additionally, 45.9% recognised the CEFR's facilitation of international mobility by providing a basis for comparing language proficiency levels. Only a small fraction (1.6%) associated the CEFR with promoting Europe as the primary source of professional language education information. This indicates that while this purpose may be recognised, it is not considered the primary aim by the majority of participants. Figure 5: CEFR purposes The last item to determine teachers’ knowledge of CEFR required the respondents to write 3 aspects of CEFR that they know apart from the reference levels. The answers given by the respondents are that the CEFR: • provides standardisation 10

• contains ‘Can Do’ statements • is internationally recognised • serves as a reference point • allows localisation • encompasses all language skills • applies to all languages All of the answers given are facts about the CEFR which seem to reflect that the respondents do have some knowledge of the CEFR. The findings above reveal a notable departure from prior studies regarding teachers' familiarity with the CEFR within the context of CELPAD. Unlike previous research by Uri and Aziz (2018), Alih et al. (2020), Ngu and Aziz (2019), and Chong and Yamat (2021), which reported limited awareness of the CEFR among educators, a significant proportion of CELPAD staff demonstrated a commendable level of familiarity with the framework. This is a significant observation, suggesting a potentially effective approach to CEFR integration within the CELPAD community. Furthermore, the active engagement of participants with CEFR-related materials is noteworthy. The tendency to review summarised CEFR documents and related materials indicates a proactive approach to understanding and applying CEFR principles. This aligns with similar findings by Sulu and Kir (2014) and Mosouve (2019), affirming that a substantial portion of educators are actively seeking knowledge and resources related to the CEFR. The mode of exposure to CEFR-related documents, primarily during in-service teacher training and conferences/seminars, suggests an effective utilisation of professional development opportunities. This stands in contrast to the findings of Sulu and Kir (2014) and Yakışık and Gürocak (2018), where a significant portion of participants did not have access to in-service training. This discrepancy highlights the potential impact of tailored training programmes on educators' exposure to CEFR-related materials. Participants frequently encountered references to the CEFR in textbooks and official institution documents, indicating widespread awareness through these sources. In contrast, references in official national documents and scholarly contexts were less common, though still significant. Finally, the unanimous agreement among respondents regarding the purpose of CEFR underscores a strong consensus on its fundamental objective: the establishment of common proficiency standards. Additionally, the majority understanding of CEFR's role in guiding language inclusion in teaching programs and its relevance for comparing language proficiency when studying or working abroad aligns with the findings of Nii and Yunus (2022), emphasizing the significance of CEFR in preparing students for global proficiency standards. Overall, the findings show that the respondents have a substantial knowledge of CEFR. They understand the purpose of CEFR, have read documents on CEFR in various forms which might have provided them with important information about CEFR. These are promising indicators of the impact CEFR integration can have on language education. 11

RQ2 - What are the teachers’ perceptions of CEFR? Figure 6 reveals predominantly favourable perceptions among teachers regarding the CEFR-aligned curriculum. A significant majority believed that the framework effectively aligns assessments with proficiency levels (76%) and ensures appropriate language targeting (54%). Moreover, a majority found it beneficial for lesson planning and objective-setting (48%), and for teaching all four language skills equally (51%). However, a notable percentage of teachers felt that the 'can do' descriptors in CEFR could be made more understandable (56%). Additionally, there is a portion of respondents who either disagreed or expressed neutrality regarding the effectiveness of the curriculum in lesson planning (52%). Furthermore, the overwhelming majority found the CEFR relevant to their work (92%). Figure 6: Benefits to teachers Figure 7 highlights teachers' perceptions regarding the benefits of the CEFR-aligned curriculum for students. A significant majority (56%) strongly agreed that it effectively covers language functions and uses that are necessary for student learning, with only 3% in disagreement. Moreover, 47% believed that the curriculum aids their students in achieving the minimum English language proficiency required by universities, while 36% disagreed with this statement. Regarding the creation of a positive, student-centered environment, 44% strongly agreed and 34% disagreed. In terms of positive implications for students' learning processes, 37% agreed, while 36% expressed disagreement. 12

Figure 7: Benefits to students The findings of teachers’ perceptions on the implementation of CEFR-aligned curriculum will be discussed in two parts: benefits to the teachers and benefits to the students. In terms of the benefits of the implementation of CEFR-aligned curriculum to the teachers, a number of points can be concluded. The findings show that teachers widely believed that CEFR offers effective means to align assessments with proficiency levels, likely influenced by their active involvement in the assessment-setting process, which directly engaged them with CEFR assessment-related materials. This differs from the uncertainty expressed by teachers in studies by Alih et al. (2020) and Musoeva (2019) about assessing students according to CEFR standards. Additionally, Nii and Yunus (2022) noted that many teachers felt underprepared for CEFR, indicating the need for more time to familiarise themselves with the marking scheme. These disparities highlight the necessity for comprehensive training and resources to aid teachers in implementing CEFR-aligned assessments effectively. However, regarding its influence on teaching and learning, teachers expressed some reservations. Less than half found it beneficial for tasks like lesson planning, objective setting, ensuring balanced skill instruction, and targeting appropriate language levels for students. This may be due to their limited involvement in preparing course materials, which were standardised by the curriculum committee. This stands in contrast to the results of Yakışık and Gürocak (2018) and Kanchai (2019), where teachers found CEFR helpful in curriculum design and material development. In addition, many faced challenges in understanding the 'can do' statements within the framework. This echoes Chong and Yamat's (2021) findings, suggesting that teachers may not be adequately prepared for CEFR implementation, potentially due to the complexity of the proficiency scale and its descriptors. These observations underscore the need for targeted training and resources, particularly in curriculum design and proficiency assessment. Despite varying perceptions of CEFR's impact, the majority agreed on its relevance to their work. As for the teachers’ perceptions on the benefits of the CEFR-aligned curriculum on the students, the findings indicate that a majority of teachers believed that the CEFR-aligned curriculum effectively covers essential language functions and usage crucial for student learning. This aligns with similar conclusions drawn in studies by Uri and Aziz (2018), Ngu and Aziz 13

(2019), and Khair and Shah (2021), all of which underscored the positive impact of CEFR on improving students' competency levels. Given that IIUM employs English as the medium of instruction and communication, it has specific language proficiency requirements for its students. In this context, nearly half of the teachers felt that the CEFR-aligned curriculum aids students in meeting the university's minimum language requirements. This is a significant concern for teachers, as ensuring students' readiness for undergraduate or postgraduate studies is paramount. However, a notable portion of teachers did not perceive positive implications of the CEFRaligned curriculum on their students' learning process. This could be attributed to the relatively short period of implementation, as the curriculum has been in place for only a year. It may still be too early to observe the full extent of its impact on students' learning processes. Further observation and assessment over an extended period may be necessary to gain a comprehensive understanding of the curriculum's effects. RQ 3 - What are the challenges faced by the teachers in implementing the CEFR-aligned curriculum? Figure 8 reveals teachers' views on the challenges in the implementation of the CEFRaligned curriculum. A notable portion (31%) felt that their training adequately prepares them for this curriculum, while a smaller percentage (5%) strongly agreed with this statement. However, a substantial proportion (34%) disagreed, indicating a need for more comprehensive training. When it comes to confidence in utilising CEFR-aligned curriculum and teaching materials, a significant majority (36%) expressed confidence, while a notable portion (42%) disagreed. Many teachers (37%) believed that extra training is necessary for effective implementation, with 9% strongly agreeing. Additionally, a considerable number (44%) found it challenging to differentiate students' performances and plan support within the CEFR-aligned curriculum. Lastly, a quarter of respondents felt confident in monitoring and evaluating students' progression in a student-centered environment, while a substantial number (41%) expressed uncertainty in this regard. Figure 8: Challenges in implementation of CEFR aligned curriculum From the findings above, we can see that teachers have identified several challenges in implementing the CEFR-aligned curriculum. These challenges include difficulties in assessing 14

their students' performance and providing appropriate support within this framework. Additionally, teachers lacked confidence in using the CEFR curriculum and teaching materials. They expressed the view that their training did not sufficiently prepare them for effective implementation. Surprisingly, despite these challenges, they did not feel the need for additional training on the new CEFR-aligned curriculum. This could be attributed to the numerous familiarisation workshops conducted during the one-year implementation period, which aimed to acquaint teachers with the CEFR framework. These observations align with the findings of previous studies by Ngu and Aziz (2019), Chong and Yamat (2021), and Khair and Shah (2021), which emphasised the importance of additional training to enhance teachers' understanding of the CEFR framework. Conclusion In conclusion, it could be seen that there was a positive trend in CELPAD staff's familiarity with the CEFR, showing proactive engagement and effective professional development utilisation. There is a widespread integration of CEFR into educational materials, highlighting its significance in guiding language inclusion. Most teachers found CEFR relevant, especially in aligning assessments with proficiency levels. However, they faced challenges in applying it to lesson planning and skill instruction. They also struggled with 'can do' statements, indicating a potential readiness gap. Addressing this and providing targeted training can enhance CEFR implementation. Teachers also believed the CEFRaligned curriculum effectively addresses language functions. However, some were still assessing its impact, considering the relatively short implementation period, thus continued observation will yield deeper insights. As for the challenges faced by teachers in implementing the curriculum, they found difficulties in differentiating students' performance and providing support, a lack of confidence in using CEFR materials, and a perception of inadequate training. Despite those challenges highlighted, there was no strong demand for additional training, which may be due to extensive familiarisation workshops that they were required to attend. Future studies in this area should consider conducting in-depth assessments of the long-term impact of CEFR implementation on language proficiency and academic performance. Additionally, it would be beneficial to explore strategies for further enhancing teachers' understanding and application of CEFR principles, especially in curriculum design and proficiency assessment. Investigating the effectiveness of specific training programmes or interventions in addressing the challenges faced by teachers could provide valuable insights. Lastly, examining the perspectives of students and their experiences with CEFR-aligned instruction would offer a comprehensive view of the framework's impact on their learning outcomes. 15

References 16 Alih, N. A. C., Yusoff, M. A. M., & Abdul, A. H. (2020). Teachers’ Knowledge and Belief on the CEFR Implementation in Malaysian ESL Classroom. International Journal of Multidisciplinary and Current Educational Research (IJMCER), 2(5), 126-134. Chong, G., & Yamat, H. (2021). Teachers' Implementation of CEFR-aligned Curriculum: A Preliminary Study. Journal of English Language Teaching and Applied Linguistics, 3(3), 5-9 DOI: 10.32996/jeltal.v3i3.1366. Kanchai, T. (2019). Thai EFL university lecturers’ viewpoints towards impacts of the CEFR on their English language curricula and teaching practice. NIDA Journal of Language and Communication, 24(35), 23-47. Kaur, P., & Mah, Z. J., (2022). The CEFR-Aligned Curriculum: Perspectives of Malaysian Teachers. Asian Journal of Research In Education And Social Sciences, 4(1), 138-145. Khair, A. H. M., & Shah, P. M. M. (January 2021). ESL Teachers’ Perceptions on the Implementation of CEFR in Malaysian Primary Schools: Issues and Challenges. Journal of Advances in Education and Research, 6(1), 35-49. DOI: 10.22606/jaer.2021.61005 Le, T. T. H., & Pham, T. H. N. (2019). Implementing CEFR at a Vietnamese university - General English Language teachers's perception. CEFR Journal - Research and Practice (1). ,https://doi.org/10.37546/JALTSIG.CEFR1-4 Ministry of Education Malaysia. (2013). Malaysia Education Blueprint 2013-2025 Executive Summary. Ministry of Education Malaysia. (2015). English Language Education Reform in Malaysia: The Roadmap 2015-2025. English Language Education Standards and Quality Council. Musoeva, A. (2019). What do Uzbekistani EFL teachers know and believe about the usefulness of the CEFR? International Journal of Academic Research and Reflection, 7(6). Ngu, M. K., & Aziz, A. A. (January 2019). English Language Teachers' Perceptions on the Implementation of CEFR-Aligned Curriculum Among Primary Schools in Malaysia. In Prosiding Seminar Wacana Pendidikan 2019(SWAPEN 2.0). Nii, A. T., & Yunus, M. M. (2022). Teachers’ Perceptions on the Implementation of Common European Framework Reference (CEFR) in an ESL Classroom: The Malaysian Context. Open Journal of Social Sciences, 10, 226-240. Sülü, A. & Kır, E. (2014). Language teachers’ views on CEFR. International Online Journal of Education and Teaching (IOJET), 1(5), 358-364. Uri, N. F. M., & Aziz, M. S. A. (2018). Implementation of CEFR in Malaysia: Teachers’ Awareness and the Challenges. 3L: Language, Linguistics, Literature, 24, 168-183. https://doi.org/10.17576/3L-2018-2403-13 Yakışık, B. Y., & Gürocak, F. Ü. (2018). A comparative study of perceptions about the ‘Common European Framework of Reference’ among EFL teachers working at state and private schools. International Online Journal of Education and Teaching (IOJET), 5(2), 401- 417. http://iojet.org/index.php/IOJET/article/view/303/243

Examining Achievability of CLOs for CEFR-Mapped English for Academic Writing Course Liza Abdullah , Rosnani Kassim and Siti Zubaidah Anuar English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) The nation’s aspiration to bring the English proficiency level among Malaysian students to international standards has led to the English Language Education Reform in Malaysia and the birth of the English Language Roadmap 2015-2025. According to the Roadmap, at tertiary level, undergraduates are expected to achieve CEFR level C1 upon graduation. Thus, there is a need for Higher Learning Institutions (HLIs) to map their English Language courses’ learning outcomes to the Common European Framework of Reference for Languages (CEFR). At the Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia (IIUM), the mapping process was done on all the English language courses and one of them was English for Academic Writing (EAW) which a university’s required course (UNICORE). After one academic session of its implementation, this study is conducted to investigate whether the Learning Outcome (LO) targeted for writing which is mapped to CEFR level C1 has been successfully attained. The LO for writing is students should be able to write clear, wellstructured exposition (cohesiveness, language, vocab) of complex subjects or topics by expanding and supporting points of view, reasons, and relevant examples. Using a qualitative research method, this study analysed 10 essays based on the marking rubric mapped against CEFR. The results revealed that the LO for writing (task, linguistic range, vocabulary range and grammatical range) targeting CEFR level C1 has been successfully achieved. These would lead to implications for any second language education programmes at large. Keywords: Common European Framework of Reference for Languages (CEFR), Learning outcomes, English Language Roadmap 2015-2025, writing assessment, Essay 18 1 2 3 1 2 3

Introduction The Common European Framework of Reference for Languages (CEFR) has steered the English Language Education Reform in Malaysia and the birth of the English Language Roadmap 2015-2025. The roadmap states that at tertiary level, undergraduates are required to achieve CEFR level C1 upon graduation. Hence, it is imperative for the Higher Learning Institutions (HLIs) to map their English Language courses to the Common European Framework of Reference for Languages (CEFR). In response to this, the Centre for Languages and Pre-University Academic Development (CELPAD) of the International Islamic University Malaysia (IIUM) has taken steps ahead in doing so. The mapping process was carried out on all its English language courses and one of them is English for Academic Writing (EAW) which is one of the university’s required courses (UNICORE). As UNICORE, EAW is a course that is required for students of Year 1 and 2 of the university, depending on the study plans of their respective faculties. Emphasizing on writing skills wherein students are further exposed to writing academic reading reports and extended essays with citation techniques and referencing, this course also incorporates academic reading and oral presentations. On top of that, topics selected for those assignments focus on the 17 goals of Sustainable Development Goal crafted by UNESCO, which is in line with the university’s Sejahtera Academic Framework. After one academic session of employing the newly crafted CLO for the course, there is a need to examine whether the CLO has been achieved. Therefore, this study seeks to investigate whether the LO for writing skills of the course mapped to CEFR level C1 is successfully attained. The LO for writing is students should be able to write clear, wellstructured exposition (cohesiveness, language, vocab) of complex subjects or topics by expanding and supporting points of view, reasons, and relevant examples. Literature Review Since the publication of CEFR, a number of empirical projects and studies on CEFR and its implementation have been carried out. Literature reviews indicated that although the focus was on the level of its impacts, interests in CEFR differed from the areas to the contexts of its implementation. Many focus on students’ learning outcomes whereas a lot are interested in curriculum design and development (Le Thi Thanh Hai, 2018). As cited in Le Thi Thanh Hai (2016), Faez, et al. (2011) observed that the focus of other researchers was more on paying attention to learner autonomy and learners’ proficiency respectively. It was later on summarised that among other domains of language education, CEFR has a much bigger impact on assessment as its impact on curriculum design and pedagogy (Little, 2007). 19

It was proven with the development of DIALANG, the free-of-charge online selftesting service, available in fourteen European languages aiming at helping learners to familiarize themselves with the six- reference- level tests (Figueras, 2007). This at the same time will benefit language learners in terms of following CEFR can-do statements which eventually leads to achieving the learning outcomes. Faez et.al (2011) studied on the impact of CEFR-informed instruction on second language (L2) instruction and learning outcomes in French as second language programmes in Ontario, Canada, and discussed its potential on communicative teaching based on CEFR task-based approach. Apart from revealing the challenges of implementing CEFR-informed instruction namely time restriction and lack of understanding on CEFR and its applicability in the ESL classrooms, it was also found that the main teaching and learning resources that promoted classroom teaching approaches aligned with the communicative learning outcomes specified by CEFR need to be developed. This shows how important it is to use CEFR can-do statements as learning outcomes to ensure teachers and learners are well-guided which can be translated into learners’ understanding and performance. A study by Glover (2011) investigated to what extent the CEFR supports language learning on how a group of learners, in the course of a term, used parts of the Common Reference Levels (CRLs) of the CEFR for self-assessment. The study focused on the extent to which the CRL facilitated in raising the learners’ awareness of their language use and learning, and looked in detail at how the learners described their speaking skills and how their use of CRL statements and their perceptions of self-assessment changed during the term. Results revealed that the students used CRL statements to write longer, more relevant, and more detailed and critical descriptions of their speaking skills. This study is in line with the study conducted by Moonen et al. (2013) that affirmed students’ encouraging performance due to more exposure to CEFR-based testing service offered. In summary, a proper and comprehensive implementation of the Common European Framework of Reference for Languages (CEFR) across all language areas has the potential to significantly benefit student performance and language learning outcomes. Among others, from the aspects of motivation and goal setting, it benefits students by giving clear ideas and setting achievable language goals which can boost motivation and engagement in their language learning journey. 20

Materials and Methods Using a qualitative research method, this study analysed 10 essays randomly selected from students undergoing the course. The essays were gathered and analysed by 3 experienced instructors, based on the marking rubric mapped against CEFR adapted from the Communicative Language Competences rubric (Council of Europe (2022), PP. 77, 130- 142 for C1) as illustrated in Table 1. The areas focused were task, linguistic range, vocabulary range and grammatical range. 21

General Linguistic Sociolinguistic Pragmatic 1.Task 2.Linguistic Range 3.Vocabulary Range 4.Grammatic al Accuracy 5.Sociolinguistic A ppropriateness 6.Coherence Can produce clear, wellstructured expositions of complex subjects, underlining the relevant salient issues. Can use a broad range of complex grammatical structures appropriately and with considerable flexibility. Has a good command of a broad lexical repertoire allowing gaps to be readily overcome with circumlocutio ns; little obvious searching for expressions or avoidance strategies. -Consistently maintains a high degree of grammatical accuracy; errors are rare and difficult to spot. -Has a good command of common idiomatic expressions and colloquialisms; can play with words/signs fairly well. Can produce clear, smoothly flowing, wellstructured language, showing controlled use of organisational patterns, connectors and cohesive devices. -Can expand and support points of view at some length with subsidiary points, reason and relevant examples. -Can select an appropriate formulation from a broad range of language to express themselves clearly without having to restrict what they want to say. -Can understand and use appropriately the range of technical vocabulary and idiomatic expressions common to their area of specialisation. 22

Excerpts Areas Analyses The widespread outbreak of Covid-19 has triggered3 drastic changes4 in the lives of many. The World Health Organizations (WHO) had officially declared3 Covid-19 as a pandemic due to the highly contagious airborne nature4,6 of the virus as well as the lethality of its infection. Many countries promptly executed3 national lockdown to contain the virus and people are forced to stay at home. This incident particularly shifted3 the focus from offline shopping to the booming5 e-commerce. E-commerce become the main alternative for many to get their needs and wants met. Having said that, all businesses in Malaysia should adopt ecommerce as it boosts profits, benefits the environment, and increase productivity and efficiency6. First and foremost6, e-commerce should be adopted by all businesses in Malaysia as it boosts profits. The global pandemic has tremendously improved e-commerce. As stated by Ismail & Gupta (2021), “E-commerce has become essential for virtually all product types and a way forward for businesses” (p. 7). Businesses should take advantage of the growing trend of e-commerce4 to rake profits3,4. Not only will their sales increase as their market reachability is widened4 to bigger audience through the internet, but e-commerce also helps to cut costs significantly. This is because e-commerce allows businesses to operate without having a physical store, and it potentially reduces waste and the need for inventory, warehouse space, and business operating cost as everything is operated online and electronically6. Thus, it is undeniable that the sales boosted, and costs saved by e-commerce will result in higher profits for businesses6. Next, e-commerce benefits the environment. This is because it helps to6 reduce carbon footprint4. A sustainable and ecofriendly business4 is much more appealing4 to customers as it instils a sense of responsibility in them hence, businesses should strive to be one. Businesses can be conducted without transporting through e-commerce (Ismail & Gupta, 2021, p. 7) and since there are less reliance on physical store, businesses can opt for having workers work virtually from home6, reducing the overall number of transports on the road. As e-commerce is digital, papers can be saved. Thus, e-commerce helps to preserve the environment and decrease of global warming6. 1.gTask Can write a suitable introduction. Complex subject matter in a specified field. 2. Linguistic Range a broad range of grammatical structure 3. Grammar Accuracy Consistently maintains high degree of grammatical accuracy. Errors are rare. 4. Vocabulary Range a broad lexical repertoire, less common words 5. Sociolinguistic Appropriatenes Idiomatic expression 6. Coherence Writing is clear and wellstructured. Employs suitable cohesive devices/phrases Results Based on the analysis, it was found that all the 10 essays have successfully met the targeted LO which was pitched at CEFR C1. Using the CEFR-mapped rubric which emphasizes on what students can do in the 6 mentioned areas in the previous section of this paper, the essays were scrutinised based on the CEFR can-do statements. From the 10 essays analysed, a few excerpts taken from 3 of the essays are presented below to represent the other essays. The analyses are presented as follow: Essay 1 23

Excerpts Areas Analyses E-commerce is an online system that allows businesses, consumers, manufacturers, and suppliers to buy and sell goods and services via the internet. According to Steward (2020), “One in seven transactions are now made online” and he also mentioned that according to estimates, global internet retail sales will reach $4.8 trillion by 2021(p.1). Because of this, it is important for all businesses in Malaysia to jump on the train5 and adopt e-commerce, it is because practicing e-commerce will bring many benefits such as provides sustainable lifelines for businesses, improve the economy of the country and environmentally friendly. 6 Firstly, online selling provides sustainable lifelines4 for businesses in times of economic uncertainty. The current pandemic that hits all the nation have concurrently accelerates online retail. According to Ismail and Gupta (2021), “e-commerce has become essential for virtually all product types and a way forward for businesses” (p.7). Companies will be in a better position to produce great results in the future if they invest aggressively in ecommerce.3 While the e-commerce market is more competitive than ever, it also offers firms incredible opportunity to differentiate itself.6 Hence, businesses in Malaysia should take the opportunity and venture along in e-commerce. Lastly, the ecosystem benefits from e-commerce. Ecommerce has the potential to boost a country's GDP while also lowering its carbon footprint.6 Because internet businesses are less reliant on physical storefronts, they have greater leeway when it comes to implementing carboncutting measures.6 This is accomplished by allowing their workers to work electronically or from home, resulting in fewer commuters on the road (Ismail and Gupta,2021). Therefore, e-commerce does not just benefit the economy but also the environment. To summarise, e-commerce unquestionably has a positive impact on the global economy and environment. As a result, businesses and consumers in Malaysia should fully embrace e-commerce for a brighter future. 1.gTask Can write a suitable introduction. Complex subject matter in a specified field. 2. Linguistic Range a broad range of grammatical structure 3. Grammar Accuracy consistently maintains high degree of grammatical accuracy. Errors are rare. 4. Vocabulary Range a broad lexical repertoire, less common words 5. Sociolinguistic Appropriatenes Idiomatic expression 6. Coherence Writing is clear and wellstructured. Employs suitable cohesive devices/phrases Essay 2 24

Excerpts Areas Analyses E-commerce has risen in popularity around the world(2) With the explosive growth(5) of the Internet, ecommerce has become the top buying and selling medium and is rapidly growing. Ismail and Gupta (2021, p. 7) stated that “E-commerce is growing at a striking pace,(4) on track to surpass $1 trillion in sales by 2022”. In addition, it also provides undeniable benefits to businesses throughout the world, changing consumer and supplier relationships, improving company processes, and even reshaping the whole industry.(6) Hence, all businesses in Malaysia should adopt e-commerce because it benefits the environment, it makes businesseasier, and it helps the economy growth. Next, businesses should adopt e-commerce as it makes business easier. Having an e-commerce platform means that the owners will not only rely on the sales from the physical stores.(2) E-commerce allows people to shop from any stores at any time without restrictions. The buyers do not have to wait for the normal operating hours, they can make a purchase in the middle of the night and just wait for the product to be delivered.3 It also helps the owners as with e-commerce applied, they will need less workers to operate compares to opening a physical store as they do not have to hire sales assistants to assist the customers. 1.gTask Can write a suitable introduction. Complex subject matter in a specified field. 2. Linguistic Range a broad range of grammatical structure 3. Grammar Accuracy consistently maintains high degree of grammatical accuracy. Errors are rare. 4. Vocabulary Range a broad lexical repertoire, less common words 5. Sociolinguistic Appropriatenes Idiomatic expression 6. Coherence Writing is clear and wellstructured. Employs suitable cohesive devices/phrases Essay 3 Discussion As the study was to examine whether the LO for the writing is achieved, the themes for the analysis set by the researchers are based on the criteria from the Communicative Language Competences (Council of Europe (2022), Pp. 77, 130-142 For C1). Based on the analyses presented above, from the aspect of task all essays analysed were found to have met to the requirements of the question. They demonstrated the ability and flair in defining the subject matter and were also able to support and elaborate well by providing relevant examples for more clarity. As far as linguistic and grammatical range is concerned, the essays exhibited the ability to apply a wide range of tenses consistently as well as accurate parts of speech needed for the writing. The essays also exhibited a broad use of higher order word choice, be it terminologies or correct registers. Grammar wise, they managed to maintain a high degree of accuracy with almost no errors. A good command of idiomatic expressions and colloquialisms is also used effortlessly which supports the sociolinguistic appropriateness of CEFR Level C1 of the course. Finally, the essays also demonstrated clear, smooth flowing and well-structured language and at the same time showed the ability to consistently maintain coherence and cohesion. It can be concluded that the learning outcome for the writing skill for the course has been attained based on students writing performance. The findings correspond with what was emphasized by Moonen et al. (2013) as cited in (Le Thi Thanh Hai, 2016), Faez et al. (2011) as well 25

as Glover (2011) in which they were moving toward suggesting that there is a need to incorporate CEFR can-do statements in teaching and assessments to see students’ promising performance in language skills. Conclusion The aim of this study was to investigate whether the learning outcome based on CEFR can-do statements for writing skill for the English for Academic Writing course of the International Islamic University Malaysia is attainable. Using the Communicative Language Competences (Council of Europe (2022), PP. 77, 130-142 for C1) presented in Table 1, 10 essays were analysed and result analyses on 3 of the essays were presented in this paper. Based on the results and evidence presented in the earlier part of this paper, it was found that the learning outcome set for the writing skill for the course was achieved successfully. From the essays, students have demonstrated the ability to write clear, well-structured exposition (cohesiveness, language, vocab) of complex subjects or topics by expanding and supporting points of view, reasons, and relevant examples. Based on these findings, it is imperative for all higher institutions to ensure the curriculum and assessment of English courses are aligned to CEFR. Specific objectives of each language skill need to be defined clearly to make sure instructors and students are aware of what they are doing which would lead to more uplifting language performance. Nonetheless, the fact that this study analysed 10 essays, they may not have comprehensively represented the whole population. Therefore, more bigger scale studies should be conducted to obtain more accurate results to validate the findings of this study. 26

References 27 Faez, F., Majhanovich, S. Taylor, S., Brown, P., Smith, M., & Crowley, K. (2011). The power of “Can Do” statements: teachers’ perceptions of CEFR-informed instruction in French as a Second Language Classrooms in Ontario. Canadian Journal of Applied Linguistics, 14(2), 1-19. Figueras, N. (2007). The CEFR, a lever for the improvement of language professionals in Europe. Modern Language Journal, 673–675. Figueras, N. (2012). The impact of the CEFR. ELT Journal, 66(4), 477–485. Oxford University Press. Jones, N., & Saville, N. (2009). European language policy: Assessment, learning, and the CEFR. Annual Review of Applied Linguistics, 29, 51–63. Le Thi Thanh Hai (2018). Impacts of CEFR-Aligned Learning Outcome Implementation on Assessment Practice at Tertiary Level Education in Vietnam: An Exploratory Study Hue University Journal of Science: Social Sciences and Humanities Vol. 127, DOI: 10.26459/hueuni-jssh.v127i6B.4899 Little, D. (2007). The Common European Framework of Reference for Languages: Perspectives on the making of supranational language education policy. The Modern Language Journal, 91(4), 645–655. Moonen, M., Stoutjesdijk, E., Graaff, de, R., Corda, A. (2013). Implementing the CEFR in secondary education: Impact on FL teachers' educational and assessment practice. International Journal of Applied Linguistics. 23(2), 226–246.

Examining the Attainment of Learning Outcome for Spoken Production in a CEFR-Mapped English Course at CELPAD, IIUM: A case study Rosnani Kassim , Siti Zubaidah Anuar , Liza Abdullah English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) English Language Division, Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia, Gombak (E-mail: [email protected]) Abstract The efforts in bringing the English proficiency level of Malaysian students to international standard have led to the advent of the English Language Roadmap 2015-2025. At tertiary level, the language proficiency targeted among graduates is CEFR level C1. As all Higher Learning Institutions (HILs) are partaking in this national agenda, the International Islamic University Malaysia (IIUM) is committed in doing so and has worked towards fulfilling it by mapping the curriculum and assessments of its English Language courses to CEFR. One of them is English for Academic Writing, a university required courses. Apart from its emphasis on academic writing, this course also incorporates oral presentation. After implementing the curriculum for 2 academic years (2020/2021 & 2021/2022), it is imperative to analyse whether the Learning Outcome (LO) for speaking skills: Can give a clear, well-structured presentation of a complex subject by expanding and supporting points of view at some length with supporting points, reasons and relevant examples, which is mapped to CEFR C1 has been successfully attained. Hence, this case study was conducted to analyse students’ performance in spoken production of 3 students. The oral presentations were recorded, transcribed and then analysed using qualitative data analysis method. The research instrument used was a marking scheme with descriptors mapped to CEFR Can-do statements for level C1 for addressing audience (giving presentation). The analysis revealed that the students demonstrated the speaking ability of CEFR C1 based on the topic, range, accuracy, fluency, coherence and phonology. This study concluded that for the 3 students, the learning outcome has been attained and they would graduate with CEFR level C1 in speaking skills. Keywords: Spoken production, Learning Outcomes, CEFR, Oral Presentation, Assessment. 29 1 2 3 1 2 3

Introduction One of the current challenges of education is to produce school leavers or graduates who can function as global citizens. For one to be a successful global citizen is via possessing English language proficiency that is of the international standard. The Ministry of Education (MOE), Malaysia has initiated the English Language Education Reform in Malaysia by setting up The English Language Standards and Quality Council (ELSQC) that produced the English Language Roadmap 2015-2025 (Mohd Don & Abdullah, 2019). The roadmap charted the English Language education in Malaysia from pre-school until university with the goal to produce students who are proficient users of English and can use English effectively in various contexts. To ensure that the English language proficiency of Malaysian students is of international standard, the targeted proficiency level for each level of education is based on the Common European Framework of Reference for Languages (CEFR). There are six levels of proficiency based on the CEFR: A1 (beginner/ elementary), A2 (preintermediate), B1 (intermediate), B2 (upper intermediate), C1 (advanced) and C2 (proficient). It is stipulated in the Roadmap that the target English language proficiency level for undergraduates is CEFR C1 upon graduation and the need to adopt a common language framework is further emphasised in the roadmap for language education reform in Malaysia where it was stated that: The common framework brings with it the advantage that it can be used by all parties working on the English language programme. It would be difficult enough for assessors and curriculum developers to integrate their work without a common framework to work to, and well-nigh impossible to bring teachers and materials developers into line. The possibility of getting assessors, curriculum developers, teachers and materials developers to produce a pedagogically ordered and internally consistent programme from preschool to tertiary level would be zero without a common framework, and not worth even considering. It might be possible to devise a framework ad hoc, but it is obviously preferable to use a framework which has already been developed and tried and tested. (English Language Education Reform in Malaysia: The Roadmap 2015-2025, p. 74) Hence, in tandem with the MOE’s aspiration and goal to align the standard with the required CEFR levels, the Centre for Languages and Pre-University Academic Development (CELPAD), International Islamic University Malaysia (IIUM) underwent a rigorous process of mapping the learning outcomes for the English 30

language courses it is offering such as the English Proficiency courses, English for Academic Writing course and English for Specific Purposes Courses to the appropriate CEFR levels. Therefore, this study will look at the English Academic Writing (EAW) course, a university required course offered by CELPAD, which is a compulsory course to be taken by all undergraduates who are in their first year. Even though this course focuses on developing students’ skills in Academic Writing, it also includes oral presentation as a way for the students to report their written products. In addition, it also carries 10% of the overall course assessment. The oral presentation scoring rubric has also been developed based on the CEFR C1 performance for spoken production (addressing audience/giving presentation). According to the CEFR, oral activities are divided into two, which are spoken production and spoken interaction. The activities included in oral productions are as follows (Council of Europe, 2020, p.61): 1. Sustained monologue: Describing experience 2. Sustained monologue: Giving information 3. Sustained monologue: Putting a case 4. Public announcement 5. Addressing audiences Meanwhile, the activities included in oral interactions are (Council of Europe, 2020, p. 71): 1. Understanding an interlocutor 2. Conversation 3. Informal discussion 4. Formal discussion 5. Goal-oriented co-operation 6. Obtaining goods and services 7. Information exchange 8. Interviewing and being interviewed 9. Using telecommunications The oral assessment for the EAW course used for this study is in the form of spoken production – addressing audiences. Having the ability to give formal oral presentations is essential among university students and due to this, it is deemed to be one of the most suitable oral production skills to be taught to and mastered by university students for their academic endeavour. The learning outcome for the speaking component of this course is students “Can give a clear, well-structured presentation of a complex subject by expanding and supporting points of view at some length with supporting points, reasons and relevant examples”. This learning outcome has been mapped to the Can-do statement for CEFR Level C1 (Addressing audiences) which is “Can give a clear, well-structured presentation of a complex subject, expanding and supporting points of view at some length with subsidiary 31

points, reasons and relevant examples; Can structure a longer presentation appropriately in order to help the audience follow the sequence of ideas and understand overall arguments.; Can handle interjections well, responding spontaneously and almost effortlessly” (Council of Europe, 2020, p. 65-66). To date, the students’ achievements in the course are reflected through their final grades which contain the total scores of their performance in all the related skills taught in the course, such as speaking, reading and writing. However, the attainment of the learning outcome for the speaking skill per se has never been investigated in detail. Thus, this study aims to examine whether the learning outcome for the oral presentation component of this course has been attained and at the same time confirms that the students have indeed attained CEFR C1 level in terms of their oral presentation skills. Literature review Since the introduction of CEFR, there have been numerous studies done at the different contexts where language education is concerned. Foley (2019) acknowledged the huge influence CEFR has brought to language education by stating that it has brought great impact to language teaching globally in three areas, which are language policies, teaching and learning and evaluation or assessment. In terms of speaking skill assessment based on CEFR, a study was conducted by Razali and Abd Latif (2019) to examine students’ own selfassessment of their spoken ability in both spoken production and spoken interaction. A total of 233 students from two Malaysian public universities took part in the study and it was found that the students felt their ability ranged between B1 and B2. So, for this group of students, they rated themselves as being at the intermediate or upper intermediate level. None of them felt that they were at C1 level (advanced). This finding is somehow expected since based on the roadmap, the target level for university undergraduate is B2. Meanwhile, Waluyo (2019) conducted a study to investigate the level of English proficiency among a group of 2248 students who were in their first year at a university in Thailand. He employed the Walailak University Test of English Proficiency (WU-TEP) as the data collection instrument. Wu-TEP is a battery of tests that is CEFR informed, and it contains tests to evaluate students’ writing, reading, speaking and listening performance from A1 to C1 level. The findings showed that with regards to speaking, most of the students were at A1 level (63.5%) and A2 level (19.7%). While it was reported that 11% were at B1, only 4.5% at B2 and 1.3% at C1. We can see that these figures are quite alarming considering that the students are all undergraduates, and they should possess a considerable English language proficiency for them to succeed academically. The author concluded that several factors 32

including traditional method of teaching and learning, Thai traditional cultures, status and hierarchy, and the characteristics of the students could be the reason for the low level of proficiency among the students. A similar study was also carried out by Wanna et al. (2016) among 368 freshman students in a university in Ethiopia. All the students sat for a proficiency test conducted by the university and the general English proficiency level of most of the students was categorised as low (81.5%) scoring between 0 – 50 in the proficiency test, with 16% and 1.6% were in the average (scoring 50 – 69) and high proficiency level (scoring 70 – 100) respectively. To assess the students’ spoken ability, due to the financial and logistic constraints, three students from each category were randomly selected to take the speaking exam which was conducted via an interview. The scoring criteria for the oral test is as shown below: (Source: Wanna et al., 2016, p. 135) The results showed that students who had high proficiency scored 3 in each of the criterion, those who had average proficiency scored 2 for each and those in the low proficiency group scored 1. Apparently, the results indicated that students with high general language proficiency also displayed high oral communication ability while students who had low proficiency had low oral communication ability. Materials and methods For this course, the students are required to give an oral presentation based on a Sustainable Development Goals (SDG) theme. They prepare the oral presentation in groups but they will be assessed on their individual performance during their individual presentation 33

Can do statements for sustained monologue (addressing audiences/ giving presentation) for C1 (Council of Europe, 2020, p. 65-66) Learning Outcome (LO) for the EAW course Can give a clear, well-structured presentation of a complex subject, expanding and supporting points of view at some length with subsidiary points, reasons and relevant examples. Can structure a longer presentation appropriately in order to help the audience follow the sequence of ideas and understand overall arguments. Can handle interjections well, responding spontaneously and almost effortlessly. Can give a clear, well-structured presentation of a complex subject by expanding and supporting points of view at some length with supporting points, reasons and relevant examples. (5 minutes each). The presentation should comprise several elements. First is the introduction that contains the background of the topic, the problem statement and an opinion-based thesis statement. Next is the body that consists of three main ideas with supporting evidence and analysis. This is followed by the conclusion which includes the summary of the three main points and some practical recommendations. For this research, three students (2 female and 1 male) were selected via random sampling from 3 different teams. All three students attained CEFR Level B2 in an internal proficiency exam that has been mapped to CEFR. Student 1 presented a topic on Mental Health (SDG3), while student 2 presented a topic on environmental sustainability via social media (SDG11) and student 3 on renewable energy (SDG7). Their presentations were recorded and later transcribed. In order to examine whether the students have successfully given a presentation that reflects the ability of C1 level performance as stated in the course learning outcome, the data were analysed using the descriptors mapped to CEFR Can-do statements for C1 level for addressing audience (giving presentation) with the addition of the qualitative features of spoken language for C1 (Council of Europe, 2020) as shown below: Table 1: The mapping of Learning Outcome (LO) against Can do statements for giving presentation CEFR C1 level Table 2: Qualitative features of spoken language Range Can show a good command of a broad range of language allowing him/her to select a formulation to express him/herself clearly in an appropriate style on a wide range of general, academic, professional or leisure topics without having to restrict what he/she wants to say. 34

Results The aim of this research is to examine whether the learning outcome for the oral presentation component of the EAW course has been attained and that the students demonstrate the performance of C1 level in giving presentation. Therefore, the data extracted from the transcribed presentations will be presented and the findings will be analysed based on the learning outcome and the qualitative features of spoken language such as range, accuracy, fluency, coherence and phonology, that have been described in detail in the preceding section. Student 1 The student chose the topic of mental health for her presentation which is a complex subject suitable for C1 level. In addition, we found that she enhanced the audience’s understanding of the concept of mental health by citing an authoritative figure in the field in defining mental health. This can be illustrated below: Accuracy Can consistently maintain a high degree of grammatical accuracy; errors are rare, difficult to spot and generally corrected when they do occur. Fluency Can express him/herself fluently and spontaneously, almost effortlessly. Only a conceptually difficult subject can hinder a natural, smooth flow of language. Coherence Can produce clear, smoothly flowing, well-structured speech, showing controlled use of organisational patterns, connectors and cohesive devices. Phonology Can employ the full range of phonological features in the TL with sufficient control to ensure intelligibility throughout. Can articulate virtually all the sounds of the TL; some features of accent retained from other language(s) may be noticeable, but they do not affect intelligibility at all. “Jean hhh Jean Hallhouse a aaa who has more than 26 years ofexperience providing terapi (therapy) and is currently regional director… a regional director… defined mental health as a physical dises (desease) of the brain that causes disturbance in thinking, behaviour, energy, or emotion and make… that makes it difficult to cope with the demands of normal life. In recent years, there has been roaring number of cases aa of people having mental health problems.” 35

This clearly indicates that she was able to support and expand her point with supporting evidence or example . Next, there is clear evidence of a wide range of vocabulary used. The use of common words or phrases such as “realise, causes” to less common ones were found. Below are some examples: “…Whether people realise it or not in line with the growing development…” “…more various causes that have triggered mmm mmm mental health issues.” “…that makes it difficult to cope with the demands of normal life.” “…there has been roaring number of cases…” In regard to accuracy, it is evident that the student was able to maintain a high degree of accuracy in a consistent manner. Most of her sentences were grammatically accurate and she was able to use different tenses with precision and ease. Moreover, compound and complex sentences were used throughout the presentation and they were formed correctly with accurate use of parallel structures and subject and verb agreement. These can be seen in her utterances below: “…there are more and more various causes that have triggered mmm mmm mental health issues.” [correct subject verb agreement and present perfect tense] “…Jean Hallhouse a aaa who has… and is currently regional director… a regional director and clinic clinic manager…” “…a physical dises (desease) of the brain that causes disturbance in thinking, behaviour, energy or emotion and make… that makes..” Furthermore, we discovered that errors were rare and when she made any errors, the student was able to correct those errors immediately. This can be seen when the student mentioned, “is currently regional director” and then she promptly corrected it by saying “is a regional director..”. Another instance is when she said “… and make…that makes it difficult..”. To examine the fluency and the phonology aspects, the researchers observed that generally, the student’s speech was fluent with very occasional uses of filler such “aaa” that did not affect the flow of her speech at all. It was also found that she enunciated most words clearly and correctly except for a couple of mispronounced words which were “terapi” (for therapy) and “dises” (for disease). Nevertheless, her speech was generally perfectly intelligible. Finally, it was also found that her ideas were well organised in the sense that since it was an introduction, there was a smooth flow from introducing the idea of mental health in general to the specific definition of mental health and then to how it is affecting people nowadays. 36

Student 2 The student chose the topic of the role of mass media in spreading information to the community about environment safety. Clearly, this is a complex subject to be researched on and presented; and the student supported his main idea with two subsidiary details. One was a study done in China which examined the role of social media in spreading information in China. He mentioned: “firstly is uh...social media, where...a place of millions of people connect and communicate across the globe uh with a single click. It is much easier uh to...spread information and knowledge. It is an effective platform uh in the current world...to reach specific audience. So, according to Weng et.al.. uh...the use of electronic device in today’s world is common in cellphone and tablet. By...uh gaining access to the internet, they can use the sosial (social) media. In China, typically people use WeChat and Weibo...an app-a social media app uh to... spread information uh regarding pollution so that it is much uh...faster...to travel and spread regarding pollution.” Apart from social media, he also expanded his supporting detail by highlighting the role of television and radio. This can be seen in the following sentence: “And the other form of different media technology is uh...television and...radio...and that they need to broadcast advertisement about the danger of pollution to increase the-uh to increase awareness among the viewers. It is clear that these two instances demonstrate that the student can expand and support points of view at some length with subsidiary points and relevant examples which is in line with the course learning outcome. The data also showed that the student used a lot of complex and compound sentences. Simple sentences were used too but not as often as the former two. A couple of examples of the sentences are shown below: “there are many technologies regarding to mass me-mass media, but according to a study by Weng et al. in 2016, um...most people in China get their information regarding pollution uh from...the news channel.” “It is apparent that uh most um information that people uh...acquired is from news channel like...TV3 if they want to know about the haze that happened before...but there should be more uh television stations and radio that promote this because for every individual... as every individual has different preference uh for them to acquire different information from sources.” 37

At the same time, the student displayed a broad range of repertoire by using words and phrases shown below: “…It is an effective platform uh in the current world...to reach specific audience.” “…as every individual has different preference…” “…for them to acquire different information…” “…they need to broadcast advertisement about..” “…including the custedian (custodian) of nature.. it is the duty of Muslims to respect and take care..” In addition, the student consistently showed excellent grammatical control throughout the presentation and was able to self-correct himself in order to deliver a good and clear presentation. This is evident from the following examples: “In China, typically people use WeChat and Weibo...an app-a social media app..” “…everyone should use the media to promote more awareness on pollution and is..its negative impact..” “So that… there should be a broadcasted…” We can see that when he uttered the phrase “an app-a social media app”, he corrected himself by inserting the adjective “social” before the word “media” to make his point clearer and more specific. One more example shown above is when he mistakenly said “...and is...” and then instantly corrected himself by uttering “...its negative…” and later in his presentation he self-corrected himself again when he mistakenly said “So that..” and then said “...there should be a broadcasted...” It was evident from the recording that the student generally delivered his presentation smoothly, fluently and effortlessly. Even though we could hear the student used a filler like “..uh..”, it did not affect the clarity of information and his smooth delivery at all. His pronunciation was clear and intelligible throughout the presentation with a slight accent which was not intrusive at all. The student’s main ideas and supporting details were presented in a highly coherent manner. There was a clear progression from the introduction and thesis statement to the elaborations of main ideas in the body and finally to the concluding section. It was found that the student used transitional signals appropriately and effectively which contributed to the coherence. Some examples are shown below: “…now I’m going to talk…” “…there are many technologies..” “…And the other form of different media technology..” “…It is apparent that…” “…Um...for example…” 38

All in all, it can be concluded that the student had delivered the presentation successfully and in doing so, displayed the abilities expected of a person with C1 proficiency. Student 3 The student presented a topic on renewable energy - a complex topic which involves abstract concepts. She argued that people should utilise renewable energy in order to improve their standard of living. The data shows that the student presented three main ideas to support her opinion and provided several supporting evidence to substantiate her view. The following excerpts taken from her second main idea demonstrate this: “…for the second main idea, renewable energy consumption can improve people … people’s standard of living because it allows people to build wealth in the long run...but how? Okay...it’s because renewable technology like the solar system and wind turbine enables people to produce their own electricity at homes.” “…the second reason...renewable energy consumption will contribute to job creation. Simple, when people start to shift their consumption to renewable energy...it creates demand. So, when there an increasing trend of demand for renewable energy, the investors and businesspeople will react to its..to this demand by upping up more businesses and companies to...to-that produce renewable energy to meet this demand.” Regarding language and vocabulary, the student displayed a broad range of language and was able to express herself clearly. Below are some of the sentences that she produced during the presentation: “…but through the government’s incentive of feed-in tariff, people can actually earn some passive income...by selling any excess of electricity generated from the solar to theelectric grid or...in Malaysia, people usually referred it to as TNB, Tenaga Nasional Berhad.” “Right, then so...when people have more wealth or income...technically or economically, people will have more ability to...buy the othe-to buy more of the other goods and services.” “…when people have income or even more income, they have more ability to spend on the other goods and services. But it doesn’t stop there...because people will have uh...but people will have more access to...non-economic benefits such as good education and um also quality healthcare. So, will having this access help people to improve their standard of living? The government on the other hand can reduce its allocation of welfare budget to the unemployed group in the country, and this allows the government...to redirect this allocated fund to be spent on the other government’s development plans.” 39

From the examples, there is clear evidence that the student could produce different types of sentences which are also grammatically accurate. For instance, the student displayed the ability to use variety of tenses, possessive noun like “government’s”, parts of speech like noun like “wealth” and adverbs like “technically” and “economically”, as well as many other grammatical items in a correct manner. Not only that, we also found that apart from using common words, the student also used less common words such as “tariff”, “consumption”, “demand”, “access”, “allocation” and “economic benefits.” Moreover, errors were also difficult to spot and when they happened, the student was able to correct them instantaneously as can be seen in the following utterances: “...enable people to produce their own electricity at homes. Though, by right...it means they can reduce their electricity bills. Not only that, but the...but through the government’s incentive...” “…people will have more ability to...buy the othe...to buy more of the other goods and services.” The utterances highlighted in bold confirm that the student was able to correct her mistakes as she was delivering her oral presentation so that she was able to relay her ideas using correct structure and grammar. Moreover, the student had delivered her presentation smoothly, fluently and effortlessly. It was very rare to find the use of fillers and if there were any, they were unnoticeable. The flow of her ideas and examples was highly coherent from the start to the end of her presentation. This was achieved through effective use of referents and transitional signals as shown below: “…the second main idea, renewable energy consumption can improve …because it allows…” “Not only that, but…” “So, imagine what would happen…” “…if people saved electricity bill…They build wealth.” “..the second reason...renewable energy…” “…when people start to shift their consumption to renewable energy...it creates…” “…So, because of this..” “...as a result, more unemployed…” “The government on the other hand can…” “Hence...it is clearly shown…” Finally, throughout her presentation, the student maintained clear and intelligible pronunciation with appropriate word and sentence stress. Hence, throughout her presentation, the student had displayed the ability as stated in the course’s learning outcoming. 40

Discussion The aim of this study is to examine whether the learning outcome for the speaking skill of the EAW course has been attained based on the students’ performance during their oral presentations. Based on the results presented in the previous section, there are several important findings that can be highlighted. The learning outcome that has been mapped to CEFR C1 level states that at the end of the course, in terms of addressing audience (giving a presentation), the students should be able to give a clear, well-structured presentation of a complex subject by expanding and supporting points of view at some length with supporting points, reasons and relevant examples (Council of Europe, 2020, p. 65-66). The results show that all the three students selected topics that are complex and related to some of the SDGs which are mental health (SDG 3: good health and well-being), the role of mass media in spreading information about environment safety (SDG 11: sustainable cities and communities) and renewable energy (SDG7: clean energy). These topics require deep understanding about abstract concepts like sustainability and its importance to individuals and communities. The students also displayed their ability to express an opinion related to a specific issue on the topic and the results confirmed that all of them had successfully supported their opinions by providing reasons and relevant examples based on facts and data that were cited properly. Hence, it can be concluded that generally, the learning outcome for speaking that is pitched at CEFR C1 level has been successfully attained. Next, the results also indicate positive performance by the students regarding the five qualitative features of spoken language which are range, accuracy, fluency, coherence and phonology. The analysis of their utterances shows clear evidence of a wide range of language used which is a distinct feature of CEFR C1 ability as indicated in the CEFR Companion Volume (Council of Europe, 2020). To illustrate, the use of simple, compound and complex sentences can be seen throughout the students’ presentations and we could see that the students displayed a broad range of repertoire while delivering their speech. The analysis did not only find ample use of common words or phrases but also found the use of less common words or phrases such as “triggered and roaring number, acquire different information , custodian of nature and allocation” in the presentations. Hence, this confirms that the students have attained the performance of CEFR C1 level in terms of language and vocabulary range. 41