P a g e 51 | 239 Furthermore, I also encouraged my students to enter competitions and exhibitions to showcase their inventions. I find that students would work hard when given the opportunity to let them showcase their work. During the preparation stage, they will find extra information, work on their product, and go beyond what is being taught. For example, in my flipped classroom, I taught students on how to develop knowledge management systems. They will use their own creativity to create knowledge management systems (KMS) using the concepts taught. They will ensure all elements are included as a complete package. They will also go beyond the miles, to learn new technology to design the KMS, and add unique features that they found useful to the target audience. Some of them took the effort to promote the KMS to public, sell them online and conduct user testing for validity. This is the criteria that I would like my students to have, which is to produce work beyond what is taught in flipped class. I have learnt that students will find more knowledge when given incentives and rewards. This include letting them to participate in competitions, exhibitions, and other avenues. Furthermore, during after flipped activities such as exhibitions, conferences, and competitions students often receive questions that test their knowledge and understanding. These questions are practical and related to industry needs. These include business idea, copyright issues and marketing plans. I find it interesting to let students think and learn beyond what could be read in books, but very much related to the real world. Moreover, students also get to connect with various people from different types of industry including investors that may be interested in their work. They also get the opportunity to network with potential funders. Most of them would share more ideas with my students on ways to improve their work and to make it marketable. Many would want to collaborate and alleviate the idea. As I could not give nor teach those, I let the experts do it!
P a g e 52 | 239 Pictures taken in various competitions that my students have entered and organized at university and national levels as part of the after flipped class activities.
P a g e 53 | 239 3.2a INNOVATIONS: BEFORE FLIPPED CLASSROOM Over the years since 2014, I have started innovating flipped classroom content and resources for students to learn and understand at their own phase and any time before the in-class learning takes place. This is important as during Self-Learning Time (SLT) lecturers are not present. Therefore, having engaged and self-explanatory materials that could guide students’ understanding is extremely crucial. I wanted them to get the fundamental theory right and to avoid any misconception, misunderstanding and wrong interpretation. The more challenging part is to create or curate learning materials that are interesting, motivating and not boring. Amateur Videos In 2014, I have started with amateur videos to teach students about many topics that include creating tables, diagrams and explaining about theories. I have started with screencast-o-matic tool, and the quality was really bad at that first point! An example of amateur video taught students on the steps to create a geodesic table, created using screencast-o-matic https://www.youtube.com/watch?v=xrw9ksiA2l0 In-class recording I have then started to capture my teaching and learning in class that happens and records them. At that point of time, the idea is to allow those who miss my class due to any particular reason to still be able to experience the learning in class that takes place. It was recorded videos that many students appreciate. Not only because to those missing the class, but for everyone to refer and watch over and over again after class and before exams for revisions. It is also being appreciated most by students who often could not focus in-class, slow in grabbing the topic and weak students. Many would go online and watch this video after class for recaps. I will always feel happy when videos like this becomes so useful and a source of knowledge to anyone.
P a g e 54 | 239 Watch - Example of a video explaining about process and steps in class. https://www.youtube.com/watch?v=xrw9ksiA2l0 A better semi-pro video The videos have to get better, so I told myself! As I have viewers from any parts of the world and students that could be using these videos to learn, it should be off better quality, that was what I told myself. I used the tripod and with better camera and some editorial help, clearer and more professional videos are created! Watch - Example of a semi-pro video explaining about knowledge codification topic https://www.youtube.com/watch?v=T5DnXUeYia0&t=143s Robots used as part of learning activities A few robots are designed and programmed using the EV3 Lego Mindstorm robotic sets. These robots are being introduced to students during lab hours to learn about certain topics. This mostly are related to programming of codes and students being able to view the output after they successfully codes the lines. This activity during lab hours has made students being able to test their codes and look at the output. For example, students will be able to check whether their codes for turning left is working. The lines of codes could be easily tested by looking at the movement of robot and turning to the left. The lines of codes could then be rectified and tested again until students get them right. This made flipped learning in class active, and 3
P a g e 55 | 239 hours of lab is often not enough. Students are seen staying till late evening to get their codes right! The LEGO EV3 Mindstorms sets are available in the Roboholic lab at the Block 02 of UTP academic block. Students can always come during non-class hour to use the lab and test the programming knowledge, debug errors, build robots and create new inventions. Through hands-on experience like, this student could apply the theoretical knowledge of programming and apply it and visualize the output for better understanding and knowledge retention. Some of the robots developed for many topics such as identifying shapes, numbers, and colors The PyBOT that teaches place value
P a g e 56 | 239 The program and code activities using Lego EV3 Mindstorm where students create robot to teach about place values Bobby the timekeeper video on how students program using Lego EV3 Mindstorm and the testing and output is done simultaneously https://www.youtube.com/watch?v=orPcxBkuXyU Students program and code using Lego EV3 Mindstorm during lab sessions during programming courses
P a g e 57 | 239 The video created in the Green Screen studio with process flow simulations topics I found that there is a need for some topics to be explained with a better graphics and process flows being shown. This is when I took the opportunity to use the green room facility and the recoding facility. The video was then edited with the process flow simulation being included for better viewing together with me giving the explanation. Watch - Example of a semi-pro video explaining about knowledge codification topic. https://www.youtube.com/watch?v=nMRFmgNRtmI&t=453s More videos created for various content When COVID-19 pandemic strikes all of us have no choice but to abruptly change the way we teach. Within short period of time, T&L has changed to fully online during MCO, CMCO and RMCO. Due to the emergency situation, I have created my Youtube channel with various videos explaining about new knowledge aligning to the course learning outcomes (CLOs) for all courses that I taught. I have created more videos to allow students to learn more at these struggling moments. I also create the YouTube channel for my online distance learning (ODL) courses. In 2020, it was the start of me teaching ODL courses. It was the year where many working adults started to take-up ODL courses during the pandemic period. Bite-sizes videos were one of the resources I made for the ODL students.
P a g e 58 | 239 My Youtube Channel stores various topics and learning materials for students to have different options of learning resources. Augmented Reality to Explain Complex Machinery/Hardwares etc. When COVID-19 strikes there is a request to leverage on augmented reality (AR) applications in teaching about complex concepts such as machinery parts and so on. That is when AR application is created for students to learn about complex ideas and processes such as to explain about machinery and parts of an equipment. From then on, AR has been gaining its momentum due to its ability to showcase complexity without the person need to be present at the dedicated location. The Augmented Reality application accessible through smartphone and tablets
P a g e 59 | 239 Example of an AR application in action https://www.youtube.com/watch?v=AABKMt3vd_c Virtual Reality to Engage Learners In the mid of 2022 there seems to be a lot of demand from the training providers to start using Virtual Reality (VR) content for flipped classroom. Their intention is to engage trainees with material before class/workshops/trainings. Trainees are expected to be ready and know the subject matter before attending the upskilling or reskilling classes. As matured and working learners, it is often difficult to make and find time for learning. Therefore, VR is seen as a way to engage matured working learners to grasp the learning at their own time and own phase. So, here I am venturing into the possibilities of VR for students. I have started exploring and doing research on VR for students. The idea has gained its popularity among students. It’s a fun and engaging learning love by many millennial age students. The only problem now that I am thinking is the availability and affordability of the VR headsets. Nevertheless, many students have their own VR headsets now. The university has bought a few sets too. It’s not impossible and it’s the future that I am excited to look upon. Student using VR for learning before class in flipped classroom – Using Oculus Quest2 The content viewed and the activity was on project pitching and presentation to panels We conducted a study to measures students’ experience (UX) and to gauge students’ acceptance towards VR as one of the resources for flipped classroom learning. A convenience sampling approach was used to recruit participants for this study. A total of twelve students volunteered and took part in this study. During the testing phase student participants tested the VR modules. At the end of the session, they had to answer a survey. The survey instrument used is the Questionnaire on User eXperience in Immersive Virtual Environments (QUXiVE) validated by Tcha-Tokey et al. (2016). This questionnaire includes 10 components/subscales:
P a g e 60 | 239 presence, engagement, immersion, flow, skill, emotion, usability, technology adoption, judgment and, finally, experience consequence. Our adopted questionnaire on User eXperience in Immersive Virtual Environments is made up of 85 items on a 10-Likert-type scale and contains three open questions in which the user is asked about their experience in the virtual environment. Part of survey results are as shown in the chart that follows. The chart is showing students’ acceptance towards the VR technology. The chart depicts on participants’ thoughts on the VR technology, interaction devices, and application platform. The highest mean is depicted from items which relates to participants feeling that if they were to use the VR application, it will be clear and not an issue, they have the skills, an easy-to-use technology and VR is found to be an interesting technology. Students like working with the gadgets for learning. 3 items are below the mean value which indicates that: (i) students do not perceive using VR is a bad idea, (ii) students do not have the VR resources (e.g., Oculus headset, gamepad and/or keyboard) and (iii) they are not sure about the compatibility of the devices as they do not own VR sets. 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 Clear Skillful Easy Bad Idea Interesting Working with devices Resources Knowledge Compatible Technology Acceptance Mean Standard Deviation Average mean Average S.D.
P a g e 61 | 239 Tested VR for flipped classroom with TAM5043 students January 2023, it may be the way to go in this near future. – Using Oculus Rift
P a g e 62 | 239 3.2b INNOVATION FOR ASSESSMENTS: DASHBOARD Having dashboard to monitor students before class activities and level of understanding is another innovation that I have introduced. After students completed their SLT and access the materials and resources (e.g., AR and VR applications, robots, videos etc.) I find the importance to review students’ level of understanding before the in-class flipped classroom takes place. Therefore, dashboard is used to check upon students’ level of understanding. This is crucial as to know whether they understand the subject matter well. It is also to identify weak areas that requires guidance and deeper explanation during in-class flipped classroom. Before Class Activities – some examples Formative assessments to check students’ level of understanding and areas of weaknesses is an example of activities given to students before class. Students will not be penalized. Most importantly is for me and my students to close the gaps and get the fundamental theory right! I have utilized a few platforms where I posted the formative assessments questions. Students are expected to provide answers on the platform. The questions are often straight forward and short answers are expected. I have used the elearning platform and a few other tools such as the Socrative. I appreciate tools that could immediately produce a dashboard that presents my students performance. With a glance, I can immediately identify questions that most students could not answer. This is important as the in-class flipped classroom will then focus on these weak areas. Sample of questions for before flipped classroom Dashboard that shows students’ answers for each question The figure below shows the overall answers from students for question 1 and 8. Students answered online and the software that I use will automatically chart students’ progress in a
P a g e 63 | 239 dashboard. We are able to view the average score for each question. The figure below (left) shows that the overall average score for question 1 is 3.8/5. This indicates that this sub-topic requires some guidance during in-class flipped classroom. I will focus on this sub-topic with elaboration and examples to make students understand the concept better. Question 1 is showing students medium understanding about this sub-topic. There will be more explanation about this sub-topic to get the understanding about the fundamental theory right. Question 8 is showing students good understanding about this sub-topic. Guidance will be less for this sub-topic, but students will be given problem solving activities on this topic to strengthen their understanding. Referring to the figure above, question 8 (right) is showing an average score of 4.9/5. This indicates that majority of the students understood the sub-topic and theory well. I will not be explaining about this sub-topic in great depth during the in-class flipped classroom. However, students may ask questions if they wanted to know anything more about this sub-topic. I will also include this part of sub-topic in the in-class activity to strengthen their understanding. With this kind of strategy, in-class flipped classroom lecture becomes more focus and we have time to conduct problem solving activities. Students seem to be more engaged to know more about what they do not know or polish the part they know they are lacking. This strategy also triggers deep learning through problem solving and focused discussion. As mentioned before, having a dashboard to monitor updated students’ performance weekly is crucial. It is to identify low performing students and pushing others to be greater. Dashboards allow us to monitor students’ performance wellbeing and find room for improvements. The weaker ones should be given immediate help. While the outstanding students should be pushed higher to excel. The mediocre group should be given more activities that open up opportunities for them to be in the excellent group.
P a g e 64 | 239 A dashboard that I have created for TEB1083 May 2019 that charted coursework marks, course outcome attainment, students’ self-reflection tool and current grade standing.
P a g e 65 | 239 3.2c INNOVATION: AFTER FLIPPED OUTSIDE CLASSROOM ACTIVITIES – COMMUNITY REACH OUT AND MORE As mentioned before, after flipped is where my teaching and learning happens outside of the classroom. I innovated part of the after flipped class to be learning that applies knowledge learnt to solve real world issues. This include helping out the communities, educational visits, entering competitions and participating in conferences. I choose to expose my students to hands on activities with the industry and community so that they could learn more than what is published in books, papers, and articles. For example, I want them to apply the knowledge learnt to help-out the underserved communities. An example of innovation for after flipped class activity that I have created was community services. Students are required to apply the knowledge from my knowledge management systems class to produce invention or applications that could help the community. It was interesting to see creative knowledge management systems being developed, utilized, and then shared with the community for various purposes as shown in the table that follow. The following events were conducted by various cohort of students of different semesters who are taking the knowledge management (KM) specialization courses: 2019 Events Students’ After Flipped Class Activity: Community Reach-out April Project Initiation with SK Coronation Park and SK Marian Convent 19 th June Learn Math with Robots 28 th June Mural painting July – Aug Coaching SK Marian Convent students for National Robotics Competition 2019 (Perak State Level) – Smart Classroom for Special Needs Kids (ASD) August Industry Recognition - Project Initiation with Sasbadi Education Sdn Bhd 25 th – 27 th Sept Showcasing the project impact with Sasbadi Education Sdn Bhd to public and Ministry of Education at SEIC Conference 28 th Sept Robotic showcase to SK Coronation Park students, teachers and parents and Awareness Talk Students learnt a lot from this innovative after flipped class activity with the theme ‘Community Reach-out’. They apply the knowledge learnt in their KM courses to help the community and improve their quality of life. This includes: ➢ Knowledge sharing and becoming student advisors to SK Marian Convent students and teachers on how KM and robots can be developed and implemented using LEGO EV3 Mindstorms ➢ Judging the robotics competitions participated by the primary school students ➢ Showcasing robotics education at SEIC Conference
P a g e 66 | 239 ➢ Demonstrating and introducing robot technologies to Sekolah Aliran Khas ➢ Giving awareness on robotic technologies to teachers of Sekolah Aliran Khas After flipped class Innovation: Community Reach-out Undergraduate and postgraduate students that I supervised are also exposed to after flipped hands-on activities through community reach-out. Below is the figure that shows different group of students starting in the year 2015 till 2020 conducting technology testing at various Sekolah Pendidikan Khas. The objective is to introduce and test the developed technologies to the underserve community. They have visited a few Sekolah Pendidikan Khas that includes: ➢ 2015 till 2018 - Perak vicinity (i.e., SK Sultan Yussuf, SK Kampung Boyan, SK Sg Siput, interested parents, etc.) ➢ 2020 - Selangor & Kuala Lumpur (i.e., Pusat Jagaan Kanak-Kanak Ceria Murni, Kepong) I am sharing another after flipped class innovation that was conducted by my students in IT Project Management course. As shown in the figure below, my students organized the event and some of them entered the mobile application development competition. It was part of the outside flipped class activity. It is for the students to learn ways to handle IT projects and dealing with various stakeholders.
P a g e 67 | 239 Picture Source: Berita Harian. My students entering and organizing a competition related to knowledge management mobile apps. We received good feedbacks from the industry who attended as panel of judges. The competition received good response and media coverage. Most of the students mentioned that after class is a platform that they enjoyed especially if they got the chance to learn outside of classroom. They honestly mentioned that after class activity taught them things that could not be taught in books and in classroom by lecturers. As one of the students being interviewed mentioned, “I learnt a lot at competitions and exhibitions. Although it is not so much about our syllabus, but it makes me confident about the product that I have developed using the concepts learn in class. I learnt how to present well, dress well, be prepared with technical and non-technical questions, prepare a budget, and defense my work. It teaches me how to think fast and be creative too…. I met a lot of people from industry. That is what I treasured most. I got to shake hand with the CEO of XXX [name of company is removed]!! I won’t be able to get this opportunity if I were to stay in class! And… I listen to many good speakers from different companies like XXX, XXX and XXX [name of company is removed]. They shared their experience on how they become successful, their secrets in business, their ups and downs, which motivates me a lot. It helps me to feel motivated to do well and be like one of them!” The interview demonstrates that students enjoyed the informal learning. After flipped class activities has taught my students to improve their soft skills and other skills that could not easily be shared through teaching and learning in class. Through these after flipped activities that happens outside classroom, students cultivate soft skills that include teamwork, leadership
P a g e 68 | 239 and being ethical aligned with UTP’s 7-attribute of well-rounded graduates’ capabilities as shown in the figure that follows. UTP’s 7-attribute of well-rounded graduates
P a g e 69 | 239 3.2d INNOVATION: BRINGING THE INDUSTRY EXPERTS TO MY FLIPPED CLASSROOM Here is something that I would like to share as part of the flipped learning conducted. As teaching and learning activities becomes interesting and more creative, we have flipped classes with industrial experts. It was so interesting to have classes with experts take overs and learning from them and they did assess students’ presentations and work. It is really an eye opening to hear and learn from businesspeople, as always. Some of the industrial experts that share their knowledge!
P a g e 70 | 239 We had industrial experts to evaluate and give marks for the final presentation of TAM5023 Course in April February 2021
P a g e 71 | 239 3.3 IN-CLASS FLIPPED CLASSROOM ACTIVITIES AND ASSESSMENTS THAT ENCOURAGE HIGHER ORDER THINKING SKILLS As mentioned, during the in-class flipped classroom students will get involve with various types of activities. Some of these activities are summative assessments and are counted as part of the coursework mark. Here I am sharing some examples of assessment activities for a few selected topicsthat encourage Higher Order Thinking skills among the students. It is not merely examination type of questions or test or quizzes that is conducted in the summative assessments. As much as I could I wanted to cultivate the higher order thinking skills through collaborative and active learning types of activities. This will create deep learners and critical thinking skills among them. Below are some examples of the examples of activities that encourage higher order thinking skills among my students. Topic: Knowledge Process and Knowledge Pillars ➢ Students have to design a mobile apps following the knowledge processes and knowledge pillars steps that they have learnt. ➢ It’s a collaborative work activity
P a g e 72 | 239 The role plays presentation style ➢ Peer Evaluation Score Sheet
P a g e 73 | 239 Topic: Knowledge Codification ➢ Students are given business case and required to create a repertory grid for the case organization. ➢ Collaborative work activity ➢ Example of submission of answers
P a g e 74 | 239 Group Presentation & debates using the Gallery Walk technique. Each group has to present their repertory grid to others. They will be challenged by other groups. Topic: Knowledge Mapping ➢ Each group received a business case study. They have to work individually and prepare one the outputs of the steps in knowledge mapping. ➢ After 30minutes the group members will discuss and share their parts. ➢ Each group is required to share the final output of the knowledge mapping process and present a combination of answers as a group. ➢ Peer evaluation as part of the evaluation.
P a g e 75 | 239 ➢ Knowledge mapping is created based on case studies given to each group. The map diagrams are representing different business processes. ➢ Think Aloud Technique Topic: Knowledge Audit ➢ Group Evaluation Scoring Sheet
P a g e 76 | 239 ➢ Evidence of Students’ Answers following the knowledge audit steps ➢ Group Activity
P a g e 77 | 239 Sample of students’ submitted work in various creative forms Topic: Knowledge Assessment ➢ Students debate titles that relates to knowledge assessment topic.
P a g e 78 | 239 3.4a FLIPPED CLASSROOM INNOVATION & ITS IMPROVEMENT ON STUDENTS’ PERFORMANCE I often conduct a review of how my flipped learning classes are performing. Again, it is not about punishing my students but to see how impactful my flipped classroom teaching and learning is. I will do the impact analyses at a few checkpoints: early, mid and at the end of the semester. I also did the comparisons between semesters with different cohort of students. I need to know where I could improve and how to impart knowledge the best way possible. I found that every semester is different although the subject being taught is the same. There are many factors contributing to students’ performance although the same materials, topics, resources, and assessments are given. The factors that make the learning outcomes differ include the type of groups, their background and the instructor who delivers. These are some of the important reasons why I need to change the flipped classroom styles conducted semesterly. For example, when the group of students that I receive in a particular semester are mostly the weak in their academics, that is when in-class flipped classroom has to be more guided. We often conduct active learning activities with one-to-one guidance to the failing ones. Motivation for them to conduct before class activities during SLT has to be different. It often comes with guided instructions, pre-learning activities and quick responses. Contrarily, the flipped classroom styles with the cohort of outstanding students will often be full of flipped activities. During in-class flipped learning, we often conduct group challenge, debates, and collaborative work. Interestingly, high achieving students will work harder when challenged. They stand up and volunteered, argued when they feel their answers should be accepted and hungry for more. Some high-achieving students will help the slower ones. This is when I mix the group of students with different levels of achievements. Each group will have the high achievers, middle group, and the weak ones. This is to create an environment of teamwork where the good helping the weak ones. Nevertheless, this might work in some cohort of students but not all. I have to make sure that the weak ones are not left behind. Therefore, observing and analyzing weak students’ performance has to be done closely throughout each semester.
P a g e 79 | 239 This group has a mix of the high achievers, the mediocre and the weak ones. The group did well at the end of the semester. From then on, I build a group with mix categories of students. Here I am sharing some of the impact analyses that I have conducted. Case Study 1: September 2019 vs January 2020 for Course SDB4313 (EA) The Case study 1 table is showing a comparison of students’ performance in September 2019 and January 2020. The course is taught by me in January 2020 and another lecturer (i.e., Mr X) during September 2019. Flipped classroom is conducted in January 2020 only. The results have shown that students’ Median Grade is B+ in both semesters. However, the overall mean average is better when flipped classroom T&L is conducted during January 2020. The standard deviation in January 2020 is small in January 2020 (σ = 6.07) as compared to in September 2019 (σ = 9.86). This indicates that flipped classroom learning allow the whole class to learn, understand the subject and perform better in summative assessments. The standard deviation shows a small standard deviation that indicates the weak are not falling far behind the good ones. The whole group of class are performing with the Median grade of B+ with a small deviation among everyone in class. This can also be seen from the table where there are no
P a g e 80 | 239 students below the B grade in January 2020 as compared to September 2019 group. The September 2019 class has about 2.44% of failures (grade F). This concludes that the flipped classroom T&L strategy is able to bring the weak ones to perform better and eliminate failures. Case Study 2: May 2019 vs January 2020 for Course SDB4313 (EA) The Case study 2 table is showing a comparison of students’ performance in May 2019 and January 2020. The May 2019 class was conducted by another lecturer (Ms X), and I conducted the class during the January 2020 semester. The SDB4413 course is conducted with online flipped learning (i.e., e-flipped classroom) T&L strategy throughout the January 2020 semester. Table Case Study 2 indicates that the e-flipped classroom strategy has a better median grade which is B+ as compared to the group without flipped classroom learning with median grade of B. The mean average results are also better which is 77.35 when online flipped classroom is conducted as compared to none flipped learning indicating mean average results of 70.62. The standard deviation is also smaller when e-flipped classroom strategy is conducted although the difference is small as compared to May 2019 semester. Most importantly, there are no failing students when online flipped classroom is conducted. This indicates that online flipped strategy allows students to learn better especially the weak ones. On the other note, Table case study 2 also indicates that the good students excel even more through flipped learning strategy. Comparing the 2 semesters cohort of students, the A and A- grade achievers are more when online flipped classroom strategy is conducted. The difference is quite big in percentage as shown in Table Case Study 2. For A achievers, the difference marked about 17.2% between the online flipped classroom and none flipped cohort. While the A- grade achievers has about 11.32% difference between the online flipped classroom semester and the none flipped semester in 2019. This makes a difference of about 29% A- and A high achievers when flipped learning strategy is injected.
P a g e 81 | 239 Case Study 3: September 2019 vs January 2020 for Course TEB3043 (ISP) The Case study 3 table is showing the results of TEB3043 (ISP) course in September 2019 and January 2020 semester taught by Ms X and myself respectively. This case study is showing that the flipped classroom strategy has shown better results in terms of students’ academic performance. The median grade and mean (average) results during the semester where flipped classroom is conducted has shown better output. The flipped classroom results achieved median grade of B+ as compared to B when flipped learning was not conducted. The overall mean (average) of summative assessment results is 76.04 during flipped learning as compared to 70.40 when flipped classroom is not conducted. There is no C grade when flipped classroom is conducted. Overall, the standard deviation is smaller when flipped learning pedagogy is employed. These results indicated that flipped classroom has shown better students’ performance not only focusing on good ones but bringing the weak ones to improve. This can be seen from the smaller standard deviation in January 2020. Besides that, the good students will also get better with flipped learning. This is evident from the percentage of A and Agraders. Previously, when flipped classroom was not conducted in September 2019, there seems to be no A students. When flipped classroom was conducted in January 2020, there are 14.29% of A graders. The number of C+ students are much lesser and there are no C graders when flipped classroom was conducted as compared to when flipped learning is not implemented. This indicates better results and performance when flipped strategy is adopted. From the three case studies, it is evident that flipped classroom strategies does gives a better impact in terms of students’ performance. It makes the good students excel to a greater height and not forgetting the weaker ones in class. The weak students improve, and flipped classroom eliminates failures! Part of these case studies are published in the Asean Journal of Engineering Education. Arshad, N. I., Abdullah, M. N.., Tengku Malim Busu, T. N. Z., M Addi, M., Mokhtar, N. I.., Kalid, K. S.., & Savita K Sugathan. (2021). Flipping the Classroom to Improve Academic Achievement of Information Systems Students with the Support of Learning Management
P a g e 82 | 239 System (LMS). Asean Journal of Engineering Education, 4(2). https://doi.org/10.11113/ajee2020.4n2.6 This document can be downloaded from this link: https://ajee.utm.my/index.php/ajee/article/view/6/4
P a g e 83 | 239 3.4b FLIPPED CLASSROOM IMPACT ON STUDENTS’ QUALITY OF WORK I often analyzed students’ performance to learnt how impactful is the flipped classroom strategy. It is not about punishing the students, but it is to know whether the learning is taking place, or some were falling behind. It is like a check and balance that I have to evaluate and improve or do something and make the learning to be better. Impact Assessment Goal: To investigate the different types of flipped classroom approaches and its impact in terms of students’ cognitive performance in learning the subject being taught. Previously most of us have conducted teacher-centered approach (i.e., giving long lecture, chalk-and-talk) and then I started to conduct the flipped classroom. Therefore, at the beginning of this stage I wanted to know whether my new teaching pedagogy “flipped classroom” is impactful for learning and gaining knowledge in better ways. Study 1: Do students perform better in flipped classroom as compared to traditional – learning (i.e., chalk-and-talk, lecture)? Results of Study 1 Group Group 1 Group 2 Approach Flipped Classroom Traditional Mean (Average) 4.69 2.75
P a g e 84 | 239 The first group of students undergone flipped classroom learning and the second group of students sat for the traditional learning (i.e., chalk and talk and lecture). At the end, a test was given to all students. The mean average results are shown in Table above. Based on the mean results, it shows that the average marks for flipped learning which is 4.69 is higher than the traditional learning mean marks which is 2.75. This indicates that students performed better in learning through flipped classroom approach as compared to traditional approach. Study 2: Do self-learning time (before class), and in-class flipped classroom learning important for a successful flipped classroom teaching and learning to happen? During Study 2, data was collected in two stages. During the first stage, students conducted independent learning during self-learning time (SLT) through the available online materials for the topic before class. They sat for a test at the end of the online session. Data collected from the online test were analysed as shown in Table below. The mean (average) marks for the independent learning test (after students undergone online learning during SLT) is 2.94. In the second stage, students attended flipped-collaborative classroom learning. During this learning, students were given group work and activities related to the topic. A test was given at the end of the class. The mean (average) marks were 3.63 as shown in Table 2. Comparing the two marks of online independent learning test and flipped-collaborative classroom learning test, it is obvious that students’ performance has improved after they get involved in the class activities. This indicates that attending flipped-collaborative classroom learning assists students to understand better about the subject being taught and perform better in flipped classroom learning. Results of Study 2 Topic Knowledge Codification Approach Online Test (After Independent Learning) Class Test (After Flipped-collaborative Classroom Learning) Mean (Average) 2.94 3.63 Correlation 0.916268516
P a g e 85 | 239 Furthermore, a correlation analyses was performed. Referring to Table 2, the online independent learning test marks and flipped-collaborative learning test is denoting a high correlation. The correlation which is 0.9162 poses a strong significant between independent learning and performing during the flipped-collaborative classroom learning. This also indicates that a successful flipped classroom requires both independent learning and in-class flipped learning. In the case of this study, online learning and flipped-collaborative learning is an example of a successful matched of flipped classroom approach. Study 3 - Flipped-collaborative learning vs Flipped-individual learning? For Study 3, students conducted two different types of flipped problem-solving activities in class: flipped-collaborative and flipped individual. Data in Table below shows the mean of both problem-solving activities results. The collaborative problem-solving denotes a slightly higher mean result as compared to the individual problem-solving results. This highlights that students solve problems better when they are given the opportunity to work collaboratively in groups as compared to when being tested individually. Results of Study 3 Topic Social Networking Analyses Approach Flipped individual Flipped collaborative Mean (Average) 3.07 3.75 Correlation 0.837536235 It is worth noting that the mean difference between flipped-individual and flipped-collaborative is rather small. This indicate that both flipped approaches could be implemented by instructors. Depending on the topic and learning content, instructors may choose suitable problem-solving activities (i.e., individual or grouping).
P a g e 86 | 239 Furthermore, correlation analysis was performed on the data collected. As shown in Table 3, the correlation result is showing strong significant between flipped-individual and flippedcollaborative approaches. This is due to the topics which are very related to each other in the case of Study 3. As mentioned before, Part 1 of the topic uses the flipped-collaborative approach while Part 2 uses the flipped-individual approach for the given problem-solving activities. Therefore, based on the result of this study, it is suggested for instructors to mix the flipped approaches when conducting problem-solving activities that covers related topics. This section has presented some of the studies that I have conducted between 2015 to 2018. I learnt a few important things from this studies and analyses that include: • With a proper planning, flipped classroom allow students to learn more effectively as compared to traditional learning, • Both independent learning and in-class flipped classroom learning are important for a successful flipped classroom to occur, • Flipped-collaborative learning can be a better approach than flipped-individual learning for students, during problem-solving activities. These three studies (Study 1, 2 and 3) have been presented at the IDE4TE X Experience Learning, Innovating Teaching Environments (ELITE Conference 2019) Organized and hosted by MMU at the Cyberjaya campus.
P a g e 87 | 239
P a g e 88 | 239 3.4c ANALYSES OF STUDENTS ONLINE BEHAVIOUR AND ITS IMPACT ON PERFORMANCE ACHIEVEMENTS I also conducted impact analyses on students’ self-learning time (SLT) that takes place before class and after class. As mentioned before, flipped classroom is made up of: (i) before class, (i) in-class and (iii) after class learning. Analyzing students’ learning that happens at their own phase via the resources that I have prepared and make it available online is crucial. I want to know the impact of my online resources and materials (e.g., VR and AR applications) that happens during SLT without the presence of instructor physically contributed to their learning. Besides that, this is important as to identify weak and students that are falling far behind the others. I realized that those that come to class unprepared and did not access the materials and resources during SLT will not be able to cope with flipped learning. Therefore, the analyses of how students learn online will be one of the review and observation to know who are doing well and one of the factors to identify potential failures. Early indicators that show signs of students not reviewing, accessing, and getting involve in self-learning online is a trigger for me to watch-out. I will push these group of students, motivates them or dig their problems before they fall even far behind. Here I am sharing one of the case studies I conducted in May 2019 for the flipped classroom class I taught for SBB4373/SCB3363 course. The analyses of students’ online behavior on e-learning platform before and after flipped classroom
P a g e 89 | 239 What do I watch-out and observe online? Reviewing students’ achievements during SLT that happens online means I have to check their participation in the e-learning platform. This is not an issue as the e-learning platform will have dashboard that allow me to easily watch all my registered students’ online behavior and involvement. I could easily review who has watch the online videos, those have accessed the online materials, and anyone fails to complete/attempt the quizzes online and more. I look at the students’ online behaviors that include their completion of pre-assigned work, participation in in-class activities and many more as shown in the figure. I also identify those who fails to complete certain activities or fails to access any shared resources. From this dashboard I can identify passive students online. I will then observe them in physical class during in-class flipped classroom. The figure below illustrates the examples of materials, resources available for a topic. Students who complete accessing most of them often will excel. As mentioned before, I anticipated that students who conducted online pre-class learning (through LMS) prior to face-to-face class will perform well in flipped learning courses. In other words, those who attend face to face classes unprepared, will not be able to fully grasp the benefit of flipped learning. As shown in figure below, this study posits learning through LMS before face-to-face class takes place, will influence students’ achievement in flipped classroom. The conceptual framework guiding my observation Based on these causal interpretations, multiple linear regression is deemed appropriate aimed to estimate the causal relationships between students’ online behavior in LMS (i.e., total views,
P a g e 90 | 239 total activities, total online quiz marks) and their achievement (i.e., students’ summative assessment score). The independent variable (IV) and dependant variable (DV) Variables Code Variables Descriptions IV1 Total Views Adding up the number of individual students’ view into the course in Ulearn IV2 Total Activities Adding up the number of individual students’ online activities occurring in the online course that include posting, views, attempt to quizzes and others IV3 Total Online Quiz Marks Adding up individual students’ online quizzes marks (this are formative assessments and not compulsory). DV Overall Final Marks Adding up individual students’ summative assessment scores: coursework marks, test marks and final exam marks. Data Collection Web-log data were collected from the Ulearn platform (the elearning platform). The IV for this study is shown in Table above and were computed from data embedded in the Ulearn data-log reports. Overall final summative assessment marks were collected as the DV of this study. The final marks were collected from another system that stores students’ grades and examination scores. All data were collected and stored in a database with the permission from the university’s examination unit, department, faculty, and center for excellence in teaching and learning. Results of simple linear regression analysis Independent Variables (IV) Crude Coefficient Standard Error t-statistic (95% CI) p-value IV1 - Total Views 0.0801 0.0406 1.9731 (-0.0029, 0.1631) 0.0581 IV2 - Total Activity 0.0767 0.0337 2.2747 (0.0077, 0.1457) 0.0305 IV3 - Total Online Quiz Marks 0.8944 0.1693 5.2840 (0.5482, 1.2406) <0.0001 Multiple linear regression analysis was conducted to find the answers to whether students’ online behavioral engagement that happens in e-learning platform relates to their achievement in flipped learning. In this study, multiple linear regression analysis is used to infer the causal relationships between the three IVs (i.e., Total Views, Total Activities and Total Online Quiz Marks) and the student’s final achievement (DV) as shown in Table above. The choice of using
P a g e 91 | 239 multiple linear regression analysis is due to the ability of this statistical analysis revealing the relationships between a DV and a collection of IV in a fixed dataset. Results are shown in Table below. Results of Multiple linear Regression Analysis Independent Variables (IV) Adjusted Coefficients Standard Error t-statistic (95% CI) p-value Total Views (IV1) -0.134 0.138 -0.974 (-0.417, 0.149) 0.339 Total Activities (IV2) 0.140 0.118 1.193 (-0.101, 0.382) 0.243 Total Online Quiz Marks (IV3) 0.815 0.183 4.464 (0.441, 1.190) <0.0001 The causal relationship between the three online behavioral engagement in the elearning platform was accounted for 53% of students’ final achievement in the flipped learning course, R2 = 0.53. Based on the results in Table above, it presents that Total Online Quiz Marks had a significant (p-value < 0.05) relationship with students’ final achievement. The confidence interval is between 0.44 and 1.19. However, the total views and total activities conducted in LMS are not contributing towards students’ final achievements. The purpose of this study is to understand whether students’ online behavioral engagement that happens in learning management system relates to their overall achievement in a flipped learning course. This would contribute towards designing an effective flipped learning environment supported by elearning platform and/or any other online technology. Based on the results, three variables are tested and confirmed that one of them is significantly correlated with the overall learning achievement. From these analyses, it is learnt that learning happens when students are given online tasks such as quizzes in the elearning platform. This type of tasks allows students to analyze, think, and find solutions that leads to deep learning. Therefore, online pre-class activities should involve such tasks that include quizzes and problem-solving activities to probe learning to happen before face-to-face in-class takes place. Based on the results, there is a reason to believe that if students put more effort in learning and perform analytical thinking and problems solving, it will drive towards excellence in flipped learning course. In the long run, students are prepared to be matured and deep learners. On the other hand, results have shown that by just viewing course materials and watching pre-recorded videos are not enough for students to excel. Activities that push them to think and solve problems are significant. It is important to note that flipped classroom has three important components: (1) pre-class: moving most lecturing before class, (2) in-class: use of class time for problem solving and collaborative learning activities, and (3) post-class: requiring students to complete and reflect pre and during class activities to fully benefit from flipped learning.
P a g e 92 | 239 Based on the case study conducted, flipped learning best practices are suggested However, this study was designed to study the pre-class factors that may only partly contribute towards the overall achievement in flipped learning. Results have also shown that the regression model with three IVs of students online behavioral engagements has accounted towards 53% of students’ academic achievement in the flipped learning course. Therefore, this study has reasons to posits that the remaining 47% of students’ achievement is attributed from in-class and post-class activities and efforts. Therefore, it is suggested for future research to include pre-class, in-class and post-class variables and investigate its contributions towards students’ overall achievement in flipped learning. This case study is presented and published in the RCEE 2020 Conference organized by the Center of Engineering Education, UTM and Society of Engineering Education Malaysia.
P a g e 93 | 239 SECTION 4:Assessment and Testimonial in Teaching & Supervision
P a g e 94 | 239 4.1a STUDENTS ASSESSMENTS AND REFLECTIONS ON FLIPPED CLASSROOM Students’ assessments and reflections on my flipped classroom strategy is something important to me. I took students’ reflections and assessments as something critical and should be done regularly. It is for me to improve and know how my students are adapting to my teaching styles. I often conduct and distribute surveys to my students to know how I am doing throughout the course. Here is an example of a reflection results and analysis of data I collected from the course I thought in 2018. Details about the course: ➢ Semester and Year: May 2018 ➢ Number of Students: 65 students ➢ Course & Code: KM Tools & Techniques (SDB4423) ➢ Year: Final Year, Final Semester ➢ Background of students: Some have undergone flipped learning with me in previous courses, but some are new to the flipped classroom environment. ➢ Note: This exercise is not anonymous as the aim is to see the pattern of data collected and gain some insights. Students’ Demographics Profiles CGPA Students’ performance level based on their current CGPA • 61% students CGPA are between 3.01 <=and <= 3.50 (Second class upper) • 31% students CGPA are between 2.51 <= and <= 3.00 (Second class lower) • 8% students CGPA are between 3.51 <= and <= 4.00 (First class) I perceive my students as good students. Their CGPA are mostly second class upper, and above which is about 69%. It helps to make my flipped and collaborative learning delivery easier. On top of that, as they are final year students, I could consider them as matured learners. Accepting flipped learning and collaborative learning was not much of an issue. It learnt that having
P a g e 95 | 239 matured learners will often turns flipped classroom and active learning more active and engaging. Gender • 63% of the student respondents are female and 37% are male. Opinion about KMTT Course Now that this course is over, what is your first thought about it? • 56.9% have very positive thoughts about this course (when it’s over). • 33.8% have mostly positive thought about this course (when it’s over). • 9.2% answered neutral.
P a g e 96 | 239 Qualitative opinion about the course upon completion Word Cloud concerning the positive thoughts after the course is over. The word cloud shows students’ positive thoughts after the course is over. They gained knowledge, great understanding, and find it is an interesting, flipped classroom. The also enjoyed the learning, the activities and find the course as useful. They believe that they will be able to apply the knowledge gained into their career. Most of them agreed that the course content is useful for organizations in managing data, information, and knowledge. They find the learning experiences as conducive, interactive, and teaching was delivered efficiently. They also mentioned that the teaching techniques were interesting and not about memorizing facts. It is geared towards problem solving, challenging their curiosity, and predicting what is coming in the future of information and knowledge management. Active learning has been applied in class together with collaborative learning and active learning. The class delivery was always interactive, and it is a two-way communication between students, peers, and the lecturer. Materials provided are up to date and engaging. Overall, students mentioned positive notes about the course, after they have experienced it. What is the strength of this course? Interestingly below are noted from students’ perspective: • 66% pointed that the LECTURER is the STRENGTH of this course • 20% pointed that the DELIVERY STYLE is the STRENGTH of this course • 9% pointed that the ACTIVITIES is the STRENGTH of this course • 5% pointed that the MARKING OF ACTIVITIES is the STRENGTH of this course
P a g e 97 | 239 What is the failure of this course? Failure of this course: • 46% pointed that the VENUE is the FAILURE • 32% pointed that the TIME is the FAILURE • 20% pointed that the MARKING OF ACTIVITIES as the FAILURE • 2% pointed that the ACTIVITIES as the FAILURE Reflections on Failure of this Course: Failure of this course: • 46% pointed that the VENUE is the FAILURE Since the class is BIG, I fail to find a collaborative setting. Collaborative and active learning required a spacious area for various types of activities. Classes which are small & lecture theater type of classroom are found to be not suitable for collaborative and active learning. • 32% pointed that the TIME is the FAILURE The class has to be conducted at 8am as scheduled in the timetable. This is very difficult to control by the lecturer as timetable has been set by a dedicated unit. Changing time is difficult as students come from different cohorts, and they took different courses. • 20% pointed that the MARKING OF ACTIVITIES as the FAILURE This could be coming from the marking of weekly activities which include peer reviews. Some students viewed peer reviews as unfair, and bias. • 2% pointed that the ACTIVITIES as the FAILURE This is something that needs to be further investigated. However, since the percentage is small, I believe it may be coming from students who have been missing a lot of flipped classrooms, therefore missing the flipped activities.
P a g e 98 | 239 Room for improvements for this course Word Cloud concerning the room for improvements suggested by students. Referring to the word cloud, when asked about room for improvements for the course, students mostly mentioned ‘nothing’ and ‘none’ which is a good remark. However, it is good to know that there are students who find the flipped classroom materials are complicated to comprehend. Clear and clarity of materials is one of the keys to a successful implementation of a flipped classroom. Some of the students also mentioned that the flipped classroom materials are disorganized and confusing. Many pointed out about the issue of inconducive venue. Most students are uncomfortable with the room setup which are not suitable for collaborative and teamwork. It is often difficult to discuss and move around. Some does not like class with big number of students and prefer to be class with smaller number for flipped classroom. Majority hate morning classes and that is something unavoidable as the schedule has been prepared by a centralized unit in UTP. Some students pointed out about the class delivery practices that could be improved, this includes the importance of giving assistance when needed. This is true when the class is big, and time is limited, assistance given to students are often lacking. They complained that session sometimes becomes very compact and less focused. Some students mentioned about not being happy when being paired with strangers in paired activities. They do not like the idea of having to discuss with someone they are not familiar with. Some are also against the idea of peer reviews where friends are allowed to give marks to their submitted work. They find it unfair and bias. On the other note, some students find the questions given during in class flipped classroom are tricky and challenging. They find that those questions seem to be harder than what has been taught and given as pre-class materials. This is true, as the objective of giving students questions and activities during flipped classroom is to challenge them with the higher bloom taxonomy that include: analyzing, creating, evaluating, and assessing. While the materials given before flipped classroom is forstudents to understand fundamental theories and concepts. The challenge is to let them think and apply those concepts.
P a g e 99 | 239 Will you encourage your juniors to take this course and why? • 65% said YES, that they will introduce to others because it is INTERSTING AND USEFUL • 35% said YES, they will introduce to others because THEY LIKE THE FLIPPED CLASSROOM STYLE Highlight of this course Word Cloud about highlight of the course The word cloud presents students’ opinion about the highlights of the course. Majority of the students enjoyed the activities conducted during in-class flipped classroom and the lecturer style of delivering. This shows that students enjoyed the active learning and flipped learning. This indicates that they accept the flipped classroom delivery. Another important note is the fact that students like the interaction that happens during in-class flipped classroom. Rather than having one-way communication (traditional lecture), they enjoyed having the ability to discuss, debate, argue, present, and talk to friends, lecturers, and other groups. Many of them also highlighted that they like the exercises given in class. This includes pop quizzes, case studies and problem-solving activities, presentations and a few more. They find
P a g e 100 | 239 that these activities pushed them to be proactive (prepared before coming to class), explaining facts to friends when needed and stimulating their thinking. Overall, they like the activities, and the atmosphere that was created during each flipped class. Some find it fun, friendly environment yet very informative. What do you think of the activities in-class? • 86% find the collaborative learning and active learning in class as USEFUL • 14% find the collaborative learning and active learning in class as HELPFUL IN SOMEWAY Majority of the students seems to like the flipped learning, where it is not about coming to class and attend a lecture. It is about doing activities in class to get better understanding of what they have read before coming to class (pre-reading materials). What do you think of the delivery of the in-class flipped classroom? • 79% find the flipped-classroom delivery in class as INTERESTING. • 20% find the delivery in class was DELIVERED WELL • 1% find the delivery in class was OK.