E-proceeding PEERS'22

E-PROCEEDING

1st POSTGRADUATE RESEARCH SYMPOSIUM
PEERS 2022
Vol 1 No 1

Copyright © 2022, PEERS

All rights reserved. No part of this publication may be reproduced or
transmitted in any form by means, electronic or mechanical, including
photocopy, or any information storage and retrieval system, without
permission from the publisher.

Perpustakaan Negara Malaysia Cataloguing-in-Publication Data

Postgraduate Research Symposium PEERS’ 22 (1st : 2022 : Shah Alam,

Selangor)

Postgraduate Research Symposium PEERS’ 22 : 1st

POSTGRADUATE RESEARCH SYMPOSIUM PEERS’ 2022, UiTM

Shah Alam, Malaysia, March 5th 2022 /Azliza Mohd Ali, Shuzlina

Abdul Rahman Editors.

Mode of access: Internet

eISBN 978-629-97265-0-0

1. Computer science--Congresses.

2. Computer security--Congresses.

3. Machine learning--Congresses.

4. Government publications--Malaysia.

5. Electronic books.

I. Azliza Mohd. Ali, 1979-. II. Shuzlina Abdul Rahman, 1972-.

II. Title.

004

Publisher:
Faculty of Computer and Mathematical Sciences
Universiti Teknologi MARA
Shah Alam, Selangor, Malaysia

Editors

Azliza Mohd Ali
Shuzlina Abdul Rahman

Co-Editors

Ariza Nordin
Nor Hapiza Mohd Ariffin
Nurhidayah A. Kadir
Nurulhuda Noordin
Nurzeatul Hamimah Abdul Hamid
Siti Nur Kamaliah Kamaruddin
Sofianita Mutalib

Reviewers

Afiza Ismail
Ariza Nordin
Azlin Ahmad
Azliza Mohd Ali
Fariza Hanis Abdul Razak
Mohd Zaki Zakaria
Nor Hapiza Mohd Ariffin
Nor Shahniza Kamal Bashah
Nur’Aina Daud
Nurulhuda Noordin
Shakirah Hashim
Shamimi A. Halim
Shuzlina Abdul Rahman
Siti Arfah Ahmad
Sofianita Mutalib
Syaripah Ruzaini Syed Aris
Yuzi Mahmud

ii

Preface

PEERS 2022 is the 1st Faculty of Computer and Mathematical
Sciences Postgraduate Research Symposium. It is a day symposium
that provides opportunities for postgraduate researchers in the Faculty
of Computer and Mathematical Sciences to share their research,
giving and obtaining constructive feedback from industry. This
symposium will be one of FSKM's visibility initiatives to the industry.
Postgraduate students in Master by Coursework and Master & PhD
by Research from Computing Sciences programmes are invited to
present their findings in this one-day symposium. The main
objectives of PEERS 2022 are to provide opportunities for students to
present their project findings and obtain feedback from the industry
and alumni. Students will also be exposed to industry demands
through the knowledge sharing session from the invited speaker. As a
result, our students' talents will be known in the industry with good
knowledge and soft skills capabilities.

iii

Table of contents

1 Sentiment Analysis of Customer Service Review 1
Towards Banking Sector

Wan Irma Liyana Zulkefli, Mohd Zaki Zakaria

2 Exploring Innovative Diagnostic Tools in the Diagnosis of 17
Malaria in Nigeria

Akinwale Charles Ojo, Nurzeatul Hamimah Abdul Hamid

3 Detecting Android Trojan using Dynamic Analysis and 24
Machine Learning

Nur Atiqah Binti Mohamad Ikhsan, Kamarul Ariffin Bin Abdul
Basit

4 The Evaluation of User Experience (UX) about an 35
Interactive Animated Comic of Paedophile Awareness for

Children (i-ComPedo)

Nur Anis Idayu Binti Mohd Nor, Fariza Hanis Abdul Razak

5 The Factors that Influence the Implementation of 46

Automation Technology Application Towards Warehouse
Productivity for Logistic Industry

Nur Ika Natasha Roslan, Fauziah Ahmad, Nor Shahida
Mohamad Yusop, Norjansalika Janom

6 Exploring Latent Dirichlet Allocation for Topic Modelling 68
in Facebook for Mental Health on COVID-19 Pandemic

Nurzulaikha Khalid, Shuzlina Abdul-Rahman, Wahyu Wibowo

7 Information Requirement for Plants in Augmented Reality 78
Application using Participatory Design

Nabihah Yusof, Rozianawaty Osman

iv

8 Predictive Analytics of Alumni Employability 86

Tuan Nur Najwa Tuan Mohammad, Sofianita Mutalib, Ariff Md
Ab Malik

9 Perceptual Aliasing Analysis Utilizing Bag of Visual Words 97
for Optimal Loop Closure Detection.

Talha Takleh Omar Takleh, Shuzlina Abdul-Rahman, Sofianita
Mutalib, Siti Sakira Kamaruddin

10 Enhancing MAAD in Streaming Data 107

Muhammad Yunus Bin Iqbal Basheer, Azliza Mohd Ali,
Nurzeatul Hamimah Abdul Hamid, Muhammad Azizi Mohd

Ariffin, Rozianawaty Osman, Sharifalillah Nordin

11 Malware Detection for Window Registry using Machine 116
Learning

Nurwahida Binti Misran, Kamarul Ariffin Bin Abdul Basit

12 Company Bankruptcy Prediction Using Supervised 125
Learning Techniques

Amin Iman Muhd Jelaini, Azlin Ahmad

13 Review on the Evaluation of Digital Storytelling Evaluation 136
Model (DSEM) as a Learning Tool

Hayati Abd Rahman, Nor Izzati Nor Zamrai

v

Sentiment Analysis of Customer Service Review
Towards Banking Sector

Wan Irma Liyana Zulkefli1, Mohd Zaki Zakaria2

Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA,
40450 Shah Alam, Selangor, Malaysia

[email protected], [email protected]

Abstract: The social media has become part of a daily routine of people
around the world. The social media platforms are generating a huge amount
of data on the internet with an abundance of opinions from the users. In
understanding the needs of internet users, sentiment analysis can be used to
benefit both the banking sector and its consumers. Customer service is a
bank department that serves to ease client problems with banking,
particularly by assisting them online or through calls. One of the key
successes of a bank is its customers’ satisfaction. Customer satisfaction and
loyalty could not be separated in achieving the service quality of a bank.
Yet, the difficulty for customers to reach out to customer service will give
not only a bad experience but also a bad reputation to the bank. This research
aimed to identify the factors that could improve the banking sector’s
customer service by comparing several machine learning techniques to find
the most suitable model. The sentiment of the data obtained was classified
as negative, positive, and neutral based on the expression and emotion of the
customer opinions. This study used 4076 data from Twitter. The data were
scraped within Malaysia’s local banks. The sentiment analysis process was
performed using Artificial Neural Network (ANN), Random Forest, Naïve
Bayes, and Support Vector Machine RBF Kernel (SVM RBF). ANN was
found to give the best level of accuracy results (90.54%) compared to other
techniques. The implementation of this study will benefit various industries,
particularly the banking sector, by concentrating on the customers’ opinions
and feedback. Such a discovery could facilitate the achievement of a better
reputation, hence profitability for the banks.

Keywords: Artificial Neural Network, Banking, Customer service,
Sentiment Analysis, Twitter

1

1. Introduction

Nowadays, people from all over the world are using social media
not just for sharing information and suggestions but also for expressing
emotions. Social media platforms such as Twitter, Facebook, and Instagram
are allowing people around the world to interact with each other. Large
companies are taking this opportunity by having an account to reach their
customers and discover their product feedback (El Rahman et al., 2019).
Sentiment analysis, also known as opinion mining, is used to gauge the
feedback and opinions shared by customers in a review form and comments
on social media. This analysis is used for predicting users’ behaviour,
assessing the impact of an event, and exploring the patterns of public
opinion on assorted issues. The comments and feedback received are helpful
in improving the service of a company and in comparing it with competing
brands (Shakeel et al., 2020). Social networks have now become an
excellent tool for customer service, particularly banking service, as the
open-sourced data could be gathered with the help of software to scrape the
data (i.e., Python).

In recent years, there has been an extensive growth of social media
around the world that has been affecting various business sectors, including
the banking sector. It is now crucial for any brand in any business sector to
take social media seriously and have an online presence to reach its
consumers (Khan & Urolagin, 2018). When it comes to banking issues and
money matters, people tend to be quite passionate or frantic; thus, they will
be sharing their suggestions or expression on social media, and Twitter is no
exception. Malaysians and Twitter are evolving their conversations about
the perks and difficulties of being a bank customer. Banking customer
services have been affected by the global pandemic COVID-19, which
worsened the sentiment (Aguilar & Torres, 2021). Customers often have
problems reaching customer service through the classic channel of calling
the customer service, which reflects a slow service by the company (Helms
& Mayo, 2008). The hospitality of the frontline employees is at the heart of
every organization; it represents the company’s first impression, which will
either increase or decrease the level of customer expectations (Zhu et al.,
2019).

A bad customer experience could lead to customers switching to
another bank. Such a departure needs to be avoided as loyal customers are
more profitable for a bank, particularly if the clients are corporate clients.

2

Such drawbacks not only affect clients but also put the bank at risk of losing
its reputation and profit. Opinions and attitudes do matter for a bank to be
successful. The banking sector is linked to news that will affect the price of
their stocks, which could even lead to bankruptcy (Nopp & Hanbury, 2015).

2. Literature Review

This paper focuses on the reviews of banking sectors together with
the development and technique of using machine learning and the
fundamentals of models used in previous studies. We will also focus on
Twitter-related studies on the customer services provided in the Malaysian
banking sector.

Reviews of people currently play a notable role in an organization.
Reviews that are published through the web are publicly viewed and
therefore could lead to the rise of a company or risk its reputation loss. In
Parveen & Pandey's (2017) study, their sentiment analysis of reviews on
particular events and opinions focused on microblogging websites such as
Twitter. The data of feedback and opinion posted by people through Twitter
could help determine future interest and satisfaction with a product or
service.

Social media has also become an incomparable channel for people
to interact on the World Wide Web, particularly by providing an
organization with their customers’ opinions and feedback. Social media has
also given its users the ability to influence other users through comments
that are posted online; their perspectives will be considered by other
customers who are considering the same product or service (Joshi & Simon,
2018).

Social media of finance has now brought people, companies, and
organizations together to share ideas, comments, and even complaints for
improvements. This media is now providing a huge amount of unstructured
data that could be analysed and processed into decision making for problem
solving Sentiment analysis has been frequently used in assessing the social
media user’s feelings and emotion towards a particular subject matter
(Sohangir et al., 2018).

In Malaysia, a few banks allow customer services to be reported via
Twitter to ease the waiting time of traditional phone calls. The banks include

3

Maybank, RHB, CIMB, Alliance Bank, etc. These banks have a Twitter
account and therefore, will benefit from this study to discover their
customers’ satisfaction, particularly by analysing the reviews. Banks that do
not have a Twitter account could also benefit from these studies by
understanding customers’ reviews on Twitter, specifically those that have
been linked to their banks with hashtags. Twitter has important information
for scholarly works as it offers data that could be gathered from social media
and consist of meaningful knowledge (Hudaefi, 2022).

2.1 Model Comparison

In this study, multiple machine learning algorithms were applied for
the purpose of training. The aim is to figure out, through a sentiment
analysis, the views of Malaysians on the country’s banking customer service
and to find any factor requiring improvement. Four machine learning
algorithms were used to compare and evaluate the most suitable algorithm
for a customer service sentiment analysis. The four algorithms are Naïve
Bayes, Random Forest, Support Vector Machine, and Artificial Neural
Network.

2.1.1 Random Forest

Random Forest is known as the easy and flexible to use algorithm
that gives good results. This model is also a supervised learning algorithm.
The “Forest” is built from the ensemble machine learning technique of
Decision Tree that was then called “Random Forest”. This ensemble is the
combination of learning models to increase the overall result by using
“bagging” method. Random Forest has been showing convincible results
surpass the performance of SVM, Naïve Bayes, and other machine learning
technique for classification function (Fauzi, 2018)

2.1.2 Support Vector Machine

SVM is one of the supervised machine learning algorithms that
could be used for classification or regression. According to Zainuddin &
Selamat (2014), previous studies have widely used the SVM to test the
various domains of customer reviews and opinion datasets such as movies,
hotels, and music. Such also indicates that the SVM could achieve higher

4

accuracies and works well for the classification of text analysis. The
advantage of the SVM is its robustness, which has been shown to give

promising results from previous sentiment analyses. There are four types of

SVM: Linear, Polynomial, Radial Basis Function (RBF), and sigmoid. In
this study, RBF was selected as it involves a nonlinear problem.

2.1.3 Artificial Neural Network

The neural network, also known as the Artificial Neural Network
(ANN), is a flexible system that simulates the human brain by its
interconnected nodes that have a layering structure. ANN blends several
processing layers that are hidden and an output layer. In this technique, the
idea of getting the output is the node, which can only be a value of 1
(activate) or 0 (do not activate). The hidden layers in ANN will then be
computed independently, using the common ANN method. ANN is
proposed to be used in this study for its increased popularity (Abid et al.,
2019).

2.1.4 Naïve Bayes

Naïve Bayes is a supervised learning algorithm based on the Bayes
Theorem, which is often used in solving classification data. Naïve Bayes
assumes the independence among the predictors of the classification
technique. The method is notably useful for large datasets. Naïve Bayes is
also known to outperform sophisticated classification methods. Naïve Bayes
will produce classification based on the Bayes formula. The output is the
likelihood of (1=activate) and (0=do not activate) and will be calculated
based on the theorem of Bayes. According to Neethu & Rajasree (2013),
Naïve Bayes works well with highly dependent features for certain
problems.

( | ) = ( | ) ( )
( )

3. Methodology

This paper aims to determine which factor that can improve the
banking service by applying suitable machine learning techniques for

5

sentiment analysis. The determined factors were then based upon to
compare and evaluate the most suitable algorithm using machine learning
techniques. The data mining techniques used to produce the models were
Naïve Bayes, Random Forest, Support Vector Machine and Artificial Neural
Network. These techniques were chosen based on past research. Most of the
previous studies used Naïve Bayes, Support Vector Machine (RBF), and
Artificial Neural Network. The methodology involved five steps, consisting
of Data Collection, Data Cleaning, Sentiment Detection, Data Visualization,
Model Comparison and Evaluation. Fig 1 illustrates the research method
adopted for this study.

Fig. 1 Business Strategies and Frameworks

3.1 Data Collection
This study used data that were scraped from Twitter with related

keywords. The data collected were tweets on Malaysian Bank customer
service from four popular banks in Malaysia, which are Maybank, CIMB,
RHB and Public Bank. The data scraped consisted of four attributes. Out of
the four attributes, only related attributes were used, and additional attributes
were added for analysis. Daily tweets were scraped in real-time from Twitter
and consisted of 4076 data, as shown in Table 1. The data consisted of
variables “Datetime”, “Tweet Id”, “Text”, and “Username”. A sample of the
original tweets is provided in Table 2.

6

Table 1. Number of Tweets Scraped Based on Banks

Maybank CIMB Customer RHB Customer Public
Customer Service Service
Service Bank
Customer
Service

2000 tweets 1001 tweets 506 tweets 569
tweets

Table 1 shows that the tweet counts for Maybank gives the highest
result considering the bank to be the oldest and biggest bank in Malaysia
with the most branches all over the country. Maybank has also gained
popularity among Malaysians. However, everyone has his or her own
preference when it comes to choosing a reliable bank to safe keep his/her
assets. The keyword Maybank gives the highest number of counts compared
to other banks.

Table 2. Sample of Original Tweets

Datetime Tweet Id Text Username
kenottahan
public bank
gjoyyy
2021-03-02 1.37E+18 has the worst haffdean
07:26:52+00:00 customer

service???

💀

Teruk betul

2019-12-20 1.21E+18 customer
01:22:39+00:00 service hotline
Public bank ni

😅

Public bank

2016-01-01 6.83E+17 customer
06:12:21+00:00 service

soooooo slow

3.2 Data Cleaning

In this stage, the data collected were cleaned by removing unrelated
symbols, advertisements, and languages other than English and Bahasa
Malaysia. The data collected were sentences written in English, Bahasa
Malaysia, Indonesia, and others. Sentences with languages other than

7

English and Bahasa Malaysia were removed, along with special characters,
emojis, punctuation, and hyperlinks. The dataset was then filtered to include
only Bahasa Malaysia and English tweets. The data were then finalized to
3769 tweets that are related to the study. The dataset was then partitioned
into a 70:30 ratio (70% of training and 30% of testing). This step yielded a
total of 2692 data for the training dataset and 1084 data for the testing
dataset, as shown in Fig 2.

Count of Data

3000 2692

2500

2000

1500 1084
1000

500

0 30 %
70% Testing

Training

Fig. 2 Split Data Testing and Training

3.3 Sentiment Detection

This stage is where the data were categorized to define the
expression or emotion of the sentence in sentiment analysis. This paper
defines the sentiment of a tweet based on the classification of the sentences.
In this study, sentiment categorization was performed manually. The
sentiment was grouped into “negative,” “positive,” and “neutral,” according
to the opinion of customers on the banking services. It is vital for an
organization to notice and learn about their customers’ feedback, whether
positive or negative. Knowing these feedbacks would allow the companies
to gauge their customers’ level of satisfaction.

8

Counts of sentiments

Positive, 587

Negative, 1716

Neutral, 1466

Negative Neutral Positive

Fig. 3 Count of Sentiments

Fig 3 shows that the sentiment counts after the data cleaning process
were from 4076 to 3769. The data were reduced after the removal of
unrelated tweets, such as advertisements and tweets not related to banking
customer services. A total of 1716 counts were negative statements; 1466
counts were neutral statements; and 587 counts were positive statements.
The trend identified from the initial data cleaning was that that the negative
feedback outnumbered the positive feedback. The trend that can be seen is
that most customers are using social media to channel their complaints more
than complements. Table 3 shows that the counts of sentiments for negative
sentiment (448), neutral statements (305) and positive statement (377)
account for 30% of the testing data.

Table 3. Count of Sentiment for Testing Data

Sentiment Counts

Negative 402

Neutral 305

Positive 377

9

3.4 Data Visualization
The purpose of sentiment analysis is to transform unstructured data

into meaningful information which could help us make better decisions with
the insight. After the data were categorised, the results were visualized using
Word Cloud. The Word Cloud shows the most frequent words used on
Twitter. In this study, few Word Clouds were generated from the negative
and positive sentiments. Word Cloud is the visual representation of text data.
The bigger the word in the cloud, the higher the frequency of that word being
mentioned. The smaller the word, the lower the frequency of the word being
used. Other than that, one could also see the importance of the words in
Word Cloud to be in different colours.

Fig. 4 Main Topic of The Dataset
Fig 4 shows the word cloud for the main topic of the dataset. The
finding indicates a mixture of Bahasa Malaysia and English for the tweet
reviews. A word cloud is a collection or cluster of words depicted in
different sizes. The bigger and bolder the word appears, the more often it is
selected or used by the Twitter user. For this study, it is the reviews by
customers. The six top words used in the tweet status were Maybank, CIMB,
call, service, customer, and RHB. Three out of five words from the word
cloud indicate that the sentiment was done on Malaysian bank, as
represented by the word maybank (Maybank), cimb (CIMB), and rhb
(RHB). The words call, customer, and service indicate that the sentiment

10

was about customer service. The combination of the two words service and
customer shows that the main topic of the data discusses customer service
satisfaction towards banks in Malaysia.

Fig. 5 Negative Word Cloud
Fig 5 shows the word cloud for negative words in Bahasa Malaysia
and English. The most frequently mentioned words are busy, worst, teruk,
bad, bodoh, and susah. These are the words that indicate dissatisfaction with
a banking customer service. The word busy shows that the banking customer
service is always too busy to cater for customers’ request, and the word
angkat indicates that the customer service phone line is hardly answered,
which then became time-consuming, as seen in the word cloud “time”.
Observing the word cloud and not just on the words with big fonts allowed
identification of small words that could harm a bank’s reputation, for
example, tinggalkan and transfer. The customers could switch to another
bank, and they may transfer their assets to a more reliable banking service.

Fig. 6 Positive Word Cloud

11

Positive feedback was also noted from the Word Cloud (Fig 6). The
positive words were good, friendly, better, best, terbaik, and bagus. These
words indicate customers’ satisfaction and appreciation for the service given
by the bank. Also noted were smaller words like happy, fast, laju, love, and
helpful. These words could help the banking service in improving their
service by attending to their customers faster. Being helpful will result in
happy customers who will love their experience in using the service.

4. Result and Analysis

A total of 3769 sentences were split with a ratio of 70:30
(70%training and 30% Testing). The accuracy results indicate that the ANN
has the highest result (90.54%). Random Forest also recorded a good result
(88.15%) followed by Naïve Bayes (72.03%) and SVM RBF (23.16%)
accuracy for training result as shown in Table 4.

Table 4. Overall Results Recall
AUC Accuracy F1 Precision
90.54%
Neural 87.16% 90.54% 89.76% 89.87%
Network 88.15%
Random 86.35% 88.15% 86.63% 86.87% 72.03%
Forest 81.30% 72.03% 75.68% 84.69% 23.16%
67.31% 23.16% 19.84% 78.44%
NBayes

SVM

As shown in Table 4, the ANN recorded the highest percentage of
Area Under the Curve (AUC) (87.16%) followed by Random Forest
(86.35%), Naïve Bayes (81.3%), and SVM RBF (67.31%). The ANN also
recorded the highest accuracy, precision, recall, and F1 compared to other
models with 90.54%, 89.87%, 90.54% and 89.76%, respectively. The
accuracy rate of the ANN model (90.54%) falls under a very good range rate
(between 87.5%–100%), as well as the Random Forest’s rate. Naïve Bayes
is also considered good (72.03%), while the SVM RBF showed a poor result
(23.16%).

12

Artificial Neural Network Random Forest

Naïve Bayes SVM RBF

Fig. 7 ROC Curve of Each Models

Fig. 7 shows that the Receiver Operating Characteristic (ROC) for
all four models (ANN, Random Forest, Naïve Bayes, and RBF). The ROC
curves divert from the equator. The curve also indicates that most of the
models are closer to the upper left corner, and thus are good models. The
best model for the ROC chart is ANN, whose curve has a gradient intercept
at 0.8 (close to 1.0). Table 5 shows the Area Under the Curve (AUC) for all
four models.

13

Table 5. Area Under Curve (ROC) of overall result

Model AUC

Artificial Neural Network 87.16%

Random Forest 86.35%
Naïve Bayes 81.30%
SVM RBF 67.31%

An AUC between 0.9 and 1.0 is considered excellent. Results 0.8
and 0.9 are considered good; 0.7–0.8 are considered fair; and 0.6–0.7 are
considered poor. Table 5 shows the results for the ANN (87.16%), Random
Forest (86.35%), Naïve Bayes (81.30%), and SVM RBF (67.31%). The
highest score for AUC testing compared to the four models is ANN, which
falls in the good range (between 80–90%). However, SVM RBF falls under
the poor category (67.3%), which is between the range of 60–70%.

5. Conclusion and Future Work

Data visualizations in this study were carried out through Word
Cloud to represent the most frequently used words in the Twitter reviews.
The findings indicate that most of the customer reviews are negative. From
the negative Word Cloud, the frequent words used are bad, busy, problem,
contact, teruk, and angkat. The word angkat was particularly mentioned by
a customer to refer to a call that was not answered by a customer care
hotline. However, some customers offered compliments as they were
satisfied with the service. The positive word cloud records frequently used
words such as good, best, settle, and bagus. The findings, therefore, suggest
that one factor that can help improve customer service in the banking sector
is the efficiency of the customer service department, which is first in line to
represent the bank (whether physically or on the line). Good response from
the customer department will lead to the banks making profits and sustaining
loyal customers. Such a reputation could bring more customers from “word
of mouth” and reduce the risk of reputational loss in an organization. Finally,
a machine learning model comparison was done between SVM RBF, Naïve
Bayes, Artificial Neural Network (ANN), and Random Forest. The result
showed that the ANN had the highest accuracy (90.54%) sensitivity
(90.54%), precision (89.87%), and Area Under the Curve (AUC) (87.16%).

14

Therefore, the ANN was deemed the most suitable machine learning
technique for sentiment analysis of customer reviews in the banking sector
and the chosen algorithm for machine learning techniques.

In conclusion, a few techniques in machine learning can be used to
identify sentiments from text datasets. These techniques can be applied in a
Twitter sentiment analysis of customer reviews in the banking sector. This
paper proposed the following four methods for comparison and evaluation:
SVM RBF, Naïve Bayes, Random Forest and Artificial Neural Network.
The four models were used to classify the sentiments with the features
included in the specified dataset. It is, however, difficult to handle tweets
with misspellings and slang used by the tweet users. Data pre-processing
needs to be done to remove all the unnecessary symbols and words. The
proposed classifier chosen for this study shows close accuracy between
ANN and Random Forest. Therefore, the ANN could perform the best in
identifying the sentiment of customer service reviews in the banking sector
domain. The work can be extended in the future by making a hybrid of few
machine-learning techniques, such as Naïve Bayes and Random Forest.

6. References

Abid, F., Alam, M., Yasir, M., & Li, C. (2019). Sentiment analysis through
recurrent variants latterly on convolutional neural network of Twitter.
Future Generation Computer Systems, 95, 292–308.
https://doi.org/10.1016/j.future.2018.12.018

Aguilar, A., & Torres, D. J. (2021). The Impact of COVID-19 on Analysts’
Sentiment About the Banking Sector. SSRN Electronic Journal.
https://doi.org/10.2139/ssrn.3880420

El Rahman, S. A., Alotaibi, F. A., & Alshehri, W. A. (2019). Sentiment
Analysis of Twitter Data. 2019 International Conference on Computer
and Information Sciences, ICCIS 2019.
https://doi.org/10.1109/ICCISci.2019.8716464

Fauzi, M. A. (2018). Random forest approach fo sentiment analysis in
Indonesian language. Indonesian Journal of Electrical Engineering
and Computer Science, 12(1), 46–50.
https://doi.org/10.11591/ijeecs.v12.i1.pp46-50

Helms, M. M., & Mayo, D. T. (2008). Assessing poor quality service:
Perceptions of customer service representatives. Managing Service
Quality, 18(6), 610–622. https://doi.org/10.1108/09604520810920095

Hudaefi, F. A. (2022). Zakat In The Virtual World: Sentiment Analysis Of

15

Netizens’ Zakat In The Virtual World : Sentiment Analysis Of
Netizens ’ Opinion On Twitter. (September 2021).
Joshi, P. P. A., & Simon, G. (2018). Generation of Brand / Product
Reputation using Twitter Data. 2018 International Conference on
Information, Communication, Engineering and Technology (ICICET),
1–4.
Khan, R., & Urolagin, S. (2018). Airline sentiment visualization, consumer
loyalty measurement and prediction using twitter data. International
Journal of Advanced Computer Science and Applications, 9(6), 380–
388. https://doi.org/10.14569/IJACSA.2018.090652
Neethu, M. S., & Rajasree, R. (2013). Sentiment analysis in twitter using
machine learning techniques. 2013 4th International Conference on
Computing, Communications and Networking Technologies, ICCCNT
2013. https://doi.org/10.1109/ICCCNT.2013.6726818
Nopp, C., & Hanbury, A. (2015). Detecting risks in the banking system by
sentiment analysis. Conference Proceedings - EMNLP 2015:
Conference on Empirical Methods in Natural Language Processing,
(September), 591–600. https://doi.org/10.18653/v1/d15-1071
Parveen, H., & Pandey, S. (2017). Sentiment analysis on Twitter Data-set
using Naive Bayes algorithm. Proceedings of the 2016 2nd
International Conference on Applied and Theoretical Computing and
Communication Technology, ICATccT 2016, 416–419.
https://doi.org/10.1109/ICATCCT.2016.7912034
Shakeel, M., Barsaiyan, S., & Sijoria, C. (2020). Twitter as a customer
service management platform: A study on Indian banks. Journal of
Content, Community and Communication, 10(6), 84–104.
https://doi.org/10.31620/JCCC.06.20/07
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018). Big
Data: Deep Learning for financial sentiment analysis. Journal of Big
Data, 5(1). https://doi.org/10.1186/s40537-017-0111-6
Zainuddin, N., & Selamat, A. (2014). Sentiment analysis using Support
Vector Machine. I4CT 2014 - 1st International Conference on
Computer, Communications, and Control Technology, Proceedings,
(I4ct), 333–337. https://doi.org/10.1109/I4CT.2014.6914200
Zhu, J. N. Y., Lam, L. W., & Lai, J. Y. M. (2019). Returning good for evil:
A study of customer incivility and extra-role customer service.
International Journal of Hospitality Management, 81(April 2018), 65–
72. https://doi.org/10.1016/j.ijhm.2019.03.004

16

Exploring Innovative Diagnostic Tools in the
Diagnosis of Malaria in Nigeria

Akinwale Charles Ojo1, Nurzeatul Hamimah Abdul Hamid2

Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA,
40450 Shah Alam, Selangor, Malaysia

[email protected], [email protected]

Abstract. Nigeria has consistently ranked highest in the global malaria
cases and fatalities over the past two decades. The management of malaria
in this country has been made worse by less-than-ideal malaria control
practices; outmigration of healthcare professionals because of poor
remuneration in the country; and a shortage of health professionals in
government health institutions. All these have led to lack of quality health
care, belated diagnoses, and an increase in misdiagnosis, leading to fatal
complications. This paper aims to explore the different available innovative
diagnostic tools that can be readily deployed for the diagnosis of malaria in
the country through a systematic literature review of expert systems
developed for the diagnosis of malaria in the country. A critical review of
the various expert systems developed for use in the health institutions and
their availability for potential individual use was also carried out. These
expert systems utilize different artificial intelligence techniques and
algorithms, with very high accuracy in the diagnosis of malaria when
compared with the expert diagnosis of tropical diseases specialists in the
country. This further supports the efficacy of the expert systems and the
need to deploy them for use in the government health institutions and
making them readily available for individual use. Further work should be on
the availability of treatment suggestions for the expert systems.

Keywords: Diagnosis, Diagnostic, Expert System, Innovative, Malaria

1. Introduction

According to the World Health Organization (WHO), 87 endemic
countries recorded 241 million malaria cases in 2020, this gave about 59
cases per 1000 population in 2020 (WHO, 2021). Just like the case in 2019,
where about 229 million cases were reported by the WHO, Africa countries
have been leading the pack of global malaria cases and fatalities in 2019 and
2020, as always. African countries also led the malaria fatalities recorded in
2019 and 2020. In 2019, about 558, 000 fatalities from malaria were
recorded (WHO, 2020), while in 2020, the fatalities increased by 12 % to
627, 000 (WHO, 2021). While Nigeria accounted for 23% of the global
malaria cases and 26.7% of the global malaria fatalities in 2019 (WHO,
2020), she also accounted for 27% of the global malaria cases and 31.9% of
the global malaria fatalities in 2020 (WHO, 2021) as shown in Fig.1. It has
been established that over 97% of the Nigerian population remains at risk of
malaria infection year-round (Adah et al., 2018).

Fig. 1 Global distribution of malaria deaths in 2020 (WHO, 2021)

The Federal Ministry of Health in Nigeria (FMoH) has provided
guidelines for the optimal management of malaria to healthcare
professionals in the country. These are expected to serve as a guide in
treating malaria patients in the country (FMoH, 2020). The guidelines
recommend that all suspected cases of malaria should go through a strict
quality assurance program to ascertain or confirm the presence of malaria in
a patient before a strict administration of artemisinin-based combination
therapies (ACTs) for effective treatment of malaria. The major diagnostic

tools currently deployed for the management of malaria in the country
include the Rapid Diagnostic Test (RDT) and the Peripheral Blood Smear
(PBS) test or microscopy.

2. Innovative Diagnostic Tools in the Diagnosis of Malaria

While existing diagnosis tools have been in use for decades,
limitations have emerged in the form of shortage of qualified health
professionals in the government owned health institutions, migration of
qualified health professionals to other countries because of poor
remuneration and poor malaria control practices. This is the reason behind
the need to develop innovative diagnostic tools based on Artificial
Intelligence (AI) which can assist and improve malaria diagnosis among
health professionals and individual users. Some Expert Systems (ES) have
been developed with different artificial intelligence techniques and
algorithms to assist in the prompt and accurate diagnosis of malaria in the
country. These expert systems include the following:

● Malaria Diagnosis Expert System (MDES) – This knowledge-
based expert system is rule based, it utilizes the forward chaining
technique through the use of VisiRule (Ojo, 2022), which is an
intelligent flowchart software. The expert system performs
diagnosis based on the symptoms of the user; the ES asks simple
questions based on the symptoms manifested by the patients, a
confirmatory Peripheral Blood Smear (PBS) test is required to
confirm the presence of malaria in a patient, this clearly supports
the strict guidelines of the Federal Ministry of Health in Nigeria to
put an end to presumptive management of malaria. MDES simply
recommends that the patient consult a doctor in cases confirmed not
to be malaria.

● Expert System for Diagnosis of Malaria and Typhoid – This
expert system explores the design and building of a model, using
the Naïve Bayes algorithm (Maidabara et al., 2021). This expert
system was developed using python, it is capable of diagnosing a
malaria patient within 30 minutes, even without a confirmatory
laboratory test. It uses three different algorithms to make this
accurate prediction: Artificial Neural Network (ANN), Support
Vector Machine (SVM) and Naïve Bayes. SVM and Naïve Bayes
give an accuracy of 100% in malaria prediction among patients.

● MALARES - This is a rule-based, forward chaining expert system
that uses the Visual Basic 6.0 programming language for the
diagnosis of malaria (Nkuma-Udah et al., 2020). It uses a
knowledge base built through the accumulation domain knowledge
from the domain experts, the internet and literature. It also employs
different production rules based on the
symptoms/signs/investigating reports for the diagnosis of malaria,
highlighting symptoms that are strongly related to malaria and
symptoms not related to malaria (Nkuma-Udah et al., 2020). The
expert system has a simple and interactive graphical user interface
with a menu.

● Joint Neuro-Fuzzy Malaria Diagnosis System – This expert
system harnesses the development of a collaborative neuro-fuzzy
diagnosis platform, which is basically the coming together of the
Neural Networks and Fuzzy Logic (Oladele et al., 2021. It is tedious
and sluggish and entails knowledge engineering from domain
experts who are medical professionals, the knowledge acquired
from the domain experts is captured in the Expertise Developed
Fuzzy Proficient Scheme (Oladele et al., 2021). The expert system
uses Microsoft Visual C # Programming Language and Microsoft
SQL Server 2012 for the database. This expert system has been
developed to support medical practitioners in their decision making
and to assist academics in their research.

● MALX – A Malaria Diagnosis System Using VP-Expert System
– This is a rule-based expert system that utilizes the VP expert shell,
the 363 rules determine if a patient is healthy or infected with simple
or severe malaria (Adamu, 2018). The rules were developed based
on direct interviewing of the domain experts. This expert system has
a very high accuracy in the diagnosis of malaria when compared
with human experts.

● Medical Expert System for Tropical Diseases Diagnosis – This
expert system provides the user with a background for the diagnosis
and treatment of malaria and typhoid diseases, which are two related
febrile diseases in Nigeria (Ibiobu & Nwiabu, 2019). It uses the
fuzzy logic type 2, which has proven to be an effective tool in the
building of intelligent decision support for approximate reasoning
where there are uncertainties and imprecisions (Ibiobu & Nwiabu,

2019). It has several stages, with the gathering of symptoms in the
first stage; analysis and investigation of the symptoms in the second

stage; and diagnosis based on the supplied symptoms in the third

stage. The expert system provides an accurate diagnosis as it yields
a better result than other classic systems, mainly because it

simulates the experts.

A summary of the expert systems, including the techniques used in
developing them, their deployment status, and comments on their

deployment, is shown in Table 1.

Table 1. Summary of the Expert Systems

No References Techniques Deployme Comments

nt Recommended
for deployment
1 Expert System for Python – using Not

Diagnosis of Malaria and the algorithms: deployed

Typhoid (Maidabara et SVM, ANN and

al., 2021). Naïve Bayes

2 Design of Expert System Microsoft Not Deployment
recommendati
for Diagnosis of Malaria Visual Basic deployed on not
suggested
(MALARES) Using VB 6.0. – using rule-
Proposed for
6.0 (Nkuma-Udah et al., based and study in the
field of
2020) forward- academia and
industry
chaining
Recommended
3 A Joint neuro-fuzzy Neural Not for deployment

Malaria Diagnosis Networks and deployed More of a
scientific paper
System (Oladele et al., Fuzzy Logic for an article

2021)

4 The Development of a Rule-based and Not

Malaria Diagnosis VisiRule deployed

Expert System (MDES)

in the South Western

Part of Nigeria (Ojo,

2022)

5 A Medical Expert Uses the fuzzy Not

System for Tropical logic rule deployed

Diseases Diagnosis

(Ibiobu & Nwiabu,
2019)

6 Development of Malaria Uses production Not Recommended
for deployment
Diagnosis System rules deployed

(MALX) Using VP-

Expert System (Adamu,

2018).

3. Conclusion

These are only several of the many recent expert systems that can
be effectively deployed for use in the diagnosis of malaria in the country.
They are effective, offering accuracy levels surpassing that of a junior doctor
in the diagnosis of malaria in the country, even though they are all available
for deployment, government bureaucracies and ethics are currently making
them unutilized, despite the obvious need for such innovation in the
management of malaria in the country. This further supports the efficacy of
the expert systems and the need to deploy them for use in both government
health institutions and individual use. Further work should focus on the
availability of treatment suggestions for the expert systems.

4. References

Adah, P., Maduka, O., Obasi, O., Doherty, O., Oguntoye, S., Seadon, K.,
Jalon, O., Zwingerman, N., & Uhomoibhi, P. (2018). The role of the
Deki Reader in malaria diagnosis, treatment and reporting: Findings
from an Africare pilot project in Nigeria. Malaria Journal. 17.
10.1186/s12936-018-2356-8.

Adamu, S. U. (2018). Development of malaria diagnosis system using expert
system. [M.Sc. Dissertation, Dept. Mechatronics Eng., Near East
University, Nicosia]. http://docs.neu.edu.tr/library/6583844230.pdf

Federal Ministry of Health (FMoH) (2020). National Guidelines for
Diagnosis and Treatment of Malaria. 4th ed. Abuja, Nigeria:
NMEP.

Ibiobu, N. A., Nwiabu, N.D. (2019) A Medical Expert System for Tropical
Diseases Diagnosis. International Journal of Computer Sciences
and Engineering, 7 (7), 386-390.
https://doi.org/10.26438/ijcse/v7i7.386390

Maidabara, A. H., Ahmadu, A. S., Malgwi, Y. M., & Ibrahim, D. (2021).
Expert System for Diagnosis of Malaria and Typhoid. Computer
Science & IT Research Journal, 2(1), 1-15.

Nkuma-Udah, K. I., Chukwudebe, G. A., Ekwonwune, E., Ejeta, K. O.,
Onwodi, G. O., & Ndubuka, G. I. (2020). Design of Expert System
for Diagnosis of Malaria Using VB 6.0. African Journal of Medical
Physics, Biomedical Engineering and Sciences, 7(1), 25-37.

Oladele, T. O., Ogundokun, R. O., Misra, S., Adeniyi, J. K., & Jaglan, V.
(2021). A joint neuro-fuzzy malaria diagnosis system. Journal of
Physics: Conference Series (Vol. 1767, No. 1, p. 012038). IOP
Publishing.

Ojo, A. C. (2022). The Development of a Malaria Diagnosis Expert System
in the South-Western Part of Nigeria. [M.Sc. dissertation, Dept.
Mathematics and Computer Science, Universiti Teknologi MARA],
Malaysia, 2022.

World Malaria Report 2020: 20 years of global progress and challenges.
Geneva: World Health Organization; 2020. Licence: CC BY-NC-
SA 3.0

World Malaria Report 2021. Geneva: World Health Organization; 2021.
Licence: CC BY-NC-SA 3.0 IGO.

Detecting Android Trojan using Dynamic
Analysis and Machine Learning

Nur Atiqah Binti Mohamad Ikhsan1, Kamarul Ariffin Bin Abdul Basit2

Faculty of Computer and Mathematical Science, Universiti Teknologi MARA,
UiTM Shah Alam, 40450 Shah Alam, Selangor, Malaysia

[email protected], [email protected]

Abstract: Due to a rapid rise in the number of smart devices and
applications, mobile security threats are a regular occurrence. Attacks take
many forms of malware using various vulnerabilities of operating systems,
especially in the widely known susceptible Android operating system. In
light of the serious security risks and the vulnerabilities inherent in Android
operating system, the effects of Android trojans are growing rapidly. There
is a need for strong protection, since this is critical to maintain the security
of smartphone users. There are multiple analytical methodologies for
detecting Android trojan. In this paper, dynamic analysis technique is
proposed to detect Android trojan. There are two datasets used in this paper,
which are trojans and benign datasets. The method proposed uses system
calls for dynamic analytical features. The Random Forest, J48, SVM and
Naïve Bayes classification algorithms were used to classify the instances
according to the features obtained from the proposed method. There were
57 significant system calls found that were able to detect Android trojans.
The Random Forest classification algorithm obtained the highest
performance accuracy rate at 93.69%.

Keywords: Android, Dynamic Analysis, Machine Learning, System Call,
Trojan

1. Introduction

The Android operating system has remained popular in the
electronics business, particularly in the smartphone sector, because some of
its development tools are free, open source and favoured by scholars
(Kocakoyun, 2017). Although, because of its openness, Android, and
despite delivering ease to people's daily lives, also poses major security and
privacy concerns (Liu et al., 2020). According to Kaspersky Mobile
Products and technologies (Chebyshev, 2021) reported that in 2020 a total
of 177418 cases are from Trojan malware alone, which focuses on banking
and ransomware Trojan. The number for mobile banking trojans in 2020,
are as twice the previous year’s figure and comparable to 2018. Android
trojan is a type of malware that runs in Android operating system and it work
by deceiving the users in thinking that the application is legitimate, causing
them to unknowingly download it. Android trojans are frequently distributed
with the same name and appearances as other well-known application
available on online platforms (F-secure, 2017). Several methods are taught
to end-users to lessen the risk from being a victim to Android trojan such as
update the latest patching system and avoid suspicious third-party
applications. Nonetheless, humans are not without a mistake and would
sometimes unintentionally install this sketchy software on the internet,
hence, making their Android devices exposed to Android trojans. One of the
major concerns of Android trojans is that such malware has been evolving
over the past decade, making it difficult to be detected in the Android
operating system.

2. Literature Review

This section contains the related literature, which includes an
overview of trojan malware and the techniques used in detection analysis.

2.1 Types of Android trojan Malware

In this section, we will discuss further three most common types of
Android trojan Malware which are Backdoor, SMS and Remote Access.

2.1.1 Backdoor Trojan

A backdoor trojan can install a "backdoor" which allows an attacker
to gain access to and control over a mobile phone. A third party might
download and steal user’s personal information. Additionally, new malware
might be downloaded by the attacker. For an instance, HummingBad
Android was one of the greatest attacks on Android (Tynan, 2016). It is a
tricky application used to fool users to click advertisements. It also creates
a backdoor that has root permissions on the device, which allows additional
applications to be secretly installed and would persistently remain in the
device even after rebooting the system.

2.1.2 SMS Trojan

An SMS trojan is a rogue smartphone software that makes money
by delivering SMS messages to premium-rate numbers silently. One
examples in Android is known as SMSSend, which makes money by
sending jumbled erotic text to phone numbers such as 9993, 9994 and 9995
(Alienvault, 2016). The functionality of other SmSSend variations is very
much the same, but the display pictures, text contents and recipient numbers
would be different.

2.1.3 Remote Access Trojan (RAT)

A Remote Access Trojan (RAT) system is a malicious software that
is inspired by remote access tools. A remote access tool is a software
application that allows an administrator to remotely operate or access a
system such as camera, key logs and microphone (Shihab, 2018). For an
example, AndroRAT, it is a built inside an application and act as a carrier.
The built-in RAT allows remote attackers to control the afflicted device after
the software is installed on a device. A recent AndroRAT variant was
discovered that has the ability to collect browser data, capture pictures from
front camera and screenshots (Palmer, 2018).

2.2 Malware Analysis

The various techniques of malware analysis will be described in this
section. There are three types of malware analysis, which are static, dynamic
and hybrid.

2.2.1 Static Analysis

In modern research, there are three main methods used in malware
analysis, one of them being static analysis. It is a method of examining a
binary file without running it. It is the simplest to carry out and allows
researchers to obtain metadata from the suspicious binary. Even if it does
not provide all the necessary information, it can occasionally offer useful
information. However, because to its relative difficulty in identifying
disguised malware, it might miss substantial malware behaviour.

2.2.2 Dynamic Analysis

Another common malware analysis method is dynamic analysis.
Dynamic analysis, often known as behavioural analysis, is a method of
running a probable binary in a controlled environment and observing its
behaviour. This is simple to do and provides useful information on the
binary's activity throughout execution. This technique is useful when an
application's source code is disguised, because the application would be
continue running and its activities are examined based on the actions. This
technique is preferred by many researchers, as it is cost-efficient in terms of
implementation and execution.

2.2.3 Hybrid Analysis

The third type, hybrid analysis, is the blend of both static and
dynamic analysis together. By combining static and dynamic approaches,
hybrid methods take use of the benefits of both. The hybrid method will
initially do a static analysis of the program before moving on to a dynamic
analysis. According to Arshad et al. (2016), the main disadvantage of this
hybrid approach is that it is a costly technique to implement due to restricted
available resources which including battery, memory, and so on.

2.3 Related Work

Static, dynamic, and hybrid analytic approaches for analysing
Android trojans have been studied in previous research. At the moment,
there is not much research being done on trojans. As a result, relevant
publications as well as studies on malware in general are examined.

The purpose of a study by Mohamad Arif et al. (2021) was to
determine how successful static analysis is in detecting Android malware
via permission-based features. Machine learning using several sets of
classifiers was used to assess Android malware detection in this study. In
this study, the feature selection approach was used to determine which traits
were best capable of recognising malware.

Chaba et al. (2017) provided a technique for creating a dataset using
system call log information in this research. After that, the dataset is run
through three algorithms. The correctness of the results is calculated once
they have been analysed. They use a filtering procedure known as Chi
Square to increase the dataset's quality.

Xu et al. (2016) proposed a HADM (Hybrid Analysis for Detection
of Malware) framework. First, a collection of static and dynamic
characteristics will be extracted. The data collection is translated into vector-
based representations for static features. As for dynamic features, the data
collection is translated into vector-based and graph-based representations,
such as system call invocations. After that, the next step would be using
deep learning techniques to build a neural network for each of the vector
sets for all vector-based representations. For each of the feature vector sets,
including system call feature vectors and static feature vectors, they train
one Deep Neural Network (DNN) built by stacking Restricted Boltzmann
Machines (RBM). The new DNN feature vector sets are created by
concatenating the DNN learnt features with the original features.

Table 1 summarizes all the related work that has been mentioned. It
includes the type of analysis that has been done in each research, the features
used, and the classification algorithm used to compare the results.

Table 1. Summary of related work

Researcher(s) Type of Features Classification
Analysis algorithm

(Mohamad Arif et Static Permission Random Forest,
al., 2021) kNN. MLP, J48

and Adaboost

(Chaba et al., Dynamic System Call SGD, Naïve
2017) log
Bayes, Random
Forest

(Xu et al., 2016) Hybrid Deep learning System calls
and Multiple invocation and

kernel SVM
learning.

3. Methodology

This part outlines how the study is carried out, including the
technique utilised in data collecting and analysis, an overview of the
research instruments and environment, and a summary of the planned
research.

3.1 Dynamic Analysis

In this paper, 1000 applications were executed with 500 applications
belonged to Trojan dataset which were taken from Drebin Project, and
another 500 applications belonged to benign dataset which were taken from
Google Play Store. The proposed model flow starts with the execution of
the .apk files from benign and Trojan datasets in the Android emulator,
Genymotion. As soon as the .apk files are installed inside the emulator, the
.apk file would be executed and the system call then be recorded. The
collection data is done by using adb shell command. It collects the data on
all system call that has been successfully detected and executed. The
metadata logs show the name of the system call, the frequency of successful
features, the interaction, the percentage of time spent on the task, and the
number of failures encountered.

3.2 Feature Extraction

The process entails extracting features from the input Android
application in a .apk file format. The collected and logged system calls from
the previous step are converted into nominal form in this phase to build a
features vector of system calls for each Android application. A number 1
indicates the presence of Trojan in the .apk file, whereas a value of 0
indicates its absence. All the nominal numbers are recorded in excel file for
collection process.

3.3 Feature Selection

In order to improve the Trojan detection of the suggested approach,
it is important to choose the most relevant features and the irrelevant system
calls will not be chosen. This approach involves the automatic selection of
characteristics from aggregated data. All the data from extraction of system
calls featured previously would be gathered into a comma separated value
(CSV) file format. In this approach, the Gain Ratio Attribute Evaluator was
utilised to conduct feature selection by using WEKA application in the
“Select Attribute” section.

3.4 Classification

Since there is variant of call features produced from benign and
Trojan datasets, machine learning techniques were employed to analyse or
categorise their behaviours based on all these combined materials. The most
popular algorithm such as Random Forest, Naive Bayes, SVM and J48
classifiers have been selected as a classification technique for monitoring
the application’s behaviours.

4. Results

In this section, the performance results of each classifier algorithm,
identification of system calls and ROC Curve are further discussed.

4.1 Performance of Classifier algorithm

True Positive Rate (TPR), False Positive Rate (FPR), and accuracy
are used to compare the performance of the patterns. TPR stands for the total
number of Android trojans that have been classified as trojans. The number
of benign apps classified as trojan is called the False Positive Rate (FPR)
and the accuracy is the proportion of times the classifier properly
categorised the instances.

Table 2. Results of different classifier performance

Classifier TPR FPR Accuracy

Naïve Bayes 0.701 0.360 67.06%

SVM 0.754 0.246 75.37%

J48 0.878 0.180 84.88%

Random Forest 0.966 0.092 93.69%

As illustrated in Table 2, the highest value in this study is 93.69
percent for Random Forest, followed by J48 at 84.88 percent, SVM at 75.37
percent, and last but not least, Naïve Bayes at 67.06 percent.

4.2 Identification of system calls

There are as many as 250 system calls which can be invoked inside
a single device when running an application. It was discovered in this study
that not all the system calls in the dataset are requested by the application;
therefore, unnecessary system calls are not selected. 57 significant system
calls were discovered during the execution of 1000 applications of Trojan
and Benign dataset. Table 3 below shows the total of 57 lists of system calls
that were selected.

Table 3. List of selected system calls

Selected System Calls

sendms sendto recvfrom ioctl epoll_wa clock_ge epoll_ct
g it ttime l

getpid munma getuid32 write read futex lseek
p
openat mprotec
open close class gettimeo clone t
writev fday
pread64
stat64 faccesssat fstata fstata64 fstat64
t64

fdatasy fsync pwrite64 chm access lstat64 socket
nc od
bind dup fcntl64
uname connect selec rt_sigret
_llseek t urn sched_yi fchmod
unlink nanosle eld at
mkdir rena umask
fchown ep me getsocko setsock
32 pt opt
getdent64 mma madvise
p2

sigprocmask

4.3 ROC Curve

ROC curves allow us to compare and assess classification methods,
assisting us in choosing the best classifier. A binary classifier's performance
is measured by the area under the ROC curve, or AUC. Perfect prediction is
defined as AUC=1, whereas a bad prediction is defined as AUC=0.6. In this
paper, it is found that the plot curve is 0.9877 in value. This means that the
curve is nearly to 1, indicating that this model was able to detect malicious
application more accurately.

Fig. 1 ROC Curve Value

5. Conclusion and Future Work

In this paper, we demonstrate that a system call is an important
dynamic element for detecting malicious applications in a behavioural-
based intrusion detection system. The presented framework makes use of
the System Call Invoked capability to detect malicious apps and has been
successful in obtaining accuracy with machine learning methods. There

were 57 significant system calls obtained while conducting the research in
this paper out of 250 system calls. From the result, the Random Forest
algorithm had the highest detection accuracy of 93.69% and the lowest false
positive rate of 0.092. This was followed by J48 with accuracy of 84.88%,
SVM with 75.37% and Naïve Bayes with an accuracy of 67.06%. The
results of tests suggest that this approach can discern between benign and
malicious applications. For future research in analysing Android trojans, it
can be upgraded with using more malware and benign datasets to get a
higher accuracy result. This may increase the reliability, although basic
logistic algorithms have been discovered to be a useful approach for
malicious programme detection. The accuracy may be increased further by
adding other features such as API calls, anomalies, signatures and so on with
the analysis of system calls. It would also be a great addition if future
analysis included by comparing all three analysis methods, including static,
dynamic and hybrid.

6. References

Alienvault. (2016). Android Fictus Malware. Retrieved from Alienvault
OTX:
https://otx.alienvault.com/pulse/56d71291aef921042423c087/histo
ry

Arshad, S., Khan, A., Ali Shah, M., & Ahmed, M. (2016, February).
Android Malware Detection & Protection: A Survey. (IJACSA)
International Journal of Advanced Computer Science and
Applications, 463. doi:10.14569/IJACSA.2016.070262

Chaba, S., Kumar, R., Pant, R., & Dave, M. (2017). Malware Detection
Approach for Android systems. ArXiv.

Chebyshev, V. (2021, March 01). Mobile malware evolution 2020.
Retrieved from Securelist by Kaspersky:
https://securelist.com/mobile-malware-evolution-2020/101029/

F-secure. (2017, January 16). classification. Retrieved from F-Secure.com:
https://www.f-secure.com/v-descs/trojan_android.shtml

Kocakoyun, S. (2017). Developing of Android Mobile Application Using
Java and Eclipse: An Application. International Journal Of
Electronics, Mechanical And Mechatronics Engineering, 7(1).
doi:DOI: 10.17932/ IAU.IJEMME.21460604.2017.7/1

Liu, P., Wang, W., Luo, X., Wang, H., & Liu, C. (2020, March 12).
NSDroid: efficient multi-classification of android malware using
neighborhood signature in local function call graphs. International

Journal of Information Security.
doi:https://doi.org/10.1007/s10207-020-00489-5
Luu, C. (2020, September 4). Ransomware Attacks: How to Protect your
Data With Encryption. Retrieved from Security Intelligence Logo:
https://securityintelligence.com/posts/ransomware-attacks-how-to-
protect-data-encryption/
Mohamad Arif, J., Ab Razak, M., Awang, S., Tuan Mat, S., Ismail, N., &
Ahmad Firdaus. (2021, September 30). A static analysis approach
for Android permission-based malware detection systems.
doi:https://doi.org/10.1371/journal.pone.0257968
Palmer, D. (2018, February 14). AndroRAT: New Android malware strain
can hijack older phones. Retrieved from ZDNet:
https://www.zdnet.com/article/androrat-new-android-malware-
strain-can-hijack-older-phones/
Shihab, M. (2018, September). Android Application to Monitor And Protect
From Remote Access Trojan. doi:10.13140/RG.2.2.36713.72809
Tynan, D. (2016, July 7). HummingBad Android malware: who did it, why,
and is your device infected? San Fancisco, California, United States.
Xu, L., Zhang, D., Jayasena, N., & Cavazos, J. (2016). HADM: Hybrid
Analysis for Detection of Malware. SAI Intelligent Systems
Conference.

The Evaluation of User Experience (UX) about
an Interactive Animated Comic of Paedophile

Awareness for Children (i-ComPedo)

Nur Anis Idayu Binti Mohd Nor1, Fariza Hanis Abdul Razak2

Faculty of Computer Science and Mathematics, Universiti Teknologi MARA,
UiTM Shah Alam Campus, 40450 Shah Alam, Selangor, Malaysia
[email protected], [email protected]

Abstract: Paedophilia can cause nightmares for child victims, as they are
likely to suffer physically and emotionally for the rest of their lives. Due to
this, the researcher developed the i-ComPedo application, a digital comic,
to provide interactive content by utilizing animated comic elements to create
awareness among children. Although children have used i-ComPedo,
whether the digital comic effectively creates awareness of the paedophile is
unknown. Therefore, this study evaluates children's experiences with the i-
ComPedo application. The Attrakdiff questionnaire was used to evaluate
four dimensions of user experience (UX): Pragmatic Quality (PQ), Hedonic
Quality Identification (HQ-I), Hedonic Quality Stimulation (HQ-S), and
Attractiveness (ATT), among children aged 9 to 12 years old. The results
revealed that primary school children users gave positive evaluation;
Attractiveness (ATT) had the highest score, followed by Hedonic Quality
Identification (HQ-I). Therefore, the researcher created the face-to-face
distributed questionnaire using Microsoft Word with five participants from
primary school teachers to suggest user experience improvements for the i-
ComPedo app. The five teachers were chosen based on their teaching
method, and the information technology (IT) currently used in their classes.
The findings show that all participants gave positive feedback and suggested
three (3) UX improvements. A future direction for this research would be to
expand the scope of the study further, improve the UX, and incorporate
qualitative methods to address the limitations of the Attrakdiff
Questionnaire Framework.

Keywords: User Experience (UX), Attrakdiff Questionnaire, Smiley-o-
meter, i-ComPedo application, children

1. Introduction

Paedophile awareness among children requires extensive efforts
and worldwide attention because the longer people keep silent about this
issue, the more widespread it may become. Paedophilia refers to a sexual
preference for young children, boys, girls, or both. It also concerns many
disciplines, including psychology, psychiatry, and criminology (Moen,
2015). Children must be aware of paedophiles, who might be harmful to
them. One approach used to increase awareness of paedophiles among
children at an early stage is by introducing digital comics that spread a
specific message using storytelling and interactive multimedia elements.

The digital comic i-ComPedo application (Ahmad et al., 2020) was
developed to provide interactive content by utilizing animated comic
elements to create awareness paedophile threats towards children. The
application can be a supportive tool for children's learning and a teaching
aid to create awareness. According to previous literature, there is still limited
research regarding the awareness of paedophiles using animated comics in
terms of user experience.

Unfortunately, there is scarce research on the use of animated comic
and awareness of paedophiles among children in the Malaysian context. It
is thus crucial to gather information from children's points of view and
perception to determine their level of awareness of paedophile issues by
evaluating the user experience. For children to be motivated to be more
careful and learn how to protect care of their bodies from strangers, this
research aims to evaluate the user experience (UX) of the i0-ComPedo
application for children. This includes determining their perceptions of
whether this app can increase awareness about a paedophile or otherwise.

1.1 Research Questions

There were two (2) research questions conducted based on the issues
discussed in the above sections of this thesis:

a) What is user experience (UX) of using i-ComPedo application?
b) How can we improve user experience (UX) of the i-ComPedo

application?

1.2 Research Objectives

The objectives were developed based on the above problem
statement to achieve the best possible outcomes. The research objectives
were:

a) To evaluate the user experience (UX) of the i-ComPedo
application.

b) To suggest user experience (UX) improvements for the i-
ComPedo application.

1.3 Research Scope

The research mainly evaluates the UX among children from 9 to 12
years old at Sekolah Kebangsaan Besut, Terengganu. This sample was
chosen because, at this age, children are often exploited by cybercriminals.
The majority heavily rely on networking sites to interact with the public
(Quayle, 2020). Children's widespread use of social networking sites and
technological tools has significantly reconfigured how they communicate,
with whom, and the consequences of this communication (Abbas, 2019).

2. Literature Review

According to Child-National Help's Child Abuse Hotline (2016) and
Giulio (2020), any child sexual abuse or paedophilia, without parents, other
protectors, or public officials to intervene, results in actual or potential crisis
for a child. It can occur in the children's family, our relationships, schools,
or the networks in which the child works and encompasses all forms of
physical abuse, sexual abuse, emotional abuse, or disregard (Cheah et al.,
2016; Ahmad et al., 2020). The i-ComPedo application (Ahmad et al., 2020)
was designed as stand-alone courseware using multimedia and animated
comics. The application utilizes animated comic elements to increase
awareness about paedophile risks to children. The i-ComPedo application is
divided into three (3) primary modules, each of which features two distinct
situations (Cerita 1 and Cerita 2) with engaging storylines involving a boy
and a girl as the victims in the first module.

Research conducted by Othman et al. (2015), a "Multimedia
Learning Application to raise children's awareness of child sexual abuse,"

was built using 2D animation and a multimedia learning application. The
report discusses the findings of a pilot study on interactive media
applications, cyberbullying, and how to increase youth awareness and
perspective. Manaf et al. investigated using "ToonDoo" as a media
technology for teaching English short stories. The proposed development of
an animated cartoon as interactive multimedia was intended to entice
children to learn. A short tale can significantly boost and develop skills
including cultural awareness, linguistic awareness, and motivation.

The Smiley-o-meter has a long history in paediatrics as a subjective
indicator of children's medical conditions (Hall et al., 2016). Recently, it has
been used to gauge children's preferences for lighter foods, more practical
experiences, and interactive products. This measurement method was
designed and verified using facial data collected from children while
viewing the i-ComPedo app. "This or That" and "Smiley-o-meter" are useful
for assessing the engagement of children with digital media degrees. Huan's
(2017) research focuses on this concept of the child's involvement in
subjects such as the Manchester Child Attachment Task (MCAST). Their
findings indicated that facial data analysis, utilizing eye-tracking and face-
to-face recognition, might be used to determine children's involvement rates.
This research also described several past research on evaluating UX for
children.

Attrakdiff was adapted and chosen for this research because the
primary objective was to provide users with a quick, immediate, and
effective tool for self-assessment of the usability and design of a
collaborative product to evaluate user experience (Schrepp et al., 2017;
Hassenzahl, 2008). Attrakdiff consists of "28 seven-step items with poles
and opposing adjectives" (for example, confusing-clear, unusual-ordinary,
good-bad), which were classified into four major categories in the current
version: Pragmatic Quality (PQ), Hedonic Quality-identification (HQ-I),
Hedonic Quality-stimulation (HQ-S), and Attractiveness (ATT).

3. Research Methodology

Figure 1 shows that this research was divided into four (4) phases
for the research framework, based on quantitative research: planning and
preparation, data collection, data analysis, and documentation.

Fig. 1 Research Framework adapted from Subbarao & Mahrin (2020)
with Research Framework of Evaluation Model

In addition, the Attrakdiff questionnaire consists of questions that
may not be easy for children to understand. The researcher translated the
English version of Attrakdiff to Malay and revised all 28 questions to ten
questions without changing the meaning. That was because the questions
would make children more engaged and understandable. The findings would
calculate into three groups on the Attrakdiff online platform: Portfolio
Presentation, The Average Value, and Description Word-pairs. Researchers
would establish whether data was feasible and consistent as a source of
information based on these findings (User Experience rating). Additionally,
the research would use descriptive analysis to present and interpret the
numerical data. Descriptive analysis is a procedure used to summarize,
organize, graph, and describe data (Schrepp, 2017). The processed data
includes mean value and standard deviation calculations.

4. Result and Discussion

The survey data were collected using an online questionnaire based
on the Atttakdiff questionnaire in the Malay language. The participants were
from primary school children from 9 to 12 years old, at Sekolah Kebangsaan
Kuala Besut, Terengganu, who did not have any experience using the i-
ComPedo application. The researcher created the online questionnaire based
on the Attrakdiff questionnaire. The researcher had face-to-face distributed
the revised questionnaire to the participants for gathering more information
about the level of user experience of the participants, as shown in Figure 4.1.
In total, 40 participants completed the questionnaire within the data
collection period, which was one day. The demographic profile was another
robust set of information retrieved from the Attrakdiff questionnaires. The
questionnaire gathered the gender and range of ages, as shown in Table 1.

Table 1. Analysis of Demographic Profile
N= 40

Demographic Category Frequency Percent (%)
Information

Gender Female 26 65
Age of range Male 14 35
9 years old 10 25
10 years old 10 25
11 years old 10 25
12 years old 10 25

4.1 Results of Research Objective 1 (RO1)

The results gathered from the analysis were used to identify the
evaluation of user experience (UX) criteria or elements of the i-ComPedo
application. While all four main groups (PQ, HQ-I, HQ-S, and ATT) in the
Attrakdiff scale individually received a positive evaluation, the
Attractiveness (ATT) and Hedonic quality identification (HQ-I) received
the highest score. Grouping scales follow Pragmatic quality (PQ) and
Hedonic quality stimulation (HQ-S), the second-highest. The results
indicated that users familiar with the technology understood it well, felt in
control, and were motivated, developing awareness about paedophiles while
using the i-ComPedo app. These results proved that four main groups (PQ,
HQ-I, HQ-S, and ATT) were the criteria or elements of user experience
(UX) in the Attrakdiff platform when evaluating the i-ComPedo application
shown in Table 2.

In addition, the results gathered from the analysis, as shown in Table
3 of the smiley-o-meter, have revealed that most participants gave positive
feedback, which was scored between 4.50 to 5.00 (90%), from 36 children
attracted to and enjoyed the i-ComPedo app, especially "The Stories" part.
But, only four females (10%) of the comments expressed negative
experiences. Some children stated that they did not like the app and found it
boring. Others felt that the drawing comic was not pretty, that some of the
features were not described or that instructions were not given properly.
Thus, some suggested that the i-ComPedo app needs enhancements on some
points, as some of them suggest that it would be great if more games in the
"Quiz" session were added. However, all of them understood that this app's
main point was to create an awareness of paedophiles in children. Therefore,
all participants were likely to engage and enjoy the i-ComPedo application.

Table 2. The results of Attrakdiff scale from 9 to 12 years old

Table 3. The average number score for smiley-o-meter positive and
negative feedbacks gathered after exploring the i-ComPedo app

4.2 Result of Research Objectives 2 (RO2)

The findings above revealed that most of the participants were
engaged and enjoyed the i-ComPedo app. Most of them found that this app
was a new technology that consisted of a combination of multimedia
principles and animated comics, the main focus to create awareness for
children about paedophile issues. They also felt that this app was suitable
for children’s learning at school. Therefore, the participants found that this
i-ComPedo app has a limitation: they cannot access it online.

Furthermore, as mentioned in Chapter One, the i-ComPedo
application was a standalone courseware using .exe format and CD-ROM.
Therefore, this app was also not distributed to the markets. Thus, it may lead
to difficulties for other users to access this app which needs permission from
the developer. So, most of the participants suggest the i-ComPedo
application to be an online learning platform using internet access. Besides,
several participants told the i-ComPedo app needs to create the ‘print out’
features, which is a resource from that app can be a guideline for teachers
and children learning in school.

The participants also suggest that this app be more beneficial if it
can be widespread become the application software available to install and
run on mobile devices. This app would make users, especially children, easy
to access wherever they go, even at school or home. However, this shows
that further improvement and the user interface (UI) needs to be made and
considered. Overall, all the participants gave positive feedback. The
research objective RO2 was also accepted, mainly suggesting UX
improvements of the i-ComPedo application.

5. Suggestions

The researcher can suggest several improvements to provide a better
user experience (UX) for the i-ComPedo application. Firstly, one suggestion
for future studies would be to apply the Attrakdiff questionnaire as part of a
usability test of the i-ComPedo application. The usability test would be part
of a qualitative study. Researchers will ask the users to perform a test task
on the i-ComPedo application and record their responses immediately. The
researcher will give the Attrakdiff questionnaire after users have finished
their usability test to understand the i-ComPedo application quickly. The
researcher should conduct face-to-face interviews with users, asking them
one by one of the 28 questions from the Attrakdiff questionnaire. Thusly,
the researcher would not need to revise the Attrakdiff questionnaire to
become ten questions. This technique will save time during the data analysis
phase, which is the researcher can direct get the findings to result
immediately without keying in one by one manually.

After completing the Attrakdiff questionnaire, users could engage in a
focus group discussion to further discuss and give opinions on areas to
improve. Using qualitative study (usability test and focus group interview),
researchers can get better feedback from users on the evaluation UX for an
i-ComPedo application and ways to improve it. In addition, the researcher
recommended that this research increase the number of primary school
children respondents for future research. In addition, the quantitative study
should be better expanded and larger the sample size that involves primary
school users, especially children. Finally, it can gather more accurate
statistics of descriptive analysis data due to evaluating user experience (UX)
for the i-ComPedo application.

6. Conclusion

The researcher evaluated the two (2) objectives identified for the
study in this chapter. First, there was the evaluation of user experience (UX)
and suggestion of UX improvements to the i-ComPedo application. The
study has a defined scope research objective summary, and hence the
researcher also discovered some limitations in this chapter. Suggestions for
improvement for future works were discussed and proposed. To that end,
the researcher was able to identify the evaluation of user experience (UX)
elements for children by using the Attrakdiff questionnaire framework and

with the UX improvements to meet the requirements based on RO1 and
RO2.

7. References

Abbas, J., Aman, J., Nurunnabi, M., & Bano, S. (2019). The impact of social
media on learning behavior for Sustainable EDUCATION:
Evidence of students from Selected universities in Pakistan.
Sustainability, 11(6), 1683. doi:10.3390/su11061683

Ahmad, S., Mohd Nor, N., & Maesin, A. (2020). An interactive animated
comic of paedophile awareness for children. International Journal
of Recent Technology and Engineering, 9(1), 332337.
doi:10.35940/ijrte. a1501.059120

Cheah, I. G.-S., & Yuen, C. W. (2016). A Review of Research on Child
Abuse in Malaysia. http://www.e-mjm.org/2016/v71s1/research-
on-child-abuse.pdf.

Hall, L., Hume, C., & Tazzyman, S. (2016). Five Degrees of Happiness.
Proceedings of the 15th International Conference on
Interaction Design and Children.
https://doi.org/10.1145/2930674.2930719

Hassenzahl, M. (2008). User experience (ux): towards an experiential
perspective on product quality.

IHM ‘08: Proceedings of the 20th International Conference of the
Association Francophone d’Interaction Homme-Machine, 11–15.
Metz & New York: ACM.
https://doi.org/10.1145/1512714.1512717 (ver. 10.12.2019).

Huan, R. (2017). Evaluating engagement in digital narratives from facial
data. Proceedings of the 19th ACM International Conference
on Multimodal Interaction.
https://doi.org/10.1145/3136755.3137036

Moen, O. M. (2015). The ethics of pedophilia. Etikk i Praksis - Nordic
Journal of Applied Ethics, 9(1), 111–124.
https://doi.org/10.5324/eip.v9i1.1718

Othman, A., & Wan Yahaya, W. A. (2015). MULTIMEIDA design
principles in developing multimedia learning application (mmla) to
increase children awareness of child sexual abuse. Jurnal
Teknologi, 75(3). doi:10.11113/jt. v75.5036