The Usability Evaluation of Adik Jom Baca: A Mobile Educational Application for
Kindergarten Malay Language Early Reading
Muhammad Modi Lakulu*1, A.A. Zidan2, B.B. Zidan3, A.J. Mohamad4, Abdul Rahim Razali5 1,2,3,4,5Sultan Idris Education University, Malaysia
modi@fskik.upsi.edu.my1
Article History: Received: 10 November 2020; Revised: 12 January 2021; Accepted: 27 January 2021;
Published online: 05 April 2021
Abstract: Many recent research emphasized the potential of mobile educational application for enhancing early reading among kindergarteners. From the Framework of Mobile Application for Kindergarten Early Reading, the researchers was developed a novel and fascinating mobile educational application for kindergarten early reading, Adik Jom Baca. This study explored the usability the mobile application by using Usability Metrics for Mobile Learning User Interface for Children. This metric evaluated the usability of application from three dimensions; effectiveness, efficiency and satisfaction. The evaluation of usability involved 30 kindergarten teachers using Nominal Group Technique (NGT). Results indicated the high level of usability for Adik Jom Baca application.
Keywords:Educational technology, mobile application, usability 1. Introduction
New advancement in educational technology has changed the landscape of early childhood education. The availability of smart devices featured with interactive and fascinating applications promised interesting learning opportunities for young learners (Forzani & Leu, 2012). These applications, if appropriately developed, have a great potential to improve early reading skills among young children (Korat, 2010). Several studies have revealed the potential of mobile application in nurturing early reading skills for young children. For example, a study by Beschorner & Hutchison (2013) on mobile application offers a novel and attractive approach to facilitate reading, writing, listening and speaking skills. This notion is supported by Yahaya & Salam (2014), which depicts that a mobile application is capable of encouraging and motivating young children to identify letters and spelling. Similarly, Neumann & Neumann (2015) have also found that mobile applications possess a great potential in enhancing children’s emergent literacy skills such as alphabet knowledge, print concepts and emergent writing. While, mobile applications (apps) directed towards young aged children have changed the digital media landscape for infants and toddlers (Papadakis, S., Zaranis, N., Kalogiannakis, M., 2019) and factors influencing the use of interactive technologyincluded the quality of the app such that creative and fun apps promoted children’s engagement in learning mathematics. The level of difficulty of an app was a second factor influencing children’s use of interactive technology. If the difficulty level was too challenging, children became disengaged with the app (Miller, Tess, 2018).On the other hands a study involving 26 kindergarten children shows that "How Far Is Up?" can engage pre-literature children while they are reading alone and also when they are reading with an adult. Based on our craft knowledge and study findings, we present a set of design strategies that aim to provide designers with practical guidance on how to create engaging interactive digital storybooks. (Sargeant, B., Mueller, F., 2018)
A mobile application for kindergarten early reading in this study was developed focuses on kindergartener at 4 years old. Specifically, the application was done aiming to enhance early reading skills in Malay language among kindergarten children in Malaysia(Mohamad & Lakulu, 2017). The development of Adik Jom Baca strictly based on A Framework of Mobile Application for Kindergarten Early Reading (Mohamad, Lakulu, & Samsudin, 2016a, 2016b, 2017). Prototype methodology was utilized in the development of the application.
2. Purpose of the Study
This paper aimed to evaluate the usability of a mobile educational application for kindergarten Malay language early reading, Adik Jom Baca.
3. Methodology
Usability evaluation methodology concerns the process of conducting evaluation; for example focus group, interview, laboratory experiment and field study (Hussain & Kutar, 2012). For this study purpose, Modified Nominal Group Technique was chosen for usability evaluation method. The questionnaires was adapted from Usability Metrics for Mobile Learning User Interface for Children(Tahir & Arif, 2014, 2015).
Procedure for Conducting Modified Nominal Group Technique (NGT)
The method used in usability testing is the Modified Nominal Group Technique. The usability testing for Adik Jom Bacamobile educational application involved 30 kindergarten teachers. This test was conducted on kindergarten teachers because kindergarten children are too young and unable to express their opinion (Hanna, Risden, & Alexander, 1997). Therefore, it is difficult to obtain opinion from them as they are unable to measure the characteristics of educational mobile application (Khanum & Trivedi, 2012).
Table 3.1.Procedure conducted a Study Using the Modified Method Nominal Group Technique
No. Steps Descriptions
1 Demonstration of Adik Jom Baca application.
Researchers started as a facilitator of workshop sessions Nominal Group Technique (NGT) to explain and demonstrate Adik Jom Baca application. Participants were given 30 minutes to try the application. 2 Explanation of the components
and elements of Adik Jom Baca.
In this step, the facilitator gave a question and answer period for participants to study things that are not understood and relatively obscure during the demonstration session.
3 Discussions between the study participants
In this step also, the participants allowed to talk among themselves about who carried out the presentation and the items contained in the usability evaluation questionnaire that was supplied in the first step Nominal Group Technique (NGT) earlier. Opinion survey participants were also recorded.
4 Voting through a questionnaire based on the component and the elements that have been presented
Next, in step, the participants are given a period of time to conduct the evaluation process through usability evaluation forms provided. Upon completion of all usability evaluation form is recovered by the facilitator.
5 Analyzing of the findings Facilitator collected the data and enter the draw made in the form of usability evaluation model by the participants into Microsoft Excel for the score and the agreement of the participants to Adik Jom Baca.
4. Results & Discussion Results of Usability Evaluation
Usability evaluation was mainly conducted to measure the usefulness of the mobile applicationfor kindergarten early reading. For this study, the usability testing has used a metric of mobile application for young children, namely Usability Metrics for Mobile Learning User Interface for Children by Tahir & Arif (2014 &2015). This metric evaluates the usability of mobile application through three dimensions; effectiveness, efficiency and satisfaction.
The Effectiveness
Effectiveness describes the way mobile application helps the early learners in accomplishing certain tasks to reach learning objective (Groth & Haslwanter, 2015). Table 4.1 demonstrates the results of usability testing from the dimension of effectiveness.
Table 4.1.The Result of Usability Testing From the Dimension of Effectiveness
No.
Effectiveness Expert Group Score Total
Score Indication Percentage A (n=15) B (n=7) C (n=8) Average Percentage
No.
Effectiveness Expert Group Score Total
Score Indication Percentage A (n=15) B (n=7) C (n=8) Average Percentage loading. 88% 82% 75% 82%
2 I found it easy to use this application.
51 23 24 98 Agree
85% 82% 75% 81%
3 The application provides clear menu.
46 22 22 90 Agree
77% 79% 69% 75%
4 The application provide understandable navigation key.
49 20 23 92 Agree
82% 71% 72% 75%
5 The app provides easy to use touch screen input.
53 22 24 99 Agree
88% 79% 75% 81%
6 The app gives feedback to my action.
51 22 25 98 Agree
85% 79% 78% 81%
7 I was comfortable with the screen orientation.
48 22 23 93 Agree
80% 79% 72% 77%
The analysis indicated that the users have agreed with seven attributes in the effectiveness elements. The users agreed that the application only takes a short time to load. At the same time, they also found that the application is easy to use. Additionally, they agreed that the application provides a clear menu and understandable navigation keys. Navigation keys assist users to move from certain scene to another scene. More than that, users found that the application provides easy to use the touch screen input. Interestingly, users agreed that the application gives feedback to their actions. Finally, majority of users agreed that they are comfortable with the screen orientation.
The Efficiency
The effeciency refers to the attributes of mobile application in supporting learners to meet the learning objectives (Groth & Haslwanter, 2015). In this study, the attributes of this mobile application are information, task, content, level, activities and text. Table 4.2 summarises the result of usability testing from the dimension of efficiency
Table 4.2.The Result of Usability Testing From the Dimension of Efficiency
No. Efficiency
Expert Group Score
Total Score Indication Percentage A (n=5) B (n=7) C (n=8) Average Percentage 1 The information was understandable. 51 19 24 94
Agree
2 It was easy to complete a task.
50 21 28 99
Agree
83% 75% 88% 82%
3 The content matches with
kindergarten curriculum. 52 21 26 99 Agree
87% 75% 81% 81%
4 The application provides different
difficulty level. 47 18 25 90 Agree
78% 64% 78% 74%
5 The learning activities were
interesting. 50 21 29 100 Agree
83% 75% 91% 83%
6 The application use appropriate text
size. 40 19 24 83 Agree
67% 68% 75% 70%
7 It was easy to read the text in this
application. 52 22 28 102 Agree
87% 79% 88% 84%
Most of users have positively perceived the efficiency of Adik Jom Bacamobile educational application. First, they agreed that the information provided was understandable. It is important to give clear information in the children’s educational mobile application. Users also agreed that the tasks in the application are easy to complete. Moreover, users found that the content of the application are aligned with curriculum for kindergarten. Besides, majority of them agreed that the application provides different difficulty level. In terms of learning experience, users found that the learning activities in the application are interesting for kindergarteners. From the aspect of legibility, users agreed that the application uses appropriate text size and suitable for young learners. On top of that, the users found Adik Jom Baca application to have a high readability features since they stated that it was easy to read the text in this application.
TheSatisfaction
In usability testing, satisfaction means the characteristics of mobile application that are affordance and comfortable for users especially young learners (Groth & Haslwanter, 2015). In this study, the characteristics measured are graphic, animation, background music, colours, design, icon and button and rewards. Table 4.3 illustrates the result of usability tesing from the dimension of satisfaction.
Table 4.3.The result of usability testing from the dimension of satisfaction
No.
Satisfaction Expert Group Score
Total Score Indication Percentage A (n=5) B (n=7) C (n=8) Average Percentage 1 I like the graphics used in the
application. 51 21 27 99 Agree
85% 75% 84% 81%
2 I like the animation used in the
application. 51 21 27 99 Agree
85% 75% 84% 81%
3 The application provides appropriate background music.
47 21 27 95
Agree
No.
Satisfaction Expert Group Score
Total Score Indication Percentage A (n=5) B (n=7) C (n=8) Average Percentage 4 The colors used in this
application are attractive. 49 23 27 99 Agree
82% 82% 84% 83%
5 I found the design of
application attractive. 49 21 27 97 Agree
82% 75% 84% 80%
6 The icon and buttons used are
recognizable. 47 20 26 93 Agree
78% 71% 81% 77%
7
The application gives interesting rewards on my performance. 45 21 25 91 Agree 75% 75% 78% 76% 8
Overall I enjoyed using the
application. 46 19 28 93 Agree
77% 68% 88% 77%
From the dimesion of satisfaction, the users obviously agreed on the eight attributes. A huge number of users expressed that they like the graphic and animation used in the application. The users also agreed that the application is featured with appropriate background music. The used of music is important to draw the attention of young learners. Moreover, majority of them found that the colours used and the design of application are attractive. In view of kindergarten teachers, most of them perceived that the icon and button used in the application are easy to be recognised. Attractively, the users found that the application give interesting rewards upon their performances. Overall, most users are satisfied and enjoyed using Adik Jom Bacamobile educational application.
5. Conclusion and Future Work
The usability testing was measuring the agreement of kindergarten teacher on the attributes of usability for mobile application; the effectiveness, efficiency and satisfaction. There are seven attributes each of effectiveness and efficiency and eight attributes of satisfaction. Most of kindergarten teacher agreed on the level of usability Adik Jom Baca because all attributes achieved the average percentage more than 70%. For further work, the researcher may discover the usability aspect form the perspective of the children itself. In other hand the researcher should identify the factors for actively engagement in using mobile application for emergent early reading skills.
6. Acknowledgement
This research is fully supported by GPU grant, 2016-0138-107-01. The authors fully acknowledged Universiti Pendidikan Sultan Idris for the approved fund which makes this important research viable and effective.
References
1. Beschorner, B., & Hutchison, A. (2013). iPads as a literacy teaching tool in early childhood. International Journal of Education in Mathematics, Science and Technology, 1(1), 16–24.
2. Forzani, E., & Leu, D. J. (2012). New literacies for lew learners: The need for digital technologies in primary classrooms. The Educational Forum, 76(4), 421–424. https://doi.org/10.1080/00131725.2012.708623
3. Groth, A., & Haslwanter, D. (2015). Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: a mobile usability study. Information Technology & Tourism, 16(2), 1–28.
https://doi.org/10.1007/s40558-015-0041-0
4. Hanna, L., Risden, K., & Alexander, K. (1997). Guidelines for usability testing with children. Interactions, 4(5), 9–14. https://doi.org/10.1145/264044.264045
5. Hussain, A., & Kutar, M. (2012). Usability evaluation of SatNav application on mobile phone using mGQM. International Journal of Computer Information Systems and Industrial Management Applications, 4, 92–100.
6. Khanum, M. A., & Trivedi, M. C. (2012). Take Care: A Study on Usability Evaluation Methods for Children. International Journal of Advanced Research in Computer Science, 3(2).
7. Korat, O. (2010). Reading electronic books as a support for vocabulary, story comprehension and word reading in kindergarten and first grade. Computers and Education, 55(1), 24–31. https://doi.org/10.1016/j.compedu.2009.11.014
8. Miller, Tess (2018). Developing numeracy skills using interactive technology in a play-based learning environment, International Journal of STEM Education, 5(1),1-12.
9. Mohamad, A. J., & Lakulu, M. M. (2017). A Framework of Mobile Educational Application for Kindergarten Early Reading. The International Journal of Multimedia & Its Applications (IJMA), 9(4), 114–119. https://doi.org/10.5121/ijma.2017.9610
10. Mohamad, A. J., Lakulu, M., & Samsudin, K. (2016a). Determining the Pillars in a Framework of Mobile Application for Kindergarten Early Reading. In 3rd International Postgraduate Conference on Engineering and Technology Research. Langkawi, Malaysia.
11. Mohamad, A. J., Lakulu, M., & Samsudin, K. (2016b). Measuring the Elements of Pedagogy in a Framework of Mobile Apps for Kindergarten Early Reading. In 7th International Conference on Postgraduate Education (pp. 1–6). Shah Alam: Universiti Teknologi Mara.
12. Mohamad, A. J., Lakulu, M., & Samsudin, K. (2017). Determining the Pillars in a Framework of Mobile Application for Kindergarten Early Reading. Journal of Informatics and Mathematical Sciences, 9(4), 1075–1079. https://doi.org/10.26713/jims.v9i4.1007
13. Neumann, M. M., & Neumann, D. L. (2015). The use of touch-screen tablets at home and pre-school to foster emergent literacy. Journal of Early Childhood Literacy, 1–18. https://doi.org/10.1177/1468798415619773
14. Papadakis, S., Zaranis, N., Kalogiannakis, M.(2019). Parental involvement and attitudes towards young Greek children's mobile usage.International Journal of Child-Computer Interaction, 22(Disember), 1-14. 15. Sargeant, B.&Mueller, F. (2018). How Far is Up? Bringing the counterpointed triad technique to digital storybook apps. In Conference on Human Factors in Computing Systems - Proceedings (CHI2018) (pp. 1-12)
16. Tahir, R., & Arif, F. (2014). Framework for Evaluating the Usability of Mobile Educational Applications for Children. In Third International Conference on Informatics Engineering and Information Science (ICIEIS2014) (ICIEIS) (pp. 156–170).
17. Tahir, R., & Arif, F. (2015). A Measurement Model Based on Usability Metrics for Mobile Learning User Interface for Children. International Journal of E-Learning and Educational Technologies in the Digital Media, 1(1), 16–31.
18. Yahaya, N. S., & Salam, S. N. A. (2014). Mobile Learning Application for Children: Belajar Bersama Dino. Procedia - Social and Behavioral Sciences, 155(October), 398–404. https://doi.org/10.1016/j.sbspro.2014.10.312
