Examining Teachers Acceptance of Exam Building Crowdsourcing Platform through
the Utaut Model: The Case of Krumun.Org
Khalizul Khalid1*, Rosmini Ismail2, Jessnor Elmy Mat Jizat3, Bahijah Abas4
1,2,3,4Faculty of Management and Economics, Sultan Idris Education University,Tanjung Malim,
Perak, Malaysia.
khalizul@fpe.upsi.edu.my*1
Article History: Received: 10 November 2020; Revised: 12 January 2021; Accepted: 27January 2021; Published online: 05April 2021
Abstract: The 21st-century learning demands for teachers to become a central agent in fostering various skills to students.
Concurrently, teachers were also encouraged to utilize ICT in scaling up learning quality. In support of this initiative, this study introduced a crowdsourcing platform called Krumun.org developed specifically for Malaysian teachers. This exam-building crowdsourcing platform enables teachers to create, edit, and share assessment instruments. However, whether teachers will use the platform will depend on their acceptance of this new technology. Therefore, the main objective of the study is to determine teachers' acceptance factors in regards to the use of this crowdsourcing platform through the Unified Theory of Acceptance Use Technology (UTAUT) model. Questionnaires were distributed to 155 teachers who participated in Krumun.org trial launch. Partial least squares (PLS) was employed to analyze the research model through Smart-PLS 3. It is conclusive that performance expectancy (PE), self-efficacy (SE), and facilitating conditions (FC) affect behavioral intention to accept the crowdsourcing platform, Krumun.org. However, there was not enough evidence to support Effort Expectancy. Therefore, to increase the number of teachers utilizing this platform, these three factors should be prioritized. The findings of the study could also be of assistance to any related party that was planning to introduce new technology to teachers.
Keywords: Crowdsourcing, assessment instrument platform, UTAUT, PLS.
1. Introduction
The internet became prominent to the public somewhere in the 1990s, and since then, began contributing to human advancement in the physical and virtual world. Consequently, with the rapid change of technology in the last 20 years, it gives birth to something astounding such as the Internet of Things (IoT), Internet of Everything (IoE) and soon, Internet of Nano Things (IoNT) (Miraz, Ali, Excell, & Picking, 2015; Srinivasan, Rajesh, Saikalyan, Premsagar, & Yadav, 2019). Due to this, human capital needs to upgrade their knowledge and skill just to compete with each other. Consequently, to accommodate future generations with what is to come, the 21st Century Learning was formulated in the hope that it could equip them with sufficient technical skill and knowledge. Concurrently, it demands teachers to acquire the same capability to coach and educate their students simultaneously to improve their quality of teaching (Syaripuddin, Ahmad, & Awang, 2019).
Consequently, one of the initiatives to improve teaching is by encouraging teachers to utilize ICT in scaling up learning quality (Ministry of Education Malaysia, 2016). However, reports were documenting low ICT usage among teachers in Malaysia (Ministry of Education, 2016). Arguably, low-level usage may due to issues related to teachers’ acceptance towards the system itself and their motivation to use the system. It is important to note that, determining relevance factors that influence teachers to adopt technology will ensure the success of embedding technology in the classroom since teachers are the ones that will guide their students (Scherer, Siddiq, & Tondeur, 2020). According to Awang et al. (2018), workload, accessibility, competency, and acceptance were significant factors for the continuing use of ICT among teachers.
There are ample studies that demonstrated a strong relationship between users' acceptance, their intention to use, and actual use of the system. These types of studies commonly applied the Technology Acceptance Model (TAM) or Unified Theory of Acceptance and Use Technology (UTAUT) model to study factors of technology acceptance. For example, Holzmann, Schwarz, and Audretsch(2020); and Khlaif(2018) documented that facilitating condition factors can be used to influence teachers to use ICT or novel technology. Akar(2019), on the other hand, found that personal innovativeness is an influential factor for teachers accepting particular technology.
This study intends to investigate factors that influence teachers to accept technology that was specifically developed to accommodate teachers' work-related routine tasks. The termed technology in this study refers to the crowdsourcing platform called Krumun.org. This exam-building crowdsourcing platform enables teachers to create, edit, and share assessment instruments not limited to exam questions but quizzes and tutorials as well. Simultaneously, the platform may benefit teachers by providing them with a tool that eases their workload in building a standard exam paper. Therefore, the main objective of the study is to determine teachers' acceptance
factors in regards to the use of this crowdsourcing platform through the Unified Theory of Acceptance Use Technology (UTAUT) model.
2. Literature Review
The National Philosophy of Education (FPK) (Kementerian Pendidikan Malaysia, 2013) aims to produce a balance generation within all aspects, namely spiritual, emotional, physical, and intellectual. Consequently, teachers are encouraged to improve the quality of teaching by selecting the best strategy and teaching method for ensuring better lessons. In support of this, the Malaysia Education Blueprint 2013 – 2025 outlines Eleven shifts to transform the education system (Ministry of Education Malaysia, 2016). One of them is Shift 7, where the blueprint emphasized on Leveraging ICT to scale up learning quality. Subsequently, it parallels with the Ministry of Education initiative to encourage teachers to apply the 21st-century learning approach in their classroom (Ministry of Education, 2016).
The 21st-century learning demands for teachers to become a central agent in implementing various skills such as excellent in communication, collaborative and problem-solving skills in the real-life (Siti Zaharah Mohid, Roslinda Ramli, Khodijah Abdul Rahman, & Shahabudin, 2018). However, to become this type of agent, teachers themselves need to embrace the very skills that they want to nurture within their students. As a result, the Malaysian education system undergoes changes that give impacts on teachers’ workload. Shafie, Kadir, &Asimiran(2017) emphasized that teacher’s workload in management and administration should be reduced to ensure that they can focus on innovation in teaching and learning. Arguably, using ICT in completing teachers’ routine tasks could also provide room for them to put in more effort for this type of innovation. However, it requires adequate support from administrators, directives to teachers to use ICT, appropriate ICT skills, knowledge, and acceptance as well as adequate resources (Mirzajani, Mahmud, Ayub, & Wong, 2016).
One of the options available to ease teachers' workload is through sharing, editing, and reuse lesson materials within the teachers’ community using ICT. This type of collaboration can be fulfilled through a platform where they can create and share work-related materials to suit their needs. Due to this, the study proposes a solution, namely a crowdsourcing platform, to accommodate teachers through the use of ICT in completing a particular work-related routine task, namely assessment instrument. The crowdsourcing platform may increase the quality of the assessment instrument and simultaneously provide the opportunity for teachers to enhance the effectiveness of teaching and learning processes. Teachers will have access to others’ instruments and share their own through the crowdsourcing platform provided.
Crowdsourcing is a type of participative online activity in which an individual, an institution, a non-profit organization, or company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, to voluntarily undertaking a task (Estellés-Arolas, Navarro-Giner, & González-Ladrón-de-Guevara, 2015) The term “crowdsourcing” was coined by Jeff Howe and Mark Robincin, editors at Wired in 2005. It described how businesses use the internet to “outsource work to the crowd” (Howe, 2006). Brabham (2013), defined it as an "online, distributed problem-solving and production model". It focuses on individuals who desire to solve a problem and willingly share the solution with the community (Estellés-Arolas et al., 2015).
Performance of ideas offered in crowdsourcing platforms is affected not only by their quality, but also by the communication among users about the ideas, and presentation in the platform itself (Guth & Brabham, 2017). Through this type of platform, it could ease teachers' workload by working together collectively in the community to complete a routine task. In recent years, many crowdsourcing platforms were utilized for educational purposes to serve this purpose (Prester, Schlagwein, & Cecez-Kecmanovic, 2019). For example, Dunlap and Lowenthal (2018) invite experienced online educators through a crowdsourcing platform to share their online teaching recommendations. Sadler, Sonnert, Coyle, and Miller (2016) invite participation from the crowd to evaluate their psychometric assessment instrument. Meanwhile, some invite crowd contributions for curriculum development (Scott, 2015), video lessons (Zahirović Suhonjić, Despotović-Zrakić, Labus, Bogdanović, & Barać, 2019) and other learning materials (Sanchez & Peraza, 2020). Most of these studies utilized third-party platforms such as Mechanical Turks, MOOC, and Coursmos.
There were a few that did not use a third party platform and instead developed their own for a specific crowd. Alghamdi, Aljohani, Alsaleh, Bedewi, and Basheri (2015) developed their crowdsourcing platform, CrowdQ, for the creation and evaluation of the examination question. The platform invites the participation of the same subject university instructors as the crowd to develop high-quality exams. In Malaysia, several crowdsourcing platforms for educational purposes were for students used (Amir, Mohd, Saad, Seman, & Besar, 2019), not teachers. The crowdsourcing platform of this study, Krumun.org is introduced exclusively for Malaysian
teachers. We developed our platform for Malaysian secondary teachers to participate in the creation and evaluation of instrument assessment. However, introducing any new technology to an organization or a community requires acceptance from its members (Rienties, Giesbers, Lygo-Baker, Ma, & Rees, 2016; Wong, 2015). Due to this requirement, studies on teachers' acceptance of new technology were ample and wide-ranging (Scherer, Siddiq, & Tondeur, 2020).
In general, there were two dominant models used in measuring technology acceptance, namely the Technology Acceptance Model (TAM) and Unified Theory of Acceptance and Use of Technology (UTAUT). The TAM model was suggested by Davis (1985), which comprises of two main variables, which are perceived usefulness (PU), perceived ease of use (PEU). This model infers that an individual’s adoption of new technology is predicted by their intention to use the innovation which is alternatively defined by an individual’s inherent beliefs about the innovation (Jan, de Jager, Ameziane, & Sultan, 2019). This information is vital in explaining the variance in innovation adoption needed for new technology acceptance by users. Many research is utilizing TAM to investigate technology acceptance among teachers. These include the study of mobile technology acceptance among practicum teachers and students (Hanafi, Zainuddin, Abd Wahab, & Ariffin, 2018; Walker, Kho, Tan, & Lim, 2019), intention to used technology in teaching (Huang & Teo, 2019), and adopting technology in teaching (Scherer, Siddiq, & Tondeur, 2019), to name a few.
Whereas, the Unified Theory of Acceptance and Use of Technology (UTAUT) is the combination of eight main models’ of user acceptance which include TAM as well. UTAUT introduces several variables as factors that influenced individuals within an organization to accept and use new technology (Venkatesh & Davis, 2000). There were four core constructs as direct determinants of user acceptance, namely Performance Expectancy, Effort Expectancy, Social Influence, and Facilitating Condition. Consequently, these constructs were moderated by four variables which were gender, age, experience, and voluntariness of use, these core constructs were employed to explain users’ intention and behavior in accepting and using technology (Venkatesh, Speier, & Morris, 2002).
The model has been applied countless times in various settings (Williams, Rana, & Dwivedi, 2015), including in educational organizations. Examples of these were investigating the use of ICT for research and learning among students (Garcia, Aunario, & Handriyantini, 2019), within the scope of social learning systems (Graham, Stols, & Kapp, 2020) and, ICT integration in teaching practices (Abd Manan & Hanafi, 2019; Khechine, Raymond, & Augier, 2020). Several studies use the UTAUT model to examine factors that motivate teachers to accept new technology. For example, Nandwani and Khan (2016) found that four out of eight factors, namely social influence, facilitating conditions, individual self- efficacy and attitude have a significant and positive impact on teachers’ intention to use technology.
In Malaysia, a study conducted by Raman et al. (2014) investigates teachers’ acceptance factors of smartboard use in school. A total of 68 teachers from five schools participate in the study. It was found a significant positive influence between the Performance Expectancy factor) and the Facilitating Conditions factor towards Behavioral Intention to use the SmartBoard during their teaching and learning process. It could be argued that, determining factors that motivate teachers to accept new technology, especially one that might help ease their workload will ensure r continuing use of ICT among teachers (Awang et al., 2018).
Therefore, the study proposes to determine factors that motivate teachers to accept new technology, namely the crowdsourcing platform Krumun.org that enables creating and sharing of instrument assessment within the teaching community. It is important to note that the crowdsourcing platform itself is just a tool to examine and understand these factors. This initiative can only be successful if we can understand the “what” and “why” factors of teachers accepting and using this type of platform.
3. Methodology
Krumun.org is a platform developed specifically for Malaysian teachers to participate in crowdsourcing activities. This platform was initiated to encourage teachers to share and use assessment materials. The platform is build using PHP and JavaScript language and MySQL for database management. Krumun.org is a platform controlled by an administrator who will monitor all the activities on Krumun.org. The platform has been introduced to a focus group consisting of 155 teachers during a trial launch. A Facebook face is created to enhance promotion and gathered participants to the platform.
Figure 1. Krumun.org Main Page
Participants have to register to Krumun.org to get into the application. Figure 1 is the main page of krumun.org interface, a potential participant has to click “register now” to get into the registration page (figure 2). A potential participant needs to fulfill all the detail needed such as name, gender, mobile number, appointment, and teachers' details and their school name, etc. All the information is needed to ensure the security of the system is guaranteed, only registered teachers will be approved by the system’s administrator. The List of schools all over Malaysia was stored and teachers have to search for their school using the search menu provided (Figure 3). If their school is not listed, the have to add the new school, and the administrator will approve it once the information is verified. All new registration will be check by the administrator by making a manual check to schools the potential participant claim they were. An approved participant is notified by email.
Figure 3. School search tool for teachers
Once approved, the participant may use the system by setting up their subjects and the level of their classes. They can only access to the subject and level they select. After completing the setting, now they can use the system in full. Teachers may build, share, edit, comment test questions in krumun.org. Each question created is stored in krumun.org database and tag by the creator’s name, schools, date, and time created. Participants can compile questions, edit questions or comment questions, and print or store them at the platform for future use. Krumun.org allows the participant to save questions into their local storage in either .pdf or .doc format. All participant activities are recorded for monitoring and research purposes. Permission from participants is acquired prior to approval processes.
The Instrument
The study examines the Krumun.org users' acceptance. Questionnaires were distributed to teachers that participate in Krumun.org platform, which consists of demographic characteristics and UTAUT constructs (Venkatesh, Morris, Davis, & Davis, 2003). Items representing each UTAUT constructs is illustrated in Table 1.The construct comprised of four predictor variables, namely performance expectancy (PE), effort expectancy (EE), facilitating condition (FC) and an additional construct, self-efficacy (SE). The response variable is the intention to use Krumun.org or denoted as Behaviour Intention (BI). Since the platform is at an experimental stage; therefore, one of the original UTAUT constructs, social influence (SI) variable is inappropriate to be measured currently and therefore omitted.
Table 1. Items and description of UTAUT constructs
Construct Code Item Description
Performance expectancy (PE)
PE1 Using Krumun.org will increase the quality of my work Krumun.org assist work performance PE2 Using Krumun.org will improve my overall work performance
PE3 Building exam questions through Krumun.org will increase the quality of my instrument.
PE4 Building exam questions through Krumun.org will increase my skills in regards to instrument building.
PE5 Building exam questions through Krumun.org will increase my knowledge in regards to instrument building.
Effort Expectancy (EE)
EE1 It is easy to register in Krumun.org Krumun.org
ease of use EE2 It is easy to build questions in Krumun.org
EE3 It is easy to edit questions in Krumun.org
EE4 It is easy to discuss with others through Krumun.org EE5 It is easy to generate output from Krumun.org Facilitating
Condition (FC)
FC1 I have adequate technical skill to use Krumun.org. Have adequate knowledge, skill and support to facilitate the use of Krumun.org. FC2 I have adequate technology equipment to use Krumun.org.
FC3 I believe I will have the support of my organization to use Krumun.org
assessment instrument.
FC5 Krumun.org helpdesk is sufficient to assist me to troubleshoot technical issues.
Self Efficacy SE1 I am able to use Krumun.org on my own. Able to learn and use Krumun.org without
assistance. SE2 I am able to teach others to use Krumun.org.
SE3 I am willing to allocate my time to explore Krumun.org. SE4 I can be skillful in using Krumun.org.
Behavioral Intention (BI)
BI1 I intend to explore further the functions of Krumun.org. Intention to participate in Krumun.org. BI2 I intend to download other teachers’ instruments.
BI3 I intend to check and edit others’ instruments in Krumun.org. BI4 I intend to use Krumun.org to build my assessment
instruments for the present school term.
BI5 I intend to use Krumun.org in the future continuously.
A pilot study was conducted to improve the questionnaire’s validity before the main data collection. Running collinearity test, and factor analysis resulted in four items were dropped from the questionnaire. Two items (PE2 and EE2) have collinearity issues with VIF above 3.3 (Kock, 2015), and another two items (PE4 and SE4) removed due to factor loading less than 0.4 (Hair, Ringle, & Sarstedt, 2011; Hulland, 1999).
The Model and Hypotheses
The study examines four relationships through five UTAUT constructs, as depicted in Figure 4. In general, PE measures the degree that the system may improve work-related tasks and EE determines the systems’ ease of use. Facilitating condition relates to organizational and technical infrastructure supporting the use of the system, and self-efficacy measures users' willingness and capability to use the system on their own. Finally BI indicates users' intention in using the systems. Consequently, the study hypothesized that all four variables, PE, EE, FC, and SE, can be used to predict the intention to use the system, Krumun.org. The following were hypotheses to be tested in the study.
Figure 4. Propose Model H1: Performance expectancy predicts the intention to use Krumun.org H2: Effort expectancy predicts the intention to use Krumun.org H3: Facilitating condition predicts the intention to use Krumun.org H4: Self-efficacy predicts the intention to use Krumun.org
The Analysis
Part A of the questionnaire which relates to respondents’ background was analyzed using SPSS for descriptive and inferential statistics. Meanwhile, partial least squares (PLS) was employed to analyze the research model through Smart-PLS 3. The study employed a two-stage analysis that involved the assessments of the measurement model for validity and reliability.
The analysis includes item loading, variance inflation factors (VIF), Cronbach alpha, andcomposite reliability (CR), convergent validity (CV), and discriminant validity (DV). Next, the assessment of the structural model was conducted to test the hypotheses.
4. Findings
There were 155 teachers participate in Krumun.org trial launch. Table 2 illustrate the distribution of participants according to demographic characteristics. Through descriptive analysis, the mean scores documented of each construct were at least 7.80 (on a scale 1-10), indicating a high degree of acceptance of the system. The highest mean score was PE (8.54), and the lowest was SE (7.80). Meanwhile, for inferential analysis, the study found that even though females have a higher mean score for PE and EE than males, the differences were not significant. Overall, there were no significant differences for all five constructs’ mean scores between male and female. Additionally, there was no significant difference for any of the demographic variables between groups.
Table 2. Respondens’ background
Background Frequency Percentage
Gender Lelaki 47 30.3 Perempuan 108 69.7 Age 21 - 25 21 13.5 26 - 30 56 36.1 31 - 35 31 20.0 36 - 40 29 18.7 Above 40 18 11.6 Teaching experience <3 years 39 25.2 4 - 10 years 52 33.5 11 - 17 years 45 29.0 Above 17 years 19 12.3
Field Art and Language 67 43.2
Humanities and Social Science 56 36.2
Science and Technology 32 20.6
Jumlah 155 100.0
Meanwhile, Table 3 illustrates the measurement model results. Factor analysis item loadings for all constructs were ranging from 0.647 to 0.913. By rule of thumb, the loading of 0.7 or higher is required, however items with loading of above 0.5 can be retained (Hulland, 1999). Another reference suggested that outer loading between 0.4 and 0.7 should be considered for removal from the scale only when deleting the indicators leads to an increase in the composite reliability (Hair Jr, Hult, Ringle, & Sarstedt, 2017, pp. 113-114). Following this recommendation, the items below 0.7 were initially removed. However, the removal did not increase the composite reliability values, and as a result, all items were maintained.
Consequently, all Variance Inflation Factor (VIF) values were lower than 3.3 and therefore, indicating free of common method bias (Kock, 2015). Meanwhile, construct reliability measured through Cronbach Alpha and Composite Reliability indicates how well its items measure a construct. Both the Cronbach’s alpha and composite reliability (CR) values have met the recommended values of between 0.70 and 0.90, thus indicating satisfactory reliability (Hair Jr et al., 2017). The final column of Table 3 showing the results for Average Variance Extracted (AVE), which measure convergent validity (CV). AVE value of 0.50 or higher indicates the construct explain more than half of the variance of its indicator (Hair Jr et al., 2017). The study’s AVE was above the cut off value of 0.5 for each construct and therefore indicating Convergent Validity (Fornell & Larcker, 1981).
Table 3. Results of the measurement model
Construct Items Item
loading VIF Cronbach Alpha >0.7 Composite Reliability >0.7 AVE >0.5 Performance PE_1 0.898 1.8 0.725 0.844 0.647
Expectancy (PE) PE_3 0.823 1.665 PE_5 0.676 1.241 Effort Expectancy (EE) EE_1 0.774 2.25 0.817 0.880 0.647 EE_3 0.862 1.894 EE_4 0.798 1.561 EE_5 0.780 1.563 Facilitating Condition (FC) FC_1 0.729 2.091 0.732 0.833 0.557 FC_2 0.646 1.342 FC_3 0.875 1.623 FC_4 0.718 1.253
Self Efficacy (SE)
SE_1 0.863 2.289 0.845 0.906 0.763 SE_2 0.842 1.958 SE_3 0.914 1.955 Behaviour Intention (BI) BI_1 0.655 1.906 0.803 0.864 0.561 BI_2 0.685 2.113 BI_3 0.817 2.061 BI_4 0.793 1.655 BI_5 0.783 1.721
Next, is the assessment of the Discriminant Validity (DV), which is the square root of the AVE of each latent construct that was compared with its inter-construct correlation. The square root of the AVE of a construct should be higher than its correlations with other constructs toachieve satisfactory DV (Fornell and Larcker, 1981; Hair et al., 2017). As shown in Table 4, for each construct, the square root of the AVE (shown diagonally with bold values) exceeded the inter-construct correlations, thereby indicating an appropriate level of DV.
Table 4. Discriminant Validity
Construct BI EE FC PE SE
Behaviour Intention (BI) 0.749
Effort Expectancy (EE) 0.630 0.804
Facilitating Condition (FC) 0.706 0.592 0.747
Performance Expectancy (PE) 0.740 0.714 0.617 0.804
Self Efficacy (SE) 0.606 0.523 0.502 0.386 0.873
The second stage of the analysis is the assessment of the structural model that tests the hypothesized relationships. The result is illustrated in Figure 6 and summarized in Table 5 showcased the relationship between variables. Overall, all of the hypotheses were supported except for H2 (β=--0.031, p>0.05. The individual construct pathcoefficient revealed that Performance Expectancy has the highest value(β=-0.473, p<0.01), followed by self-efficacy (β=-0.297 p<0.01) and later facilitating condition(β=-0.283, p<0.01). The value of R2 indicates the strength of the model. In this study, the value of R2 is 0.711, which implies that the independent variables can predict around 71.1% of the dependent variable, BI.
Figure 6. PLS Algorithm Result Table 5. Results of Structural Model
Path Coefficients t value p values R2 adj. R2
PE - BI 0.473 6.55 0.000
0.711 0.703
EE - BI -0.031 0.391 0.696
FC - BI 0.297 5.288 0.000
SE - BI 0.283 4.706 0.000
5. Discussions and Conclusions
The descriptive analysis of the four constructs demonstrates that teachers have high acceptance towards the use of the crowdsourcing platform, Krumun.org. Further results suggested that three of the hypotheses are supported while one is rejected. Performance Expectancy (PE), Facilitating Condition (FC), and Self Efficacy (SE) can be used to predict teachers Behavioural Intention. Meanwhile, there was not enough evidence to suggest that Effort Expectancy having a significant relationship with Behavioural Intention. This result is consistent with some of the teacher technology acceptance studies where EE has an insignificant effect on the intention to use technology (Nandwani & Khan, 2016; Raman et al., 2014).
The overall result indicates that teachers' behavioral intention to use Krumun.org is mainly influenced by how they perceived the platform might improve their work-related task. Next is the self-efficacy factor which signifies users' capability and willingness to explore the platform on their own. Acceptance to use Krumun.org also influenced by this factor where a positive relationship denotes it, thus indicating the higher self-efficacy, the greater the acceptance. Finally, the technical infrastructure to support the use of the platform was also a factor that motivates teachers to accept it. It is conclusive that performance expectancy, self-efficacy, and facilitating conditions affect behavioral intention to accept the crowdsourcing platform, Krumun.org..
Therefore, to increase the number of teachers utilizing this platform, these three factors should be prioritized. Consequently, if any related party were to introduce new technology to teachers, the findings of the study could be of assistance.
6. Acknowledgments
This work was supported by the Research Management & Innovation Centre, Sultan Idris Education University, Malaysia under the Fundamental Special Grant Scheme 2017-0204-106-01.
References
1. Abd Manan, N. Z., & Hanafi, H. F. (2019). Google Classroom: Student’s Acceptance using UTAUT Model. Journal of Applied Arts, 1(1), 64-72.
2. Akar, S. G. M. (2019). Does it matter being innovative: Teachers’ technology acceptance. Education and Information Technologies, 24(6), 3415-3432.
3. Alghamdi, E. A., Aljohani, N. R., Alsaleh, A. N., Bedewi, W., & Basheri, M. (2015). CrowdyQ: a virtual crowdsourcing platform for question items development in higher education. Paper presented at the Proceedings of the 17th International Conference on Information Integration and Web-based Applications & Services.
4. Amir, R. I. M., Mohd, I. H., Saad, S., Seman, S. A. A., & Besar, T. B. H. T. (2019). Perceives Ease of Use, Perceive Usefulness, and Behavioural Intention: the Acceptance of Crowdsourcing Platform by Using Technology Acceptance Model (TAM): EasyChair.
5. Awang, H., Aji, Z. M., Yaakob, M. F. M., Osman, W. R. S., Mukminin, A., & Habibi, A. (2018). Teachers’ intention to continue using Virtual Learning Environment (VLE): Malaysian context. JOTSE, 8(4), 439-452.
6. Brabham, D. C. (2013). Crowdsourcing: Mit Press.
7. Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results. Massachusetts Institute of Technology.
8. Dunlap, J., & Lowenthal, P. (2018). Online educators\u2019 recommendations for teaching online: Crowdsourcing in action. Open Praxis, 10(1), 79-89.
9. Estellés-Arolas, E., Navarro-Giner, R., & González-Ladrón-de-Guevara, F. (2015). Crowdsourcing fundamentals: Definition and typology Advances in crowdsourcing (pp. 33-48): Springer.
10. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of marketing research, 18(1), 39-50.
11. Garcia, J. G., Aunario, C. C., & Handriyantini, E. (2019). ICT Infrastructure Set and Adoption of Filipino and Indonesian SHS Students: Application of UTAUT. Paper presented at the 2019 Fourth International Conference on Informatics and Computing (ICIC).
12. Graham, M. A., Stols, G., & Kapp, R. (2020). Teacher Practice and Integration of ICT: Why Are or Aren't South African Teachers Using ICTs in Their Classrooms. International Journal of Instruction, 13(2).
13. Guth, K. L., & Brabham, D. C. (2017). Finding the diamond in the rough: Exploring communication and platform in crowdsourcing performance. Communication Monographs, 84(4), 510-533.
14. Hair Jr, J. F., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) (2nd edition ed.). Los Angeles: Sage publications.
15. Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing theory and Practice, 19(2), 139-152.
16. Hanafi, H. F., Zainuddin, N. A., Abd Wahab, M. H., & Ariffin, A. H. (2018). Technology Acceptance of a Novel Mobile Learning Application among University Undergraduates. International Business Education Journal, 11(1), 16-24.
17. Holzmann, P., Schwarz, E. J., & Audretsch, D. B. (2020). Understanding the determinants of novel technology adoption among teachers: the case of 3D printing. The Journal of Technology Transfer, 45(1), 259-275.
18. Howe, J. (2006). The rise of crowdsourcing. Wired magazine, 14(6), 1-4.
19. Hulland, J. (1999). Use of partial least squares (PLS) in strategic management research: A review of four recent studies. Strategic management journal, 20(2), 195-204.
20. Jan, M. T., de Jager, J. W., Ameziane, A. M., & Sultan, N. (2019). Applying technology acceptance model to investigate the use of smartphone advertising in Malaysia. Journal of Economics and Behavioral Studies, 11(1 (J)), 202-210.
21. Khechine, H., Raymond, B., & Augier, M. (2020). The adoption of a social learning system: Intrinsic value in the UTAUT model. British Journal of Educational Technology.
22. Khlaif, Z. (2018). Teachers' perceptions of factors affecting their adoption and acceptance of mobile technology in K-12 settings. Computers in the Schools, 35(1), 49-67.
23. Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of e-Collaboration (ijec), 11(4), 1-10.
24. Ministry of Education Malaysia. (2016). Malaysia Education Blueprint 2013 - 2025. (Pendidikan Prasekolah hingga Lepas Menengah). Putrajaya: KPM.
26. Miraz, M. H., Ali, M., Excell, P. S., & Picking, R. (2015). A review on Internet of Things (IoT), Internet of everything (IoE) and Internet of nano things (IoNT). Paper presented at the 2015 Internet Technologies and Applications (ITA).
27. Mirzajani, H., Mahmud, R., Ayub, A. F. M., & Wong, S. L. (2016). Teachers’ acceptance of ICT and its integration in the classroom. Quality Assurance in Education.
28. Nandwani, S., & Khan, S. (2016). Teachers’ intention towards the usage of technology: an investigation using UTAUT model. Journal of Education & Social Sciences, 4(2), 95-111.
29. Prester, J., Schlagwein, D., & Cecez-Kecmanovic, D. (2019). Crowdsourcing for Education: Literature Review, Conceptual Framework, and Research Agenda.
30. Raman, A., Don, Y., Khalid, R., Hussin, F., Omar, M. S., & Ghani, M. (2014). Technology acceptance on smart board among teachers in Terengganu using UTAUT model. Asian Social Science, 10(11), 84. 31. Rienties, B., Giesbers, B., Lygo-Baker, S., Ma, H. W. S., & Rees, R. (2016). Why some teachers easily
learn to use a new virtual learning environment: a technology acceptance perspective. Interactive Learning Environments, 24(3), 539-552.
32. Sadler, P. M., Sonnert, G., Coyle, H. P., & Miller, K. A. (2016). Identifying Promising Items: The Use of Crowdsourcing in the Development of Assessment Instruments. Educational Assessment, 21(3), 196-214.
33. Sanchez, A., & Peraza, J. (2020). Self-sustaining Crowdsourcing: Beyond the Wikipedia Model to Make pK-12 Computer Science Education Universal in Developing Countries. EPiC Series in Education Science, 3, 200-203.
34. Scherer, R., Siddiq, F., & Tondeur, J. (2019). The technology acceptance model (TAM): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education. Computers & Education, 128, 13-35.
35. Scherer, R., Siddiq, F., & Tondeur, J. (2020). All the same or different? Revisiting measures of teachers' technology acceptance. Computers & Education, 143, 103656.
36. Scott, C. (2015). Designing Mathematics Instruction Utilizing Crowdsourcing as a Professional Development Model. Journal of Higher Education Theory & Practice, 15(2).
37. Shafie, S., Kadir, S. A., & Asimiran, S. (2017). Workload of Technical Secondary School Teachers: Management and Administration’s Perceptions. MOJEM: Malaysian Online Journal of Educational Management, 2(4), 21-35.
38. Siti Zaharah Mohid, Roslinda Ramli, Khodijah Abdul Rahman, & Shahabudin, N. N. (2018, 7 August 2018). Teknologi Multimedia dalam Pendidikan Abad 21. Paper presented at the 5th International Research Management & Innovation Conference (5th IRMIC 2018), Palm Garden Hotel, Putrajaya. 39. Srinivasan, C., Rajesh, B., Saikalyan, P., Premsagar, K., & Yadav, E. S. (2019). A review on the
different types of Internet of Things (IoT). Journal of Advanced Research in Dynamical and Control Systems, 11(1), 154-158.
40. Syaripuddin, R., Ahmad, A. R., & Awang, M. M. (2019). The Use of Video in Teaching and Learning 21st Century History Education in Malaysia. Paper presented at The 2nd International Conference on Sustainable Development & Multi-Ethnic Society.
41. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204.
42. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 425-478.
43. Venkatesh, V., Speier, C., & Morris, M. G. (2002). User acceptance enablers in individual decision making about technology: Toward an integrated model. Decision Sciences, 33(2), 297-316.
44. Walker, Z., Kho, H. H., Tan, D., & Lim, N. (2019). Practicum teachers’ use of mobile technology as measured by the technology acceptance model. Asia Pacific Journal of Education, 1-17.
45. Williams, M. D., Rana, N. P., & Dwivedi, Y. K. (2015). The unified theory of acceptance and use of technology (UTAUT): a literature review. Journal of Enterprise Information Management, 28(3), 443-488.
46. Wong, G. K. (2015). Understanding technology acceptance in pre-service teachers of primary mathematics in Hong Kong. Australasian Journal of Educational Technology, 31(6).
47. Zahirović Suhonjić, A., Despotović-Zrakić, M., Labus, A., Bogdanović, Z., & Barać, D. (2019). Fostering students’ participation in creating educational content through crowdsourcing. Interactive Learning Environments, 27(1), 72-85.
