Research Article
Evaluating Game-Playing Activities of a MOBA Game with Gameplay
Experience Instruments
Maizatul Hayati Mohamad Yatimˡ, Tan Wee Hoe2 1,2Universiti Pendidikan Sultan Idris
maizatul@fskik.upsi.edu.my1, whtan@fskik.upsi.edu.my2
Article History: Received: 10 November 2020; Revised: 12 January 2021; Accepted: 27 January 2021; Published online: 05 April 2021
Abstract: This article reports the findings of a study that aimed to evaluate gamers’ experiences in playing MOBA and
DOTA2 games using gameplayexperience instruments. This study was based on a qualitative approach involving interviews and observations. The study sample consisted of 10 participants who were divided into two teams comprising five players who competed with one another in the best of five matches. Data collection through observations was performed by video recording gamers’ activities during the games and by checking them against a checklist of play experiences. Interviews were then followed that asked the players several pertinent questions to gather their opinions and feedback regarding the common experiences they gained by playing the games. The narrative analysis of recorded activities and interview feedback showed five major factors had significant impacts on players’ overall experience in playing the MOBAgame, namelycompetitiveness, mechanic, socialization, emotion, and immersion.In particular, the mechanics and emotions of the game had the most and least impacts on their overall experience, respectively.
Keywords:DOTA game, MOBA game, user experience (UX), game-play experiences, instrument. 1. Introduction
The gameplay is a form of interaction thorough which players collaborate and interact with a structured form of play (Becker, 2017; Macklin & Sharp, 2016, Salen& Zimmerman, 2004). Such interaction is defined by several components, such as patterns of game rules (Michael & Chen, 2010), interconnections between players and the game (McEvoy, 2016), obstacles and challenges(Denisova et al., 2020), storytelling(Hand, 2018), cognitive skills acquisition (Wasserman & Koban, 2019), and immersive experiences (Muntean et al., 2018). The natural connection between these components and gameplay experience gradually leads to a meaningful play that increases satisfaction among players (Landers et al., 2018; Schrader et al., 2017).
Studies of gameplay experiences in game-playing activities in today’s cultures have been gaining strong traction of late (Abeele et al., 2020; Klabbers, 2018; Deterding, 2017).In the context of gameplay experiences, most studies have attempted to examine the motivation of playing games based on the player-centered design approach(Todello&Nacke, 2019). The approach initially focuses on the need of players to have the option to evaluate game needs, solve game obstacles or challenges, and react with proper choices of decisions with intended skills (Tondello et al., 2019; Turner, 2018). Thus, understanding gameplay experiences is a basic skill that all game developersmustacquire. Essentially, they can design and develop new gameplay or new games by observing how players play and react togameplay experiences in game-playing activities (Wong, Maizatul, Tan & Yap, 2019; May Asliza, 2018; Ghavifekr, 2018; Nijholt, 2014). Moreover, game-playing activities require players to havestrongcritical thinking, aptitude, and skills to examine the levels of logic and rationale of games, fix errors and bugs in programs, focus on game mechanics, aesthetics, playability, and a wide range of subtleties (Nur, Nur, &Laili, 2020; Ampatzidou&Gugerell, 2019; de Souza, Monteiro&Jucá, 2019; Haz, et al. 2018; Mossman, et al., 2017; Farber, 2015).
The following three sections discuss the motivation that compelled the researchers to conduct this study, namely the components of gameplay, the gameplay experience, and the need for game designers to understand the requirements of gameplay. The discussion of this article centers on a deep understanding ofplayers’ gameplay experience in game-playing activities.
2. Motivation
Many consider the study of gameplay experience falls under the user experience (UX) study (Bernhaupt, 2015)that investigates the emotions and attitudes displayed by end-users when using a product, service, or system (Nacke, Mirza-Babaei&Drachen, 2019; Norman & Nielsen, 2019). Some even believe that any study within the UX field is subjective as individual users may experience UX design in different ways (Shin, 2019). Despite such subjectivity, UX is an important factor in designing a product, service, or system, as it “includes the
practical, experiential, affective, and meaningful aspects of human-computer interaction and product ownership.”(Olawole, 2018).
As a product consisting of complex systems, games also require good UX design for players to feel comfortable in using and interacting with them (Ghavifekr, 2018; Beckert; Grebing&Bohl, 2014). In the games research, many studies on UX rely on the knowledge of neuroscience and psychology and apply game user research methodologies (e.g. playtests and analytics) to create games with good usability and immersive property to support a gamified system (Morales, Rusu, Botella&Quiñones, 2019; Hodent, 2017; Evans, 2017).
Over recent years, the findings of UX research have helped designers to understand players and assess their needs to create high-quality games (Craftsman, 2017). A review of the current literature of previous studies of related topics helped guide the researchers to determine the focus of this study. In particular, the researchers analyzed the findings of studies in the literature based on a systematic review approach to address issues relating to gameplay experiences(Wong &Ghavifekr, 2018; Alvarez-Xochihua, et al., 2017).
Arguably, the lack of evidence-based research or publication has motivated many scholars to carry out studies on gameplay experiences, which have grown rapidly due to the technological advancements and the convergence of different systems into several socio-economic domains, effectively paving the way to create gratifying game designs to meet a players’ gaming needs (Azizan et al., 2019; Hartsone&Pyla, 2018). To this end, many studies of gameplay experience have been carried out since early 2000. In this study, the researchers chose several articlesthat deal with the latest gaming trends and technological convergence to analyze the components of the gameplay experience, as shown in Table 1.
Table 1.Analysis of Gameplay Experience Components
Components Ermi& Mäyrä (2005) Wang & Sun (2011) IJsselsteij n, Poels& de Kort (2008) Sánchez et al. (2012) Norman (2013) Nagalingam & Ibrahim (2015) Wang & Goh (2020) Accessories ü Achievement ü After-game activities ü Appeal ü Audio ü ü Balance ü Challenge ü Efficiency ü Emotions ü ü ü Engagement ü ü Fantasy/Imaginative ü ü Feedback/Affect ü ü Flow ü ü Fun ü ü Game rules ü Game space ü Game story/Narrative ü ü ü Immersion ü Interface ü ü Mental skills/Cognitive ü ü ü ü Motivation ü ü ü Motor skills/Competence ü ü ü ü ü Reward ü Satisfaction ü Social Influence ü Social Interaction ü ü ü ü Tension ü ü Time ü Visual ü ü ü
Table 1 summarizes the analysis of components of the gameplay experience based on studies of gameplay experiences.According toErmi&Mäyrä (2005), a gameplay experience can be divided into three dimensions, namely sensory immersion, challenge-based immersion, and imaginative immersion, with each dimension consisting of separate fundamental components that shape a gamer’s gameplay experience as highlighted in Table 1. Wang & Sun (2011) proposed fivedesign considerationsfor a reward system to increase the level of gameplay experience, namely life constraints, autotelic experiences, balance, uncertainty and secrecy, accumulated versus instant feedback, social purposes, and physical world activities.Earlier, IJsselsteijn, Poels& de Kort(2009) developed and validated the Game Experience Questionnaire (GEQ) that consists of seven dimensions of players’ experiences, namely sensory and imaginative immersion, tension, competence, flow, negative affect, positive affect, and challenge. Later, Sánchez et al. (2012) proposed a framework that characterizes the experience using attributes as summarized in Table 1.In 2013,
Norman analyzed two instruments of game engagement and game experience that led to him proposing four factors, namely visual presence, audio presence, sensory engagement, and sense of control, to make a game immersive by focusing on game attributes. Nagalingam&Ibrahim (2015) compiled elements of gameplay experience from related UX models and frameworks. In 2020, Wang &Goh collected several online reviews on resources relating to the feedback of game experiences.
As the game industry ushered in an era of high-budget production, UX designers have been added to the development workflow and game production team (Coleman, 2018).Admittedly, such a change in the game production is inevitable due to the technological growth and the convergence of different systems. As summarized in Table 1, the components of gameplay experience must be re-examined and revised to take into account the advancements of gaming technologies.
Surely, the analysis of the components of gameplay experience canhelp games researchers to continue keeping track and producing suitable models, guidelines, or frameworks within the field of Games User Research (GUR) (Stahlke et al., 2020; Nacke, 2015; Mirza-Babaei et al., 2013, El-Nasr et al., 2013). In addition to concentrating on the specific content to improve games through game evaluation, playtesting, user experience testing, usability testing, and other related game evaluation methods, GUR also explores game testing using physiological evaluation, including electroencephalography (EEG) technology (Drachen, Mirza-Babaei, &Nacke, 2018; Kerous, Skola, &Liarokapis, 2018; Mandryk, Nacke, &Mandryk, 2016).
In this study, the researchers testedgame-playing activities of amultiplayer online battle arena (MOBA) game called DOTA2 using the instruments of gameplay experience, which have been designed and validated specifically forthe competition of electronic games, the design and validation of which are also discussed in this article. In the gaming industry, electronic game tournamentsare also known as esports (or electronic sports), a term coined in 1999 by Online Gamers Association (OGA) (Rogers, 2019).The researchers chose DOTA2 as the game of the study due to its huge popularity among your gamers throughout the world (Gough, 2020). Moreover, this game won the Best MOBA game award at the 2015 Global Game Awards and has continued to be the best MOBA game ever since (Global Game Awards, 2020).
The Origin of DOTA2
Since 1998, Valve Corporation has successfully developed and commercialized many games,including Half-Life, Counter-Strike, Portal, and DOTA2, which evolved from MOBA gameplay (Valve Corporation, 2020). In 2008, Steam was created as a digital content distribution hub or channel to deliver high-end gaming entertainment by connecting players and games distributors. Subsequently, through this hub, DOTA2 was released and emerged as the most widely used game among players all over the world (Gough, 2020).
Prior to DOTA2, a classic real-time strategy game known as Warcraft III: Reign of Chaos (2002) and its successor called Warcraft III: The Frozen Throne (2003) were among the leading multiplayer games developed by Blizzard Entertainment (Blizzard, 2020). Together with Saffire Corporation, Blizzard Entertainment developed a real-time strategy game known as Starcraft: Brood War (2003). Additionally, a custom map known as the Aeon of Strifewas created forStarcraft: Brood War, which is considered to be the first DotA map(Liquipedia, 2018).Inspired by the commercial success of Warcraft III, Blizzard Entertainment features a series of DOTA, the first of which is DotA that permits players to make their own maps utilizing a map-creation tool embedded in the game (Gaming Tools, 2020).Encouraged by the players’ enthusiastic reactions to the map and gameplay, the game had gone through several modifications and expansions using different titles and maps until DOTA2 was released in 2013 (DOTA2, 2020). The evolution of maps involved several changes, which
ranged from simple aspects of the game such as terrain design to complex aspects such as new rules, events, and interactive experiences. Figure 1 shows the map preview of Warcraft III, DotA, and DOTA2.
Figure 1. (left) Warcraft III; (middle) DotA; (right) DOTA2
As stated, DOTA2 was chosen for this study due to its popularity among young gamers of MOBA games. In this study, the participants were also asked to rate their most preferred MOBA games, which showed the top four games were DOTA2, League of Legends, Mobile Legends, and PlayerUnknown's Battlegrounds (PUBG). In the interview sessions, the selected participants indicated that they preferred personal computer (PC) games to mobile games, citing better network accessibility and immersion of the former compared to those of the latter.
3. Research Method
This section describes the data collection method and data analysis used in this study. This qualitative study was based on an exploratory approach to help the researchers betterunderstand gamers’ experiences in playing DOTA2. Given the lack of self-reported studies in gameplay experience (Basto et al., 2019), this study aimed to analyzegamers’ meaningful gameplay experiences gained by playing DOTA2.
Participants
The study involved the analysis of game-playing activities carried out by a study sample consisting of 10 gamers, who were divided into two teams of five players each to compete against one another in five best matches. The participants were purposively selected and screened from a pool of 63 diploma students of Game Design and Development program. Purposive sampling was used to select only ten participants who were categorized as hardcore MOBA gamers. This sampling technique is one of the most cost-effective and time-effective sampling methods available (Osman, 2014). A quick survey was carried out to determine their levels of involvement in playing MOBA games. Additional selection criteria were the number of playing hours in MOBA games, students’ attitude toward game-playing, and their ability to give constructive opinions and suggestions.
Procedures
The study was carried out in three phases, namely Phase 1, Phase 2, and Phase 3. Phase 1 involved the initial requirements process through which the researchers reviewed several articles relating to previous studies. Such a review helped identify appropriate elements of gameplay experience prior to developing the instruments of this study, which were validated by a panel of content experts. This phase also involved the shortlisting of participants and the selection of the MOBA game for the study.
Phase 2 involved several sessions of game-playing activities performed by the 10 selected gamers, who were divided into two teams consisting of five players that competed against one another in the best of five matches. Each match was recorded with a video camera in a closed environment in a computer laboratory. The gameplay from all matches was also automatically saved onto each player’s computer as game logs. Both video recordings and game logs were used as evidence in the next phase of the study.
In every match, every player controlled his or her characters (player’s characters) based on a set of attributes that definedthe play styles, such as strength, agility, and intelligence), and a unique set of five play styles, such as fighting, farming, supporting, pushing, and versatility skills. In DOTA2, players could choose 113 selectable characters (known as heroes). Each team would have to play several roles, such as a damage dealer (who attacks the enemies in a fight), a supporter (who heals and helps teammates), a caster (who relies on magic spells), and others. Figure 2 shows the basic abilities of the selectable characters and their playing styles.
Figure 2.(left)Basic abilities of selectable character; (right) Play style for selectable character After choosing a character, each team began playing the game from their own base, which had a monument called Ancient, located in the southwestern and north-eastern part of the map. True to the name of the game Defense of the Ancients, the goal of the game was “to defend the Ancient from the opposing team, while at the same time trying to destroy their opponent’s”(Valve Corporation, 2020). The total game-playing sessions of the five matches lasted for six hours, with each lasting for almost one hour to finish and declare the round’s winner. All matches were conducted in the daytime in a games lab in the university campus.
Phase 3 involved collecting data through interviews and observations. A focus group approach was used to gather the participantsto collect their feedback on the game-playing activities. Data gathered from the interview sessions were analyzed with a data checklist, which was derived from the recorded video footage captured during the gameplay in all matches. Figure 3 summarizes the flow of the study consisting of three phases and processes.
Figure 3. The phases and sequences of processes Data Collection
As highlighted, the data collection process through observations and interviewswere carried out during the game-playing activities and after they had ended, respectively. Data collected were used to analyzeparticipants’ feedback on common gameplay experiences by using a gameplay experience checklist, which had been prepared in the early stage of the study involving the video recording of observations. Feedback in a form of transcripts from the interview sessionswas examinedusinga narrative analysis, with each narrative being interpreted verbatim using codes. Additionally, the same checklist was used to categorize the specific behaviorsdetected in the video recordings.
Beforethe interviews, a set of interview protocols consisting of a list of interview questions and instructions to guide the participants before, during, and after interviews was prepared. As highlighted, theinterviews were conducted based on the focus group approach to ensure meaningful discussions between the participants and researchers. Also, a letter of consent and a consent form detailing the terms of agreement of confidentiality were given to all participants. Table 2 shows the preparation procedure used to guide the conduct of the interviews in this study.
Table 2.Preparation procedure used for the interview session Procedure
a) Choose a setting with the least distraction. b) Explain the purpose of the interview. c) Address terms of confidentiality. d) Explain the format of the interview.
e) Indicate how long the interview usually takes. f) Provide contact information of the interviewer.
g) Allow interviewee to clarify any doubts about the interview.
h) Prepare methods for recording data such as take notes and audio recording.
Instruments
In this study, two instruments were used, namely a set of interview protocols and a checklist for video observations, the construct validity of which were verified by a panel of five experts, who had more than five years of experiences in game designs and game studies. Three of the experts worked in the game industry, and the remaining two were lecturers in the field of games and creative media. Admittedly, the selected experts came from different academic backgrounds; however, they were experienced gamers and actively involved in game study. Both instruments were sent to the experts through email, and their feedback was received within two weeks.
The instruments’ construct validity was verified statically and logically based on their listed scores. The study used construct validity as a measure of reliability because such a concept has been widely used in social sciencesthat deal with many subjective aspects (Osman, 2014). The analysis of experts’ feedbacks helped identify and select related elements or constructs of gameplay and eliminate those that were unrelated. Additionally, the relationships between elementswere analyzed by focusing on their sub-elements that contributed to their similarities.Table 3 summarizes the experts’ feedback that had been analyzed to highlight the common elements of five major factors that helped measure a players’ overall experience in playing the MOBA game.Also, Pearson’s Correlation Coefficient analysis was used to determine the correlation between the score of each item and the total score that yielded a coefficient of 0.89, signifying that the construct validityof both instruments were high. Table 3 shows a strong agreement among the experts concerningthe indexof the five major factorsof the gameplay experience.
Table 3.Construct validity index both instruments
Item/Elements Expert1 Expert2 Expert3 Expert4 Expert5 Comply CVI
Mechanic ü ü ü ü - 4 0.80
Competitiveness ü ü ü ü ü 5 1.00
Immersive ü ü ü ü ü 5 1.00
Social ü ü ü ü ü 5 1.00
Emotion ü ü - ü ü 4 0.80
* CVI, construct validity index
As shown, bothinstruments had five major factors that helped measure a player’s overall experience in playing the game, namely mechanic, competitiveness, immersion, socialization, and emotion, which were selected based on the literature review and whose construct validity were verified by a panel of experts.
Data Analysis
Data collected from the interview sessionswere analyzed based on a narrative analysis by focusing on the participants’ experiences and opinions.Specifically, the analysis was carried out by interpreting their experiences and opinions verbatim using codes. Additionally, the feedback on the video recordings was analyzed by checking the players’ behaviorsagainst the gameplay experience checklist. As predicted, there were significant differences in their opinions between the two teams, which is consistent with that of another study by Maizatul (2019).
4. Results
Ten participants were divided into two teams to compete in the DOTA2 game. Data relating to their attitudes toward game-playing and their gameplay experiences were collected through interviews and video recording,
respectively, which were then analyzed narratively using codes.The demography of the participants is summarized in Table 4.
Table 4.Participants’ demography data
Group/Participant Age Gender Average Hours in Playing DOTA2 (per day) A1 19 Male 5 A2 18 Male 5 A3 18 Male 6 A4 19 Male 6 A5 20 Male 7 B1 19 Male 5 B2 20 Male 5 B3 18 Female 6 B4 18 Male 5 B5 19 Male 6
In this study, the researchers used a gameplay experience checklist to observe players’ behaviors and actions, which had been previously recorded during the game-playing sessions. As discussed earlier, this checklist helped list five major factors that significantly impacted the players’ overall experiences in playing DOTA2. Table 5 showsthe five main factors, the elements of gameplay experience, observable behaviors, and actions of the players.
Table 5.Five main factors including gameplay experience elements and listed behaviors indicating the gameplay experience in playing DOTA2
Main Factors Gameplay Experience
Elements Behaviour/Actions Mechanic Functionality Game rules Prior experience Functionality Game rules Confidence Competitiveness Challenge Intrinsic motivation Learning Efficiency Flow Complaisant Aggressive
Physical abuse (hitting, fighting, teasing, bullying, angry)
Verbally attack (argue, name calling, teasing, provokes) Immersive Artistic Interaction Interface Story Persona Graphics/visual effects Audio/narrative Mesmeric Manipulative Confuse Forgetful Lost Deeply engage Offensive Social Cooperative Collaborative Teamwork Obliging Uncooperative Blaming others Team spirit Harassment Centre of attention Emotion Psychology Personal Satisfaction Connection Enjoyable Fun/serious fun Calm Discipline Disruptive Interrupts intentionally Attention seekers Telling lies
The study showed that all participants were aggressive in playing DOTA2. In particular, Team A constantly intervenedthe gameplay by teasing and provokingtheir opponents, which occurred without any physical intimidation. The above scenario was discussed during the interview, indicating that both teams verbally attacked one another because they were on the opposite teams and located in the same location, which contrasted with how multiplayer games are played. Overall, the players of both teams played the game fairly in five matches by not interrupting other players intentionally. However, they did talk silently with one another about their gameplay using headphones. Remarkably, both teams showed strong communication skills and team spirit, as exemplified by their restraint in notblaming or criticizing each other when they lost a game.
Both teams put a strong effort to win a game;unfortunately; unfortunately, Team B kept on losing as Team Aseemed more experienced in playing of DOTA2, with two of its players having participated in a DOTA2 competition in the states of Sabah and Perak. Not surprisingly, their players showed some signs of inflated ego as they tauntedthe other team’s members.On a more positive note, players from both teams showed strong interpersonal skills during and outside the games, as demonstrated by their willingness to take turns and follow their friends’ lead in playing the games.
The interview protocols helped elicit respondents’ natural responses when they were asked to state their opinionson the impact of playing DOTA2 on learning. Interestingly, their opinions revolved around some aspects of computing and English learning. They indicated that playing DOTA2 entailed them to make proper decisions to solve obstacles in the game, which reminded them of the skills in making decisions and solving problems that they had learned and acquired in programming classes. As such, playing this challenging game helped them appreciate and value such skills needed in developing efficient algorithms for programs.Additionally, they indicated that playing DOTA2 helped them gain the confidence to converse in English, as indicated by one of the players’ comments that says“I want to learn English while playing DOTA2”.
Likewise, they stressed that playing DOTA2 entailed good communication because they had to respond to questions raised in discussions on public forums, among others. Equally revealing, a majority of the participants indicated that they had to repeat several exercises to achieve their goals, learn from their past mistakes, and learn how to work in a team. They also opined that playing DOTA2 entailed them to have vast experiences in gameplay, which they claimed had been accrued since 2015 when their friends introduced them to the game at the university. Ever since, they continued to play the game, citing their fascination with the characters, maps, and patterns of the game.
5. Conclusion
In the online gaming world, MOBA games, especially DOTA2, have a huge fan base, seemingly suggesting something special about their gameplay that makes them exciting to play. In this study, the researchers used a playtest to gather some feedback on the gameplay of such games from two teams of players who played DOTA2. In the gaming industry, games developers use of the playtest to identify flaws or bugsin games under development. In this study, it was observed that the players kept on repeating several exercises to achieve their goals, learn from past mistakes (after losing a game), and learn to work in a team. Arguably, multiplayer games or MOBA games are not suitable for those who are faint-hearted, given the sexist, racist, or domineering behaviors of players when they play these games, such as DOTA2. However, in this study, the participants did not exhibit such behaviorsdue to several factors, such as they knew each other, they were playing in the same location, and they were being closely watched by the researchers and facilitators. As observed, experienced players were more engrossed and excited in playing DOTA2 as they gained more rewards for their gaming efforts. Furthermore, experienced players tended to be more critical and creative in their gameplay that led to improved self-esteem and self-confidence. These findings suggest that the longer the gameplay of DOTA2, the more experienced playerswill become, allowing them to use gaming characters at a higher level of complexity, ultimately making them more skillful and competent.
6. Acknowledgement
This study was supported by Universiti Pendidikan Sultan Idris through a GeranPenyelidikan Universiti (GPU 2017-0026-109-01).
References
1. Abeele, V.V., Spiel, K.,Nacke, L.E., Johnson, D., &Gerling, K. (2020) Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human-Computer Studies 135, 102370.
2. Alvarez-Xochihua, O., Munoz-Merino, P., Munoz-Organero, M., Kloos, C., & Gonzalez-Fraga, J. (2017). Comparing Usability, User Experience and Learning Motivation Characteristics of Two Educational Computer Games. In proceedings of the 19th International Conference on Enterprise Information Systems. 3, pp. 143-150.
3. Ampatzidou, C. &Gugerell, K. (2019). Participatory game prototyping–balancing domain content and playability in a serious game design for the energy transition. CoDesign, 15(4), 345-360.
4. Azizan, U.H., Yatim, M.H.M., Ibharim, L.F., &Zain, N.Z.M. (2019). Analysis of Game Elements in Digital Educational Game According to Gagne Nine Events of Instruction. International Journal of Academic Research in Business and Social Sciences, 9(7), 131–135.
5. Bastos, A.S., Santos, E., Gomes, G.A.M., &Mourao, M.A. (2019). Evaluating the Use of Affordable User Testing and Visualization Techniques in Level Design of a Hardcore 2D Platform Game. In proceedings of SBGames 2019. Brazil, October 28th – 31th, 2019, 145- 154.
6. Becker, K. (2017). Digital Game-Based Learning: Learning with Games. In Choosing and Using Digital Games in the Classroom (pp. 25-61). Springer, Cham.
7. Beckert, B., Grebing, S., &Bohl, F. (2014). How to Put Usability into Focus: Using Focus Groups to Evaluate the Usability of Interactive Theorem Provers. In C. Benzmuller and B. WoltzenlogelPaleo (Eds.): Eleventh Workshop on User Interfaces for Theorem Provers (UITP 2014). EPTCS 167, pp. 4–13. 8. Bernhaupt, R. (2015). Game User Experience Evaluation. Springer.
9. Blizzard Entertainment (2020). Blizzard Website. Retrieved June 10, 2020, from: https://www.blizzard.com/en-gb/
10. Coleman, K. (2018). Designers' Influence on Attitude Change towards User Experience (UX) in a Software Development Environment.Doctoral dissertation, University of Pretoria.
11. Craftsman, J. (2017). UI/UX Designer’s Daily Sketchbook. New York: CreateSpace Independent Publishing Platform.
12. de Souza Filho, J. C., Monteiro, I. T., &Jucá, P. M. (2019). Game for PLAYability Heuristic Evaluation (G4H-PLAY): Adapting G4H Gamification to Allow the Use of Playability Heuristics. In proceedings of the International Conference on Human-Computer Interaction (pp. 68-84). Springer, Cham.
13. Denisova, A., Cairns, P., Guckelsberger, C., &Zendle, D. (2020). Measuring perceived challenge in digital games: Development & validation of the challenge originating from recent gameplay interaction scale (CORGIS). International Journal of Human-Computer Studies, 137, 102383.
14. Deterding, S. (2017). The pyrrhic victory of game studies: Assessing the past, present, and future of interdisciplinary game research. Games and Culture, 12(6), 521-543.
15. DOTA2 (2020). DOTA2 Website. Retrieved June 10, 2020, from: http://www.dota2.com/international/battlepass/
16. Drachen, A., Mirza-Babaei, P., &Nacke, L.E. (Eds.). (2018). Games User Research. Oxford University Press.
17. El-Nasr, M.S., Desurvire, H., Aghabeigi, B., &Drachen, A. (2013). Game Analytics for Game User Research, Part 1: A Workshop Review and Case Study. IEEE computer graphics and applications, 33(2), 6-11.
18. Ermi, L. &Mäyrä, F. (2005). Fundamental Components of the Gameplay Experience: Analysing Immersion. D.C. Suzanne & J. Jennifer (Eds.). Changing Views: Worlds in Play. Selected Papers of the 2005 Digital Games Research Association (DiGRA) International Conference, pp. 15–27.
19. Evans, D.C. (2017). Bottlenecks: Aligning UX Design with User Psychology. Apress.
20. Farber, M. (2015). Gamify Your Classroom: A Field Guide to Game-Based Learning. New York: Peter Lang.
21. Gaming Tools (2020). Dota Map. Retrieved June 10, 2020, from: https://gaming-tools.com/warcraft-3/dota-maps/
22. Ghavifekr, S. (2018). User Experience Design of History Game: An Analysis Review and Evaluation. International Journal of Distance Education Technologies, 16(3).
23. Global Game Awards (2020). Global Game Website. Retrieved June 10, 2020, from: https://www.game-debate.com/awards/2019
24. Gough, C. (2020). DOTA 2 Peak Concurrent Player Number on Steam 2016-2020. Retrieved June 10, 2020, from: https://www.statista.com/statistics/807457/dota-2-number-players-steam/
25. Hand, J. (2018). Gameplay Narratives: The Storytelling of Play. Doctoral dissertation, ResearchSpace@ Auckland.
26. Hartson, R. &Pyla, P. (2018). The UX Book: Agile UX Design for a Quality User Experience. MA: Morgan Kaufman.
27. Haz, L., Guarda, T., Leon, M., Chalen, M., &Arguello, L. (2018). Playability Applied in Education. In proceedings of the International Conference of Research Applied to Defense and Security, pp. 299-306. 28. Hodent, C. (2017). The Gamer's Brain: How Neuroscience and UX Can Impact Video GameDesign.
CRC Press.
29. IJsselsteijn, W., Poels, K., & de Kort, Y.A.W. (2008). The Game Experience Questionnaire: Development of a self-report measure to assess player experiences of digital games. Eindhoven: TU Eindhoven. FUGA Deliverable D3.3. Technical Report.
30. Kerous, B., Skola, F., &Liarokapis, F. (2018). EEG-based BCI and video games: a progress report. Virtual Reality, 22(2), 119-135.
31. Klabbers, J.H. (2018). On the architecture of game science. Simulation & Gaming, 49(3), 207-245. 32. Landers, R.N., Auer, E.M., Collmus, A.B., & Armstrong, M.B. (2018). Gamification science, its history
and future: Definitions and a research agenda. Simulation & Gaming, 49(3), 315-337.
33. Liquipedia (2018). Liquipedia Website. Retrieved June 10, 2020, from: https://liquipedia.net/dota2/Dota_History/Part_1
34. Macklin, C.& Sharp, J. (2016). Games, Design and Play: A detailed approach to iterative game design. Addison-Wesley Professional.
35. Mandryk, R.L., Nacke, L.E., &Mandryk, R.L. (2016). Biometrics in Gaming and Entertainment Technologies (pp. 197-224). CRC Press, Boca Raton FL.
36. May Asliza, T.Z. (2018).
PenjelasanIdentitiAsasPengaturcaraandanPerubahanKonseptualMelaluiAdeganPermainan. Journal of ICT in Education. 5(1), pp. 48-57.
37. McEvoy, L. (2016). Understanding Socialization for Young Adults with High Functioning Autism Spectrum Disorder Utilizing Massively Multiplayer Online Role-Playing Games. A PhD dissertation. Arizona: Northcentral University.
38. Michael, D. & Chen, S. (2010). Serious Games: Games that Educate, Train, and Inform. Ohio: Course Technology Cengage Learning.
39. Mirza-Babaei, P., Nacke, L.E., Gregory, J., Collins, N., & Fitzpatrick, G. (2013). How does it play better? exploring user testing and biometric storyboards in games user research. In proceedings of the SIGCHI conference on human factors in computing systems (pp. 1499-1508).
40. Morales, J., Rusu, C., Botella, F., &Quiñones, D. (2019). Programmer eXperience: A Systematic Literature Review. IEEE Access, 7, 71079-71094.
41. Mossman, J., Reategu, E., Barbosa, D., Fonseca, R., Cardoso, C., & Valadares, V. (2017). Evaluation of the Usability and Playability of an Exergame for Executive Functions Stimulation and its Development Process. In proceedings of the International Conference on Engineering Psychology and Cognitive Ergonomics, pp. 164-179.
42. Muntean, C.H., El Mawas, N., Bradford, M., & Pathak, P. (2018). Investigating the impact of an immersive computer-based math game on the learning process of undergraduate students. In 2018 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). IEEE.
43. Nacke, L.E. (2015). Games user research and physiological game evaluation. In Game User Experience Evaluation (pp. 63-86). Springer, Cham.
44. Nacke, L.E., Mirza-Babaei, P., &Drachen, A. (2019). User Experience (UX) Research in Games. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-4). 45. Nagalingam, V., & Ibrahim, R. (2015). User Experience of Educational Games: A Review of the
Elements. Procedia Computer Science, 72, 423-433.
46. Nijholt, A. (2014). Playful User Interfaces: Interfaces that Invite Social and Physical Interaction. Singapore: Springer.
47. Norman, D., & Nielsen, J. (2019). The Definition of User Experience (UX). Retrieved June 10, 2020, from https://www.nngroup.com/articles/definition-user-experience/
48. Norman, K.L. (2013). GEQ (Game Engagement/Experience Questionnaire): A Review of Two Papers. Interacting with Computers, 25(4), 278-283.
49. Nur Fasiha, S.Z.T, Nur Azimah, K., &LailiFarhana, M.I. (2020). Penerapan Model RESPECTFUL dalamPembelajaranBerasaskanPermainan Digital bagiKursusKaunselingPelbagaiBudaya. Journal of ICT in Education, 7(1), 1-8.
50. Olawole, D.O. (2018). User experience: tool for Human-Computer Interaction (HCI) design. AFRREV STECH: An International Journal of Science and Technology, 7(2), 61-66.
51. Osman, L. (2014). PenyelidikanKualitatif: PengenalankepadaTeoridanMetode. Tanjong Malim: UPSI. 52. Rogers, R. (2019). Understanding eSports: An Introduction to the Global Phenomenon. Lanham:
Lexington Books.
53. Salen, K. & Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. The MIT Press. 54. Sánchez, J.L.G., Vela, F.L.G., Simarro, F.M., & Padilla-Zea, N. (2012). Playability: Analysing User
Experience in Video Games. Behaviour & Information Technology, 31(10), 1033-1054.
55. Schrader, C., Brich, J., Frommel, J., Riemer, V., & Rogers, K. (2017). Rising to the challenge: An emotion-driven approach toward adaptive serious games. In Serious Games and Edutainment Applications (pp. 3-28). Springer, Cham.
56. Shin, D. (2019). How do users experience the interaction with an immersive screen?. Computers in Human Behavior, 98, 302-310.
57. Stahlke, S.N., Mirza-Babaei, P., Gerling, K., Toups, Z.O., Johnson, D., &Mandryk, R.L. (2020). Games and Play SIG: Shaping the Next Decade of Games & HCI Research. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-4).
58. Tondello, G.F. &Nacke, L.E. (2019). Player Characteristics and Video Game Preferences. In proceedings of the Annual Symposium of Human-Computer Interaction in Play – CHI PLAY ’19. ACM. 59. Tondello, G.F., Kappen, D.L., Ganaba, M., &Nacke, L.E. (2019).Gameful Design Heuristics: A Gamification Inspection Tool. In: Kurosu M. (eds) Human-Computer Interaction. Perspectives on Design. HCII 2019. Lecture Notes in Computer Science, 11566. Springer, Cham.
60. Turner, P., Johnston, E., Kebritchi, M., Evans, S., Heflich, D., & Wang, S. (2018). Influence of Online Computer Games on the Academic Achievement of Nontraditional Undergraduate Students. Cogent Education, 5(1).
61. Valve Corporation (2020). At Valve We Make Games, Stream, and Hardware. Retrieved June 10, 2020, from https://www.valvesoftware.com/
62. Wang, H. & Sun, C.T (2011). Game Reward Systems: Gaming Experiences and Social Meanings. In proceedings of the 8th International Conference on Digital Games Research Association (DiGRA): Think Design Play. 1-15.
63. Wang, X., & Goh, D. H. L. (2020). Components of Game Experience: An Automatic Text Analysis of Online Reviews. Entertainment Computing, 33, 100338.
64. Wasserman, J.A., & Koban, K. (2019). Bugs on the Brain: A Mental Model Matching Approach to Cognitive Skill Acquisition in a Strategy Game. Journal of Expertise/June, 2(2).
65. Wong, S.Y. &Ghavifekr, S. (2018). User Experience Design of History Game: An Analysis Review and Evaluation Study for Malaysia Context. International Journal of Distance Education Technologies.16(3).
66. Wong, Y.S., MaizatulHayati, M.Y., Tan, W.H., & Yap, L.C. (2019). A Game-Based Learning Assessment Framework for Learning Ubiquitous Computational Thinking. In proceedings of the International Conference on Interactive Collaborative Learning (pp. 607-615). Springer, Cham. 67. Yatim, M.H.M. (2019). Lessons Learned In Game Playing Activities Using Playtesting In A
Game-Based Learning Approach. In proceedings of the 8th UPI-UPSI International Conference 2018 (UPI-UPSI 2018). Atlantis Press.
