AUGMENTED REALITY CHILDREN’S BOOK
A THESIS SUBMITED TO THE GRADUATE
SCHOOL OF APPLIED SCIENCES
OF
NEAR EAST UNIVERSITY
By
AHMED LOQMAN MUSTAFA ALYOUSIFY
In Partial Fulfilment of the Requirements for
the Degree of Master of Science
in
Software Engineering
NICOSIA, 2019
AHM E D LO QMA N M UST AF A ALYOUS IFY AU GMENTE D R E ALIT Y C HIL DR E N’ S B O OK NEU 2019AUGMENTED REALITY CHILDREN’S BOOK
A THESIS SUBMITED TO THE GRADUATE
SCHOOL OF APPLIED SCIENCES
OF
NEAR EAST UNIVERSITY
By
AHMED LOQMAN MUSTAFA ALYOUSIFY
In Partial Fulfilment of the Requirements for
the Degree of Master of Science
in
Software Engineering
Ahmed Loqman Mustafa ALYOUSIFY: AUGMENTED REALITY CHILDREN’S BOOK
Approval of Director of Graduate School of Applied Sciences
Prof. Dr. Nadire ÇAVUŞ
We certify this thesis is satisfactory for the award of the degree of Masters of Science in Software Engineering
Examining Committee in Charge:
Prof. Dr. Rahib Abiyev Department of Computer Engineering, NEU
Supervisor, Department of Information Systems Engineering, NEU
Assoc. Prof. Dr. Kamil Dimililer Department of Automotive Engineering, NEU
I hereby declare that all information in this document has been obtained and presented in accordance with academic rules and ethical conduct. I also declare that, as required by these rules and conduct, I have fully cited and referenced all material and results that are not original to this work.
Name, Last Name: AHMED LOQMAN MUSTAFA ALYOUSIFY
Signature:
ii
ACKNOWLEDGMENTS
First and foremost, I want to thank God for giving me the opportunity to study at this University and specially at the department of Software Engineering and helping me throughout my life and guided me every time.
I want to thank my family, my Mom and Dad for believing in me and pushing me to my limits to achieve the goals I have set for myself. I will forever be thankful to you and will never forget your actions and prayers.
To my dear Supervisor and Advisor, Assist. Prof. Dr. Boran Sekeroglu, I am proud and thankful that in this journey, I have met you and any word of wisdom that you have taught me will be with me and will teach it to others.
Thanks to Lana for her voice used in this project.
To all my friends and relatives who prayed for me and were with me throughout my study.
iii
iv
ABSTRACT
This thesis describes the development of an Augmented Reality Children’s Book. The application uses Augmented Reality technology to track Image Targets on the book and display 3D objects and models of the Turkish Alphabet and each alphabet has an example with it for better understanding the use of that specific letter from the Alphabet. The phonetic is also considered in this application. When an Image target is found, along with the 3D model of the letter of the Alphabet, the sound (Phonetic) of that letter is played. The application was developed using Unity3D game Engine with Vuforia SDK, and 3D models of Turkish Alphabets and the design of the book were developed using Adobe Photoshop. The developed system is the first AR Turkish Alphabet Learning Tool attempting to teach children of age 5 to 7 the Turkish Alphabets and Phonetic by using 3D objects, 3D letters, and sound of the pronunciation of each letter and object in the book.
Keywords: Augmented Reality; virtual reality; mobile learning; language learning; turkish language learning; children’s book; teaching children
v
ÖZET
Bu tez, Artırılmış Gerçeklik Çocuk Kitabı'nın geliştirilmesini anlatmaktadır. Uygulama, Kitaptaki Resim Hedeflerini izlemek ve Türk Alfabesinin 3B nesnelerini ve modellerini görüntülemek için Artırılmış Gerçeklik teknolojisini kullanır ve her bir alfabe, Alfabedeki söz konusu harfin kullanımını daha iyi anlamak için buna bir örnektir. Fonetik de bu uygulamada kabul edilir. Bir Görüntü hedefi bulunduğunda, Alfabenin harfinin 3D modeli ile birlikte, o harfin sesi (Fonetik) çalınır. Uygulama Vuforia SDK ile Unity3D oyun motoru kullanılarak geliştirildi ve Türk Alfabelerinin 3 boyutlu modelleri ve kitabın tasarımı Adobe Photoshop kullanılarak geliştirildi. Geliştirilen sistem, 5-7 yaş grubundaki çocuklara Türk alfabelerini ve fonetiği 3D nesneler, 3D harfler ve kitaptaki her harf ve nesnenin telaffuz seslerini kullanarak öğretmeye çalışan ilk AR Türk Alfabesi Öğrenme Aracıdır.
Anahtar Kelimeler: Artırılmış Gerçeklik; sanal gerçeklik; mobil öğrenme; dil öğrenmek; türkçe öğretimi; çocuk kitabı; çocuklara öğretim
vi TABLE OF CONTENTS ACKNOWLEDGMENTS ... ii ABSTRACT ... iv ÖZET ... v TABLE OF CONTENTS ... vi
LIST OF TABLES ... viii
LIST OF FIGURES ... ix
LIST OF ABBRIVATIONS ... xi
CHAPTER 1: INTRODUCTION 1.1 The Importance of Phonics Reading to the Child ... 2
1.2 Problem Statement ... 3
1.3 The Aim of the Thesis... 3
1.4 Significance of the Thesis ... 3
1.5 Limitations of the Study... 4
1.6 Overview of the Thesis ... 4
CHAPTER 2: LITERATURE REVIEW……… 5
CHAPTER 3: METHODOLOGY 3.1 Theoretical Framework ... 10
3.2 Augmented Reality ... 10
3.2.1 Objectives of augmented reality………..……….………... 11
3.2.2 Types of augmented reality………..……….……. 11
3.2.3 Importance of augmented reality ... 13
3.2.4 Advantages of augmented reality…..……….………... 15
3.2.5 Augmented Reality Software Development Kits (SDKs)………....…... 16
3.3 Virtual Reality ... 19
3.4 Mobile Phones/ Smartphones ... 22
CHAPTER 4: SYSTEM DEVELOPMENT AND IMPLIMENTATION 4.1 System Development ... 24
vii
4.1.1 Architectural System Design ... 25
4.1.2 System description ... 26
4.2 Implementation of AR Children’s Book in Unity3D with Vuforia SDK ... 28
4.2.1 Steps for configuring Vuforia into the project ... 28
4.2.2 Steps of development in Unity3D Editor ... 31
4.3 Implementation of the Application ... 35
4.3.1 Pages of AR Children’s Book ... 36
4.3.2 AR Children’s Book in Use ... 37
CHAPTER 5: RESULTS AND FINDINGS………..………… 39
CHAPTER 6: CONCLUSION AND RECOMMENDATIONS 6.1 Conclusion ……….…… 44
6.2 Recommendations.………...……….. 44
REFERENCES:………..… 45
APPENDICES Appendix 1: Book Design (Image Targets)……….. 54
Appendix 2: 3D Models While System Is In Play……… 64
Appendix 3: 3D Models of Turkish Alphabet……….…………... 80
viii
LIST OF TABLES
Table 3.1: Types of AR applied in Education ... 13 Table 5.1: Recognition Probability based on Vuforia ... 40 Table 5.2: Image Targets Recognition Test Results………. 41
ix
LIST OF FIGURES
Figure 3.1: The mechanical head position sensor in use (Sutherland, I. E., 1968) ... 10
Figure.3.2: Advantages of augmented reality from motivation perspective ……….…… …………....(Masmuzidin, et al., 2018) ………...….… 15
Figure.3.3: Diagram of the process of application development platform Vuforia...… …………... (G.Jack, 2013) ... 18
Figure.3.4: Reality-virtuality continuum (Milgram, P., & Kishino, F., 1994), (Flavian,…….. ……….C., 2018) ... 19
Figure 3.5: Google Daydream View (Googlecom. 2019) ... 20
Figure 3.6: Samsung Gear VR (Samsung, 2019) ... 20
Figure 3.7: Google Cardboard (Google.com, 2019) ... 20
Figure 3.8: HTC VIVE (Vive.com, 2019) ... 21
Figure 3.9: Oculus Rift (Bbystaticcom. 2019) ... 21
Figure 3.10: Oculus Go (ESSA KIDWELL, 2018) ... 21
Figure 3.11: Oculus Quest (Oculus.com, 2018) ... 21
Figure 3.12: IBM Simon Personal Communicator (T. Caudell and D. Mizell, 1992)…. 22 Figure 3.13: Number of smartphone users worldwide from 2014 to 2020 (in billions), …... ……….(Statista, 2016) ... 23
Figure 4.1: System Development Tools ... 24
Figure 4.2: Vuforia Core Components of the AR-Children's Book Application ... 25
Figure.4.3: Mobile Augmented Reality Block Diagram Architecture (Dipti Raja ……. ……….Dhotre, 2016) ... 26
Figure 4.4: Flowchart of the developed system ... 28
Figure 4.5: License Key Creation (Vuforia) ... 29
Figure 4.6: Database Creation ... 29
Figure 4.7: Add Targets ... 30
Figure 4.8: Download Database ... 30
Figure 4.9: Importing Vuforia Assets ... 31
Figure 4.10: Import Database ... 31
Figure 4.11: Add image target ... 32
x
Figure 4.13: 3D model object ... 33
Figure 4.14: Default Trackable Event Handler in Visual Studio (C#) language ... 34
Figure 4.15: Switching to Android Platform ... 34
Figure 4.16: First page of the Book ... 35
Figure 4.17: Capital and Small Letter A ... 36
Figure 4.18: An example of a car (Araba) on the Letter A ... 36
Figure 4.19: Letter A (Capital, Small)... 37
Figure 4.20: Car (Araba) ... 37
Figure 4.21: Threes Models Showing at the same time ... 38
xi
LIST OF ABBRIVATIONS
3D: Three Dimension
AR: Augmented Reality
DOF: Degrees of Freedom
FOSS: Free and Open Source Software
GIS: Geographic Information System
IBM: International Business Machine
ICT: Information Communication Technology
IOS: iPhone Operating System
IT: Information Technology
NEU: Near East University
OOSE: Object orientated software engineering OS: Operating System
PC: Personal Computer
SDK: Software Development Kits VR: Virtual Reality
1
CHAPTER 1
INTRODUCTION
Educational content can be experienced through a wide variety of media, ranging from non-interactive books to highly non-interactive digital experiences that fully engage the user’s senses. This paper is concerned specifically with adding value to the field of education in the emerging medium of augmented reality. A relatively high amount of research studies have investigated the potential impact of augmented reality to benefit student learning. These diverse research programs can provide useful information for educators and technology designers interested in enriching young students’ minds through novel technologies Radu, I. (2014). In recent years, technology-enhanced learning (TEL) research has increasingly focused on emergent technologies such as augmented reality, ubiquitous learning (u-learning), mobile learning (m-(u-learning), serious games and learning analytics for improving the satisfaction and experiences of the users in enriched multimodal learning environments. Research on AR has also demonstrated its extreme usefulness for increasing the student motivation in the learning process (Bacca, et al. 2014).
Technology of Information and Communication play a very important role in the education sector, they can improve the knowledge of the student and at the same time the teaching methods used by teachers. Likewise, many applications can be seen using augmented reality, however are not oriented towards educational sector The nursery school "Juana Alarco de Dammert" was founded in 1993 in the city of Trujillo, this educational institution hosts children from 2 to 5 years of age every year with the aim of providing quality education, this research focused on children older than 4 years, because they had a little difficulty at the time of learning, the main factor was that they were too restless and were distracted very quickly with the teaching method taught by the teacher through drawings made on paper, printed images and pictures with images of the vowels or numbers. Therefore, the children should learn in a more dynamic and fun way, where they can interact with the learning material in the same educational institution and in the comfort of their home; preventing them from being easily distracted (Cieza, et al., 2018).
2
1.1 The Importance of Phonics Reading to the Child
Phonics is a branch of linguistics where the sounds and physical properties of human speech sounds are studied. Phonics reading is highly essential in every child education. It is not uncommon to find parents who question the importance of phonics reading in the education of their children. Such parents believe that children will naturally master the different sounds of human speech since the ability to use language is innate in every human being. Their view may look plausible but they are not actually correct (Clicknkidscom, 2019). Indeed, phonics reading is very important in the education of children. The report of National Reading Panel indicates that teaching children phonics will help them in many ways in life. In the first instance, phonics reading is very important in helping children to learn how to spell words. It will be impossible for a person to spell any word correctly if the person is not able to recognize the sounds of the letters used in forming the words. When a child is taught phonics, the child will be able to recognize sounds in words and will be able to spell them correctly (Clicknkidscom, 2019).
Children have problem in reading because they are not able to recognize the sounds of the letters of the alphabet in the words they read. Phonics reading will help children to recognize and associate sounds of the letters of the alphabet in the word they read. This will help them to improve in their reading skills and efficiency. In other words, it will be difficult for a child to improve in his reading skills if the teaching of phonics is removed from their curriculum (Clicknkidscom, 2019). Phonics reading is also necessary for the improvement of a child's reading comprehension. It is impossible for somebody to understand a word that is not properly pronounced. When a child learns how to pronounce a word very well, the child will be able to comprehend what he or she reads. When a child is able to pronounce a word correctly, the child will be able to understand the word. Children normally use in their words that they understand in their daily speech (Clicknkidscom, 2019).
Children have to develop more confidence in themselves before they begin to vocalize more. This begins the moment they realize that they can pronounce words correctly like older people. It is only through phonics reading that children will develop the ability to pronounce words very well (Clicknkidscom, 2019).
3
In order to make phonics learning becomes more effective, here comes a project with aims of designing an interactive Turkish Language phonics learning Book for Children using Augmented Reality (AR) technology. AR technology allows the physical images or object mixed with a virtual layer.
1.2 Problem Statement
Augmented Reality has not yet been known or its near absent in the field of Turkish Language teaching and learning.
There is a lack of learning system-based approaches to mobile applications which are using AR technology for Turkish Language.
The objective of this thesis is to develop an AR mobile application based on Turkish Alphabet and a Book to interact with the application. The project was developed for children aged 5 to 7 years so that they can learn interactively using images, 3D models and sound (phonetics).
1.3 The Aim of the Thesis
The aim of this thesis is to develop an Augmented Reality application and book for children that teaches them Turkish Alphabets with phonics, and each letter with an example.
1.4 Significance of the Thesis
With the increased improvements in technology, it is reasonable to make use of the technology in all areas of life where applicable. One of the fields where it needs more implementation of technology is education. Using different techniques in the learning process is a huge demand and once they practice using the technology in education and make the most of it, learners tend to show more willingness in learning and in getting the knowledge.
Being a child means playing and learning. These two things should be considered when approaching any methods for teaching children. The lessons prepared and given by educators and parents should focus on the playfulness and the fun factor (Rahmat, R. F., Akbar, F., Syahputra, M. F., Budiman, M. A., & Hizriadi, A., 2018, March). Removing those factors
4
will simply result in a bored child and eventually will stop following what the lesson is being given. The basics of each language is the Alphabet. Those two factors mentioned earlier are considered when developing the application to ensure that the children using the application are enjoying the learning process.
Developing the Augmented Reality Children’s book for the Turkish Alphabet will be the first application of such kind. No previous Augmented Reality Book Application of the kind developed have been created for teaching Turkish Alphabet for children which gives more significance to this thesis and motivation of building such application.
1.5 Limitations of the Study
Despite the fact that the thesis reaches its goals, if different designs were selected in making the pages of the book, the tracking of the image targets would leave no chance for having less features of an image target to be detected and get recognized better in placing 3D objects over the targets.
1.6 Overview of the Thesis
The written thesis consists of six chapters, a brief insight on each chapter is written below:
Chapter 1: Chapter one gives a general introduction on Augmented Reality and the new
digital media, and how this technology can help in different areas of the current age. This chapter also describes the aim of the thesis, the limitations, and the importance of the thesis.
Chapter 2: Past related research to the topic are reviewed, the techniques they used and their
results are also mentioned.
Chapter 3: This chapter focuses on the theoretical background of Augmented Reality,
Virtual Reality, and Mobile Learning and how AR helps in education sector.
Chapter 4: The process of developing the system, and the implementation of Augmented
Reality Children’s Book application is explained in details in this chapter.
Chapter 5: The results and finding of the thesis are discussed.
Chapter 6: Finally, the conclusion is explained in chapter six and also the future works is
5
CHAPTER 2 LITERATURE REVIEW
This chapter presents previous works and studies on Augmented Reality applications, interactive books and learning languages through Augmented Reality. It talks about what is already available and which areas need more research and applications. Augmented Reality is relatively new to consumers and it started getting public’s notice in the past couple of years, this is one of the reasons the time range of the chosen literature presented is 5 years from 2014 until 2019. In the end of this chapter the suggested application is presented to fill that gap between what is already there and what is needed.
(Gil, et al., 2014) in a short-term experiment, presented an Augmented Reality story book called AR Petite Theater, that enabled role-play via augmented reality technology. Through interactive reading experience it provided an opportunity for children to learn the ability of empathy by mocking and copying the way the character’s role of the story was thinking and speaking. After conducting an experiment with twenty-four young children, they measured the participation and perception of children’s role-playing. Empathic behavior was found in the groups were Augmented Reality was used, and the children using the technology showed less unconnected perspectives than those who did not use the Augmented Reality technology.
(Al-Ali, et al., 2016) presented MyVision AIR, an interactive Augmented Reality book (application), aiming to enhance the reading experience of adult-learners through integration of Augmented Reality technology to improve the interaction between the user and normal books. My Vision written by the Vice-President and Prime Minister of the UAE and Ruler of Dubai, H.H. Sheikh Mohammed Bin Rashid Al Maktoum, was the book they applied Augmented Reality to it. The added features to the normal book were 3D models, animations, audio and video. In result of using Augmented Reality it improved the user experience with traditional textbooks and the findings were found to be very positive as stated by the authors.
(Zainuddin, N., & Idrus, R. M, 2016) in their study described the development process of an augmented reality enhanced flashcards for Arabic vocabulary acquisition, to improve student’s knowledge and memorization of basic Arabic Vocabulary. The Augmented Reality
6
element provided an engaging, varied learning environment with a mixture of audio, visual or multimedia content. The use of augmented reality was considered as one of the platforms that can be used to help the students in memorizing certain information and maintained their knowledge of a vocabulary in a language, in their example, the Arabic vocabulary.
Findings of the research conducted indicated that AR enhanced flashcards help in supporting the knowledge of students regarding the Arabic vocabulary acquisition. Furthermore, they also found that with the help of the AR enhanced flashcards the process of learning is easier.
(Dalim, et al., 2016) Presented An interactive augmented reality tool for teaching basic English to children who do not come from an English-speaking background called TeachAR which is an interesting challenge for educators. The system as the authors claimed was “the first AR language learning tool attempting to teach young children, 4 to 6 years old, about spatial relationship and shapes.” The results indicated a potentially better learning outcome using the TeachAR system than the traditional system. It also showed that children enjoyed using AR-based methods. However, it also showed a few usability issues with the TeachAR interface such as issues in Speech-Input.
(Majid, et al., 2016) developed an Augmented Reality mobile application for Pre-Literacy Kit. The developed AR application was based on three main things: contents, design, and tools. The results show that the children and teachers gave positive feedbacks, children were having fun in using the application and the teachers after using the mobile app for the pre-literacy also liked the addition of a variety of teaching aids.
(Hsu, T. C., 2017) In an attempt for third grade students to learn English vocabulary in free and situated environments, they developed and evaluated two Augmented Reality educational game systems. The students had excellent learning effectiveness regardless of whether they used the self-directed or the task-based AR educational game system in their study. It was also found that the learning styles of the students had a critical role in the foreign language learning anxiety and in their mental effort as mentioned in the study.
(Hashim, et al., 2017) developed ARabic, a Mobile Augmented Reality application for teaching and learning early Arabic language and to also study its effectiveness. The application had two main modules; Learning and Exercise Module which consisted of objects, graphics, animation, video, audio, and 3D objects to keep the student’s interest.
7
Because of the printed textbook with digital information, and based on the user testing, it showed that users were very interested in ARabic and learned the modules quickly. The authors also mentioned that this application can also help parents to teach their children Arabic at home, because it is a complete learning system that used the teaching modules in school and it comes with Malay translation to help parents.
(Lee, et al., 2017) developed a prototype of an Augmented Reality application to teach Kindergarten Students English vocabulary in an interactive way. The application allows kindergarten students to learn English vocabulary in any place and at any time using a mobile device. They also integrated a monitoring system into the application to address the parents’ concerns on their children’s health. The monitoring system allows the parents to monitor their child’s usage and stop the application in real time online. This act would reduce the negative effect on children’s health while they still enjoy the experience. In their preliminary assessment, it showed that the effectiveness of the application was satisfactory.
(Sidi, J., et al., 2017) developed an interactive English Phonics Learning for Kindergarten using Augmented Reality. With aims to make phonics learning more interesting, interactive, and effective. The marker-based technique of Augmented Reality technology used for tracking allowed children to interact with virtual phonics content through physical manipulation. Phonics card here being the image marker that triggers the tracking process. They were able to read CVC words with the phonics sound along with the related animated 3D model presented through the triple phonics card interaction. Children could answer the questions with 3D model presented through assessment. This interaction method provided a better learning experience for children. The results of performance evaluation showed that children had reading improvements after using the mentioned courseware. It is considered by the authors as a usable courseware through SUS evaluation held by teachers.
(Rahmat, et al., 2018) Developed an interactive Augmented Reality android application of Hijaiyah alphabet (letters used in the Qur’an) for children education. The application used the Smartphone and marker as its medium. 3D objects Hijaiyah can appear on the marker, and each virtual button can make a sound based on the button selected by the hand. Using the application, the user can learn and understand the shape and pronunciation of Hijaiyah
8
alphabet by touching the virtual button on the marker. One of the limitations of this application is light, in the case of the dark or having less light, the application cannot run.
(Zainuddin, et al., 2018) developed an Augmented Reality application for Personalized Arabic Vocabulary Learning for non-native students of Universiti Sains Islam Malaysia (USIM). Findings indicated that the AR enhanced flashcards help in supporting the knowledge regarding the learning of the Arabic vocabulary. Furthermore, the findings showed that the AR enhanced flashcards facilitate the process of personalized learning. The study also showed that the augmented reality application could be considered as one of the personalized learning platforms that can be used to help students in memorizing certain information and maintained their knowledge of Arabic vocabulary, and creating different sentences by using the target vocabularies more than half of the time.
(MacCallum, et al., 2018) conducted a research on exploring Augmented Reality in Mobile applications in early childhood literacy learning. The authors developed an early prototype of an Augmented Reality application as the first phase to explore the concepts of exploring and identifying the potential of AR in pre-literacy learning. The prototype made use of markers for tracking purposes, in the form of cards showing a letter of the alphabet with an associated image representing that letter to activate the content and to track it. Using the application, a 3D animation of the object appears along with the phonetic sound of the letter. The approach they used provided the children a context when getting familiar with the sounds of the specific letter makes when pronouncing the letter phonetically.
(Cieza, et al., 2018) Developed an Educational Augmented Reality Mobile application based on markers to improve the level of understanding of the usage of vowels and numbers for children of a kindergarten in Trujillo. The scenes were developed in a way that when each time you focus on a marker through the camera of the mobile application the respective image is projected in Augmented Reality will be reflected. For example, using a marker with an image of a number would present on the mobile application the 3D object of that number, and the same case goes for the vowels. Authors indicated in their research that after implementing the application, it was possible to increase the level of academic performance of the use of numbers by 22.60% and vowel usage by 27.60% in the Juana Alarco de Dammart nursery school children.
9
In this paper, an Augmented Reality Children’s Book was developed. It helps children learn the Turkish Alphabets along with an example of a letter being used in a word. First, it displays 3D objects of the Turkish Alphabet once the marker image is found, then sound of the pronunciation of that letter is played, and the example of a 3D object of a word using that letter is displayed. After following up with the recent research it has been found that there is a huge demand of Augmented Reality in Educational sector, and for teaching children even more.
It was also found that there is none-previous research nor application towards teaching children learn Turkish Alphabet using Augmented Reality, and this was one of the reasons this application was developed.
10
CHAPTER3 METHODOLOGY
3.1 Theoretical Framework
This chapter will explain the theoretical framework used in this study, thus explaining augmented reality, a brief history of AR, an introduction to its applications, augmented reality usage and the advantages of having AR in the selected framework, Mobile application development, types of mobile applications, smartphones, and the usage of smartphones in the world.
3.2 Augmented Reality
Augmented reality is a technology that overlays virtual objects onto the real-world three-dimensional environment. These virtual objects appear in real time and space that coexist in the same place and can be collaborated with continuously to make it an immersive experience (Akçayır, M., & Akçayır, G., 2017), (Kiryakova, G., Angelova, N., & Yordanova, L., 2018).
The history of augmented reality usage goes back to the time when Ivan Sutherland created the first augmented reality system, which is also the first virtual reality system (see Fig.1). It used an optical see-through head-mounted display that was tracked by one of two different 6-Degrees-of-Freedom trackers or (6DOF): a mechanical tracker and an ultrasonic tracker. Only very simple wireframe drawings could be displayed in real time Due to the limited processing power of computers at that time (Sutherland, I. E., 1968, December).
11
Augmented Reality is considered to be one of the fastest growing branches of ICT that is visible in our everyday lives. With its assistance, the physical world can be expanded with virtual elements (e.g. 2D images, videos, 3D models, and animations) that merge into the real-life environment. The technology is halfway between the real and the virtual world, and the virtual objects are displayed and generated with the help of a computer screen, a mobile phone, or a special head-mounted device (Biró, et al., 2017).
3.2.1 Objectives of Augmented Reality:
To enhance the imagination of youths, to implement it into day by day lives to help the majority to accomplish and attain what is a constraint in the real-world environment, challenge the unthinkable, make new realities out of the real-world environment, and to make a virtual objects more entertaining for client experience (Mohammed, M. M. ,2018).
3.2.2 Types of Augmented Reality
Fundamentally, AR applications when it comes to tracking are of two types of implementations: The marker-based, and the marker-less. The marker-based makes use of the camera and a visible indication of (e.g., An image, or a shape) to serve as the target to be tracked, while the marker-less deals with the mobiles’ global positioning system and compass from positioning information to be able to track and position the virtual objects. These two main types can be derived to 4 types:
3.2.2.1. Fiducial Marker Based Tracking
The most used technique to achieve AR is this type. In AR they are used in the field of view for easy recognition and typically have high contrast. By using them we can relate to point in space and also calculate distance and the angle at which we look. The typical markers used in AR are black and white squares with geometric Figures. The use of black and white gives high contrast compared to having a background environment and can therefore be quickly recognized. One of the common downfalls in fiducial markers technology is that they always have to be seen and cannot be obscured by other objects during the augmentation. This problem can be partially alleviated by remembering marker position and refreshing its position accordingly with device movement (Amin, D., & Govilkar, S. ,2015).
12
3.2.2.2. Hybrid Based Tracking
Hybrid Based Tracking usually combines two or more data sources such as Compass, GPS, Accelerometer to calculate the actual position and orientation. Using compass, we can tell the direction where the device is pointing at and check if that route has any objects to be augmented on. Global Positioning System (GPS) identifies the current location of the device, with this location information, we can find objects that are to be augmented in our area. The accelerometer is used to calculate orientation of the device using gravitation to its advantage. Combining all information, we can calculate what should be augmented in the field of view without any actual processing of the real image but of course the real image is used for placing the layer of augmentation. (Amin, D., & Govilkar, S. ,2015).
3.2.2.3. Modeled Based Tracking
A model-based approach uses prior understanding of 3D objects within the environment along with their appearance. Using geometrical representation of 3D objects, we are able to manipulate their position and orientation matching them with their counterparts in the field of view. Model approach works using edge detection for construction of 3D models, in some instances the model is provided to track resemblance with regards to its object in the environment e.g. tracking a moving car on the streets, but this approach usually requires more processing power (Amin, D., & Govilkar, S. ,2015).
3.2.2.4. Natural Feature Tracking
This technique allows AR to be achieved using objects in real world as markers by recognizing the markers natural characteristics. Based on some mathematical algorithms, we find “interesting features” of the image that are highly distinguishable. A Feature descriptor of a given image is saved for additional recognition purposes. Based on this particular feature set we can recognize the exactly same image from different orientation, distances, and illumination levels even with some occlusion due to the fact that the descriptor is invariant to those changes (Amin, et al., 2015).
(Bacca, et al., 2014), in the results of their analysis found that most of the studies on the use of Augmented Reality in Education used “Marker-based AR” by (59.3%) markers are most used in the development of the AR applications towards educational settings. A probable explanation for this result is that compared to the marker-less tracking techniques currently
13
the tracking process of markers is better and more stable. Therefore, Marker-based AR could be recommended for a better experience with the technology for students until better techniques for tracking can be developed for marker-less AR. In educational settings “Markerless AR” has not been widely used based on the average percentage (12.5%) from the analysis.
The development of “Location-based AR” applications is major (21.8%) compared to marker-less AR applications. This is because of the availability of multiple sensors in today’s smartphones and devices such as the accelerometer, digital compass, gyroscope, and usually the GPS. These technological advancements enable developers to develop applications of AR that are aware of the user’s location and in return be able to show information according to the position and/or orientation based on the data given by the sensors (Bacca, et al., 2014).
Table 3.1: Types of AR applied in Education
Code Sub-category Number of studies Percentage (%)
SC1 Marker-based AR 19 59.38%
SC2 Marker-less AR 4 12.50%
SC3 Location-based AR 7 21.88%
SC4 Not specified in the study 2 6.25%
3.2.3 Importance of Augmented Reality
Augmented Reality applications are important and are being used in many different areas and fields among which are (Medical Training, Retail, Repair & Maintenance, Design & Modeling, Business Logistics, Tourism Industry, Classroom Education, Entertainment, etc. …).
One of the main areas of focus in this research of where Augmented Reality is being used is Education.
There is a wide variety of media in which educational content can be experienced. Student’s traditional method of learning is through interaction with teachers. And also, through non-interactive media like textbooks, slides and instructional videos. Digital media has gradually
14
made its way into educational settings in the last half century, providing students with different learning opportunities around interactive simulations and educational games. Those experiences have typically been accessible only in classrooms equipped with desktop computers and some with interactive whiteboards. More recently, students can experience different kinds of learning experiences and access them portable devices such as smartphones and tablets (Radu, I., 2014).
There is an increase interest in Augmented Reality to develop effective learning experiences. For teachers considering to implement AR experiences with their students, learning theories can serve as an approach to accomplish that goal (Wang, et al., 2018).
For preschool children in the learning process, the role of AR is for sure to capture their attention. Giving children information continuously in an attractive way will prepare them for the future and motivate them to explore their surroundings and the world in their own ways. Based on (Rasalingam, et al., 2014) research, they found that the use of flashcards could enhance active learning. If learning is conducted in an educative environment where technology is easily accessible, it tends to be more efficient, effective, and dynamic for children. Teachers and parents should work together in the child’s learning process and should make sure that the environment they are being thought in is a suitable one without any disturbance.
In (Rasalingam, et al., 2014) they had an interview with a 6-year old child, preschooler and an IPad user named Siti, they asked her about her experience using Augmented Reality technology in the classroom and, and this was her reply:
“The class was nice. I see animals on the cards. I love to learn this way. I have IPad at home but I only play games with the IPad……I play the Angry Bird game and car racing games. I like to learn this way because there is animal on the card when I bring the cards near to the camera. Yes, easy to learn this way. I had fun today. I want to use this in class. “
According to their interview, Siti seems to prefer learning using the AR technology in her classroom rather than the traditional learning methods. She was excited to use this technology in her learning process. She was also asked at the end of the class about the alphabets, her response was good and gave an outstanding feedback.
15
3.2.4 Advantages of Augmented Reality
As mentioned before, the features and potential of AR technology has shown a need in the field of education. Because there is an increase usage of AR technology, many reports by researchers and educators have been made about the advantages of Augmented Reality. According to (Masmuzidin, et al., 2018) and from motivation perspective, the major advantages of AR are shown in Figure 3.2.
Figure 3.1: Advantages of augmented reality from motivation perspectives (Masmuzidin,
……… et al., 2018)
According to (Safar, et al., 2017)., in 2012 in Bulgaria an empirical research was implemented on 26 children at the primary stage to find if whether or not children who are learning a language thorough the use of AR technology comprehend more than those who learn languages by traditional methods. Their findings supported the argument that using AR technology helps in promoting vocabulary and that children are satisfied after the educational experiment was done in an AR environment.
To obtain knowledge and support learning, Books have long been used. When technology advanced, books were available in an electronic form or as called e-books. These e-books can be viewed on different types of multimedia platforms and devices which makes reading experience more accessible and interesting. Thou the sense of having and touching a real physical book is more preferred as mentioned in (Matcha, et al., 2012, June). In Books that
16
are using AR or AR-Books, virtual information and objects are added, they enhance the reading experience.
One of the earliest works implementing AR technology in Books is the "MagicBook" which used pop-up books as the concept (Billinghurst, et al., 2001). Users were able to read the book like a regular one while being able to view the virtual contents (animations) through the use of a Hand Held Display. In AR books, the way it works is that users can interact with the physical book and use it as the interface to show the virtual content augmented through a displaying device.
3.2.5 Augmented Reality Software Development Kits (SDKs)
In (M. Romilly, 2019) in his presentation, he talked about 12 Best Augmented Reality (AR) SDKs available in 2019. The right SDK will depend entirely upon the exact project requirements. Here the focus will be on Vuforia SDK used in the creation of the application.
“An augmented reality SDK (software development kit) is the core technological software engine that powers the development and creation of new AR apps and experiences.” (M. Romilly, 2019)
The role of the AR SDK is to perform the task of fusing digital content with the real world. The capabilities of the AR SDKs determine the features and functionality within your AR application, so it’s essential to choose the correct platform based on the requirements of your project.
Each AR SDK is equipped with its own unique properties that enable developers to track, recognize, and render the application in the most ideal manner possible.
Below is the list of the 12 Best Augmented Reality SDKs available in 2019:
1) Vuforia 2) ARKit 3) ARCore 4) Wikitude 5) EasyAR 6) Kudan 7) Onirix
17
8) MaxST
9) Pikkart AR SDK 10) DeepAR
11) Lumin (Magic Leap)
12) MixedReality Toolkit (HoloLens)
Vuforia is an augmented reality SDK that lets app developers and businesses to quickly make highly-reliable, mobile-centric, immersive AR experiences The Vuforia AR SDK uses computer vision technology to detect and track image targets and 3D objects in real-time. This can help developers and businesses to place and orient virtual objects. Including 3D models and other content such as 2D videos or images to the real-world environment. Virtual objects are overlaid on top of the real-word environment scene and can be viewed using an AR-enabled device such as a smartphone or a tablet (M. Romilly, 2019).
Vuforia’s augmented reality SDK has the ability of supporting a various range of 2D and 3D targets, including multi-target 3D configurations, marker-less image targets, and fiducial markers also known as a ‘VuMark’. Other additional features available in the Vuforia augmented reality SDK include localized occlusion detection by using virtual buttons, the capacity to calibrate target sets and develop them at runtime, and target image selection at runtime (M. Romilly, 2019).
Vuforia provides different application programming interfaces (API’s) in C++, Objective C++, Java, and .NET via an extension of the Unity game engine. Vuforia SDK has the ability to support both native development for Android and iOS and the development of AR applications and prototypes in Unity’s game engine that can easily be ported across both of the mentioned platforms (M. Romilly, 2019).
This gives a great option for brands and businesses looking to develop apps that cover both Android and iOS platforms whilst minimizing commercial and technical risk. The means that AR apps can be developed in short period of time seamlessly for the widest possible number of target mobile devices (M. Romilly, 2019).
18
Figure 3.2: Diagram of the process of application development platform Vuforia (G.Jack,
……… 2013)
To track a target, a developer uploads an input image. The mobile application can access target resources in two ways:
• Using web services, it can be accessed from a cloud target database.
• It can be downloaded in a device target database to be incorporated with the mobile app.
In the case of text recognition, the developer specifies a set of words that can recognize, using the following text data sets:
• Word lists in the Vuforia Word List (VWL) binary format. • Using simple text files, additional word lists can be specified.
• Optional list filters for words (black or white lists) to explicitly include or exclude the recognition of specific words.
The mobile app and the word lists and filter files are bundled together and loaded at runtime using the API (G.Jack, 2013).
19
3.3 Virtual Reality
Often times when Augmented Reality is mentioned, Virtual Reality (VR) is also mentioned with it, here Virtual Reality will be explained in brief and also to differentiate between both AR and VR technologies.
Virtual Reality is 100% digitally generated environment where users can interact with, VR contains real-time immersive simulations through digital graphics. In other words, Virtual Reality provides users with a feeling of actual presence and being immersed psychologically in that virtual environment (Huang, et al., 2019).
Between the real environment and the virtual environment there is augmented reality and virtual reality, they both gave birth to mixed reality.
Figure 3.3: Reality-virtuality continuum (Milgram, P., & Kishino, F., 1994), (Flavian, C.,
……… 2018)
VR headsets can be divided into three categories:
• Mobile headsets
A Mobile headset is basically used by placing the smartphone in a case with lenses, usually there is a divider between the two lenses, and the related VR app on the smartphone divides the screen into two separate images, each for an eye to make an immersive experience on a smartphone. Some of the example of mobile headsets are Google Daydream View, Samsung Gear VR, and also the cheap headset to enter the VR world is Google Cardboard.
20 • Tethered (PC/Laptop Powered) headsets
They are called Tethered headsets because they are physically connected to a PC or (PlayStation – PlayStation 4 for the PS VR). The major differences between Tethered headsets and Mobile headsets is that they use 6DoF (6-Degrees-of-Freedom) Controllers, and built-in headset cameras and sensors. Examples on Tethered headsets are HTC VIVE, and Oculus Rift. Both the HTC VIVE and Oculus Rift required powerful processing power from a PC to tun.
Figure 3.4: Google Daydream View
………… (Googlecom. 2019) Figure 3.5: Samsung Gear VR (Samsung, ……… 2019)
Figure 3.6: Google Cardboard
21
Figure 3.7: HTC VIVE (Vive.com, 2019) Figure 3.8: Oculus Rift (Bbystaticcom.
……… 2019) • Stand-alone headsets
Standalone VR headsets are new on the market of VR headsets and provide a median option between the Mobile-Based headsets and Tethered headset. No additional hardware is required. They are basically VR headsets with built-in displays and a custom-built android operating system. Examples on Stand-alone headsets are Oculus Go, and the most recent release of the Oculus Family with new features and improvements, the Oculus Quest.
Figure 3.9: Oculus Go (ESSA ………
KIDWELL, 2018)
Figure 3.10: Oculus Quest (Oculus.com,
……… 2018)
Whether it’s a Mobile-Based, Tethered, or a Stand-alone headset, the development of these devices is in the benefit of end user and in return the more advanced they get the more it can help the areas we need the most like Health sector, Education, etc.
22
3.4 Mobile Phones/ Smartphones
In (Arth, et al., 2015)., the authors talk about when the first ever smartphone was introduced by IBM and Bellsouth at COMDEX in 1992, It was called the IBM Simon Personal Communicator. The phone had a 1 Megabyte of memory equivalent to 1000 Kilobytes and a Black & White touch screen with 160 x 293 pixels as resolution. The IBM’s Simon worked as a phone, calculator, pager, address book, a fax machine, and as an e-mail device. It weighted 500 grams and cost around 900 United States Dollars.
Figure 3.11: IBM Simon Personal Communicator (T. Caudell and D. Mizell, 1992)
A mobile phone is considered to be as one of the essential components of modern life. The continuous evolution of mobile phones has changed the shape of the world and how we interact with it (michaelcon123, 2019). What makes smartphones different from traditional mobile phones is that smartphones offer more functions for the end user, including internet connection, multimedia players for music, pictures, and videos (michaelcon123, 2019). One of the standing innovations about smartphones is the ability for the users to download various types of applications on different platforms. This ability opens the door to a lot of possibilities for the developers to develop applications in different fields and for different aspects of life and for the user to use and make their life easier.
In a survey done by Statista in the duration of 2014 to 2015, (Statista, 2016) in June, 2016, they published statistics on the total number of smartphone users worldwide from 2014 to 2020 as shown in the Figure 3.13.
23
Figure 3.12: Number of smartphone users worldwide from 2014 to 2020 (in billions)
……… (Statista, 2016)
In their findings, it appears that the number of mobile phone users in the world was expected to reach around 2.5 billion mark by 2019 (Statista, 2016).
Operating system of mobile phones:
To run a smartphone, an operating system is required, there are three major operating systems available in the market and are widely spread worldwide.
1. Android 2. iOS
3. Windows OS
When searching for mobile operating systems, other operating systems might appear, but they are not as strong and widespread as the other top three, other OS for mobile are but not limited to (Bada, Palm OS, Open WebOS, Maemo, MeeGo, Verdict, Symbian, etc…).
24
CHAPTER 4
SYSTEM DEVELOPMENT AND IMPLEMENTATION
This chapter explains and navigates through the developed Augmented Reality mobile application the Augmented Reality Children’s Book and talks about the system development, system architecture, the implementation and how it works.
4.1 System Development
To achieve the desired application, different tools were used for both the software and the book itself. In Figure 4.1 it shows all the tools used in the development of the Augmented Reality Children’s Book.
The system development contains five steps.
Step 1: Modeling 3D Letters, via Adobe Photoshop, and preparing the related 3D objects from ready to use 3D models.
Step 2: Drawing and creating image targets for the Book via Adobe Photoshop.
Step 3: Preparing and editing sound using Adobe Audition.
25
Step 4: System programming of the augmented reality based on Vuforia SDK and Unity Game Engine as the system development environment.
Step 5: Finally, Unity is used as system building platform and integrated with the Android SDK, the final product application is available for installation and for usage.
4.1.1 Architectural System Design
The AR-Children’s Book is designed using Vuforia SDK that has the ability to store multiple image targets and also capable of communicating to the database connected with the application on the user’s device. After tracking image targets and a successful detection, 3D objects are placed in real life through a screen display to make the immersion possible. The AR-Children’s Book application architectural system design is described in the bellow Figure.
26
The user opens the application to detect an image target in the Book through the camera, the camera then captures the preview frame and passes it to the image tracker. The image convertor uses the pixel format convertor and converts the default format of the camera and changes It to a suitable format for openGLES rendering and tracking. Tracker, detects and tracks real world objects in the preview frame of the camera using image recognition algorithms, then the results are stored in the state object which is passed to the video background renderer. Rendered the camera image stored in object state and the virtual objects are augmented on the real world displayed in the camera screen of the user’s device.
4.1.2 System description
Mobile phone AR applications with Marker-based feature search for an image target through the camera. It compares the live feed from the camera with the stored image target in the database. Once the target image is found, the software brings the digital object/ a 3D model and places it on to the real world. Marker-based AR are also called Image recognition or Recognition based AR. The working principle of how Marker-based AR applications work is demonstrated bellow in Figure 4.3.
Figure 4.3: Mobile Augmented Reality Block Diagram Architecture (Dipti Rajan Dhotre,
27
The Architectural block diagram of the developed AR above shows the following important parts:
Camera:
When the camera is opened in the device which the user is using, a real-world live video is presented as an input from the device (In this case an Android phone) camera and passes the information to the Camera module.
Image Capturing Modules:
The live video feed from a mobile device camera is the input to the Image Capturing Module, by examining and processing each frame in the video (Dipti Rajan Dhotre, 2016)
Image Processing Modules:
The images from the Image Capturing Module are the inputs to the Image Processing Module. Using an image processing technique these images are processed to detect the AR camera. The detection of the AR camera is necessary to identify the position, and where to play the virtual object. The moment an AR object is detected, its location is given as an input to the Tracking Module (Dipti Rajan Dhotre, 2016).
Object Tracking Modules:
The tracking module is the essential core of the augmented reality system; the relative position of the camera in real time is calculated (Dipti Rajan Dhotre, 2016).
Rendering Module:
Rendering Module has 2 inputs. The first one is the calculation of the position from the Tracking Module, and the other input is the Virtual Objects to be Augmented. The Tracking Module then combines both the original image and the virtual components using the calculated position and displays the augmented objects on the display screen of the mobile device (Dipti Rajan Dhotre, 2016).
28
Figure 4.4: Flowchart of the developed system
4.2 Implementation of AR Children’s Book in Unity3D with Vuforia SDK 4.2.1 Steps for configuring Vuforia into the project
• License key
o A license key is needed to be able to use Vuforia in the application.
o Create and copy a license key from Vuforia’s developer portal and past It In the unity software
29
Figure 4.5: License Key Creation (Vuforia)
• Database
o To create a new Database, go to Vuforia’s developer Portal o Go to Target Manager
o Click on Add Database o Give a name to your Database o Choose the type of the Database o Click Create to create the Database
Figure 4.6: Database Creation
• Add Targets to Vuforia
30 o Select a Database
o Click on Add Target
o Choose an image file form your device o Give dimensions(width) to your target o Give your target a name
o Click Add
o For multiple targets, repeat the same actions in the same Database uploading different Targets with different names
Figure 4.7: Add Targets
• Download desired Database
o After uploading all the targets wanted, you can start downloading the Database to use it in your project
o Click on Download Database (All)
o Choose the Development platform you wish to work on, in this case Unity Editor
31
4.2.2 Steps of development in Unity3D Editor
• Add AR, and Vuforia assets to your Unity project
o Since Unity version 2017.2, Unity integrated Vuforia Engine for easy development of AR experiences
o In the Hierarchy Area, right click on an empty space and hover on Vuforia Engine, a List will show, click on Add Camera
o It will ask you to import some quired assets, click import
Figure 4.9: Importing Vuforia Assets
• Add Vuforia Database to Unity Editor
o In the Assets folder, right click on an empty space, then Hover on Import Package, then click on Custom package and select the database you’ve downloaded earlier
32 • Add Image Targets
o Right click on an empty space in Hierarchy area and hover on Vuforia Image, then click on Image
o When the Image Target is added, If the Database is loaded before this step, it automatically assigns an image for detection to the Image Target
33 • Modify Image Target settings
o In-order to modify the settings of the Image Target, Vuforia has to be enabled in the Player Settings
o File->Build Settings->Player Settings
o In XR settings, check Vuforia Augmented Reality o Remove the main camera, and leave only the AR camera
Figure 4.12: Image Target with Database loaded
• Adding 3D models to Image Targets o Load your 3D models into Unity
o Drag and Drop your Object/3D model and place it on top of your image target o Use the Scale tool if the Model is too Large or Too Small
34 • Adding Sound
o Click on the Image Target you want to add sound to
o Find Default Trackable Event Handler and double click on the script
o This will take you to your preferred script editor, In the case of this project it was Visual Studio, and the script language is C#
o The notable mention here when coding to add sound is when image target is found, it should activate the sound file, if not found it will not do anything until image target is found
Figure 4.14: Default Trackable Event Handler in Visual Studio (C#) language
• Compile the project with Android
o This project is for Android platform, so naturally we should change the project settings in Unity accordingly
o File->Build Settings->Android o Click Switch platform
o For the project to run with no errors, Android SDK is advised to be installed in the PC before starting the development process.
35
4.3 Implementation of the Application
AR children’s book was designed based on the Turkish Alphabet and according to each Alphabet a respective representation of that letter is presented in an example. When the Camera detects and Image Target, the 3D model/Object of that assigned to the Image Target will appear in the camera. At the same time when Image Target is found, a sound is played which pronounces that letter in the Alphabet or the example related to that letter.
36
4.3.1 Pages of AR Children’s Book
The first page when you open the book as shown in Figure 4.17. is the letter A, Capital Letter A, and Small Letter a in Turkish language. The Next Page or each page to the right is an example of that specific letter, in this example a Car (Araba) in Turkish representing the letter in the Turkish Alphabet. See Figure 4.18.
Figure 4.17: Capital and Small Letter A
37
4.3.2 AR Children’s Book in Use
After a user points the camera towards the image target, the letter in this case, the 3D model of the letter will appear on the smartphone’s screen along with a sound pronouncing that letter. The same thing goes for the other page. The user will point the camera towards the image target, towards to the image of the car in this example and the 3D model of the car will appear on the device (smartphone) with the sound of the pronunciation of that word containing the previous letter learnt from the application. See Figure 4.19. And Figure 4.20.
Figure 4.19: Letter A (Capital, Small)
38
Another example on the application being used is shown below in Figure 4.21.
Figure 4.21: Three Models Showing at the same time
In the above Figure, two, 3D models are shown at the same time since the letters from the Alphabet are shown as one object connected with a single image target. The program has been set to track only 2 targets at the same time because the way the book looks is two image targets can be seen at once in the book when opened, which makes it reasonable to only track 2 targets, maximum at the same time. This also reduces resource consumption because after reaching the limit of showing objects at the same time, it won’t be looking for another object to find. Although the Letters and the example can be tracked simultaneously, it is advised for the user to point the camera to only one page and interact with for better learning experience.
39
CHAPTER 5
RESULTS AND FINDINGS
One of the requirements of this study was to make a new augmented reality children’s book application that teaches children Turkish Alphabet.
R1. Generate 3D models of Turkish Alphabet Letters
R2. Generate 3D models of examples for each Turkish Alphabet Letter
R3. Play the respective sound once an image is recognized for pronunciation
3D models of the Turkish Alphabet letters have been created from scratch and have been implemented in the application.
3D models of examples on each Turkish Alphabet Letter have been gathered and implemented successfully in the developed application.
The implementation of sound in this application has also been successful. After each successful recognition of an Image Target, related sound of the letter or the example will be played accordingly.
In the below table it is shown that the Image target representing letter A is not likely to appear based on the image’s feature points, and gave its system gave it 0 out of five stars.
The Second example is the Red Car, though the system deals with the image in black and white, the color of the image here does not affect the performance. The only thing the system to recognize an Image target is its feature points. Here the system gave this example 4 out of five stars, which means that its probability of recognizing the Image Target is high.
The Third example is of a flower. In this example a change has been made on the original chosen Image Target for more probability of recognition. In return, the Vuforia’s system in measuring the probability of an Image Target being recognized gave this image 4 stars out of 5.
40
Table 5.1: Recognition Probability based on Vuforia
41
Comparing these results with the actual developed application results show that unlike the percentage given by Vuforia’s system, the Letter A is always recognizable when pointing the camera to the related Image Target. More test findings show that although it is advised to update the Image Tracker once a low recognition rate is given. Not every low recognition rate means that the respective Image Tracker will not be recognized.
Table 5.2: Image Targets Recognition Test Results
Target Image (Letter) Recognized Note Target Image (Example) Recognized Note
A Yes None Araba Yes None
B Yes None Balloon Yes None
C Yes None Ceket Yes None
Ç Yes None Çîçek Yes None
D Yes/No D struggles until it gets recognized Defter Yes/No Struggles until it gets recognized, this could be because of the size or details of the 3D object
E Yes None El Yes None
F Yes None Fil Yes None
G Yes None Güneş Yes None
Ğ Yes None Ağaç Yes None
42
I Yes None Irmak Yes None
İ Yes None Înek Yes None
J Yes None Jöle Yes None
K Yes None Kedî Yes None
L Yes None Lamba Yes None
M Yes None Masa Yes None
N Yes None Nar Yes None
O Yes None Ot Yes
The presented 3D model object contains a lot of details which makes the application a bit slower than usual
Ö Yes None Ördek Yes None
P Yes None Pencere Yes None
R Yes None Roket Yes None
S Yes/No At times, instead of recognizing S, it recognizes Ş because of the
43 shape
similarity
Ş Yes None Şemsiye Yes None
T Yes None Tavşan Yes None
U Yes None Uçak Yes None
Ü Yes None Üzüm Yes None
V Yes None Vişne Yes None
Y Yes None Yıan Yes None
Z Yes None Zil Yes None
D struggles until it recognizes the image target, but after a few tries, it shows.
Defter also struggles until it is recognized, and when it is finally recognized, the 3D model shakes, it is assumed that it is due to the details and the size of the object and the rendering time.
Even though S letter image target has been modified, the lack of distinguished features of the letter makes it hard to be recognized. In times it miss-represents it with Ş and sometimes it correctly recognizes S.
Because the application was developed to interact with a physical book, Image Targets were somehow forced to be in a certain shape. Almost every issue can be avoided if the designs are only serving the detection purpose nothing else in mind.
All other letters and shapes are recognized instantly when the camera is directed towards the Image Targets.
Based on the above table, only 10.34% of the Image Targets face issues when trying to be recognized, which in return gives an 88.66% of correct recognition of Image Targets.
