The use of electronic curriculums in occupational education in order to evaluate and improve the cognitive capacity of candidate software engineers

(1)

T. C.

Bahçeşehir Üniversitesi

THE USE OF ELECTRONIC CURRICULUMS IN

OCCUPATIONAL EDUCATION IN ORDER TO

EVALUATE AND IMPROVE THE COGNITIVE

CAPACITY OF CANDIDATE SOFTWARE

ENGINEERS

Master Thesis

Barış YÜCE

(2)

T. C.

Bahçeşehir Üniversitesi

The Graduate School of Natural and Applied Sciences

Information Technologies Graduate Program

THE USE OF ELECTRONIC CURRICULUMS IN

OCCUPATIONAL EDUCATION IN ORDER TO

EVALUATE AND IMPROVE THE COGNITIVE

CAPACITY OF CANDIDATE SOFTWARE

ENGINEERS

Master Thesis

Barış YÜCE

Supervisor: Assoc. Prof. Dr. Adem KARAHOCA Co-supervisor: Dilek KARAHOCA

(3)

T. C

BAHÇEŞEHİR ÜNİVERSİTESİ

Graduate School of Natural and Applied Sciences Information Technologies

Title of the Master’s Thesis :The Use of Electronic Curriculums in Occupational Education in Order to Evaluate and Improve the Cognitive Capacity of Candidate System Analysts

Name/Last Name of the Student : Barış YÜCE Date of Thesis Defence : 01.09.2009

The thesis has been approved by the Institute of Graduate School in Sciences.

Signature

Asst. Prof. Dr. Tunç Bozbura Director

This is to certify that we have read this thesis and that we find it fully adequate in scope, quality and content, as a thesis for the degree of Master of Science.

Examining Committee Members:

Assoc. Dr. Adem KARAHOCA (supervisor) :

Asst Prof. Dr. Yalçın Çekiç :

(4)

ACKNOWLEDGEMENTS

I would like to take this opportunity to thank the people who have helped me with this thesis. Firstly, I would like to thank my supervisor Assoc. Prof. Dr. Adem Karahoca and also Dilek Karahoca for their continued support and guidance throughout this research project. Throughout this project they were always available to offer help and advice without which the completion of this study would not have been possible.

I would also like to thank my co-workers for their help; in particular I would like to thank İlker Yengin and Tamer Uçar for the help and motivation which he provided.

Finally, I would like to express my thanks to my family and my fiancée Lynne Cairns for the love and support they have provided me, not only whilst completing this thesis but throughout my life. It is thanks to their encouragement and never-ending belief in me that I have been able to achieve all that I have.

(5)

ÖZET

SİSTEM ANALİSTİ ADAYLARININ YAZILIM MÜHENDİSLİĞİ MESLEKİ EĞİTİMİNDE BİLİŞSEL YETENEKLERİNİN ELEKTRONİK MÜFREDAT

YARDIMI İLE GELİŞTİRİLMESİ VE DEĞERLENDİRİLMESİ

Yüce, Barış

Bilgi Teknolojileri Yüksek Lisans Programı

Tez Danışmanı: Doc. Dr. Adem Karahoca

Ağustos, 2009, 50 sayfa

Bu çalışma 2009 yılı güz döneminde Sistem Analizi dersinin içeriğini oluşturan kavramların ve teorik bilgilerin diğer mesleki disiplin alanlarında ve uygulamada aktif kullanımını arttırmak üzere geleneksel yöntemlerin ötesinde tasarlanan bir eğitim ortamının tasarlanması ve uygulanması hedeflenmiştir. Tasarlanan Harmanlanmış Öğrenme sisteminde sınıf içi yüz yüze ders anlatımlarının ve vaka analizi çalışmalarının yanı sıra elektronik ortamda, öğrencilerin web üzerinden eriştikleri uygulamalar ve testler yürütülmüştür. Öğrencilerin kısa dönem hafıza yetenekleri ile öğrendiklerini vaka analizi çalışmalarındaki tekniklerle pekiştirerek; hafıza kartları ve test uygulamalarındaki kavramları ve vaka analizlerini, farklı durumlarda kullanarak problemleri farklı yöntemlerle çözmeye çalışmaları beklenmiştir.

Anahtar Kelimeler: Harmanlanmış Öğrenme, Elektronik öğrenme, Kritik Düşünme, Sistem Analizi, Öğrenme Stratejileri

(6)

ABSTRACT

THE USE OF ELECTRONIC CURRICULUMS IN OCCUPATIONAL EDUCATION IN ORDER TO EVALUATE AND IMPROVE THE COGNITIVE CAPACITY OF

CANDIDATE SYSTEM ANALYSTS

Yüce, Barış

Information Technologies Master Program

Supervisor: Assoc. Prof. Dr. Adem Karahoca

August 2009, 50 pages

This research is aimed to go beyond the traditional techniques of design and to apply an educational system. This system was designed to allow students to apply the theoretical knowledge and jargon of the System Analysis and Design course in Bahçeşehir University, Engineering Faculty usefully to other branches, in different situations. This course was taken by students in the fall term of 2008. The learning system which has been designed involves some electronic applications and tests in order to make the content more understandable and to improve student’s knowledge. At the end of term it was measured to what extent students were able to use the skills which they had learned from the course. Every week repeated case studies and other tools were employed to exercise students’ short-term memories. Their knowledge was gained through a combination of face-to-face sessions (information from the formal teaching methods and case studies) and online tools (jargon based word games and tests).

Keywords: Blended learning, E-Learning, Critical Thinking, System Analysis, Learning Strategies

(7)

LIST OF TABLES

Table 3. 1: Descriptive Statistics of H1 ... 28

Table 3. 2: Correlations of the scores of online tests and the scores of final grades ... 29

Table 3. 4: Correlations of attendance and the scores of final grades ... 30

Table 3. 6: Correlations of between their attendance and the scores of final grades ... 31

Table 3. 8: Correlations between Case study success and Final grades ... 32

Table 3. 10: Correlations between the final score of the System a Analysis and Design course and the scores on Data Base Management courses... 33

Table 3. 11: Descriptive Statistics of H5... 34

(9)

LIST OF FIGURES

Figure 2.1: How system works ... 16

Figure 2.2: Flash Cards ... 23

Figure 2.3: Who Wants to Be a Computer Genius ... 24

(10)

1. INTRODUCTION

1.1. LITERATURE REVIEW

The central focus of this study is essentially learning and methods which may be used to improved the learning process. In education, a common definition of learning is a process that brings together cognitive, emotional, and environmental influences and experiences for acquiring, enhancing, or making changes in one's knowledge, skills, values, and world views. In addition, Chih-Ming Chen has referred to learning as ‘a process that continues formally or informally throughout our lives’ (Chen 2009) highlighting the fact that the learning process does not end when a person graduates from school or university.

Traditionally the majority of learning has been carried out in the classroom. This is known as classroom learning (c-Learning), it involves face-to-face contact between the teacher and the students. The concept of “distance learning” was coined in the late 1800s, when the idea arose that it was possible to use an instruction method in which the teacher and the learner were not present in the same place at the same time. It was based on the idea that communication could be carried out through letters between the teacher and the learner. This process of distance education continued with radio in the 1930s and was further developed in the 1950s as television began to be used as a means of instruction, thus distance learning has been developed in parallel with technological developments.

The 1990s was an important decade in the development of technology and subsequently in the development of distance learning. In the early the 1990s interactive television was introduced as a means to send information to the distance learner, shortly after the use of closed-circuit television. Other developments included microwave transmission, video recording, and satellite transmission. Furthermore, as a result of the electronic revolution in the 1980s, which saw the development of a range of technologies such as personal computers and CD-ROM, the internet and www, by the mid 1990s the

(11)

transmission of knowledge to the learner could be carried out more rapidly and with more flexibly (Chen 2009).

1.2. DISTANCE LEARNING

In the twenty first century face-to-face learning in classrooms and the use of textbooks continue to be the primary media for formal learning. Nevertheless, the proportion of learning been carried out via distance learning is increasing. However, classroom teaching and textbooks remain important for adult distance learners because they are familiar to them. Indeed distance learning can use face-to-face learning environments to help learners to acquire skills, construct knowledge through discussions and real-life situations, and to provide a social learning context. In addition to textbooks, journals or other printed media can also help learners to study in their own time and place. Thus distance learning does not necessarily mean that traditional methods of teaching and learning need to be abandoned (Chen 2009).

As mentioned above, the use of technologies as a means of learning has greatly increased in recent years. There are two types of available technologies which can be used in distance education; these two groups are synchronous and asynchronous. Synchronous technology is a mode of online delivery where all participants are "present" at the same time requiring a timetable to be organized. Asynchronous technology is a mode of online delivery where participants’ access course materials at anytime they wish. Thus in contrast to synchronous methods students are not required to be together at the same time. Below are some examples of a synchronous and asynchronous technology.

Synchronous technologies • Web based VoIP • Telephone

• Video Conferencing • Web Conferencing

(12)

Asynchronous technologies • Audiocassette • E-mail

• Message Board Forums • Print Materials

• Voice Mail/fax • Videocassette/DVD

According to Bates one reason for the increasing level of interest in some of the newer technologies such as computer-conferencing and the World Wide Web is their potential for asynchronous two-way communication, which allow the student to interact directly and flexibly with a teacher or other students, either in real time or asynchronously. When this is combined with high quality asynchronous interaction with learning material, such as a CD-ROM or a World Wide Web site, then learners can be provided with a very powerful learning environment, whether on or off campus (Bates 1997).

1.3. E-LEARNING

E-Learning allows learners to have access to educational content and have either one way or two way communication with other learners and instructors. This can be carried out through computer networks, intranet, internet and www, providing quick, easy and flexible access for all kinds of content through digital devices, such as CD-ROMs, DVDs, computers and mobile phones. Computer based learning (CBL) has the advantage of allowing self-paced learning using tutorials, drill-and-practice software which can be loaded or installed from CD, DVD or network. As a result of CBLs biggest advantage of allowing learners to learn in their own time and place it has been widely used in distance learning (Karadeniz 2009).

A modified version of CBL is computer-based training (CBT), through CBT a student learns by carrying out special training programs on a computer relating to their occupation. CBT is especially effective for training people to use computer applications because the CBT program can be integrated with the applications so that students can

(13)

practice using the application as they learn. It is used for web-based distance education and involves no face-to-face interaction. However, in addition to this understanding there are also much broader definitions. For example, the term is also used to include all types of technology enhanced learning (TEL), where technology is used to support the learning process (Nichols 2007). Furthermore it is often used interchangeably with various other related terms, such as distance learning, distributed learning, and electronic learning.

Many for-profit higher education institutions are now offering on-line classes. Whereas by contrast, only about half of private, non-profit schools offer such classes. One of the reasons why private institutions may become more involved with on-line education may be the fact that the cost of instituting such a system decreases. This coupled with the fact that The Sloan report, which was based on a poll of academic leaders, found that students generally appear to be at least as satisfied with their on-line classes as they are with traditional ones; this may be a good incentive for such institutions to offer on-line classes (Hebert 2007).

However, CBL does have some disadvantages such as a lack of social interaction, a difficulty in upgrading the content and copyright issues. Nevertheless as online learning technologies such as computer networks, intranet, internet and www have developed and become more widespread, they have become a key media in distance learning environments. Although internet or web based learning have many benefits such as flexible interaction and various media usage, there are still some obstacles in designing online learning. One such obstacle is the fact that it is not feasible to design the content based on video, animation or simulations or to use video conferences frequently because of the low bandwidth of networks. Thus a careful design which weighs the various media use should be taken into consideration (Karadeniz 2009).

1.4. MOBILE LEARNING

In addition to computer based learning M-Learning has started to become an education sector with the rapid increase in the numbers of mobile telephones (Keegan, 2002). The

(14)

wireless handheld devices such as PDA, mobile phone, wireless laptop and tablet PC; which are always on and always with the learner, location aware and individualized, allow learner-learner and learner-instructor interaction from any place and at anytime. However, there are also disadvantages to M-learning, these include limited storage and memory capacity in wireless handheld devices, too small screens, insufficiencies in the usability of user-interface, SMS and MMS capacity, a problem of battery power duration, security, insufficiencies in cross-platform solutions in connection to LMS, high prices of WAP access and devices (Quinn 2000, Berger et al. 2003, McLean 2003, Roberts et al. 2007, Homan & Wood 2003, Luna 2005, Motiwalla 2007, Karadeniz 2009).

1.5. VIRTUAL-BASED TRAINING

Virtual-Based Training is a form of e-Learning that involves a formal online and offline training environment. It resembles a workshop environment but without face-to-face interaction. Here computers and software enable instructors, learners, and peers to interact with one another by sending and receiving messages, reading and writing comments on training materials, taking tests online and giving feedback without having to attend the workshop at fixed schedule (Dabbagh & Bannan-Ritland 2005). In virtual based training the teacher provides course content through course management applications, multimedia resources, the Internet, video conferencing, etc. Students receive the content and communicate with the teacher via the same technologies (Kurbel 2001).

Many virtual study programs are mainly text based, using HTML, PowerPoint, or PDF documents. Multimedia technologies have been investigated for many years and eventually found their way into practice. Virtual-Based Training (VBT) could be used to develop post-secondary students’ abilities in critical thinking (CRT), in their 2005 study Dabbagh and Bannan-Ritland found that their VBT was effective for teaching language and mathematics (Dabbagh & Bannan-Ritland 2005).

(15)

1.6.SELF-REGULATED LEARNING (SRL)

In recent years, learning modes have experienced a revolution due to the rapid growth of internet technologies as introduced above. In conventional classroom learning, learners typically play a passive role as teachers are used to convey knowledge and experiences to learners (Shih et al. 2005). In contrast to this in modern education, learners have a primary role in learning, with learners being directed to construct knowledge based on learner self-analysis, self-examination, and autonomous exploration abilities. Notably, the internet has been a revolutionary tool in education as it can be used to overcome the limitations of time and space.

In recent years self-regulated learning (SRL) has received considerable interest in the education and psychology fields (Zimmerman, Bonner & Kovach 1996, Zimmerman & Schunk 1989). Zimmerman has been at the forefront of such research, self-regulated learning was defined by Zimmerman as ‘the degree to which learners are metacognitively, motivationally and behaviorally active participants in their own learning’ (Zimmerman 1986a, 1986b). That is, self-regulated learning refers to a learning situation in which learners set their own learning goals, plan, and then regulate and evaluate the learning process independently (Narciss, Proske & Koerndle 2007). The goal of SRL is to allow learners to become their own teachers (Zimmerman & Schunk 1989; Torrano Montalvo & Gonzalez Torres 2004).

Zimmerman et al. (1996) referred to various aspects of SRL in different studies when developing their SRL model. Their model contains the following four interrelated learning processes: self-evaluation and monitoring; goal setting and strategic planning; strategy implementation and monitoring; and, strategy outcome monitoring. Below is a further explanation of these processes.

1. Self-evaluation and monitoring: learners assess their learning performance based on self-observations of learning achievement.

2. Goal setting and strategic planning: learners set learning goals by analyzing their learning missions, and then generate appropriate learning strategies to set learning goals.

(16)

3. Strategy implementation and monitoring: learners execute planning learning strategies and monitor the effectiveness of learning strategies via self-monitoring.

4. Strategy outcome monitoring: learners assess learning effects of learning strategies by observing the relationship between learning achievement and learning strategies.

Chen (2009) found that the SRL model can efficiently be used to help learners self-examine and self-evaluate their learning performance by monitoring learning goals set by the individual learners during learning processes. Self regulated learning encourages learners to become active learners. Passive learners have low spontaneous learning abilities and do not know how to plan for autonomous learning. Active learners usually achieve higher learning performance than passive learners. Furthermore, according to Chen learners with a good self-regulated learning ability have a strong will to learn and achieve the goal of lifelong learning.

1.7. BLENDED LEARNING

Hybrid learning and blended learning are two terms which are often used interchangeably to mean the same thing. This can lead to some confusion; therefore for the sake of clarity this paper shall adopt the term hybrid learning. However, it should be noted that is term is intended to cover the term blended learning.

Hybrid learning has been referred to as the “third generation” of distance education systems. The first and second have been discussed above, the first being correspondence education which utilized a one-way instructional delivery method, including mail, radio, and television. The second was distance education with single technology, such as computer-based or web-based learning, (Phipps & Merisotis. 1999).

In general, hybrid learning means any combination of learning delivery methods, mostly including face-to-face instruction with asynchronous and/or synchronous computer technologies (So & Brush 2008).

(17)

Hybrid courses combine face-to-face (FTF) classroom instruction with electronic online devices. A significant amount of learning in a hybrid course occurs online. Hybrid learning can be seen as maximizing the best advantages of face-to-face learning and multiple technologies to deliver learning (Lindsay 2004). Such courses are most frequently used in college or other higher education courses. Institutions of higher education choose this delivery method for various reasons, including the following:

1. Hybrid courses appeal to the market of busy working adults who choose to complete their college-level education beyond their late teens and early twenties. Hybrid courses allow these adults to fit occasional class time into their busy schedule while completing the remainder of the course work over the internet.

2. Hybrid courses reduce pressure on university classrooms. The costs to build and maintain a university are high. Hybrid courses provide a solution to crowded classrooms, since much of the course work is completed on a virtual campus. Recently, the virtual classroom/campus environments as used in colleges and universities, including The University of Edinburgh (UK), Harvard University (USA), and The Open University (UK) are becoming increasingly popular (Parker 2007). Language learning in virtual worlds is the most widespread type of education in virtual spaces, with many universities, mainstream language institutes and private language schools using virtual environments to support learning (8D Taps Language Learners, Bots, Microtransactions 2009).

3. They bring students together only where and when needed, allowing them to self study otherwise. Hybrid learning by means of online discussion forms allows teachers to monitor class discussions. These discussions help teachers to identify topics that need to be clarified or that have captured the interest of students, and they can use the insights gained by them to structure class time. Also, if the teacher uses the Web discussion area to address some of the

(18)

more straightforward student questions, they can make better use of class time.

A typical example of a hybrid learning methodology would be a combination of technology-based materials and face-to-face sessions to present content to students. An instructor can begin a course with a well-structured introductory lesson in the classroom, and then proceed with follow-up materials online. Hybrid learning can also be applied to the integration of e-learning with a Learning Management System using computers in a physical classroom, along with face-to-face instruction (Worthington 2008). It is suggested that face to face instruction should be given early in the process, and to be used less as learners gain expertise (Kirschner, Sweller & Clark 2006). In hybrid learning courses instructors find themselves more in the role of assisting students with computer skills and applications, helping them access the internet, and encouraging them to be independent learners. As outlined above there are many advantages of hybrid learning, and it could be said to ‘offer the best of both worlds’. However, it takes time for both the instructor and learner to adapt to this relatively new instructional concept.

1.8. CRITICAL THINKING

Critical thinking is a thinking skill which consists of mental processes of discernment, analysis and evaluation.

‘Critical thinking is a composite of knowledge, attitudes, and application of skill’ (Staib 2003). Guiller, Durndell and Ross (2008)refer to it as ‘a skill essential in understanding theories in all fields’. It is based on a belief that peer interaction will help learners to assess their current level of understanding.

Kuhn (1991)notes that although there have been many studies carried out on critical thinking in face-to-face and classroom contexts until now there have been very few studies in virtual contexts. He goes on to state that unlike face-to-face situations in the classroom, discussions through the web provide a new way of learning in that the

(19)

learning is still collaborative but asynchronous. A key advantage being that there is more time available for learners to think and search for evidence before they express their thoughts through writing. Subsequently, the style of arguments on the web board will be based more on research-based evidence and will be more formal. According to Kuhn this is a characteristic of good critical thinking.

‘CRT is like a scientist – able to critically examine data and information, to draw the right conclusions and to make important decisions. Critical thinking deals with the use of reasoning in the pursuit of truth’ (Kuhn 1991). Training on critical thinking helps people to form strong defendable opinions and conviction whilst strengthening the feeling of intellectual responsibility. The fourteen components of critical thinkers are as follows: 1. Habitually inquisitive, 2. Well-Informed, 3. Trustful of reason, 4. Open-Minded, 5. Flexible, 6. Fair-Minded in evaluation, 7. Honest in facing

8. Prudent in making judgments, 9. Willing to reconsider

10. clear about issues,

11. Orderly in complex matters,

12. Diligent in seeking relevant information, 13. Reasonable in the selection of criteria and

14. Persistent in seeking results that are as precise as the subject and circumstances of inquiry permit (Zimmerman et al.1996, Jitgarun & Tongsakul 2009)

It is widely accepted that one aspect of critical thinking is the ability to analyze, understand, and evaluate an argument. Gelder (2005) claims that students are actually

(20)

improving their abilities on online discussions, chat, and face to face discussions. Whilst acknowledging that there are difficulties Gelder is optimistic in that he asserts that ‘critical thinking may be difficult but it is certainly not impossible’.

Watson-Glaser (1980) previously defined critical thinking as ‘a composite of attitudes, knowledge and skills which includes attitudes of inquiry that involve an ability to recognize the existence of problems and acceptance of a general need for evidence in support of what is asserted to be true knowledge of the nature of valid inferences, abstractions and generalizations in which the weight or accuracy of different kinds of evidence are logically determined and skills in employing and applying the above attitudes and knowledge’ (Evancho, 2000).

In a recent study, Wang & Woo (2007) studied the effectiveness of class discussion in face-to-face and online settings. Their focus was on how both settings have context-specific advantages. They found that face-to-face discussions generally have greater efficiency, and the advantage of immediate feedback whilst being free from the technological issues which can be confronted when dealing with online settings. Furthermore face-to-face discussions provide a greater perceived interactivity and important verbal and non-verbal communication cues (Akyuz & Samsa 2009).

Critical thinking may involve determining the meaning and significance of what is observed or expressed, or concerning a given inference or argument, determining whether there is adequate justification to accept the conclusion as true. Hence, Fisher & Scriven (1997) define critical thinking as "Skilled, active, interpretation and evaluation of observations, communications, information, and argumentation. Whereas Parker & Moore define it more narrowly as the careful, deliberate determination of whether one should accept, reject, or suspend judgment about a claim and the degree of confidence with which one accepts or rejects it (Brooke Noel Moore 2004).

According to Sumner critical thinking calls for a persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusions to which it tends. It can occur whenever a person judges, decides, or

(21)

solves a problem. In general terms it occurs whenever one must figure out what to believe or what to do, and do so in a reasonable and reflective way. Reading, writing, speaking, and listening can all be done critically or uncritically. Critical thinking is crucial to becoming a close reader and a substantive writer. Expressed most generally, critical thinking is “a way of taking up the problems of life.” (Sumner 1906).

Accordingly Fisher & Scriven(1997), claim that critical thinking is an important element of all professional fields and academic disciplines (by referencing their respective sets of permissible questions, evidence sources, criteria, etc.). The process of critical thinking involves the careful acquisition and interpretation of information and use of it to reach a well-justified conclusion. The concepts and principles of critical thinking can be applied to any context or case but only by reflecting upon the nature of that application.

'Critical thinking forms a system of related, and overlapping, modes of thought such as anthropological thinking, sociological thinking, historical thinking, political thinking, psychological thinking, philosophical thinking, mathematical thinking, chemical thinking, biological thinking, ecological thinking, legal thinking, ethical thinking, musical thinking, thinking like a painter, sculptor, engineer, business person, etc.'(Fisher & Scriven 1997).

In other words despite the fact that critical thinking principles are universal, their application to disciplines requires a process of contextualization. Critical thinking is important, because it enables a person to analyze, evaluate, explain and restructure their own thinking thus decreasing the risk of adopting, acting on, or thinking with, a false belief (Fisher & Scriven 1997).

Parker and Moore (2005) identify two phases to the learning of content. The first involves a process of internalization. It occurs when learners, for the first time, construct in their minds the basic ideas, principles, and theories that are inherent in content. The second phase takes place when learners effectively use those ideas, principles, and theories as they become relevant in the learners’ own lives. This is a process of

(22)

application. Parker and Moore believe that good teachers cultivate critical thinking at every stage of learning, including initial learning. At the heart of critical thinking is the belief that all students must do their own thinking; they must be responsible for their own construction of knowledge.

1.9. AIMS AND HYPOTHESIS

Following hypothesis are considered to prove the achievements of the e-learning supported learning methodologies.

H 1. The success of the System Analysis and Design course affects the results of the Data Base Management System course which has an advanced content of system analysis and design.

H 2. The tests on the online system increase student’s success on final grades. H 3. The students, who attend the course more, succeed with higher final grades. H 4. The students, who have more flashcard scores, also have higher final grade

scores.

H 5. The success of case studies effect final grades.

H 6. The gender is an important factor which affects the success of System Analysis and Design course.

(23)

2. METHODOLOGY

2.1. FACE TO FACE DISCUSSIONS

This research is based on Bahçeşehir University’s System Analysis and Design course. Every week there were three hours of face to face sessions. In these sessions a lecturer taught the content to the students for two hours every week with the materials such as PowerPoint presentations, glossaries and by other virtual methods. In the other hour we discussed how to solve a problem regarding a case which was either fictional or non-fictional; students were required to solve these problems using their current knowledge.

2.2. CASE STUDIES

A case study is one of several ways of carrying out research whether it is social science related or even socially related. It involves the intensive study of a single group, incident, or community (Shepard & Greene 2003).

Rather than using samples and following a rigid protocol to examine a limited number of variables, case studies involve an in-depth, longitudinal examination of a single instance or event i.e. a case. They provide a systematic way of looking at events, collecting data, analyzing information and reporting the results. As a result of the information gained from the case study the researcher may gain a better understanding of why events happened as they did and what might become important to look at in more in future research. Case studies involve both the generation and the testing of hypotheses (Flyvbjerg 2006).

Case studies produce the type of context-dependent knowledge that research on learning shows to be necessary to allow people to develop from rule-based beginners to virtuoso experts. According to Flyvbjerg (2001) in the study of human affairs, there appears to exist only context-dependent knowledge, which presently rules out the possibility of epistemic theoretical construction.

(24)

It is also claims that in a teaching situation, well-chosen case studies can help the student achieve competence, whereas context-independent facts and rules will bring the student just to the beginner’s level. However, only few institutions of higher learning have taken the consequence of this in to account. Here both teaching and research in the professional schools are modelled to a wide extent on the understanding that case knowledge is central to human learning (Christensen, 1987; Cragg, 1940).

Flyvbjerg (2006) does not claim that rule-based knowledge should be discounted on the contrary he believes that it is important in every area and especially to novices. Carefully chosen experiments, cases, and experience were also critical to the development of the physics of Newton, Einstein, and Bohr, just as the case study occupied a central place in the works of Darwin, Marx, and Freud. In social science as in other fields the strategic choice of case may greatly add to the generalizability of a case study.

Flyvbjergs’ (2006) conclusion is that conventional wisdom about case-study research is wrong and often misleading. He sees the sharp separation between qualitative and quantitative methods as a false one which has occurred as a result of power relations and time constraints in graduate training. He does not see this to be a logical consequence of what graduates and scholars need to know in order to do their studies and to do them well.

Kuhn (1987) claimed that in the social sciences the balance between case studies and large samples is currently biased in favour of the latter, to such an extent that it puts case studies at a disadvantage within most disciplines. Kuhn believes that a discipline without a large number of thoroughly executed case studies is a discipline without systematic production of exemplars, and that a discipline without exemplars is an ineffective one. In order to solve this problem in the social sciences a greater number of good case studies are needed (Flyvbjerg 2006).

In this vein it is claimed that more often than not, a combination of qualitative and quantitative methods will do the task best. Indeed the situation is changing as there

(25)

seems currently to be a general relaxation in the old and unproductive separation of qualitative and quantitative methods.

(26)

2.3. ONLINE COURSE MATERIALS

As the knowledge source, tests and practice materials we used the book of Shelly

Cashman’s Systems Analysis and Design (Seventh Edition) as a guideline.

2.3.1. Notes

In every week and for each unit, students were able to reach course notes and presentations which allowed them to study and follow the course. These documents are basically the summaries of the chapters which the students were studying in class. According to the authors of the course book and the educational designer group of “System Analysis and Design” the objectives were determined for every chapter. This course involves six chapters, the learner objectives of which are shown below (Cashman, Cashman & Rosenblatt 2007):

Objectives of chapter 1

• Discuss the impact of information technology on business strategy and success • Define an information system and describe its components

• Explain how profiles and models can represent business functions and operations

• Explain how the Internet has affected business strategies and relationships • Identify various types of information systems and explain who uses them • Explain modeling, prototyping, and CASE tools

• Distinguish between structured analysis and object-oriented analysis

• Compare the systems development life cycle waterfall model to adaptive models • Discuss the role of the information technology department and the systems

analysts who work there

• Explain the concept of a business case and how a business case affects an IT project

(27)

• Explain the purpose of a mission statement

• Describe the SDLC, and explain how it serves as a framework for systems development and business modeling

• Describe risks and risk management features

• List the reasons for information systems projects and the factors that affect such projects

• Describe systems analysis phase activities and the end product of the systems analysis phase

• Explain joint application development (JAD) and rapid application development (RAD)

• Understand how systems analysts use a functional decomposition diagram (FDD)

• Describe the Unified Modeling Language (UML) and explain use case diagrams and sequence diagrams

• List and describe system requirements, including outputs, inputs, processes, performance, and controls

• Explain the importance of scalability in system design

• Use fact-finding techniques, including interviews, documentation review, observation, questionnaires, sampling, and research

• Define total cost of ownership (TCO) and explain the concept • Conduct a successful interview

• Develop effective documentation methods to use during systems development Objectives of chapter 4

• Describe data and process modeling concepts and tools, including data flow diagrams, a data dictionary, and process descriptions

• Describe the symbols used in data flow diagrams and explain the rules for their use

• Draw data flow diagrams in a sequence, from general to specific • Explain how to level and balance a set of data flow diagrams

(28)

• Describe how a data dictionary is used and what it contains

• Use process description tools, including structured English, decision tables, and decision trees

• Describe the relationship between logical and physical models Objectives of chapter 5

• Explain how object-oriented analysis can be used to describe an information system

• Define object modeling terms and concepts, including objects, attributes, methods, messages, classes, and instances

• Explain relationships among objects and the concept of inheritance • Draw an object relationship diagram

• Describe Unified Modeling Language (UML) tools and techniques, including use cases, use case diagrams, class diagrams, sequence diagrams, state transition diagrams, and activity diagrams

• Explain the advantages of using CASE tools in developing the object model • Explain how to organize an object model

• Describe software trends, including the concept of software as a service

• Explain software acquisition alternatives, including traditional and Web-based software development strategies

• Describe software outsourcing options, including offshore outsourcing and the role of service providers

• Explain advantages and disadvantages of in-house software development • Explain cost-benefit analysis and financial analysis tools

• Explain the differences between a request for proposal (RFP) and a request for quotation (RFQ)

• Describe the system requirements document

• Explain the transition from systems analysis to systems design, and the importance of prototyping

(29)

• Discuss guidelines for system design

• Describe future software development trends

According to the curriculum of Bahçeşehir University, these objectives constitute the fundamentals of the courses of Database Management Systems and Human Computer Interaction. In order to be able to design a more effective database, it is essential to fully understand the way in which the system works. Therefore the final scores of “System Analysis and Design” and “Database Management Systems” courses were compared.

It has been suggested that one of the most efficient ways to create an efficient learning system is to employ a method of packaging whereby content can be reused. Packages of learning content and associated descriptive metadata are produced to facilitate the portability of learning content between different Learning Management Systems (LMS). The packages usually contain additional information about how the amalgamated learning content should be sequenced and additional metadata that describes the learning resources.

As the learning content, may be stored in searchable repositories, the course author may simply have to search for pre-authored material and combine it into a new offering. The course author may produce additional packages tailored to specific learning approaches and learner prior knowledge, while still supporting a common learning objective in the different course offerings. This may be done by re-sequencing the learning content in the package or by adding additional content, alternatively additional content that supports different approaches to learning may be added.

Multiple content packages may be aggregated together to assemble an aggregated content package. This allows a learner’s requirements to be met from more than one package (Conlan, Dagger & Wade 2002).

The students in the test group we studied with were from two different departments: software engineering and computer engineering. The System Analysis is normally an

(30)

Industrial Engineering originated topic and these students did not have any prior knowledge about this topic unlike the Industrial Engineering students.

For this reason it was decided that the information which was provided to students in the course book, using complicated terms, should be simplified. The way in which this was done was by providing students with more basic and more understandable chapters. This can be seen as a form of packaging, as the information was summarized into smaller parts for students with less prior knowledge. The aim was to teach the topic to these students with less information, relying more on practices and case studies.

2.3.2. Glossaries

In university engineering faculties students have to take mostly numerical courses. In system analysis topics, it is consequentially difficult for the candidate engineering students to learn and to memorize the jargon that is largely unknown to them. Therefore we needed some other applications, such as flash cards and puzzle games, in order to motivate the students to learn.

2.3.3. Games

Games can be a valuable and fun tool to aid students in the learning of new concepts and to facilitate students to memorize new glossary terms. In 1997 Keys carried out extensive research on the use of concepts, cases and games in learning environments in order to assess how these can be best employed. His findings are summarized below:

1. Cases and games were found to be superior to cases alone in a business policy course.

2. A multiple choice or objective test is not subtle enough to capture game or case-type learning.

3. In terms of faculty student time tables, a simple game and cases may be more worthwhile than a complex game and cases.

4. A case-type class is superior to a game-only class in teaching business policy, corporate strategy, and production management fact mastery.

(31)

5. There is some indication that game time can be effectively substituted for lecture time in a marketing class, when similar game decisions, objectives, and readings are assigned to both classes.

6. A game-only business policy class produces superior results to a case-only business policy class where significant instructor guidance is provided in the game-only class.

7. A case-game class is superior to a game-exercise class in teaching business policy concepts when the instructor remains relatively passive in the game- exercise class.

When the total teaching mixture is examined in each of the studies reviewed, with the role of the instructor carefully examined, there appears to be no conflict in the findings. In teaching business policy concepts, cases, and games appear to be superior to cases alone, games alone, or games plus exercises, if the instructor is allowed to play an active and significant role in guiding game learning. In addition, given a reasonably effective blend of game complexity, cases and instructor guidance, course time can usually be made more effective by trading reflective learning, assignments, and discussions for game complexity. There is some indication that functional courses can afford to trade class lecture time for game activities and still produce the same effectiveness on essay-type exams (Keys 1997).

Bikramjit Banerjee and Peter Stone (2007) have also researched the use of games as learning aids. They present a reinforcement learning game player that can interact with a General Game Playing system and transfer knowledge learned in one game to expedite learning in many other games.

2.3.4. Flash cards

Flashcards are used as a learning drill to aid memorization. A description is given on one side of a card and an answer is provided overleaf. Students played these cards after face to face discussions; they were intended to help students to memorize the terms in every chapter. As Prensky (2004) emphasizes, the use of flashcards is particularly

(32)

effective with regard to the learning of acronyms, foreign languages, business or professional jargon.

Figure 2.2: Flash Cards (Rosenblatt, n.d.)

2.3.4.1.Who Wants to Be a Computer Genius

This Discovering online game tests the students' computer knowledge as they try to answer 15 questions correctly to become a Computer Genius on a specific topic from the System Analysis Course. Each student should attempt this at least once a week (current chapter assigned) and keep their own score.

(33)

Figure 2.3: Who Wants to Be a Computer Genius (Rosenblatt, n.d.)

2.3.4.2.Wheel of Terms

This is another game aimed to teach important keywords to users. In this game there is a wheel of points, and with a random point, students try to guess a letter which belongs to the keyword. If students guess correctly they increase their total points.

2.3.4.3. Crossword Puzzle Challenge

Some researchers including Prensky claim that puzzle games, such as the one shown below, can be an effective tool in the topics of strategic and tactical thinking and quality analysis (Prensky 2004).

(34)

Figure 2.4: Crossword Puzzle Challenge (Rosenblatt, n.d.)

2.3.5. Tests

2.3.5.1. Multiple choice test

Multiple choice tests are a form of assessment in which respondents are asked to select the best possible answer or answers out of the choices from a list. There are several advantages to multiple choice tests. Provided test writers are well trained and the items are quality assured, multiple choice tests can be a very effective assessment technique (Item Writing Manual 2008).

According to the Item Writing Manual, which was published by the National Board of Medical Examiners, if students are instructed on the way in which the item format works and myths surrounding the tests are corrected, they will perform better on the test (Beckert, Wilkinson & Sainsbury 2003).

Downing 2004 found that on many assessments, reliability was shown to improve with larger numbers of items on a test. Furthermore with good sampling and care over case specificity, overall test reliability can be further increased (Downing 2004).

(35)

A key advantage of multiple choice tests is that they often require less time to administer for a given amount of material than would tests requiring written responses. This results in a more comprehensive evaluation of the candidate's knowledge. Even greater efficiency can be created by the use of online examination delivery software.

Benjamin et al (1984) claim that the much cited theory that a student should trust their first instinct and stay with their initial answer on a multiple choice test is indeed a myth. On the contrary researchers have found that changing answers generally results in a higher test score.

2.3.5.2. True / False test

In contrast to some other test forms true/false questions can be easier to prepare and quicker to read and answer. This means that the number of questions in a test can be increased more easily with the result that the test offers a more comprehensive evaluation of the candidate's knowledge. Moreover, with true/false tests, it is possible to ensure that partial knowledge is restricted to a ‘lack of confidence’.

2.3.5.3. Short Answer test

Pressley et al found that students had great confidence that their answers to thematic questions in both short-answer and multiple-choice formats were correct, even when they were wrong. Importantly, students' overconfidence in answers to thematic questions was not related to their verbal ability. The authors conclude that when adults read challenging, inconsiderate texts, they may often be unaware of gross comprehension problems. There is a limited amount of research on this topic, thus future research is necessary in order to determine how common such serious misperceptions are among adults (Pressley et al. 1990).

One problem with multiple choice and true/false tests is that they commonly contain only a small proportion of all the questions that could be asked, meaning that an examinee may be lucky or unlucky in the selection, in addition to this drawback random guessing further reduces test reliability when it is not effectively discouraged. Burton

(36)

2001 tested the effects of question selection and guessing, both separately and together and concluded that as a result of test length and the number of answer options per question, many multiple choice tests are too unreliable for their purpose as currently conducted (Burton 2001).

2.3.5.4. Practice Tests

Donald Bacon compared student scores on multiple-choice and short-answer portions of a midterm exam in consumer behavior in terms of completion time, reliability, and validity. The multiple-choice format was shown to provide equivalent reliability and validity in a shorter amount of test-taking time than short answer sections. A further finding was that in contrast to some earlier studies, no gender effect was found (Bacon 2003).

Practice tests, as shown below, were offered to allow students to learn their level and also to identify their mistakes. The system allows them to take these tests many times and thus gives students an opportunity to correct their errors. These tests were intended to increase students’ achievements by showing their weak sides to them. They were encouraged to work on their weaknesses and were able to take the same tests again.

(37)

3. FINDINGS

In this section, we argued the hypothesis of this study and tried to realize our observations in the classes.

H1. The tests on the online system increase student’s success on final grades.

The correlation tests revealed that there is a positive relationship between the scores of online practice tests and the scores of final grades.

r₍ ₎ =. 713, p <. 01

Table 3. 1: Descriptive Statistics of H1

Mean Std. Deviation N

Tests 63,2897 20,56423 145

Final 50,0434 17,54904 145

According to the results of the “test success and final scores test results statistics” as displayed in the table above, the average test scores and standard deviation of 145 students are 63.28 and 20.56 respectively. The final grades and standard deviations are 50.04 and 17.54.

(38)

Table 3. 2: Correlations of the scores of online practice tests and the scores of final grades

Tests Final

Tests Pearson Correlation 1 ,713**

Sig. (2-tailed) ,000

N 145 145

Final Pearson Correlation ,713** 1

N 145 145

**. Correlation is significant at the 0.01 level (2-tailed).

The correlation results show that; at the 0.01 level, there is a significant relationship (.713) between training tests and final results.

(39)

H2. The students, who attend the course more, succeed with higher final grades.

The correlation tests showed that, as hypothesized there is a positive relationship between attendance and the scores of final grades.

r₍ ₎ =. 657, p <. 01

Attend 65,10 21,802 145

Final 50,0434 17,54904 145

According to results of the “measured course attendance and final scores test results statistics” displayed above, the averages of the attend score and the standard deviation of 145 students is 65.10 and 21.80 correspondingly. The final grades and standard deviations are 50.04 and 17.54.

Table 3. 4: Correlations of attendance and the scores of final grades

Attend Final

Attend Pearson Correlation 1 ,657**

N 145 145

The correlation result shows that; at the 0.01 level, there is a significant relationship (.657) between attendance and final results.

(40)

H3. The students, who have more flashcard scores, also have higher final grade scores.

As hypothesized, it was shown through the correlation tests that there is a positive relationship between their attendance and the scores of their final grades.

r₍ ₎ =. 668, p <. 01

Flash Cards 76,2621 17,03306 145

Final 50,0434 17,54904 145

According to the results of the “measured Flash cards and final scores test results statistics”, the averages of the Flashcard scores and standard deviation of 145 students are 76.26 and 17.03. The final grades and standard deviations are 50.04 and 17.54 respectively.

Table 3. 6: Correlations of between their attendance and the scores of their final grades

Flash Cards Final

Flash Cards Pearson Correlation 1 ,668**

N 145 145

**. Correlation is significant at the 0. 01 level (2-tailed).

The correlation results tabled above show that at the 0.01 level, there is a significant relationship (.668) between flashcard results and final results.

(41)

H4. The success of case studies effect final grades.

As hypothesized, the correlation tests revealed that there is a positive relationship between their case study grades and the scores of final grades.

r( ) =. 710, p <. 01

Case Studies 50,98 18,637 145

Final 50,0434 17,54904 145

According to results of the “measured case study and final scores test results statistics”, the averages of the Flashcard scores and standard deviation of 145 students are 50.98 and 18.63. The final grades and standard deviations are 50.04 and 17.54 respectively.

Table 3. 8: Correlations between Case study success and Final grades

Case Studies Final Case

Studies

Pearson Correlation 1 ,710**

N 145 145

**. Correlation is significant at the 0. 01 level (2-tailed).

The correlation results in the above table show that; at the 0.01 level, there is a significant relationship (.710) between case study results and final results.

(42)

H5. The success of the System Analysis and Design course affects the results of the DBMS course which has an advanced content of system analysis and design.

The results of the correlation tests confirmed the hypothesis that there is a positive relationship between the final score of the System a Analysis and Design course and the scores on Data Base Management courses.

( ) =. , <. 0

System Analysis 50,1013 19,15764 79

DBMS 57,8354 24,44273 79

According to the results of the “System Analysis and Design” course final grades and the “DBMS course final scores test results statistics”, the averages of the Flashcard scores and standard deviations of 79 students are 50.10 and 19.15 respectively. The final grades and standard deviations are 57.04 and 24.44.

Table 3. 10: Correlations between the final score of the System a Analysis and Design course and the scores on Data Base Management courses.

System Analysis DBMS System Analysis and Design Pearson Correlation 1 ,560** Sig. (2-tailed) ,000 N 79 79 Database Management Systems Pearson Correlation ,560** 1 Sig. (2-tailed) ,000 N 79 79

(43)

H6. Gender is an important factor which affects the success of System Analysis and Design course.

An independent sample T-test revealed that, as hypothesized there is a an important difference between females (M=56. 01; S. D. = 15. 04) and males (M=47,99; S. D. = 17,93) of the students which took the System Analysis and Design course on their final grades. As Independent Samples Test results show (t( ) = 2. 440, p =. 016, p <

. 05), female students achieved higher final grades than male students.

t( )= t ., p <

t( )= 2. 440, p =. 016, p <. 05

gender N Mean Std. Deviation Std. Error Mean

Final 1 108 47,9973 17,93741 1,72603

2 37 56,0157 15,04469 2,47333

According to a comparison of the gender and success of students, we found the average grades of 108 male students to be 48.99 with a standard deviation of 17.93 and standard error mean values of 1.72, on the other hand the 37 female students have an average grade of 56.01, with a 15.04 standard deviation, and 2.47 standard error mean.

(44)

Table 3. 12: Independent Samples Test results of gender and success relationship Levene's Test for Equality of Variances

t-test for Equality of Means

F Sig. t df Sig. (2- tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper F ina l Equal variances assumed 2,201 ,14 -2,440 143 ,016 -801,836 328,688 -1,451,551 -152,121 Equal variances not assumed -2,659 73,72 ,010 -801,836 301,605 -1,402,835 -200,838

(45)

4. CONCLUSIONS AND FUTURE PLANS

The aim of this research was to improve the cognitive capacity of software engineers (candidate system analysts) via face to face discussion environments with an e-learning system support. These students (n=147) were 3rd and 4th year students, in the fall semester of 2008.

This research sought to discover, what effect if any students approach to the course had on their performance. E-learning materials were used to increase the motivation of students’ who were used to taking numerical courses in System Analysis and Design Course which involve mostly verbal knowledge. The values which were measured were about the behaviors of the students regarding the course. These behaviors were categorized under attendance, joining the tests and playing games. It also researched the relationship between their success and gender. Another measurement is their ability to use the knowledge that they had learned in the System analysis and design course in different course, “Data base management systems”

Consequently according to statistic calculations, their attendance, test scores, flash card scores and case study scores are all strongly related to their final grades as the results below:

1. There was found that there is a positive correlation between the scores of online practice tests and the scores of final grades,

2. There is a positive relationship between attendance and the scores of final grades,

3. There is a positive correlation between the student’s attendance and the scores of their final grades,

4. There is a positive relationship between their case study grades and the scores of final grades,

5. There is a positive relationship between the final score of the System a Analysis and Design course and the scores on Data Base Management courses. These

(46)

results show that the students who attitude to the course responsibilly, succeded higher marks above.

6. There is another significant difference between females and males in their training tests and final grades.

As result, we found a significant correlation between these two lessons. This result shows us, the system which is used affects students’ long-term learning success in an affirmative way.

All these features show how students interact with the course. It is clear from the results of this research that mostly the students who were more responsible to the course, had succeeded with higher grades. All these arguments are related to their short-time learning memory. In order to measure the effects of this blending learning system on the teaching of the System Analysis and Design course, we compared their success in DBMS course.

Another interesting finding is that, female students achieved higher final grades than male students. An important detail about this finding is the number of the girls in the course: in the course only %40 of students were boys. We suggest researching this result in a course with equal number of each gender.

(47)

5. BIBLIOGRAPHY

Akyuz, H.I. & Samsa, S., 2009. The effects of blended learning environment on the critical thinkingnext term skills of students. In World Conference on Educational

Sciences., 18 March 2009.

Bacon, D.R., 2003. Assessing Learning Outcomes: A Comparison of Multiple-Choice and Short-Answer Questions in a Marketing Context. Journal of Marketing Education, 25(1), pp.31-36.

Banerjee, B. & Stone, P., 2007. General Game Learning using Knowledge Transfer. In

The 20th International Joint Conference on Artificial Intelligence. pp.672-77.

http://www.cs.utexas.edu/~pstone/Papers/bib2html-links/IJCAI07-bikram.pdf. Bates, T., 1997. The impact of technological change on open and distance learning. Distance Education.

Benjamin, L.T., Cavell, T.A. & Shallenberger, W.R., 1984. Staying with the initial answers on objective tests: Is it a myth?. Teaching of Psychology, pp.133-41.

Berger, S., Mohr, R., Nösekabel, H. & Schäfer, K.J., 2003. Mobile Collaboration Tool for University Education. Proceedings of the Twelfth International Workshop on

Enabling Technologies: Infrastructure for Collaborative Enterprises, p.77.

Brooke Noel Moore, R.P., 2004. Critical Thinking. McGraw-Hill; 7th edition (August 1, 2007).

Burton, R.F., 2001. Quantifying the Effects of Chance in Multiple Choice and True/False Tests: Question Selection and Guessing of Answers. Assessment &

Evaluation in Higher Education, pp.41-50.

http://pdfserve.informaworld.com/490802_758064766_713611459.pdf.

Cashman, S., Cashman, T.J. & Rosenblatt, H.J., 2007. Systems Analysis and Design,

Seventh Edition. Course Technology; 7 edition.

Chen, C.-M., 2009. Personalized E-learning system with self-regulated learning assisted mechanisms for promoting learning performance. In Expert Systems with Applications:

An International Journal. Ch. 5. pp.8816-8829.

http://dx.doi.org/10.1016/j.eswa.2008.11.026.

Downing, S.M., 2004. Reliability: on the reproducibility of assessment data Medical Education. Ch. 9.

Fisher, A. & Scriven, M., 1997. Critical Thinking: Its Definition and Assessment,

Center for Research in Critical Thinking (UK). Edgepress (US).

Fisher, A. & Scriven, M., 2002. Critical Thinking. Its Definition and Assessment. In Glopper, K.d. Argumentation. Springer.

Flyvbjerg, B., 2001. Making social science matter: Why social inquiry fails and how it