Evaluation of Students’ Learning Behaviour and
Success as a Prerequisite for Modernizing Practical
on Campus Networking Courses in Higher Education*
Yüksekö¤retimdeki kampüs içinde uygulamal› a¤ kurma derslerini modernlefltirmede bir ön gereklilik olarak ö¤rencilerin ö¤renim davran›fl›n›n ve baflar›s›n›n de¤erlendirilmesiJens Haag1,2, Christian Witte1, Stefan Karsch1, Harald Vranken2, Marko van Eekelen2,3
1Cologne University of Applied Sciences, Cologne, Germany, 2Open Universiteit, Heerlen, The Netherlands, 3Radboud University, Nijmegen, The Netherlands
C
C
omputer science curricula for students at universities nowadays include courses on networking and infor-mation technology (IT) security. Teaching theory on networking and IT security is usually done by means of textbooks and classes (either face-to-face classes or virtual classes, which are popular at universities for distanceeduca-tion). To anchor and deepen the acquired theoretical knowl-edge, a commonly used teaching method is to hand out prac-tical assignments. While solving the assignments can be vol-untary, a mandatory practical course can be used to guide the students at the university while working on the assignments and to finally verify that the students can successfully apply
Yüksekö¤retimdeki bilgisayar bilimleri müfredat›, ö¤rencilerin teorik bilgi-lerini uygulayabildikleri, derinlefltirebildikleri ve sa¤lamlaflt›rd›klar› uygula-mal› dersleri içermektedir. Üniversiteler, e¤itim sistemine uygun ö¤renim ortam› sunmak ve ayr›ca ö¤rencilerin ihtiyaçlar›n› karfl›lamak zorundad›r. Ö¤rencileri destekleyecek çeflitli teknik konseptlere sahip olman›n yan› s›ra e¤itim personeli, bu konseptlerin mevcut bir ö¤renim ortam›na uygun olup olmad›¤›na ve bunlar›n gelifltirilmeye müsait olup olmad›klar›na karar vere-bilmek amac›yla ö¤rencilerin ö¤renim davran›fllar›n› vere-bilmek zorundad›r. ‹ki-yüzün üzerinde kat›l›mc›ya sahip uygulamal› bir üniversite dersi, ö¤renim baflar›s›, ö¤rencilerin ö¤renim davran›fl› ve tercih ettikleri ortam bak›m›ndan de¤erlendirilmifltir. De¤erlendirme, dersin genel olarak baflar›l› oldu¤unu, ço¤u ö¤rencinin dersi geçti¤ini ve bireysel ö¤renim baflar›lar›n› yüksek sevi-yeye ç›kard›¤›n› göstermektedir. Ancak yüksek derecede esneklik göz önüne al›nd›¤›nda ö¤rencilerin büyük bir ço¤unlu¤u, arzu ettikleri ö¤renim ortam› olarak, ders dan›flmanlar›yla desteklenen üniversite laboratuvar›nda grup ha-linde çal›flmay› seçmifltir. Karma ö¤renim yöntemlerini kullanarak bir kam-püs içi uygulamal› dersi modernlefltirirken, bu sonuçlar dikkate al›nmal›d›r. Anahtar sözcükler:e-Ö¤renim, ö¤renim davran›fl›, uygulamal› ders, a¤ kur-ma, k›lavuzlu ö¤renim, ders dan›flmanl›¤›, de¤erlendirme.
Computer science curricula in higher education include practical cours-es, where students can apply, deepen and anchor their theoretical knowl-edge. Universities have to provide a learning environment which fits into the educational system and also meets the needs of the students. While having various technical concepts to support the students, educational staff has to know the students’ learning behaviour in order to decide, whether these concepts fit into and can improve upon an existing learn-ing environment. A practical university course with more than 200 par-ticipants was evaluated with regard to learning success, students’ learning behaviour and their preferred environments. The evaluation shows that the course was generally successful, most of the students passed and rate their individual learning success as high. Although given a high degree of flexibility, a majority of students chose working in a group at the univer-sity’s laboratory supported by course advisors as their desired learning environment. These outcomes should be considered when modernizing a practical on-campus course by introducing methods of blended learn-ing.
Keywords:e-Learning, learning behaviour, practical course, networking, guided learning, course advisory, evaluation.
‹letiflim / Correspondence: Jens Haag
Cologne University of Applied Sciences, Cologne, Germany
e-mail: jens.haag@outlook.de
Yüksekö¤retim Dergisi 2014;4(2):83–90. © 2014 Deomed
Gelifl tarihi / Received: Kas›m / November 1, 2013; Kabul tarihi / Accepted: Ocak / January 24, 2014
*This work has been done by first author as a PhD student of the Open Universiteit, Heerlen, The Netherlands. This paper is an extension of a presentation at the Second International Conference on e-Learning and e-Technologies in Education (ICEEE), Sept. 23-25, 2013, Lodz, Poland.
Öz Abstract
their theoretical knowledge. The Open Universiteit (The Netherlands) as well as the Cologne University of Applied Sciences (Germany) offer courses in the field of computer net-works and IT security. While the Open Universiteit is a uni-versity for distance education, the Cologne Uniuni-versity of Applied Sciences is a traditional on-campus university. Since both offer practical courses in networking and IT security, these universities started a research cooperation in 2009 in order to improve the quality of learning by means of better support the students and to grant them more flexibility with respect to their learning environment. The Cologne University of Applied Sciences provides its students with a classic computer lab to work in while the Open Universiteit has developed a virtual computer security lab (VCSL) that can be run on any student's personal computer, so that there is no need for university-supplied work station computers (Vranken and Koppelman, 2009). During the research cooperation this VCSL was further developed into a distributed VCSL (DVCSL) as a proof of concept (Haag et al., 2011; Vranken et al. 2011). This DVCSL enables students to work together in a virtual lab even if they are physically distant from each other. Furthermore, a concept for an electronic course advisor (Haag et al., 2012) was developed and introduced that can guide stu-dents in working on their assignments, regardless of the lack of a physically present human course advisor.
In a steadily changing world with new technologies, grow-ing mobile connectivity and every time-everywhere character it seems natural that these changes may impact the students’ needs and requirements regarding their learning environ-ment. With respect to students working on assignments in a physical lab at a traditional university, a growing demand for computer supported learning environments can be expected.
The obvious question is whether the aforementioned con-cepts and prototypes fit into and can improve upon a classic on-campus learning environment. A way for educational staff to answer this question is to interrogate the students about their opinions (acceptance research). In the winter semester 2012 a practical course was evaluated, where students have to work out and solve assignments, and to defend their solutions. A special property of this course is that the students are given a high degree of flexibility during their assignment prepara-tion time. In order to evaluate and optimize the alignment of course concepts and technical implementations in place, the success of courses and actual learning behaviour of the stu-dents were analysed using a questionnaire.
The remaining sections are organized as follows: first, an overview of related work will be given followed by an introduc-tion to the observed course and its design, accompanied by an example exercise. To interrogate the learning behaviour and success, the questions used in the evaluation questionnaire and
the results of this evaluation are then shown and interpreted. Finally the findings will be summarized and concluded.
Related Work
Computer security labs are of great value in courses that teach networking or security of computer systems (Sloan and Schlindwein, 2004; Gaspar, Langevin, and Armitage, 2007; Standard et al., 2013). Initially, a computer security lab was a room containing computer systems connected in a network, completely isolated from the outside world (Bishop and Heberlein, 1996). Some modernized labs apply virtualization in order to reduce the administration effort (Bullers, Jr., Burd, and Seazzu, 2006; O'Leary, 2006; Gephart and Kuperman, 2010). Isolated computer security labs at universities are however not suited for distance teaching, since students may not be able to travel to the lab due to restrictions on time or distance. Distance teaching is accommodated by providing remote access to the lab over secure network connections. Examples of such computer security labs with remote access have been reported in some studies (e.g. Hu, Cordel, and Meinel, 2005; Keller and Naues, 2006; Border, 2007). Some universities recently adopted a dif-ferent approach for providing a virtual computer security lab (Li, 2009; Vranken and Koppelman, 2009). Instead of moving students to the lab, either physically or by remote access, the lab is moved to the students by providing an isolated, secured soft-ware environment on a DVD, that each individual student can easily install on his/her computer. This decentralized approach is suited to accommodate any number of students, providing students the freedom to run the lab whenever and wherever they want. Current research indicates that the development of computer labs involves the adoption of latest technology trends, e.g. cloud computing (Xu, Huang, and Tsai, 2012).
Supplemental to the development of computer lab architec-tures, researcher also addresses the impact and the integration of computer labs in the field of education. Especially in the higher education, hands-on experiences and practical skills from working on lab assignments are crucial to illustrate, deep-en and anchor studdeep-ents’ theoretical knowledge (Yu, 2007; Vranken and Koppelman, 2009). Students showed a clear pref-erence for experiential learning in a lab (Helps and Ekstrom, 2008).
Current research also indicates a field of assessment strate-gies to get clues on improving hands-on courses. The effective-ness of a course can be assessed and compared using the learn-ing outcome (Yuan, Schlough, and Anderson, 2013). A key fac-tor is the students’ learning style. Courses’ learning outcomes can be improved by adapting educational content to students’ different learning styles (Vassileva, 2011). This adaptation process imports an assessment of students’ needs and actual learning behaviour (Haag et al., 2013).
Course Design
All 3rd semester students of the Bachelor programs at the department of computer science at the Cologne University of Applied Sciences in Germany have to take part in the course “Communication technology and networks
(Kommunikation-stechnik und Netze)”, where they learn about concepts and
standards of computer networks, hosts and intercommunica-tions. The course consists of two hours of weekly lectures, accompanied and supplemented by one additional hour, where students work on their assignments in a practical course (“Praktikum”). Students are required to pass the prac-tical course as a prerequisite to take the exam. According to the curriculum, the practical course’s outcome is either pass or fail while the exam is being graded. It is planned but not mandatory to participate in the lecture and practical course within the same semester. The practical course is organized as follows: students have to register to take part. In a kick off meeting they will get to know the course advisors, the assign-ments and the computer laboratory. The course advisors are members of the academic staff at the university, have expert knowledge about the course content and are able to support and guide the students. The assignments are related to cer-tain theoretical concepts of the lecture and facilitate that stu-dents have to apply their previously acquired knowledge, e.g. by setting up and configuring real world networking scenar-ios or by analysing network traffic. These kinds of assign-ments require a safe playground, which is generally known as a computer lab (Bishop and Heberlein, 1996; Hill et al., 2001; Wagner and Wudi, 2004). This lab is usually a room at the university, containing computer systems connected to a net-work which is completely isolated from the outside world. In the lab, students can carry out exercises with the freedom to make mistakes, deal with attacks, deal with misconfigurations and modify system configurations at will. The lab is designed to ensure that no other systems will be harmed (e.g. the uni-versity network or internet hosts). In addition, a malfunction-ing lab can easily be restored to an initial workmalfunction-ing state. In order to solve the assignments without the need for more than one physical machine and a custom wiring, students are asked to solve this exercise using the open source network vir-tualization and simulation environment Netkit (Pizzonia and Rimondini, 2008) which is available for free. Netkit applies virtualization based on User Mode Linux (UML) (Dike, 2006) and allows to setup and configure UML virtual machines with virtual network interfaces, and to connect them into virtual networks. A UML virtual machine is creat-ed by running a Linux kernel as a user process in the virtual host machine. Multiple UML virtual machines can easily be run simultaneously, while using minimal resources. The file system is shared by all UML virtual machines using the
copy-on-write (COW) mechanism. Hence, the file system is shared read-only by all UML virtual machines. Each UML virtual machine has a second separate file system in which only the local changes to the shared file system are stored. This saves both disk space and memory, and simplifies management of multiple UML virtual machines. Restoring an initial clean system means to simply remove the second file system. Netkit was preinstalled on the laboratory’s workstations and could also easily be installed and run on the students’ own comput-ers if they wish to work independently from the lab, e.g. at home. After the kick off meeting the students have to work out and solve the assignments within a specified time (cur-rently 3 weeks). Students were told that they are free to choose their learning environment; the assignments were prepared not to require any special setting. Students can work e.g. alone, in a learning group, at home, at the lab or in any combination. No matter what specific learning environment they end up choosing, they work on the same assignments. The students that choose to work in groups are free in form-ing groups of any size and organize the group and work as they please. Students were also told to be able to get support in guided learning hours, which are regularly offered by the course advisors. The students are given this flexibility, so that they can chose the learning environment that best suits their respective learning. The aim of the practical course is to make sure that every student has learned the concepts related to the assignment, has an understanding of the solution, and is able to reproduce and defend the solution. To successfully com-plete the course, each student has to demonstrate and defend the solved assignment in a final bilateral expert talk with a course advisor. The course advisor knows the solution and possible ways of solving. He is able to judge whether a stu-dent has successfully acquired and applied theoretical con-cepts of the lecture, or not.
Networking Assignment Example
Most of today's intra- and internetworking is set up and con-figured in a decentralized manner by multiple different IT net-work administrators. This decentralized approach net-works, because that interconnection of computer hosts is based on commonly accepted and applied standards. In our practical course, the students were given real world assignments in order to practice the application of the aforementioned networking standards and the configuration of modern networks. One such assignment, taken from the examined course, is:
“Set up and configure a scenario with at least four hosts (e.g. client, router1, router2, server) to demonstrate routing behaviour. The client and the server should be located in different networks. The client should be able to intercommunicate with the server by using the intermediate router1 and router2.”
The minimal requirement for this setup is shown in TTT Figure 1, consisting of at least four hosts. The client and the server have one network interface; the routers are equipped with two network interfaces. A valid and straightforward solu-tion for this networking assignment example solved in the vir-tual environment Netkit is stated in TTTTable 1. In order to accomplish the given task, students will have to start four virtu-al hosts and interconnect them accordingly within three virtuvirtu-al networks. They will then have to assign appropriate addresses to these networks and hosts, and ultimately configure the rout-ing on each host. Once the network is configured properly, stu-dents can demonstrate the validity of their solution by sending network packets between client and server, and by using a suit-able tool (e.g. tcpdump) to analyse the packet flow. Students have to assure that the packets match their expectations based on the aforementioned standards.
After working through this exercise the students have learned about network classes, the OSI layer model, the con-cept of routing, basic network configuration tools in Unix-based systems, routing tables and their manipulation and tools to analyse network packets and network behaviour.
Evaluation
In the winter semester 2012, 249 male and female students signed up for the practical course “Communication technolo-gy and networks”. One hundred and seventy-eight of them (71%) participated in the evaluation process. The students were divided into smaller groups with reserved timeslots to get an evenly distributed utilization of the laboratory. These groups were created for organizational purpose only and were not related to any didactical concept. Three course advisors were working at the laboratory which provides 15 computer workstations for the students.
After the students completed their last expert talk at the end of the winter semester, a free and anonymous questionnaire was distributed among the students. The aim of this summative evaluation was to evaluate the learning behaviour of the stu-dents when they work on the assignments, and the success of the practical networking course. The major motivation was to
determine whether the learning situation and environment in our practical course meet the needs of the students and to get findings about key factors with respect to possible improve-ments to the course. For that purpose, questions were designed to interrogate about different parameters, which should pro-vide answers to the following questions:
Q1: Which learning environment did the students choose with respect to learning location, learning method and guided learning, and which environment would they prefer with respect to form of education?
Q2: What is the objective and subjective success of the course?
For Q1, 6 questions were designed to interrogate the stu-dents. To assess the course success (Q2) 3 questions were designed for the students in order to interrogate their subjec-tive course success. These questions were used to create a printed questionnaire handed out to the students. All questions were multiple-choice questions with predefined answers. The
TTT Figure 1.Example network setup.
TTTTable 1.Example solution for the networking assignment
Create the virtual environment vstart client --eth0=network1
vstart router1 --eth0=network1 --eth1=network2 vstart router2 --eth0=network2 --eth2=network3 vstart server --eth0=network3
Configure the client
ifconfig eth0 150.0.0.1 up route add default gw 150.0.0.2 Configure router1
ifconfig eth0 150.0.0.2 up ifconfig eth1 160.0.0.1 up
route add –net 170.0.0.0 netmask 255.255.0.0 gateway 160.0.0.2
Configure router2
ifconfig eth0 160.0.0.2 up ifconfig eth1 170.0.0.1 up
route add –net 150.0.0.0 netmask 255.255.0.0 gateway 160.0.0.1
Configure the server
ifconfig eth0 170.0.0.2 up route add default gw 170.0.0.1
NET1 ROUTER1 CLIENT SERVER ROUTER2 NET2 NET3
answers of all questionnaires were collected in a datasheet and analysed. In addition, the course’s list of passed and failed stu-dents was utilized in order to get an objective success rate.
Learning Location
The students were able to work on the assignments without the need for own equipment. The university provided all resources needed (e.g. computer lab with host and network infrastructure, staff, schedule) so that students could work according to their fixed lesson plan. In addition to this, the assignments were pre-pared to be solved without the need to be at the lab physically by using virtualization technology. The aim was to give students as much flexibility as possible while working on the assign-ments. In order to find out how this flexibility was accepted by the students, they were asked: “Did you work primarily at home, in the laboratory, or in a combination of the two?”TTT Figure 2 shows that a little more than half (51%) of the students worked in the laboratory almost exclusively, 22% worked only at home and 25% used a combined approach. This shows that 76% of students rely on utilizing the university’s resources by work-ing in the laboratory, while 47% of our students accept the offer to work at home independently from the lab, at least partially.
Learning Method
The students were free to choose whether to learn and work on their own or in a group setting. In order to find out, which learning method they chose for working on the assignments, given that they had to defend their solution individually, they were asked: “Did you work primarily alone, in a group, or a combination of the two?” TTTFigure 3 shows that 16% learned on their own, 66% learned in a group setting, and 17% used a combination of both approaches. This adds up to 83% of the students, for whom a group setting is an important part of their learning process, despite being tested individually.
Guided Learning
The students had the opportunity to get support when work-ing on their assignments by means of gettwork-ing help and guidance from the course advisors. Guided learning hours were sched-uled for this, which were open to all of the students. Their pur-pose was to enable all students that needed help to ask for and to receive it, and they were made voluntary, so that not every student needed to commute to the lab, i.e. the university, despite not needing any guidance. To find out how much of a demand for these hours existed among the students, and how these hours were perceived by them, they were first asked: “How often have you utilized the guided learning hours?” TTT Figure 4 shows, that there was an even distribution of students who felt they needed a lot of help, students that needed very lit-tle help or no help at all, and students that fell just in between.
That shows that these guided learning hours are very valuable to a lot of students, and that their existence seems vital to the course design. It also shows that a voluntary nature is very rea-sonable when it comes to guided learning hours.
TTT Figure 2.Learning location.
TTT Figure 3.Learning method.
In order to verify, if students are indeed able to get support if they need to, they were asked to assess the availability of the course advisors: “How do you rate the availability of the course advisors?” While most of the students (85%) were being able to get support if they needed to, nobody was left alone (TTT Figure 5). Next, the students were asked: “How do you rate the quality of support during the guided learning hours?”
TTTFigure 6 shows, that the vast majority felt that the sup-port's quality was very good, and only 2% thought poorly of it.
Preferred Style of Education
To find out, what form of education the students would prefer with respect to the blended learning approach, where conven-tional classroom settings and e-learning will be combined, the students were asked: “In which of the following scenarios would you expect your learning success to be highest: working online with an e-learning system, working in the lab in a face-to-face setting, or in a combination of the two?” According to TTTFigure 7, 8% of the students would prefer to learn ly in an e-learning environment, 43% prefer to learn exclusive-ly in a conventional face-to-face classroom setting and 41% of the students think it would be beneficial to combine these two approaches. While this shows that the majority (84%) of the students think of a classical face-to-face environment as essen-tial to their learning process, 49% would welcome the intro-duction of an additional e-learning environment.
The objective success rate of the course was 77%. Out of 249 registered students, 191 students passed, i.e. they worked out the assignments, and demonstrated and defended their solu-tions successfully. Most of the 23% unsuccessful students regis-tered but did not participate at all; some seemed to have other shortcomings; a few tried but could not defend their solution properly. Because of the anonymous nature of our evaluation and because the practical course is not graded but has a result of either passed of failed, it cannot be deduced whether or how the success rates differ among different groups of students.
To find out what students thought about their subjective learning success, they were first asked: “How would you rate your own knowledge acquisition with respect to the practical course?” According to TTTFigure 8, a majority of 78% rated their personal knowledge acquisition as high. To contextualize that number, the students’ skills before taken the course should be taken into consideration. They were therefore asked: “How would you assess your level of familiarity with the course’s con-tents prior to the course?” The evaluation in TTTFigure 9 shows that 21% of the students assess their own familiarity with the course content as high, 25% as medium and nearly half of the students assess their pre-course knowledge as low. In a third question, students were asked: “How do you rate the difficulty
level of this practical course?” TTTFigure 10 shows that a major-ity of the students (70%) assessed the difficulty level of the practical course as medium. Only 9% found it easy and only 13% struggled and found the assignments hard to solve.
TTT Figure 5.Availability of the course advisors.
TTT Figure 6.Quality of support.
Conclusion
With a participation rate of over 70% of 249 students in the evaluation, a representative sample of the learning behaviour in a practical networking course was acquired. With respect to
the second research question Q2 it found out that an objective-ly successful course was observed, as participants had a success rate of over 75% in the winter semester 2012. The flexibility granted to the students in preparing their assignments was gen-erally well accepted and the students utilized all possibilities to varying degrees. Without being recommended certain learning environments or a certain learning behaviour, the students seemingly chose what they deemed best for them personally, resulting in a high success rate of the course.
Results were also found regarding the research question Q1. While the preferences for learning environments and behaviour were fairly distributed, a predominant majority of the students thought of working in groups as well as receiv-ing guidance and feedback as crucial to their learnreceiv-ing success. Students are also interested in new and modernized learning environments. It is important that these new environments do not replace more traditional ones, but rather add on to them.
One way of modernizing the practical course would be the introduction of an e-learning system, which would be explicit-ly welcomed by 49% of the students. In addition to that, near-ly half of the students said that they would like to work inde-pendently from the lab at least partially, which they would be enabled to do by the introduction of such a system.
Given the students’ preference for group working and guided learning, one should take these two key factors into account when introducing an e-learning system. This means that, in order to gain the students’ acceptance, an e-learning system should enable collaboration as well as guidance, and should not be limited to simply providing an environment for solving assignments online.
References
Bishop, M., and Heberlein, L. (1996). An isolated network for research. In 19th National Information Systems Security Conference (pp. 22-25). October 22-25, 1996, Baltimore, MD, USA.
Border, C. (2007). The development and deployment of a multi-user, remote access virtualization system for networking, security, and sys-tem administration classes. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (pp. 576-580). New York, NY: ACM.
Bullers, Jr., W., Burd, S., and Seazzu, A. (2006). Virtual machines - an idea whose time has returned: application to network, security, and data-base courses. In Proceedings of the 37th SIGCSE technical symposium on Computer science education (pp. 102-106). New York, NY: ACM. Dike, J. (2006). User mode Linux. Upper Saddle River, NJ: Prentice Hall. Gaspar, A., Langevin, S., and Armitage, W. (2007). Virtualization tech-nologies in the undergraduate IT curriculum. IT Professional, 9(4), 10-17.
Gephart, N., and Kuperman, B. A. (2010). Design of a virtual computer lab environment for hands-on information security exercises. Journal of Computing Sciences in Colleges, 26(1), 32-39.
TTT Figure 8.Knowledge acquisition.
TTT Figure 9.Preknowledge.
Haag, J., Horsmann, T., Karsch, S., and Vranken, H. (2011). A distrib-uted virtual computer security lab with central authority. In Proceedings of the Computer Science Education Research Conference (pp. 89-95). Heerlen, The Netherlands: Open Universiteit.
Haag, J., Karsch, S., Vranken, H., and van Eekelen, M. (2012). A virtual computer security lab as learning environment for networking and security courses. In Proceedings of the 3rd Annual International Conference on Computer Science Education: Innovation and Technology (pp. 61-68). Singapore: Global Science & Technology Forum. Haag, J., Witte, C., Karsch, S., Vranken, H., and van Eekelen, M. (2013).
Evaluation of students' learning behaviour and success in a practical computer networking course. In Proceedings of the 2013 Second International Conference on e-Learning and e-Technologies in Education (ICEEE) (pp. 201-206). Lodz, Poland.
Helps, C., and Ekstrom, J. (2008). Evaluation of a computer networking class. In Proceedings of the 9th ACM SIGITE Conference on Information Technology Education (pp. 259-268). New York, NY: ACM.
Hill, J., Carver, Jr., C., Humphries, J., and Pooch, U. (2001). Using an isolated network laboratory to teach advanced networks and security. In Proceedings of theTthirty-Second SIGCSE Technical Symposium on Computer Science Education (pp. 36-40). New York, NY: ACM. Hu, J., Cordel, D., and Meinel, C. (2005). Virtual machine management
for tele-lab "IT-Security" server. In Proceedings of the 10th IEEE Symposium on Computers and Communications (pp. 448-453). Washington, DC: IEEE Computer Society.
Keller, J., and Naues, R. (2006). Design of a virtual computer security lab. In Proceedings of the Third IASTED International Conference on Communication, Network, and Information Security, (pp. 211-215). Calgary, AB, Canada: ACTA Press.
Li, P. (2009). Exploring virtual environments in a decentralized lab. ACM SIGITE Research in IT, 6(1), 4-10.
O’Leary, M. (2006). A laboratory based capstone course in computer security for undergraduates. In Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (pp. 2-6). New York, NY: ACM.
Pizzonia, M., and Rimondini, M. (2008). Netkit: easy emulation of com-plex networks on inexpensive hardware. In Proceedings of the 4th
International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities (Article No. 7). Brussels, Belgium: ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).
Sloan, J., and Schlindwein, C. (2004). TCP/IP laboratory exercises for use with a remotely accessible networking laboratory. In Journal of Computing Sciences in Colleges, 19(3), 68-78.
Standard, S., Greenlaw, R., Phillips, A., Stahl, D., and Schultz, J. (2013). Network reconnaissance, attack, and defense laboratories for an introductory cyber-security course. In ACM Inroads, 4(3), 52-64. Vassileva, D. (2011). Evaluation of learning styles adaption in the
adop-ta e-learning platform. In Proceedings of the 12th International Conference on Computer Systems and Technologies (pp. 540-545). New York, NY: ACM.
Vranken, H., and Koppelman, H. (2009). A virtual computer security lab for distance education. In Proceedings of the 5th European Conference on Internet and Multimedia Systems and Applications (pp. 21-27). Calgary, AB, Canada: Acta Press.
Vranken, H., Haag, J., Horsmann, T., and Karsch, S. (2011). A distrib-uted virtual computer security lab. In Proceedings of the 3rd interna-tional Conference on Computer Supported Education (pp. 110-119). Noordwijkerhout, The Netherlands: SciTePress.
Wagner, P., and Wudi, J. (2004). Designing and implementing a cyber-war laboratory exercise for a computer security course. In Proceedings of the 35th SIGCSE technical symposium on Computer science education (pp. 402-406). New York, NY: ACM.
Xu, L., Huang, D., and Tsai, W.-T. (2012). V-lab: a cloud-based virtual laboratory platform for hands-on networking courses. In Proceedings of the 17th ACM annual Conference on Innovation and Technology in Computer Science Education (pp. 256-261). New York, NY: ACM. Yu, Y. (2007). Designing hands-on lab exercises in the network security
course. Journal of Computing Sciences in Colleges, 22(5), 105-110. Yuan, D., Schlough, S., and Anderson, B. (2013). Using outcome based
learning assessment to evaluate and improve the hands-on network-ing course. Journal of Computnetwork-ing Sciences in Colleges, 29(2), 220-228).
