Registration and map building for autonomous mobile robots using 3D LIDAR point clouds

ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL OF SCIENCE ENGINEERING AND TECHNOLOGY

M.Sc. THESIS

MARCH 2020

REGISTRATION AND MAP BUILDING FOR AUTONOMOUS MOBILE ROBOTS USING 3D LIDAR POINT CLOUDS

Betül PEHLİVAN

Department of Control and Automation Engineering

Department of Control and Automation Control and Automation Engineering Programme

MARCH 2020

ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL OF SCIENCE ENGINEERING AND TECHNOLOGY

REGISTRATION AND MAP BUILDING FOR AUTONOMOUS MOBILE ROBOTS USING 3D LIDAR POINT CLOUDS

M.Sc. THESIS Betül PEHLİVAN

(504151139)

Kontrol ve Otomasyon Mühendisliği Anabilim Dalı Kontrol ve Otomasyon Mühendisliği Programı

3D LİDAR NOKTA BULUTU KULLANAN OTONOM MOBİL ROBOTLAR İÇİN KAYITLAMA VE HARİTALAMA

MART 2020 YÜKSEK LİSANS TEZİ

Betül PEHLİVAN (504151139)

Tez Danışmanı: Prof. Dr. Hakan TEMELTAŞ

v

Thesis Advisor : Prof. Dr. Hakan Temeltaş Istanbul Technical University

Jury Members : Prof. Dr. Müjde Güzelkaya Istanbul Technical University Doç. Dr. Cenk Ulu Yıldız Technical University

Betül Pehlivan, a M.Sc. student of ITU Graduate School of Science Engineering and Technology student ID 504151139, successfully defended the thesis entitled “REGISTRATION AND MAP BUILDING FOR AUTONOMOUS MOBILE ROBOTS USING 3D LIDAR POINT CLOUDS”, which she prepared after fulfilling the requirements specified in the associated legislations, before the jury whose signatures are below.

Date of Submission : 27 December 2019 Date of Defense : 06 March 2020

vii

ix FOREWORD

This project is provided as a part of a master thesis at Istanbul Technical University. During the thesis, methodologies about robotics applications, which are subsampling methods, registration methods, and 3D mapping methods, have been examined. I would like to thank all my friends who have provided their support every time, have believed in me, and show that they are true friends. I also would like to express my gratitude to my dear family, who is always with me. Additionally, I would like to express my appreciation to my supervisor Hakan Temeltaş for his endless support, help, and guidance.

March 2020 Betül PEHLİVAN

xi TABLE OF CONTENTS Page FOREWORD ... ix TABLE OF CONTENTS ... xi ABBREVIATIONS ... xiii SYMBOLS ... xv

LIST OF TABLES ... xvii

LIST OF FIGURES ... xixvii

SUMMARY ... xxi

ÖZET ... xxiii

INTRODUCTION ... 1

1.1 Thesis Outline ... 3

LIDAR SENSORS & POINT CLOUD ... 5

LIDAR Sensors ... 5

Point Cloud ... 10

FILTERING & SUBSAMPLING ... 11

Systematic Sampling ... 12

Stratified Random Sampling ractical Applications ... 13

Decimation ... 13

3.4 Averaging ... 14

REGISTRATION ... 15

Iterative Closest Point (ICP) ... 16

Normal Distributions Transform (NDT) ... 21

4.2.1 Local probability density funtions (PDF) ... 25

4.2.2 Average vector and covariance matrix ... 25

3D MAPPING METHODS ... 29

Octree ... 31

KD Tree ... 33

EXPERIMENTAL STUDIES ... 37

Test Data and Environment ... 37

6.1.1 Test data by LIDAR ... 37

6.1.1.1 HUSKY A200 ... 38

6.1.1.2 LIDAR VLP-16 ... 38

6.1.1.3 Fittings ... 39

6.1.1.4 IMU and GPS sensors ... 40

6.1.1.5 Other equipment and connection between all equipment ... 40

6.1.2 Environment ... 42

6.1.2.1 Indoor environment ... 42

6.1.2.2 Outdoor environment ... 43

Case Study 1 : Indoor Environment ... 44

6.2.1 Subsampling methods in indoor environment and results ... 45

6.2.2 Registration methods with subsampling method in indoor environment and results ... 47

6.2.3 3D mapping methods with registration and subsampling methods in indoor environment and results ... 53

Case Study 2 : Outdoor Environment ... 57

xii

6.3.2 Registration methods with subsampling method in outdoor environment

and results ... 61

6.3.3 3D mapping methods with registration and subsampling methods in outdoor environment and results ... 63

CONCLUSION ... 69

REFERENCES ... 71

xiii ABBREVIATIONS

DEM : Digital Elevation Model DSM : Digital Surface Model ICP : Iterative Closest Point KD TREE : K-Dimensional Tree

LIDAR : Light Detection and Ranging NDT : Normal Distributions Transform PCL : Point Cloud

PDF : Local probability density functions SLAM : Simultaneous Localization and Mapping ToF : Time of Flight

xv SYMBOLS

c : The Speed of Light

M : Distance i : Sampling interval ∆P : Phase shift t : Time f : Frequency R : Rotation matrix T : Translation vector H : Covariance matrix

pdf : Local probability density function

J : Jacobian matrix

xvii LIST OF TABLES

Page Key features of HUSKY A200. ... 38 Key features of LIDAR VLP-16. ... 38 Compatibility of each subsampling method and registration method... 53 Table 6.4 : Detail list of applied eight different cases for 3D mapping methods with registration and subsampling methods in indoor environment. ... 53 Table 6.5 : Compatibility of each subsampling, registration and 3D mapping method . ... 57 Table 6.6 : Compatibility of each subsampling method and registration method... 63 Table 6.7 : Compatibility of each subsampling, registration and 3D mapping

xix LIST OF FIGURES

Page Representation of basic working principle of classical LIDAR (ToF

technique) ... 8

Representation of basic working principle of AMCW LIDAR ... 9

Figure 5.1 : Representation of octotree algorithm. ... 32

Figure 5.2 : Representation of KD Tree algorithm. ... 35

Figure 6.1 : 16-channel LIDAR sensor with 3D conical measuring capability: (a) General tool and rotation axis. (b) Sensor parameters in the vertical plane. (c) Sensor parameters in the horizontal plane ... 39

Figure 6.2 : Velodyne VLP-16 ... 39

Figure 6.3 : HUSKY A200 equipped with all equipment ... 41

Figure 6.4 : Case study in indoor environment: (a) Route followed. (b) Physical environment... 43

Figure 6.5 : Case study in outdoor environment: (a) Route followed. (b) Physical environment... 44

Figure 6.6 : Representation of the decreasing of the number of PCL data set in terms of percentage after subsampling method in indoor application. ... 45

Figure 6.7 : Representation of total time spent in subsampling method in indoor application ... 46

Figure 6.8 : Representation of the average of RMSE values above threshold in registration method with subsampling method in indoor application ... 48

Figure 6.9 : Representation of the number of RMSE situations above threshold in registration method with subsampling method in indoor application ... 49

Figure 6.10 : Representation of the average of the difference between actual X value of the PCL data and the new X value of the PCL output obtained as a result of registration method with subsampling method in indoor application. ... 51

Figure 6.11 : Representation of total time spent in registration method with subsampling method in indoor application. ... 52

Figure 6.12 : Representation of the number of points obtained at the end of each application of 3D mapping methods with registration and subsampling methods in indoor application... 55

Figure 6.13 : Representation of total time spent in 3D mapping methods with registration and subsampling methods in indoor application ... 55

Figure 6.14 : In indoor environment random sampling: (a)ICP + KD Tree. (b)ICP + Octree. (c)NDT + KD Tree. (d)NDT + Octree. ... 56

Figure 6.15 : In indoor environment grid average: (a)ICP + KD Tree. (b)ICP + Octree. (c)NDT + KD Tree. (d)NDT + Octree. ... 57

Figure 6.16 : Representation of the decreasing of the number of PCL data set in terms of percentage after subsampling method in outdoor application ... 59

xx

Figure 6.17 : Representation of total time spent in subsampling method in outdoor application ... 60 Figure 6.18 : Representation of the average of the difference between actual X value of the PCL data and the new X value of the PCL output obtained as a result of registration method with subsampling method in outdoor application. ... 61 Figure 6.19 : Representation of total time spent in registration method with

subsampling method in outdoor application. ... 62 Figure 6.20 : Representation of the number of points obtained at the end of each application of 3D mapping methods with registration and subsampling methods in outdoor application ... 64 Figure 6.21 : Representation of total time spent in 3D mapping methods with

registration and subsampling methods in outdoor application ... 65 Figure 6.22 : In indoor environment, random sampling: (a)ICP + KD Tree. (b)ICP + Octree. (c)NDT + KD Tree. (d)NDT + Octree ... 65 Figure 6.23 : In indoor environment, grid average: (a)ICP + KD Tree. (b)ICP + Octree. (c)NDT + KD Tree. (d)NDT + Octree ... 66

xxi

REGISTRATION AND MAP BUILDING FOR AUTONOMOUS MOBILE ROBOTS USING 3D LIDAR POINT CLOUD

SUMMARY

Different studies and experiments are carried out in many different areas related to mobile robots. Almost all of the reviews have a common aim is to work with a fully automated, independent mobile robot. In this context, it is an important parameter to define the working environment very well. Regardless of the environment, the obstacles and movable areas in the environment must be identified. Different methods have been developed to define environments in this way. Within the scope of this thesis, these methods have been examined and then compared.

Following the scope of the thesis study, a 3D map with 3D LIDAR point cloud data set has been tried to obtain the most realistic virtual copy of the environment studied. In this context, the environments to be studied were scanned in point cloud structure. During this process, 3D LIDAR sensors were used. The data obtained in the form of point cloud are not meaningful data by itself. For this reason, some processes are applied to them. After the processes called registration, meaningful results are obtained from point cloud data. In the registration process, the output closest to reality is tried to be obtained from the point cloud data sets. Two most known registration methods are the subject of this thesis. These are ICP and NDT methods. The characteristic feature of both methods is that they work with iterative refinement technique; moreover, one output closest to reality is tried to be obtained by trying to converge point cloud data sets that are inputs within the framework of the common target. Although a meaningful result was obtained after the registration process within the scope of the thesis, the results were taken one step further, and the 3D mapping of the environment was attempted. Two conventional methods have been studied as 3D mapping method. These are Octree and KD Tree methods. In addition to the ICP and NDT methods applied to the system, Octree and KD Tree methods were added, respectively.

The methods examined within the scope of the thesis are briefly mentioned above. During the implementation of these methods in a real environment, two different environments were studied. These are indoor and outdoor environments within the ITU campus. Three different situations were investigated in these environments, respectively. The first is the case in which only subsampling methods are studied. Here, random sampling method and grid average methods are studied; furthermore, they are compared. It is observed that the grid average method gives more successful results compared to random sampling method by using two different metrics. Then, the second case was studied, and registration methods were applied in addition to the subsampling methods. This time, the applied methods are compared by using four different metrics. In the second case study, the most successful results were obtained when random sampling method and ICP method were applied together. Finally, 3D mapping methods were applied to the methods applied in the second stage. A total of eight different situations occurred and tried with the addition of 3D mapping methods. It was observed that the most successful result among eight different cases evaluated using two separate metrics were obtained when random sampling, ICP, and Octree methods were applied together.

xxii

As a result, it has been observed that research topics, which are both theoretically examined and supported by experimental studies, give different results in different situations. It is possible to decide which methods should be used within the framework of the subject, environment, and expectations to be studied. In the light of the studies, it can be easily decided which methods should be used in which situations.

xxiii

3D LİDAR NOKTA BULUTU KULLANAN OTONOM MOBİL ROBOTLAR İÇİN KAYITLAMA VE HARİTALAMA

ÖZET

Günümüzün en önemli çalışma konularından birisi robotik çalışmalardır. Bu kapsamda da en çok üzerinde çalışılan alanlardan birisi mobil robotlardır. Mobil robotlar alanında yapılan çalışmaların en öncelikli amacı ise; robotların kullanıcıdan bağımsız bir şekilde, olabildiğince tam otomasyonlu olarak çalışabilmesidir. Bunun sağlanmasında çalışılan ortamın çok iyi tanınması büyük bir öneme sahiptir. Çünkü ortamın yapısı, sahip olduğu her türlü engel ve bozucular robotun çalışmasında engele yol açacaktır. Bu amaç çerçevesinde üzerinde çalışılan ortamların tanınması ve 3D bir sanal kopyasının oluşturulması için farklı yöntemler geliştirilmiştir. Hatta bu konuda çalışmalara hala devam edilmektedir.

Bu tez kapsamında da otonom bir mobil robotun 3D LİDAR teknolojisi yardımıyla bulunduğu ortamın nokta bulutu kaydının ve 3D haritasının elde edilmesi üzerinde çalışılmıştır. Üzerinde çalışılan ortamın sanal bir kopyasının oluşturulmasında kullanılanacak olan sensörlerden, haritalama yöntemlerine kadar bir çok adımda farklı yollar izlenebilir. Her adımda seçilen yöntem doğrultusunda da elde edilecek olan sonuçlar ciddi farklılıklar gösterebilmektedir. Bu kapsamda hangi yöntemin hangi durumlarda daha doğru ve kararlı sonuçlar verdiği incelenmiştir. Yapılan çalışmalarda farklı yöntemler için farklı ortam ve parametrelerde çalışmalar yapılmıştır. Yapılan çalışmalardan elde edilen sonuçlar da belirli metrikler çerçevesinde değerlendirilmiştir ve her durum için hangi yöntemin en ideal yöntem olduğu tespit edilmeye çalışılmıştır. Mobil robotun bulunduğu ortamın 3D bir kopyasının ve haritalamasının çıkarılması sürecinde takip edilen adımların ilk başında bir nokta bulutu veri setinin elde edilmesi gelmektedir. Nokta bulutunu elde edebilmek için de uygun ekipmanların sistem içerisinde bulunması ve tanımlanmış olması beklenmektedir. Bu noktada, sistemin içerisinde bulunması beklenilen ve nokta bulutu veri setinin elde edilmesinde asıl role sahip olan sensörlerin seçimi çok önemlidir. Büyüklük, kullanım şekli ve içerik açısından farklı çeşitte sensör seçeneği bulunmaktadır. Fakat bunlar arasından bu proje kapsamında teknik literatürde kısaca LIDAR olarak tanımlanan teknolojiye sahip sensörler tercih edilmiştir. LIDAR kelimesinin açılımı Light Detection and Ranging olup, Türkçe olarak karşılığı ise Işık Tespiti ve Değişimi şeklindedir. Kelime tanımının ötesinde anlamsal olarak LIDAR olarak isimlendirilen teknoloji, lazer ışınları sayesinde ortamda bulunan veya hedef alınan cisimlerin tespit edilip tanımlanması ve sisteme olan uzaklığının da hesaplanmasıdır. LIDAR’ı bu alanda tanımlanmış diğer teknolojilerden ayıran en önemli özelliği lazer ışınlarını kullanmasıdır. LIDAR’ın tercih edilme sebebi ise; daha istikrarlı, hata oranı düşük, esnek, çevre koşullarından daha az etkilenen, fiyat-performans açısından düşük fiyat - yüksek performansa sahip bir teknolojiye sahip olmasıdır. Ek olarak, koordinat bilgisini de içermesi LIDAR teknolojisinin tercih edilmesi noktasında önemli bir özelliktir. Genel olarak LIDAR sistemi içerisinde bir lazer tarayıcı, genel literatürde GPS olarak bilinen küresel konumlama sistemi, genel literatürde IMU olarak isimlendirilen ataletsel navigasyon ölçüm birimi bulunmaktadır. Bunların dışında sistem içerisinde kullanılacak olan lazer türü de çalışılacak ortama göre farklılık gösterebilmektedir.

xxiv

Üzerinde çalışılacak ortam için en uygun LIDAR sistemi oluşturulduktan sonra artık bu teknoloji yardımıyla ortamdaki cisimlerin, cisimsiz bölgelerin yani boş alanların tespiti yapılabilir. Tespit sonucunda da bu bilgileri içeren bir nokta bulutu veri seti çıktı olarak sunulmaktadır. Fakat somut bir nesneden sadece noktalardan oluşan bir veriye geçildiğinde anlamlı gelmemesi çok doğal bir durumdur. Tek başına nokta bulutu veri seti sadece rakamlardan ibaret bir şeydir. Bu nokta bulutu veri setini anlamlı hale getirecek olan şey bu noktaların kaydedilip anlamlandırılmasıdır. Kayıtlama işlemi olarak farklı yöntemler türetilmiştir. Bunlar içerisinde en bilinenleri ve en çok tercih edilenleri kısaca ICP ve NDT olarak isimlendirilmiş olan yöntemlerdir. ICP’nin anlamı iteratif en yakın noktadır. NDT’nin tanımsal olarak ifadesi ise; normal dağılım dönüşümüdür. Her iki yöntem de kayıtlama işlemlerinde kullanılması dışında başka ortak bir noktaya daha sahiptir. O da ikisinin de arka planda sahip olduğu tekniğin yinelemeli iyileştirme tekniğine dayanmasıdır. Matematiksel ifadeler açısından ve izlenilen süreçler açısından ayrışmaktadır bu iki yöntem aslında. Bu yöntemler içerisinden ilk olarak ICP yöntemi geliştirilmiştir. Bu proje kapsamında saf ICP yöntemi incelenmiştir ve çalışmalarda kullanılmıştır. Saf ICP yöntemi matematiksel açıdan oldukça basit ve temel formüllere dayanmaktadır. Bu özellik ICP yönteminin tercih edilme sebebini arttırmaktadır. Ek olarak, çalışma alanı olarak geniş bir yelpazeye sahip olan ICP yöntemi altı serbestlik dereceli nesnelerle uyumlu çalışabilmektedir. Bu durum da onun kullanım alanlarındaki sınırlarını azaltmaktadır. Diğer taraftan üzerinde çalışılan hedef nesneyi nokta temelli olarak ele alıp bu şekilde bir yaklaşıma sahip olduğundan cismin sahip olduğu yüzeyi bir bütün olarak ele almamaktadır. Bütün bu olumlu ve olumsuz yanları ICP yönteminin birer parçasıdır ama yöntemin nihai hedefine ulaşmasını engellememektedirler. Sadece hedefe ulaşmaktaki yolu ve sonucun verimini etkilemektedir. ICP yöntemini uygulamadaki nihai hedef ise; iki ayrı nokta bulutu veri setinden (referans ve model) anlamlı ve gerçek objeye olası en yakın bir nokta bulutu sonucu elde etmektir. Hatta mümkünse bire bir aynı yani çakışma oranı yüzde yüz olan bir sonuç elde edilmesi bu yöntemin ortaya çıkış noktasıdır. İki nokta bulutu setinin birbirine mümkün olan en yüksek oranda yakınsaması sırasında nokta bulutu veri setleri içerisinde bulunan her bir noktadan yararlanılmaktadır. ICP yönteminin bir alternatifi olan NDT yöntemi, ICP’den daha yakın geçmişte şekillendirilmiştir. 2000’li yılların başında öne sürülen NDT yöntemi yukarıda da bahsedildiği gibi temel çalışma mantığı olarak ICP yöntemiyle aynı mantıkta ilerlemektedir. Yani, yinelemeli iyileştirme tekniğini temel olarak almaktadır. NDT yönteminde de iki ana girdi bulunmaktadır. Bunlar ICP yöntemi ile birebir aynı olup referans ve model nokta bulutu veri setleridir. Yöntemin sonunda da belirlenen hedef bu iki ayrı nokta bulutu veri seti arasındaki en uygun eşleşmenin sağlanmasıdır. Bu eşleşmede rol alacak olan matematiksel fonksiyonlar ise farklılık göstermektedir. Matematiksel açıdan NDT yöntemi ızgara yöntemi ile benzerlik göstermektedir fakat NDT yöntemi, diğer yöntemin daha gelişmiş bir modelidir. Bu yöntemde nesne sadece noktalar olarak ele alınmaz. Belirli miktarda noktaya sahip hücreler olarak ele alınır. Dolayısıyla, uygulanacak olan işlemler her bir nokta için tekrarlanmak yerine bu hücreler özelinde uygulanır. Bu durum ICP’den farklı olarak ilgili nesnenin yüzeyi hakkında bütünsellik sağlamaya yardımcı olmaktadır. Bu yöntem kapsamında sonuç olarak en uygun pozisyonu sağlayacak olan dağılım fonksiyonu elde edilmeye çalışılır. ICP yöntemiyle ortak hedef çerçevesinde en uygun pozisyona sahip nokta bulutu seti elde edilmeye çalışılır. En uygun pozisyon ile hedeflenen ise referans değere olası en yakın sonucu elde etmektir.

xxv

Kayıtlama işlemlerinin sonucunda artık üzerinde çalışılan nesnenin nokta bulutu veri setinin ötesinde anlamlı bir veri seti bulunmaktadır. Bu veri setleri tek başına görsel olarak anlamlı olabilir fakat daha da ileri seviyede sonuçlar da elde etmek mümkündür. Bu sonuçlardan en başlıcası kayıtlama işlemi sonucunda elde edilen sonuç üzerinden 3D haritalama uygulamasıdır. Böylece, ortam hakkında daha fazla bilgi sahibi olunabilir. Bu kapsamda da, farklı yöntemler geliştirilmiştir. Bu yöntemler arasında en çok bilinen ve tercih edilen octotree ile KD Tree yöntemleri bu çalışmanın konusu olmuştur. Octotree yönteminde çalışılan nesne sekizli setler halinde sürekli olarak alt bölümlere ayrılır. Her bir alt parça nesne hakkında farklı bilgiler elde edilmesini sağlamaktadır. Bu parçalar üzerinden bütün hakkında ve hareket hakkında bilgi sahibi olunarak ortamın 3D haritası oluşturulur. KD Tree yönteminde ise; k boyutta bölme işlemi uygulanır. Ardından her bir bölme sonucunda üzerinde çalışılacak yani incelenecek olan noktanın pozisyonu tespit edilir. Burada pozisyon ile kastedilen ilgili noktanın bölümleme işlemi sonunda ağacın hangi tarafında yer alacağıdır. Bu bilgiler ışığında her bir nokta için en yakın nokta taraması yapılır ve ortam hakkında daha detaylı tanımlama yapılmasını da sağlayacak bilgiler elde edilir. Ardından, diğer yöntemde olduğu gibi haritalama yapılır. Haritalama konusunda seçilecek yöntemin tekniğinden çok özellikleri ön plana çıkmaktadır. Kullanılacak yöntemin ortamdaki farklılıklara çabuk adapte olabilecek, ortamda karanlık yani bilinmeyen hiçbir bölge bırakmayacak şekilde bilgi verebilecek ve esnek yapıda olması tercih sebebidir. Son olarak, yukarıda biraz detaylandırılmış olarak bahsedilen yöntemler kullanılarak tezin deneysel çalışmaları gerçeklenmiştir. Deneysel çalışmalar iç ve dış ortam olmak üzere iki ayrım ortamda tekrarlanmıştır. Her bir ortamda ise; üç farklı senaryo üzerinde çalışılmıştır. İlk olarak sadece alt-örnekleme (subsampling) yöntemleri çalışılmıştır. Bu kapsamda rastgele örnekleme (random sampling) ve ızgara ortalaması (grid average) yöntemleri iç ve dış ortamda iki ayrı parametre değeri ile uygulanmıştır. İki ayrı metrik yardımıyla yapılan kıyaslamaya göre grid average yöntemi en başarılı sonuçlara sahip olarak tanımlanmıştır. Deneysel çalışmaların bir sonraki aşamasında yine aynı iç ve dış ortam için bu sefer ilk aşamada kullanılan subsampling yöntemlerinin yanında kayıtlama (registration) yöntemleri sisteme uygulanmıştır. Yani, sırasıyla ICP ve NDT registration yöntemleri random sampling ve grid average yöntemleri ile beraber uygulanmıştır. Bu aşamada farklı olarak her bir subsampling yöntemi özelinde iki ayrı parametre tanımlanmamıştır. İlk aşamada yapılan çalışmalar sonucunda başarılı bulunan parametreler üzerinden subsampling yöntemleri uygulanmıştır. Özetle, her bir subsampling yöntemi için bir parametre tanımlanmıştır ve sırasıyla ICP ile NDT yöntemleriyle beraber uygulamalı olarak denenmiştir. Yapılan çalışmalar sonucunda elde edilen çıktılar ise dört ayrı metrik yardımıyla kıyaslanmıştır. Sonuç olarak, subsampling yöntemi ile registration yöntemleri bir arada uygulandığında en başarılı sonucun random sampling ile ICP yönteminin bir arada uygulandığı durumlarda elde edildiği gözlemlenmiştir. Son olarak, yine aynı iç ve dış ortam için 3D haritalama yöntemleri de sisteme uygulanmıştır. Bu kapsamda ikinci aşamada yapılan her bir çalışmaya ek olarak sırasıyla Octree ve KD Tree haritalama yöntemleri eklenmiştir ve uygulanmıştır. Sekiz farklı durum ortaya çıkmıştır ve her bir durum sonucunda elde edilen çıktılar iki ayrı metrik ile değerlendirilmiştir. Bu çalışmanın sonucunda ise en başarılı sonucun random sampling method, ICP ve Octree yöntemlerinin bir arada uygulandığı durumlarda elde edildiği gözlemlenmiştir.

xxvi

Sonuç olarak, gerek teorik özellikler gerekse yapılan deneysel çalışmalar sonucunda bazı yöntemlerin diğer yöntemlere kıyasla daha başarılı sonuçlar verdiği gözlemlenmiştir. Fakat her bir yöntem de kendi özelinde farklı metriklerle incelendiğinde daha baskın özelliklere sahip olduğu gözlemlenmiştir. Dolayısyla, yapılacak olan çalışmanın kapsamı, yapılacağı ortamın şartları, beklenilen sonuçlar ve bu projede elde edilmiş deney sonuçları göz önüne alınarak kullanılacak yöntemler hakkında karar verilebilir. Seçilecek yöntemden bağımsız olarak da her bir çalışmada genel olarak başarılı sonuçlar elde edilmiştir.

1 1. INTRODUCTION

Technology and technological advances are well-known words in today’s world. Technological studies have progressed rapidly over the last century. The progress made in the technical field from the beginning of the century to the present has brought with it new inventions. As a result of this, new working areas have been created. In the content of this thesis, some of the more recent study topics are examined. The topics can be classified as mobile robots, point cloud, LIDAR, registration, and mapping. In addition to being so popular with mobile robots, different studies have been done on point cloud in our laboratory studies. Within the scope of this thesis, the chance to move the studies about point cloud one step further was the biggest source of motivation, and it was interesting to examine the different results that can be obtained when different methods are applied. As a result, the works started from the point cloud data set were enriched and applied by registration and 3D mapping applications, respectively. In this way, the opportunity to add innovative approaches to the work with mobile robots from a different perspective was found. The fact that the parts missing in previous studies could be completed was the biggest incentive in the thesis study.

The main element used in this study is a mobile robot. Mobile robots are now being used in many different areas. Mobile robots are suitable to use in both indoor and outdoor environments. It has a wide range of uses, from NASA studies to domestic applications such as house sweeping [1]. Independent of these usage areas, it is aimed to enable mobile robots to work independently and autonomously in their environments [1, 2]. The mobility of a mobile robot mustn't be limited because of working in a previously undefined environment. At this point, different methods are appearing. However, in general, this problem and its solution are collected under the same title called SLAM. The SLAM problem is, in fact, equivalent to the problems of mapping and localization [3]. A few examples of these problems are the inability to determine the proper orientation, the inability to move autonomous without hitting obstacles, and the limitation of movement due to problems in positioning [2]. Point cloud data set and registration of this point cloud data set is one of the methods

2

suggested to solve these problems; moreover, they provide solutions by identification of movements and environment. The registration of point cloud method, which is encountered in most applications, has a significant effect in solving the SLAM problem [3].

The relationship between the SLAM, as mentioned earlier, and point cloud applications, the different paths followed in the problem-solution approaches, are the subject of the project and will be discussed in detail in the following chapters. To summarize here, the environment in which the mobile robot will operate is scanned utilizing several sensors. LIDAR method was preferred in this project for this screening process. The scanned data is available in point cloud type and is stored. Next, methods called registration are defined to obtain meaningful results beyond a set of points from the scanned data. These are NDT, ICP, and Softassign and Ransca methods. Among these methods, NDT and ICP methods were used and examined in this study. Both of these methods that have iterative refinement techniques provide a meaningful result from two-point cloud scanning data at the end of the day. Then, this result can be combined with different mapping methods, and a 2D/3D map of the working area is obtained. Many different methods have been developed as mapping methods. In this project, only Octree and KD Tree methods were examined.

The methods discussed within the scope of the thesis were tried experimentally as well as theoretical analysis. In experimental studies, data from real environments were processed with the help of MATLAB and ROS programs. The ROS program is mainly used to make sense of the data received from the real environment and save it in point cloud format. Data converted into point cloud format with ROS application is now available to MATLAB. At this point, the methods handled within the scope of the thesis via MATLAB were tried respectively. Navigation Toolbox, developed within the scope of MATLAB 2019b, was used while applying the methods. For two environments, subsampling, registration, and 3d mapping methods have been applied in different combinations in three different situations. Besides, comparisons between methods with the help of the results of each study applied were made by using different metrics. In this way, the positive and negative sides of each method and situation were examined.

3

As a result, how a mobile robot models the 3D form of the working environment by using data obtained from the same environment, were studied from end-to-end. Theoretical and experimental studies on the methods used are explained in detail in the following chapters.

1.1 Thesis Outline

This thesis consists of six chapters. The first section is the introduction. Background of the subject, purpose of thesis, motivation, SLAM problem, and solutions for the problem are explained. In Chapter 2, LIDAR sensors and point cloud is stated. The point cloud is the data type investigated in this thesis, and it is obtained by using LIDAR sensors. Methods used at the thesis are explained in Chapter 3, Chapter 4, and Chapter 5, according to the order of use. These methods are subsampling, registration, and 3D mapping methods, respectively. In the last chapter, Chapter 6, the experimental part of the thesis is examined. All studies and their results are explained in Chapter 6.

5 2. LIDAR SENSORS & POINT CLOUD

The point cloud data set can be defined as the subject of this thesis study. The definition and technical features of the point cloud are explained in this chapter; moreover, LIDAR sensors which are preferred in obtaining the point cloud data set are also investigated in detail below.

2.1 LIDAR Sensors

The acronym that is mentioned in the sources, LIDAR, means Light Detection and Ranging. As the name suggests, the process that detects the distance through the instrument of light is called LIDAR [4]. A more commonly known method in this field would be a radar method, and one might think LIDAR and radar method are very similar to each other; there are significant differences that separate the two. Radar technology, on its basis, identifies the object with the distance and angular values using the microwave rays; on the other hand, LIDAR identifies only the distance to objects using the optic laser rays [5]. One of the common features between LIDAR and radar technology can be that both methods identify humans as an object. Another common feature is, due to their technical specifications, both methods are suitable to use at operations with the line-of-sight feature [6]. However, the statement above should not create the impression that LIDAR has a minimal usage. LIDAR technology, which provides the opportunity to scan in different dimensions from 2D to 3D [6], can be used on all objects natural or human-made [7]. Preferred especially by the scientists or the map experts [7], LIDAR, thanks to the remote sensing feature, is used at different operations such as measuring the number of gases like ozone, CO2 in the atmosphere,

various laser applications on the military targets, observing the changes of ice caps in the northern arctic area -especially preferred by NASA-, monitoring and tracking satellites and space debris, mapping the wind actions at and around the airports, mapping the coastlines and modeling the storm surges, tracking and identifying of unknown objects, situational tracking of chemical fumes and creating hydrodynamic

6

models [7,8,4]. As can be seen from the examples of usage areas, LIDAR has a wide usage from the space to deep seas, from deep into wild forests to chemical plants and many more. Beyond all these, LIDAR was first created only for a single purpose, which was the determination of clouds. LIDAR was the name of the light beams used for this single purpose in the 1950s [8]. The usage of LIDAR, closer to the definition of today, started a decade later. In the 1960s, LIDAR started being used to identify objects and measure the distance using the laser beams[5,8]. Additionally, a wide array of objects started getting tested with LIDAR, and the method is started being used on more rigid objects like the moon, compared to clouds [8]. In the following years, LIDAR kept evolving its name and functionality. Before being called LIDAR, it was called with different names such as ladar (laser detection and ranging), laser remote sensing, laser radar [8]. As the names change, the functionality and usage areas changed and evolved as well, and by the 1970s, it started being used at mapping lands. It is also necessary to state that, with the technological development -or the lack thereof- the data that was collected was not 100% correct geographically. Likewise, the technological structure of LIDAR was also different in those years than what it is today, which was formed of only a single beam and was more frequently used in bathymetric systems. Nowadays, as a result of the constant development of technology, LIDAR -being integrated with systems like GPS, IMU, and having a more developed technological structure- provides more accurate results [5].

At present, the essential elements of LIDAR following the improvements in the light of technological developments can be listed as follows:

Lasers: Lasers are the main component of the LIDAR system. They are used to measure the object and the distance. There are two main kinds of lasers in the applications of LIDAR. The first one is the infrared laser, which is mostly used in topographical studies and mapping of specific areas. The second type is the green laser, which is used mostly with bathymetric studies. The reason why green laser is preferred for water bodies is because of its water-penetrating characteristic [7].

Laser scanners: Laser scanners are one of the essential elements of a LIDAR system [7]. Laser scanners help the relevant laser beams to reach the target area and object of study by using the mirrors. Besides, they provide the means to measure the angle of outgoing laser beams and help with the receiving of the incoming laser beams after it reflects from the surface of interest [5].

7

GPS: GPS is the last of three essential elements of a LIDAR system [7]. GPS can be said to be the supporting element for the following system components. It helps with obtaining the location information during the scanning process. It saves the location with the x, y, and z coordinates [5].

IMU: IMU stands for the “Inertial navigation Measurement Unit.” IMU is the component responsible for the measurement of angular orientation values -also known as pitch, yaw, and roll- in different coordinate planes [5].

High precision clock: High Precision Clock is the component used for measuring and recording the precise time between the firing of a laser beam until it hits the target object and returns to the scanner, which is used in the calculations and carries high importance. Except for them, it also contains GPS ground stations, data storage, and management systems.

After having all these components, it is possible to use LIDAR to do accurate measurements regarding object scanning and distance. At the basis of operations done using LIDAR, lies the principle of time of flight (ToF) [9]. Every calculation is based on the time passing through a laser ray shooting out of the system, reaching to the object, and returning to the system. First, the pre-identified located laser is sent out of the system towards the area and object of interest. Then, the laser would get reflected off of the object, and the reflected laser would be received from the system again. During this process, data regarding the location, angular orientation, and time would be gathered by the components: GPS, IMU, and high precision clock. For each laser ray that is sent out, another set of data relevant to that ray would get recorded. By using the collected data of numerous rays sent out of the system, the data set called “point cloud” (PCL) is obtained. PCL data includes the external surface information of the area and object, the location information using the cartesian coordinates (x, y, z coordinates), and the information of the angular orientation. PCL calculates the surface features with detailed data using the time information obtained by the clock component [5,6,7,9].

The simplest form of the relevant formula used to calculate the surface information is shown in Equation 2.1.

tmeasured = 2•M

8

In the given formula, M is the distance to the target, c is constant, and the speed of light (c=3.108 m/s) in free space. tmeasured is the total measured time of travel. Since the

total time includes the time passed for laser rays to travel from the system to the object and return time from the object back to the system, the right-hand side of the equation is multiplied by 2. The process mentioned above is repeated for every position. It means that each measurement corresponds to only a single area. [6].

The system mentioned above is the most commonly known and the most basic form of LIDAR systems. It is also known as the classic LIDAR, and it is pictured in Figure2.1. There are various other LIDAR systems, and these are briefly: Fluorescence LIDAR, Differential Absorption LIDAR (DIAL), Raman LIDAR, Doppler LIDAR, and Emission-based LIDAR. Different types of LIDAR systems have the same fundamental purpose, which is the identification of the target object and measurement of distance. However, these systems differ from each other in terms of their interest and operation areas. That is the reason why there was a need to define different types of LIDAR systems [8].

Figure 2.1 : Representation of the basic working principle of classical LIDAR (ToF technique)

In addition to different types of LIDAR systems, there are also various techniques used as well. Other than the aforementioned ToF technique, which is mostly associated with the classic LIDAR method, other techniques can also be used with LIDAR systems: Amplitude Modulation of a Continuous Wave (AMCW) and Frequency Modulated Continuous Wave (FMCW). Briefly, the AMCW technique has the modulation of amplitude in its basis. The primary operating mechanism of its system is shown in Figure 2.2. With the distance calculation using this technique, instead of using the

9

amount of time for the laser to go and return, the phase difference between the continuously going and returning waves is used. The equation used in calculation with phase difference value is shown in Equation 2.2 [9].

∆P =

c (2.2)

In this given equation;

M is the distance to the target,

c is constant and the speed of light (c = 3.108 m/s) in free space,

f_modulationis the modulation frequency of the amplitude of the signal, and ∆P is the phase shift.

Figure 2.2 : Representation of the basic working principle of AMCW LIDAR In the FMCW technique, the continuous waves in the same frequency level (outgoing and incoming) are aligned, and the difference in frequency is used in the calculation of distance [9].

Having many different useful techniques and various usage areas shows that the LIDAR method has many positive features and extensive application ability. Also, when looked into the results of studies that used LIDAR methods, it is possible to say the level of flexibility, accuracy, and sensitivity is very high with LIDAR compared to other methods. Independent from the surrounding conditions, even under shadow or

10

vertical direction, it can be used efficiently, and in any environment, it can always obtain the specified location coordinate data. When considered the price-performance ratio, in comparison to other methods such as photogrammetric methods, it is both low cost and has a high speed of return. On top of that, it can be integrated with additional models such as DEM (Digital Elevation Model) and DSM (Digital Surface Model), and these models can generate data over LIDAR [5]. On the other hand, generating a massive amount of data [5] and not having 100% wavelength independence [8] may cause problems in systems with LIDAR.

Considering all the features and the usage abilities, LIDAR can be said to be a preferable and efficient method. At the systems applied, the LIDAR method creates consistent and useful point cloud results and helps with the objective of the system.

2.2 Point Cloud

As mentioned in the LIDAR section, 3D scanners represent their outputs as point cloud types. A simple structured point cloud data type contains x-y-z coordinates [10]. Assuming that the obtained data is taken from the Earth's surface, it would not be wrong to say that the information point cloud data contains conjugate of latitude, longitude, and altitude values [7]. Point cloud data does not contain any characteristic inputs about the studied object. When a situation of the necessity of some information about the object such as color, raw material, etc. has occurred, point cloud data does not serve this purpose. However, if there is a need for information about the surface of the object, it is the correct address [10].

The point cloud can be seen in different modeling studies such as canopy, digital elevation, mapping studies, object recognition, and many other areas [7,11]. Point cloud, which has a wide usage area in comparison, owes this to its structural features. Point cloud is preferred because of its simplicity, flexibility, and powerful representation. In addition to these, it doesn’t have certain basic requirements like the triangle cage, which is produced with similar usage purposes. By requirements, what is meant is polygonal mesh connection and topological consistency. These basic requirements may cause vulnerability and weaknesses in terms of performance and workload during the operation. Therefore, PCL stands out compared to other methods, in the projects where performance low and high workload needs to be avoided [11]. In this project as well, for the digital modeling of the objects, PCL data sets are preferred.

11 3. FILTERING & SUBSAMPLING

As mentioned above, PCL is gaining more importance and being used in more and more areas every passing day. However, this does not mean that PCL is flawless, unfortunately. PCL also has its problems and hardships, and the main problems are explained in the following section.

One of the main problems is, with the developing technology, the number and quality of data points that can be scanned and stored has increased. Correspondingly, storing, using, and processing the collected data has gotten more complicated. In more detail, when the scanned data points have higher quality, there is a need for more storage space to store the obtained data. Unfortunately, most of the computers will have a hard time supporting this type of storage area and quality of storage units; the ones who can support is very costly, and it will bring the cost of the project overall to a higher number. Even if these technical problems are somehow solved, analyzing this massive amount of data and creating meaningful outcomes is very difficult and exhaustive. Hence; to work with a big set of data can be seen negatively in terms of time, cost, and obtaining meaningful results.

Another problem with the PCL data is a common problem with the data of similar kinds other than PCL, which is, they contain useless (not useful, meaningless) data as well as the meaningful data sets (points). Not useful or meaningless data means the data that won’t serve the purpose of the study. On top of these, there is also the data that is called the noise data. They are sometimes caused by the inadequacy of the sensors [1] and sometimes caused by the environmental conditions. These meaningless data and noise data should be sorted out and cleaned as well, to obtain a result with high precision and quality. There are different methods to solve the two main problems and hardships mentioned above. These methods are generally called under one definition, which is downsampling or subsampling. Then, the main question is what is downsampling or subsampling and what the purpose of them is.

12

If the data set with all the problems and noise data included is called a sample, down/subsampling would sort out the data with the consideration of the purpose of the study in mind and create the data sample (subset) from the vast array of data. In this sense, sampling is how these methods are called, generally [2].

Goal of Sampling:

More accurate, economical, [2]

Robust, effectiveness and efficiency [1] The main problems with Sampling: Loss of fine details

Feature protection

3.1 Systematic Sampling

Systematic sampling is quite similar to simple random sampling. The two have the same basic properties, and they only have one single point. They differ from each other. In the simple random sampling method, n number of elements were chosen in a completely random way from a set with N elements. With systematic sampling, n elements are chosen again from a set with N elements, but this time, they are chosen with certain, specified intervals. For example, if the goal were to choose five persons among a group of 20, the interval size would be 4 (20/5). It means; first, the group of 20 persons would be divided into four groups, and one person would be chosen from each group. If, for example, the 2nd person from the first group is chosen, then the rest of the chosen people would be the one with the number of 6, 10, 14, and 18 (the interval size was calculated to be 4). The most generalized formula is shown in Equation 3.1.

i = N

n (3.1)

where i is sampling interval (interval size), N is the total size of the population, and n is the sample size (how many elements should be left at the end of the elimination). There are a couple of important points worth mentioning. One of them is; if the sampling would be performed according to systematic sampling, the elements of the leading group should not be put in order with a system of rules. It should be done randomly and not according to another system of rules or features. Otherwise, the

13

sample that is obtained would not give the correct representation of the group and would not produce a correct result because the sample that is separated according to another rule. It would not represent the overall collected data; instead, it would represent a section of data separated with the specific rule, and the ones who would not follow the rule would be selected out without having a chance to be analyzed. Another point is; in systematic sampling, all the elements of one set have the same possibility of being selected, like in random sampling method. Moreover, when the same set is analyzed using the simple random method or systematic sampling method, it is observed that the error rate would be equal to each other with both methods. As expected, this is only valid when the elements are put in order randomly, not following an absolute rule.

3.2 Stratified Random Sampling

Although stratified random sampling is very similar to the systematic sampling method, it can be seen as an improved version of the systematic sampling method. Notably, it has similarities with the systematic sampling method. The stratified method contains subsets that also exist at systematic sampling method applications; however, the decision of subsets is related to specified properties in this method. Then, random selection is made among these designated, specific, and homogeneously distributed subsets, as in other methods. It has similar disadvantages as it has similar properties with the other two methods mentioned before; however, the biggest problem in this method is the determination of the subsets. For that, all the elements have to be done; moreover, the properties of them must be known beforehand. Hence, the subsets can be determined. It is, unfortunately, not possible for every system. Some systems may be unparseable into subsets. In the case of systems have decomposable features, it may not be possible to have information about each element in advance.

3.3 Decimation

The most prominent feature of this method is that the data set is defined as a matrix. Thinking that it consists of rows and columns, a starting point is selected from the data set, and this point is accepted as (0,0) point. Then, each data point is numbered sequentially either to the right/left or up/down in the frame of this reference point. For

14

example, in the case of the lower-left corner, be selected as the (0,0) point; furthermore, each point is enumerated incrementally to the right, such as 1, 2, 3. Each data point is now represented by a number. Elimination can be done with four different combinations from representative numbers. For example, it can be said that only the odd numbers remain, and the others are eliminated. In this case, data points corresponding to an odd number such as 1, 3, 5, 15, etc. are kept, and the rest is eliminated. Similarly, even numbers can be kept; or odd numbers in the row and even numbers in the column can be defined as remained values. In all of these cases, the initial data set is halved.

The disadvantageous and challenging part of the decimation method, which is based on a relatively simple and understandable logic, is that a certain mass can be eliminated due to the arrangement of the elements of the data set. For example, there is a data set that includes only red and white balls; moreover, the elements of the data set are arranged in order one red and one white. Then, when only the single digits are selected from this set, only the red data points remain, and all whites are eliminated. It makes it appear as if there is only a master data set consisting of red. Naturally, a false and undesirable result will be obtained at the end.

3.4 Averaging

In this method, the initial data set is defined as a matrix. But this time, instead of directly selecting the specific ones from each of the numbered data points, four adjacent neighboring data points are selected, and then they will be reduced to a single data point. The density of the last data point that is reduced from four points to one must be equal to the average of the density of the four selected data points. In this method, the original data set, which is 4X4, is reduced to 2X2 at the end of the process. It is similar to the decimation method.

15 4. REGISTRATION

The ultimate goal in PCL studies is to obtain a 3D image of the solid object being scanned with the most optimal error possible. The most important milestone on the way to this end is the registration step. Because every work done at this stage has a significant effect on the final result. Depending on each decision taken and applied in the registration step, it is possible to achieve almost the same result as the scanned object. At the same time, it is also possible to achieve a result that is entirely irrelevant to the actual object. Moreover, although these two results may appear to be so distant from each other, it should be emphasized that the difference between the paths followed in achieving two separate results is rather small and subtle. For this reason, the registration stage is named as the building block of PCL studies [14]. For registration, having this kind of critical role leads to have advantages to be preferred at some hard, complicated applications such as recognition problems, object detection, and of course, SLAM problems [14]. It is called registration only in terms of the dictionary, but its scope and importance is crucial, it is seen that it consists of three main subjects. They are two PCL data with some degree of overlap, reference PCL scan, and model PCL scan.

In summary, meaningful data extraction from these three objects is called registration [15,16]. In other words, one of two separate PCL scans, which are overlapped at certain angles and belongs to a solid object whose 3D output is desired, is chosen as the reference. The most optimized match and position between the referenced scan and the other one is tried to be found [14,15,16]. Different methods have been developed for this process. The most preferred and naturally known methods are ICP (Iterative Closest Point), NDT (Normal Distributions Transform), Softassign, and Ransca methods.

16

In this study, ICP and NDT methods among these methods have been focused. Although these two methods are based on the same logic named as iterative refinement techniques, both methods reach the result in different ways [14]. Naturally, it is not surprising to await differences between the results. Each method will be examined in detail below.

4.1 Iterative Closest Point (ICP)

In the first quarter of 1992, two scientists named Paul J. Besl and Neil D. McKay introduced a new approach to the registration of 3D objects with their publication ‘A method for registration of 3-D shapes’. In those times, although it had not been known that it would get attention by a big group, the iterative closest point registration method, ICP, has emerged in the registration of 3D objects. Right after Besl and McKay, in the second quarter of the same year, Yang Chen and Gerard Medioni bring ICP method into the forefront by publishing another paper about the same subject [15]. Within years, new approaches have been developed, as well as arrangements and improvements have been made on the method. Different and new publications about ICP have been published and continue to do so, but these two studies are the basis of the ICP method. The ICP method, in which Besl and McKay ignited the first spark, is now one of the most well-known and widely used methods for registration of 3D point cloud data sets.

An addition of pure ICP method, GP-ICP (Geometric Primitive ICP), feature-based ICP, and Go-ICP (Globally Optimal ICP) methods, etc. are available as registration methods today’s world. But all these methods are mainly based on the pure Besl and McKay's ICP algorithm [15,17]. In this project, a simple ICP method was utilized. There is one and apparent reason for choosing ICP method for registration, and it is the simplicity of the algorithm. Also, it follows an easily understandable path. It should also be emphasized that these two most essential features of the method are, of course, possible in ideal cases. This method also has problematic, and challenging parts like almost everything has. As mentioned above, different versions of ICP method are developed to solve, at least minimize, problems. Each of the different versions is focused on different problems mainly and is created to get rid of them [18].

17

Apart from the two most striking features mentioned above, the ICP method has many impressive advantages [16]. These advantages make it attractive and a point of interest in registration. The advantages of 3D point cloud registration operation using the ICP method can be summarized as follows [19]:

Any preprocessing steps do not require at 3D point cloud data set applications. It is the case when external disturbances are so small that they can be ignored, that is to say, in terms of statistical convergence to zero.

It provides the opportunity to examine the targeted object as a 6-DOF, not only a limited degree of freedom such as 2-DOF or 3-DOF.

It does not consist of any differential mathematical calculations. It does not contain any extraction, such as the local feature.

The target object can have a form of faceted surfaces, point sets, parametric surfaces, implicit curves, polylines, line segment sets, or any random shape. Object’s shape does not affect the application of ICP method in any aspect, positive or negative [19]. Besides these advantages, ICP method has some disadvantages that lead to generating some studies about fields open to development. They can be summarized like that: It is not compatible with methods such as SVD, fast least-squares quaternion. It makes it so challenging to bring up an advanced level and to use advanced fast algorithms, which help to understand which option can be chosen in uncertain and unclear situations.

It is not a method that is resistant to external disturbances, especially against significant external factors, statistically. A statistical and robust method can be implemented to increase the resistance points [19].

At the above, it is mentioned that it is a surface independent method as significant advantages; however, it leads to becoming a point-based method. It means that, in some cases, the surface around each point is not taken into account [15]. In summary, the calculations are made from a narrow point of view in contradiction to a general view.

In terms of time and cost, the nearest neighbor point research step is the most problematic part of this method.

18

Doing all these steps for each point in each iteration is another challenge for ICP method. In other words, reference and model scans have r and m number of points, respectively. The required time to complete each research in each iteration is equal to O(r

•

Some methods and applications exist to improve or neutralize the disadvantages, but because of the extent of this study, just pure ICP method has been investigating. Pure ICP method has been applied in the research without add-ons.

The theory of simple ICP method is not so complicated. ICP method also consists of two main elements, which are also part of registration. They are two PCL data set. One of them is labeled as reference data set -R- and the other one is named as model data set -M-. In some references, the model data set may also be referred to as the current scan. End of the day, when all procedures for this method are done, three outputs are expected. They are transformation parameters and meaningful 3D scanned object output, which is obtained from two primary data set and overlapping in the optimal level as possible. Transformation parameters are the translation vector and a rotation matrix. In reaching these three results by following the path from the two initially accepted data sets, the nearest neighbor in the reference scan has been searched for each point of the model scan. Research of the nearest neighbor is repeated for each point at each iteration iteratively. Euclidean distance displayed at Equation 4.1 is preferred when these calculations are made. It means that the minimum of the sum of the square distances of the distance between the two corresponding points is calculated. Then, translation parameters, which are rotation matrix and translation vector, are calculated by using the result of the calculation of this distance. Related calculations and mathematical demonstrations are represented at the below [15,16].

𝑚𝑖𝑛𝑑𝑚 = min[√(𝑚𝑚− 𝑟𝑟)

•

At the Equation 4.1;

Minimum distance, 𝑚𝑖𝑛𝑑𝑚, is calculated by using Euclidean distance. There is the

primary condition that minimum distance must be bigger than the threshold value, ts. Otherwise, that point does not count. Moreover, 𝑚_𝑚 is mth point in model scan, m. 𝑟_𝑟 is also rth point in reference scan, r. m and r number of points located at the model scan and reference scan, respectively.

19

After that, now, other elements named rotation matrix represented with R, and translation vector represented with t, can be calculated by using Equation 4.2 and Equation 4.3. Before that, to define R and t, some other values must be found first. They are centroids of related points showed at Equation 4.4, Equation 4.5, and then the covariance matrix is calculated at Equation 4.6. When doing them, Singular Value Decomposition (SVD) displayed at Equation 4.7 is used to combine the covariance matrix and rotation matrix.

𝑆𝑟= 1 𝐾 ∑ 𝑟𝑘 𝐾 𝑘=1 𝑆𝑚= 1 𝐾 ∑ 𝑚𝑘 𝐾 𝑘=1 where 𝐾 = ∑ ∑ 𝑤_𝑟𝑚 𝑟 𝑛=1 𝑚 𝑛=1 {𝑤𝑟𝑚= 0 𝑤_𝑟𝑚= 1| 𝑓𝑜𝑟 𝑝𝑎𝑖𝑟 𝑝𝑜𝑖𝑛𝑡𝑠 𝑒𝑙𝑠𝑒 }

Thus, each point at the reference and model scan can be reformed according to their centroid values.

𝑅_𝑐 = {𝑟_𝑐 = 𝑟_𝑘− 𝑆_𝑘} 𝑤ℎ𝑒𝑟𝑒 𝑘 = 1, 2, … 𝐾

𝑀_𝑐 = {𝑚_𝑐 = 𝑚_𝑘− 𝑆_𝑘} 𝑤ℎ𝑒𝑟𝑒 𝑘 = 1, 2, … 𝐾 Now, the covariance matrix can be calculated.

𝐶𝑀 = 𝑀𝑐

•

From SVD perspective, it also equals to 𝐶𝑀 = 𝑈

•

•

where 𝑈𝑇

_•

_•

𝐶𝑜𝑙𝑢𝑚𝑛𝑠 𝑜𝑓 𝑈 = 𝑡ℎ𝑒 𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟𝑠 𝑜𝑓 𝐶𝑀

•

20

𝐶𝑜𝑙𝑢𝑚𝑛𝑠 𝑜𝑓 𝑉 = 𝑡ℎ𝑒 𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟𝑠 𝑜𝑓 𝐶𝑀𝑇

_•

𝑆𝑖𝑛𝑔𝑢𝑙𝑎𝑟 𝑣𝑎𝑙𝑢𝑒𝑠 𝑖𝑛 𝑆 = 𝑠𝑞𝑢𝑎𝑟𝑒 𝑟𝑜𝑜𝑡𝑠 𝑜𝑓 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐶𝑀𝑇

•

At this point, everything is defined to provide a rotation matrix and a translation vector. They are represented at the Equation 4.8 and Equation 4.9.

𝑅 = 𝑉

•

•

When applying the ICP method, there are several crucial steps to be taken into account to ensure that this theory is functional. They are the correct definition of rejection rules, and convergence precision can be named as the threshold in some references. For these two definitions, it is possible to call them as the "unsung heroes" that keep the method running flawlessly. Furthermore, these are the accuracy and stability-enhancing properties of the registration process [16]. What these are two definitions and why they are used is explained briefly.

Threshold: While applying the ICP method, it is seen as a control mechanism in controlling whether or not suitable points are selected. When the minimum distance between points in reference and model data sets is calculated at each iteration, the moment when the correct distance is identified, displays that it converges to define threshold distance. In other words, in that time, the distance between the two points is less than the threshold value; moreover, in that moment, it can be said that matching of two points is adequately done. Thus, that iteration can be terminated, and the procedure is followed up with the next iteration. Otherwise, all calculations must be done over and over until proper distance is found.

Except for these calculations, predefined error metrics is also existing inside of the method. In parallel with calculations of proper distance, the error is also calculated and controlled with the threshold value. It has the same rule with distance value; so, it has been expecting that error value must be less than the threshold. The iteration cannot be terminated as long as big numbers are seen at error value, and it continues until the error value go down below the threshold.

(4.7b) (4.7c)

(4.8) (4.9)

21

Rejection rules: Two main rules may be emphasized. First of them is the rejection of the points in the model scan, which are located at the boundary of the relevant reference data set. This procedure must be applied without any exceptions. Otherwise, it provokes a systematic congestion and error called drag bias, due to mismatch.

The other rule is related to the threshold. It consists of the same logic with threshold rule that the calculated distance between two points must be smaller than the threshold value. Pairs of matched point pairs with a distance above the Threshold value must also be rejected without exception.

It should be underlined that the definitions of rejection rule and threshold mentioned in detail above are very general. Also, the definitions that should be assigned a numerical value such as threshold should be determined according to the characteristics of the data being studied.

Besides these, there are some crucial points to achieve a more accurate and highly matched registration result. These are as follows:

The threshold value might be decreased between each following iteration. Thereby, disturbing external data is eliminated. The aim here is that matching of point pairs at long distances has been completed quickly. Then over the iterations, more optimum results can be obtained by studying a smaller area instead of putting account point pairs at long distances. Choosing threshold values that are decreasing by iterations pass and even converges to zero, is a useful application for achieving low error rate results when iterations are close to the end [15].

The other subject is that the decision of the initial value will be used at the first iteration. It is critical to complete this definition. If this definition is made wrong, the result obtained is probably incorrect when all iterations are terminated. It is also possible to observe results such as going to local optimum values within the iteration [16].

4.2 Normal Distributions Transform (NDT)

Scientists always keep studying research on data registration with different perspectives. Even they are not satisfied with present approaches and not limited themselves with known methods. They always follow the question of what can be differentiated to discover new methodologies. 2D or 3D registration of objects is an