### ADAPTIVE TECHNIQUES IN COMPRESSED SENSING BASED

### DIRECTION OF ARRIVAL ESTIMATION

### a thesis submitted to

### the graduate school of engineering and science of bilkent university

### in partial fulfillment of the requirements for the degree of

### master of science in

### electrical and electronics engineering

### By Berkan Kılı¸c

### July 2021

ADAPTIVE TECHNIQUES IN COMPRESSED SENSING BASED DIRECTION OF ARRIVAL ESTIMATION

By Berkan Kılıç July 2021

We certify that we have read this thesis and that in our opinion it is fully adequate, in scope and in quality, as a thesis for the degree of Master of Science.

Orhan Arıkan(Advisor)

Approved for the Graduate School of Engineering and Science:

Director of the Graduate School ii

Simfu Gezici

U Ezhan Kar~an

### ABSTRACT

### ADAPTIVE TECHNIQUES IN COMPRESSED SENSING BASED DIRECTION OF ARRIVAL

### ESTIMATION

Berkan Kılı¸c

M.S. in Electrical and Electronics Engineering Advisor: Orhan Arıkan

July 2021

Direction of arrival (DOA) estimation is an important research area having exten- sive applications including radar, sonar, wireless communications, and electronic warfare systems. Development and popularization of the compressed sensing (CS) theory has led to a vast literature on the use of the CS techniques in DOA esti- mation which has been shown to be superior over the classical techniques under various scenarios. In the CS based techniques, measurement matrices determine the received information while sparsity promoting reconstruction algorithms are used to estimate the unknown DOAs. Hence, design of measurement matrices and sparse reconstruction algorithms are among the most important aspects of the CS theory. In this thesis, both aspects are investigated and novel techniques are proposed for improved performance.

Following a brief explanation of the classical and the CS based DOA estimation techniques, a new optimization perspective is introduced on the Capon’s beam- former by using the minimum mean square error criterion. After that, a mea- surement matrix design methodology exploiting prior information on the source environment is introduced. Hardware and sofware implementation constraints of the introduced method are investigated and more efficient alternatives are pro- posed. Additionally, an adaptive dictionary design algorithm is introduced for more effective use of the prior information. Lastly, the Cramer-Rao Lower Bound expression for the compressed DOA signal models is derived and its implications on the measurement matrix design are investigated leading to a sector based mea- surement matrix design technique along with a novel reconstruction algorithm.

Keywords: Compressed sensing, direction of arrival estimation, measurement matrix design, sparse reconstruction.

### OZET ¨

### SIKIS ¸TIRILMIS ¸ ALGILAMA TABANLI GEL˙IS ¸ AC ¸ ISI KEST˙IR˙IM˙INDE UYARLANAB˙IL˙IR TEKN˙IKLER

Berkan Kılı¸c

Elektrik ve Elektronik M¨uhendisli˘gi, Y¨uksek Lisans Tez Danı¸smanı: Orhan Arıkan

Temmuz 2021

Geli¸s a¸cısı kestirimi (GAK); radar, sonar, kablosuz haberle¸sme ve elektronik harp gibi alanlarda geni¸s kapsamlı uygulamalara sahip ¨onemli bir ara¸stırma alanıdır.

Sıkı¸stırılmı¸s algılama (SA) teorisinin geli¸simi ve pop¨ulerle¸smesi, klasik tekniklere kıyasla pek ¸cok senaryo altında daha y¨uksek ba¸sarım sa˘gladıkları g¨osterilen geni¸s bir SA tabanlı GAK literat¨ur¨un¨un olu¸smasına yol a¸cmı¸stır. SA tabanlı tekniklerde, alınan veri ¨ol¸c¨um matrisleri tarafından belirlenirken kestirilmek iste- nen geli¸s a¸cıları seyreklik te¸svik edici geri¸catım algoritmaları ile elde edilmektedir.

Bu sebeple, ¨ol¸c¨um matrisi tasarımı ve seyrek geri¸catım algoritmaları SA teorisinin en ¨onemli etmenlerindendir. Her iki konu da bu tezde incelenerek, y¨uksek ba¸sarım g¨osteren yeni teknikler ¨onerilmektedir.

Klasik ve SA tabanlı GAK tekniklerinin kısaca a¸cıklanmasının ardından, Capon huzmeleyici tekni˘gine en k¨u¸c¨uk ortalama kareler hatası tabanlı yeni bir optimizasyon bakı¸s a¸cısı getirilmektedir. Daha sonra, ortamdaki kaynak da˘gılımının ¨onsel bilgisinden faydalanabilen bir ¨ol¸c¨um matrisi tasarımı sunul- maktadır. Sunulan metodolojinin yazılımsal ve donanımsal uygulama kısıtları ara¸stırılarak, hem yazılımsal hem de donanımsal a¸cıdan daha verimli alterna- tifler ¨onerilmektedir. Aynı zamanda, ¨onsel bilginin daha etkin kullanımı i¸cin uyarlanabilir bir s¨ozl¨uk tasarımı da ortaya konmaktadır. Son olarak, sıkı¸stırılmı¸s GAK i¸saret modelleri i¸cin bir Cramer-Rao Alt Limiti ifadesi t¨uretilmekte, t¨uretilen ifadenin ¨ol¸c¨um matrisi tasarımı ¨uzerindeki sonu¸cları ara¸stırılmakta, ve bu sonu¸clar kullanılarak sekt¨or tabanlı bir ¨ol¸c¨um matrisi tasarım tekni˘gi ile bir- likte yenilik¸ci bir geri¸catım algoritması sunulmaktadır.

Anahtar s¨ozc¨ukler : Sıkı¸stırılmı¸s algılama, geli¸s a¸cısı kestirimi, ¨ol¸c¨um matrisi tasarımı, seyrek geri¸catım.

### Acknowledgement

Prof. Dr. Orhan Arıkan has my deepest gratitude for his persistent support and guidance throughout my graduate studies. This thesis is just a tiny subset of what I learnt from him. He always allocated time for me even though he often had a tight schedule and it was a great honor and pleasure for me to study with him. Apart from all, I truly enjoy being a researcher in this area, and Prof.

Orhan Arıkan is the main reason.

During the early stages of my career, Dr. Mert Kalfa has been a great mentor and a precious friend. He taught me so much things and always valued my opinions even though he had much more experience. His big influence on my career is unquestionable. I also learnt a lot from another invaluable friend Alper G¨ung¨or. I am thankful to both. Some of our collaborative works also contributed to this thesis.

Dr. Can Barı¸s Top, our supervisor at ASELSAN Research Center, has always been supportive of me. He always followed and appreciated my studies that made me study harder. For me, he is definitely much more than a supervisor.

I am grateful to my thesis committee members Prof. Dr. Sinan Gezici and Prof. Dr. C¸ a˘gatay Candan for kindly accepting to review this thesis.

I am thankful to my employer ASELSAN Inc., for supporting this thesis by providing powerful workstations, allowing me to follow my graduate classes and conduct my research.

I would like to thank The Scientific and Technological Research Council of Turkey (T ¨UB˙ITAK) for the financial support provided under the 2210-A Schol- arship Program.

I have many wonderful friends. During the preparation of this thesis, we drank gallons of coffee with Berkay, Burak Alptu˘g, Do˘gan Can, Eralp, Erdem, Erdin¸c, G¨okhan, Kutay, Sarp, Umut, and ¨Umitcan. I had the best times with Adem,

vi

Ayten, Benan, Burak, Cem, Ece, Emir, Halil, Korda˘g, Mehmet Berkay, Mert, M¨usl¨um Emir, O˘guz, Onur, Ozan, Ozancan, ¨Ozkan, Recep, Serdar, and Serhat at Bilkent.

My mom and dad, I love you and always will. I am well aware that you dedicated your lives to me. As a very humble and negligible return, this thesis is dedicated to you.

## Contents

1 Introduction 1

1.1 Contribution and Organization . . . 4

1.2 Notation . . . 6

1.3 Abbreviations and Symbols . . . 7

2 Direction of Arrival Estimation 9 2.1 Compressed Sensing . . . 9

2.2 Signal Model for Direction of Arrival Estimation . . . 13

2.2.1 Wideband DOA Estimation . . . 17

2.2.2 Coherent Source Scenarios . . . 18

2.2.3 The Method of Performance Evaluation . . . 19

2.3 The Classical Direction of Arrival Estimation Techniques . . . 19

2.3.1 The Bartlett Beamformer . . . 20

2.3.2 The Capon’s Beamformer . . . 21

CONTENTS viii

2.3.3 The Multiple Signal Classification (MUSIC) . . . 23 2.3.4 Minimum Mean Square Error Criterion in DOA Estimation 24 2.4 The Compressed Sensing based Direction of Arrival Estimation . . 26 2.5 Numerical Simulations . . . 32

2.5.1 Comparison of the Capon’s Beamformer and the MMSE based DOA Estimation . . . 32 2.5.2 Comparison of the Classical and the CS based Techniques 35 2.5.3 Summary of the Comparison Study . . . 38

3 Adaptive Measurement Matrix Design 40

3.1 Stable Recovery Conditions . . . 41 3.2 Review of the Measurement Matrix Design Approaches . . . 42 3.2.1 The Mutual Coherence based Measurement Matrix Design 43 3.2.2 The CRLB based Measurement Matrix Design . . . 45 3.2.3 The Mutual Information based Measurement Matrix Design 46 3.3 The Proposed Measurement Matrix Design Methodology . . . 47 3.4 The Dictionary Optimization . . . 52 3.5 Adaptive Measurement Matrix Design with Hardware Constraints 54 3.5.1 Proposed Hardware-Efficient Methodology . . . 55 3.6 Simulation based Numerical Results . . . 58

CONTENTS ix

3.6.1 Performance of the A-MMD, the A-MMD-UD, and the EA-

MMD . . . 58

3.6.2 Performance of the A-MMD-BD . . . 75

3.6.3 Summary of the Comparison Study . . . 79

4 Analytical Performance Bounds and Numerical Results 81 4.1 Related Works . . . 82

4.2 The CRLB Derivation . . . 83

4.3 The Proposed Measurement Matrix Design Technique . . . 84

4.4 The Proposed DOA Estimation Methodology . . . 89

4.5 Numerical Results . . . 91

4.5.1 The CRLB Comparison . . . 91

4.5.2 DOA Estimation Comparison . . . 98

4.5.3 Summary of the Comparison Study . . . 104

5 Conclusions and Future Research Directions 106 Appendix A 121 Appendix B 123 B.1 . . . 128

B.2 . . . 130

CONTENTS x

Appendix C 133

C.1 . . . 134

Appendix D 135

Appendix E 137

## List of Figures

2.1 Comparison between the `_{1} and the `_{2} norm regularizations by
using a simple example. Left figure shows the solution when the `1

norm of x is minimized subject to an affine constraint. Right figure
shows the solution when the `_{2} norm of x is minimized subject to
the same affine constraint. . . 12
2.2 Illustration of a simple DOA estimation scene . . . 14
2.3 Geometrical interpretation of the Capon’s beamformer for the two-

dimensional case . . . 23 2.4 Hardware implementation of the presented system for M = 4 and

m = 2 . . . 29 2.5 Performance comparison of the Capon’s beamformer and the

MMSE based technique for K = 1, and N_{s}= 100. . . 33
2.6 Performance comparison of the Capon’s beamformer and the

MMSE based technique for K = 2, and N_{s}= 100. . . 34
2.7 Performance comparison of the Capon’s beamformer and the

MMSE based technique for K = 1, and N_{s}= 2. . . 35
2.8 Performance comparison of the classical and the CS based tech-

niques for K = 2, and N_{s} = 10. . . 36

LIST OF FIGURES xii

2.9 Performance comparison of the classical and the CS based tech-
niques for K = 2, and N_{s} = 2. . . 37
2.10 Performance comparison of the classical and the CS based tech-

niques for K = 2, and N_{s} = 10 under multipath effects. . . 38

3.1 Schematic diagram of an adaptive DOA estimation system . . . . 51 3.2 The Adaptive Grid Selection Algorithm . . . 53 3.3 Hardware implementation of the simplified system for M = 4 and

m = 2 . . . 55 3.4 The left figure shows the measurement matrix structure when each

sensor is connected to one channel only (r = 1). The right fig- ure shows the measurement matrix structure when each sensor is connected to two channels (r = 2). The gray boxes denote the non-zero entries of the measurement matrix while the white boxes show the zero entries. . . 57 3.5 RMSE vs K for the scenario described in Section 3.6.1.2.1 . . . . 66 3.6 RMSE vs Q for the scenario described in Section 3.6.1.2.2 . . . . 67 3.7 RMSE vs L for the scenario described in Section 3.6.1.2.3 . . . 69 3.8 RMSE vs m for the scenario described in Section 3.6.1.2.4 . . . . 70 3.9 RMSE vs σs for the scenario described in Section 3.6.1.2.5 . . . . 72 3.10 RMSE vs SNR for the the scenario described in Section 3.6.1.3 . . 73

LIST OF FIGURES xiii

3.11 Parameter selection for the EA-MMD. The parameters are chosen
as equal α_{1} = α_{2} as before and the step size is set to 0.05. The
left figure shows the RMSE values for the tracked sources and the
right figure shows the RMSE values for all targets. Note that the
color bars are in different scales for a better visualization. . . 74
3.12 RMSE vs SNR for the scenario given on TABLE 3.12 for M = 36

and m = 12 . . . 77 3.13 RMSE vs SNR for the scenario given on TABLE 3.12 by replacing

Source 2 with an emerging source for M = 36 and m = 12 . . . . 78 3.14 RMSE vs SNR for the scenario given on TABLE 3.12 for M = 36

and m = 6 . . . 79

4.1 The CRLB Comparison among Φ_{dir}, Φ_{rg}, Φ_{ber}, and Φ_{des} for dif-
ferent sector indices by fixing M = 50, m = 10, K = 2, N_{s} = 10,
L = 100 and L^{(i)}r = 20 . . . 93
4.2 The CRLB Comparison among Φdir, Φrg, Φber, and Φdes for dif-

ferent m values by fixing M = 50, K = 2, N_{s} = 10, L = 100 and
L^{(i)}r = 20 . . . 94
4.3 The CRLB comparison among Φ_{dir}, Φ_{rg}, Φ_{ber}, and Φ_{des} for differ-

ent N_{s} values by fixing M = 50, m = 10, K = 2, L = 100 and
L^{(i)}r = 20 . . . 95
4.4 The CRLB comparison among Φ_{dir}, Φ_{rg}, Φ_{ber}, and Φ_{des} for differ-

ent K values by fixing M = 50, m = 10, N_{s} = 10, L = 100 and
L^{(i)}r = 20 . . . 96
4.5 The CRLB comparison among Φ_{dir}, Φ_{rg}, Φ_{ber}, and Φ_{des} for differ-

ent M values by fixing m = 10, N_{s} = 10, K = 2, L = 100 and
L^{(i)}r = 20 . . . 97

LIST OF FIGURES xiv

4.6 DOA estimation performance comparison among the FDOA, the
IRG, the A-MMD, the EA-MMD and the RGD for K = 1 emerging
at the 2^{nd} sector with L = 100 and L^{(i)}r = 20 . . . 99
4.7 DOA estimation performance comparison among the FDOA, the

IRG, the A-MMD, the EA-MMD and the RGD for K = 2 where
the first and the second target emerge at the 2^{nd} and the 4^{th} sector
respectively with L = 100 and L^{(i)}r = 20 . . . 100
4.8 DOA estimation performance comparison among the FDOA, the

IRG, the A-MMD, the EA-MMD and the RGD for K = 3 where
the first, the second and the third target emerge at the 2^{nd}, the
3^{rd}, and the 4^{th} sector respectively with L = 100 and L^{(i)}r = 20 . . 101
4.9 DOA estimation performance comparison among the FDOA, the

IRG, the A-MMD, the EA-MMD and the RGD for K = 4 where
the first two targets emerge at the 2^{nd} sector and the remaining
two targets emerge at the 4^{th} sector with L = 100 and L^{(i)}r = 20 . 102
4.10 DOA estimation performance comparison among the FDOA, the

IRG, the A-MMD, the EA-MMD and the RGD for K = 4 where
the first two targets emerge at the 1^{st} sector and the remaining
two targets emerge at the 4^{th} sector with L = 120 and L^{(i)}r = 30 . 103
4.11 DOA estimation performance comparison among the FDOA, the

IRG, the A-MMD, the EA-MMD and the RGD for K = 4 where
the first two targets emerge at the 1^{st} sector and the remaining
two targets emerge at the 3^{rd} sector with L = 120 and L^{(i)}r = 40 . 104

## List of Tables

1.1 List of Symbols . . . 7 1.2 List of Abbreviations . . . 8

3.1 DOA Distribution of the Sources to Investigate the SNR Depen-
dency (1) . . . 61
3.2 The RMSE^{o} Results for the Scenario Explained in Section 3.6.1.1.1 61
3.3 DOA Distribution of the Sources to Investigate the SNR Depen-

dency (2) . . . 62
3.4 The RMSE^{o} Results for the Scenario Explained in Section 3.6.1.1.2 63
3.5 DOA Distribution of the Sources to Investigate the SNR Depen-

dency (3) . . . 63
3.6 The RMSE^{o} Results for the Scenario Explained in Section 3.6.1.1.3 64
3.7 The RMSE^{o} Results for the Scenario Explained in Section 3.6.1.1.4 65
3.8 DOA Distribution of the Sources to Investigate the Q Dependency 67
3.9 DOA Distribution of the Sources to Investigate the M/m Depen-

dency . . . 70

LIST OF TABLES xvi

3.10 DOA Distribution of the Sources to Investigate the σ_{s} Dependency 71
3.11 DOA Distribution of the Sources to Investigate the Parameter Se-

lection for the EA-MMD . . . 74

3.12 DOA Distribution of the Sources to Investigate the Performance of the A-MMD-BD . . . 76

4.1 ω Intervals for Each Sector when L = 100 and L^{(i)}r = 20 . . . 91

4.2 ω Intervals for Each Sector when L = 120 and L^{(i)}r = 30 . . . 102

4.3 ω Intervals for Each Sector when L = 120 and L^{(i)}r = 40 . . . 103

## Chapter 1

## Introduction

Compressed sensing (CS) theory has brought new perspectives to encoding and decoding of sparsely representable signals [1–6]. The successful recovery requires the use of a sparsifying signal dictionary, a proper sampling strategy, and a spar- sity promoting reconstruction algorithm. There is a vast literature on the choice of an appropriate dictionary, which may either be learned by using the training data or pre-defined by exploiting the received signal characteristics [7, 8]. The encoding strategies are determined by measurement matrices that are desired to be incoherent with the chosen dictionary [9]. The restricted isometry property (RIP) [4] and the mutual coherence [10] are two well-known concepts to quantize the stable recovery performance provided by measurement matrices. Since ran- dom Gaussian and Bernoulli matrices satisfy the RIP with a high probability, they are the most commonly used measurement matrices in the CS literature [3, 11].

Despite the strength of the random measurement matrices with respect to the RIP, it is possible to exceed their reconstruction performance by using various design criteria such as the mutual coherence [12], the Cramer Rao Lower Bound (CRLB) [13], and the mutual information [14]. In the decoding process of the CS techniques, sparsity is used a priori leading to the stable recovery guarantees by using far fewer number of measurements compared to the Nyquist sampling the- orem provided that the designed measurement matrix successfully captures the relevant information in the environment and the signal is sparsely represented

by using an appropriate dictionary. In the CS literature, various reconstruc- tion algorithms have been proposed including the Orthogonal Matching Pursuit (OMP) [15], the Least Absolute Shrinkage and Selection Operator (LASSO) [16], and the Basis Pursuit Denoising (BPDN) [17].

Direction of Arrival (DOA) estimation for narrowband signals is one of the re- sarch areas where the CS theory has been succesfully applied. Sensor arrays are typically used in DOA estimation where an impinging waveform is received with relative phase shifts at each sensor location enabling estimation of its DOA. There are classical techniques such as the Bartlett beamformer [18, 19], the Capon’s beamformer [20], and the MUltiple SIgnal Classification (MUSIC) [21] that are being used for many decades in the DOA applications. The Bartlett beamformer is one of the most popular DOA estimation techniques due to its simplicity. How- ever, the angular resolution provided by the Barttlet beamformer is limited by the beamwidth of array [18]. Hence, the Barttlet beamformer cannot differentiate between two targets having angular separation less than the array beamwidth, i.e., it cannot achieve super-resolution. To overcome that resolution issue, the Capon’s beamformer has arised where the beamformer coefficients are computed in a more complicated way. The Capon’s beamformer can provide much higher resolution compared to the Bartlett beamformer. However, the resulting beam- former coefficients require the inverse of the covariance matrix of the array data which deteriorates the robustness of the Capon’s beamformer. Hence, the robust Capon’s beamformer techniques have continued to be studied even decades after its introduction [22]. Apart from these beamforming techniques, the MUSIC has emerged as a subspace-based technique exploiting the eigen decomposition of the covariance matrix of the array data. Unlike the Capon’s beamformer, the MU- SIC does not require inverting the covariance matrix which improves robustness of the algorithm. On the other hand, computation of the covariance matrix itself actually limits both the MUSIC and the Capon’s beamformer in many differ- ent ways. First of all, the computation of a reliable covariance matrix estimate requires more than a single snapshot. Therefore, the covariance matrix based techniques assume a stationary source environment at least for a duration that is sufficient to accurately estimate the covariance matrix of the array data. In

many applications of DOA estimation, the dynamic source environments must be handled and the performance of these techniques degrades under such circum- stances [23]. Furthermore, the existence of coherent sources results in a rank deficient covariance matrix estimate and this severely affects the performance of the MUSIC and the Capon’s beamformer. Coherent source scenarios are also commonly encountered in many practical applications which may happen as a result of multipath effects [24] or smart jamming attacks [25]. In short, although the Capon’s beamformer and the MUSIC can achieve super-resolution, they also have practically important disadvantages.

The CS based DOA estimation techniques can outperform the classical tech- niques under various scenarios [26]. Most notably, the CS based DOA estimation techniques can provide improved angular resolution, they do not require accurate covariance estimates of the array data, and they provide reliable estimations even with a single snapshot. In the DOA estimation systems, measurement matrices undersample the analog sensor outputs with the aim of not losing any informa- tion, which can be realized in hardware by using low noise amplifiers, attenuators and phase shifters [27]. Since the use of measurement matrices allows for reliable estimations by using the undersampled data, the CS based DOA estimation en- ables simplified hardware and software implementation compared to the classical techniques. Various measurement matrix design techniques have been studied in the CS based DOA estimation literature mostly to outperform the random Gaussian measurement matrices [28–32].

Adaptive techniques, that have been proven useful in the classical DOA estima- tion, have a significant potential to increase the performance of the CS based DOA estimation techniques as well [33]. There are alternative adaptive reconstruction strategies which can be employed in the CS applications. In one such approach, prior information on the signal support over the dictionary is used to formulate a weighted optimization problem [34]. Another approach is the adaptive design of the measurement matrices in the data acquisition phase [35]. Moreover, a spar- sifying dictionary can be designed by using the prior information [36]. In short, adaptive techniques are studied in every aspect of the CS based techniques.

### 1.1 Contribution and Organization

The remainder of the thesis is organized as follows. Chapter 2 starts with the in- troduction of the CS theory and the derivation of the signal model for narrowband DOA estimation. Later, the well-known classical DOA estimation techniques, namely, the Bartlett beamformer, the Capon’s beamformer, and the MUSIC are described and a new derivation of the Capon’s beamformer by using the min- imum mean square error (MMSE) criterion is presented. This derivation also introduces an MMSE based DOA estimation technique which requires the signal characteristics of the sources. Afterwards, the signal model for the CS based DOA estimation is derived and the optimization problems that need to be solved to find the DOAs are introduced. Chapter 2 is concluded with two sets of numerical simulations. In the first set, the MMSE based DOA estimation is compared with the Capon’s beamformer. In the second set, the superior performance of the CS based techniques over the classical techniques is demonstrated.

Adaptation strategies are most commonly provided by adaptively designing the measurement matrices, the dictionaries, or the reconstruction algorithms.

In Chapter 3, a novel adaptation strategy for the measurement matrix design that incorporates the prior information on the signal support over the dictionary is proposed. Hence, the proposed methodology establishes the relationship be- tween these adaptation strategies and combines them as a single technique. The proposed adaptation methodology can be used in real-time applications since a computationally feasible, closed-form expression is provided for the construction of the measurement matrix. A more software-efficient algorithm is also derived in the case that stricter computational complexity requirements are demanded.

The proposed methodologies allow for significant reduction in the number of dig- ital channels. Afterwards, the hardware implementation constraints of the pro- posed technique are investigated. Then, a hardware-efficient measurement matrix design technique is introduced. Furthermore, an adaptive dictionary design al- gorithm is proposed. The performance improvement provided by the proposed designs is demonstrated with extensive simulations by comparing them with var- ious alternatives.

In Chapter 4, a generic CRLB expression for the compressed DOA signal models, i.e., the signal model with the measurement matrix, is derived. By using the implications of the derived CRLB expression, another measurement matrix design methodology is proposed which can be used in both adaptive and non- adaptive designs. The proposed methodology focuses on particular sectors on which the sources are likely to be located. These sectors may either be adaptively changed depending on the prior information on the source scene, or pre-defined leading to non-adaptive measurement matrix designs. The measurement matrix proposed in this chapter does not have a closed-form solution. However, when the sectors are not adaptively changed, computational complexity is not an issue since a measurement matrix for each sector can be designed off-line and used in real-time. Besides, a CS based reconstruction algorithm is introduced which can take advantage of the proposed measurement matrix design methodology. In the conventional CS based DOA estimation, only the magnitudes of the reconstructed elements are used and the phase information is ignored whereas the proposed reconstruction technique exploits the phase information as well. The chapter is concluded with two sets of simulation based numerical results. First, the CRLB values achieved by various alternatives are compared and it is shown that the lowest CRLB values are achieved by the proposed measurement matrix design.

Second, the DOA estimation performance of the proposed measurement matrix design combined with the proposed reconstruction technique is demonstrated by comparing it with various alternatives including the techniques proposed in Chapter 3.

The final remarks and the prospective future studies are presented in Chapter 5. The main contributions of the thesis are listed below:

1. A new perspective on the Capon’s beamformer that exploits the MMSE criterion is presented. (Chapter 2)

2. The relation between the reweighted `_{1} minimization and the masurement
matrix design is demonstrated. An adaptive measurement matrix design
methodology is proposed by using the implications of the derived relation.

(Chapter 3)

3. An adaptive measurement matrix design methodology that is computa- tionally more efficient than the previous item is proposed while providing comparable DOA estimation performance. (Chapter 3)

4. An adaptive grid selection algorithm leading to an adaptive dictionary de- sign is proposed. (Chapter 3)

5. An adaptive measurement matrix design methodology by considering the hardware constraints is proposed. (Chapter 3)

6. The CRLB expression for the compressed DOA signal models is derived under fewer assumptions compared to those in the literature. (Chapter 4) 7. A measurement matrix design methodology by using the implications of the

previously derived CRLB expression is proposed. (Chapter 4)

8. A reconstruction algorithm that can be used together with the measure- ment matrix design methodology proposed in the previous item is presented.

(Chapter 4)

### 1.2 Notation

In the thesis, uppercase bold characters denote matrices and lowercase bold char-
acters denote vectors. i’th entry of a vector x is denoted by x_{i}, i’th column of a
matrix X is denoted by x_{i}, i’th entry in the j’th column of a matrix X is denoted
by Xij. The `q norm of x for q = {1, 2} is denoted by kxkq which is defined as
kxk^{q}_{q} ≡ P

i|x_{i}|^{q} where |x_{i}| is the magnitude of x_{i}. The `_{0} norm of x is denoted
by kxk_{0} which is the number of non-zero entries in x. kXk_{F} is the Frobenius
norm of X which is defined as kXk^{2}_{F} ≡P

i,j|X_{ij}|^{2}. X^{T}, X^{∗}, and X^{H} denote the
transpose, the conjugate, and the conjugate transpose of X respectively. Tr(X)
computes the trace of X and diag(X) is the vector consisting of the diagonal
entries of X. diag(x) = X where X is a diagonal matrix with Xii = xi. The
vector of ones is denoted by 1 and the vector or matrix of zeros is denoted by 0.

X 0 means that X is positive semi-definite. E[r] denotes the expected value of a random variable r.

### 1.3 Abbreviations and Symbols

TABLE 1.1 and TABLE 1.2 include the list of important symbols and abbrevia- tions used in the thesis.

Table 1.1: List of Symbols

a Steering vector

A Steering matrix

B Effective dictionary

D Dictionary

D_{r} Sub-dictionary

n Noise vector

N Noise matrix

Rxx Covariance matrix of array data

s Source signal vector

S Source signal matrix

W Whitening matrix

x Received signal vector

X Received signal matrix

y Received signal vector after compression

Y Received signal matrix after compression

Φ Measurement matrix

K Number of sources

L Number of grid points

m Number of digital channels

M Number of sensors

N_{s} Number of snapshots

θ/ω Direction of arrival

Table 1.2: List of Abbreviations

ADMM Alternating Direction Method of Multipliers

A-MMD Adaptive Measurement Matrix Design

A-MMD-BD Adaptive Measurement Matrix Design hav-

ing the Block Diagonal structure

A-MMD-UD Adaptive Measurement Matrix Design with

Updated Dictionary

BP Basis Pursuit

BPDN Basis Pursuit Denoising

CRLB Cramer-Rao Lower Bound

CS Compresed Sensing

DOA Direction of Arrival

EA-MMD Effective Adaptive Measurement Matrix De-

sign

ETF Equiangular Tight Frame

ETFD Equiangular Tight Frame Design

EVD Eigenvalue Decomposition

FDOA Focused Direction of Arrival

IMD Idendity Matrix Design

IRG Ibrahim, Roemer, Galdo Design

LASSO Least Absolute Shrinkage and Selection Op-

erator

LZKR Li, Zhang, Kirubarajan, Rajan Design

ML Maximum Likelihood

MLE Maximum Likelihood Estimator

MMSE Minimum Mean Square Error

MSE Mean Square Error

MUSIC MUltiple SIgnal Classification

OMP Orthogonal Matching Pursuit

RGD Random Gaussian Design

RIC Restricted Isometry Constant

RIP Restricted Isometry Property

RMSE Root Mean Square Error

SNR Signal-to-Noise Ratio

SVD Singular Value Decomposition

ULA Uniform and Linear Array

## Chapter 2

## Direction of Arrival Estimation

The main interest of the presented thesis is the CS based DOA estimation. Hence, the CS theory is explained first and then the well-known classical DOA estimation techniques are presented along with a novel derivation of the Capon’s beamformer that is based on the MMSE criterion. After that, the use of the CS theory in DOA estimation is introduced. The chapter is concluded with simulation based comparative analyses.

### 2.1 Compressed Sensing

In many engineering problems, solving linear systems with fewer equations than the number of unknowns is of great interest. These systems are called as the

“underdetermined systems” and can be expressed as:

y = Bx, (2.1)

where y ∈ C^{m×1}, x ∈ C^{M ×1} with m < M which implies that there are infinitely
many x’s satisfying (2.1). To find a unique solution, regularization techniques
are used. The conventional approach is to use the `_{2} norm regularization which

leads to the following optimization problem:

ˆ

x = arg min

x

kxk^{2}_{2}

subject to y = Bx. (2.2)

The popularization of the `_{2} norm regularization is because (2.2) has a unique,
closed form solution which can be achieved by writing the Lagrangian of (2.2):

L(x, λ) = kxk^{2}_{2}+ λ^{H}(y − Bx). (2.3)
Hence, the following dual function can be written as:

g(λ) = inf

x

L(x, λ) = L 1

2B^{H}λ, λ

, (2.4)

where ˆx = 1

2B^{H}λ is achieved by setting ∂L(x, λ)

∂x = 0. By using the primal feasibility,

y = B ˆx = 1

2BB^{H}λ ⇒ λ = 2(BB^{H})^{−1}y

⇒ ˆx = B^{H}(BB^{H})^{−1}y = B^{†}y. (2.5)
Note that ˆx is achieved by solving the dual problem. However, since the Slater’s
condition for (2.2) implies the primal feasibility, the dual and the primal solutions
are the same [37], i.e., ˆx = B^{†}y is the optimal solution for (2.2). The physical
meaning of the `_{2} norm regularization is the minimization of energy. Hence, in
order for this approach to be applicable, the unknown signals must tend to have
minimum energies.

The CS theory enables reconstruction of signals by exploiting their sparse rep- resentation in a known “dictionary” [1–6]. In other words, (2.1) can be rewritten as:

y = ΦDγ, (2.6)

where D is the dictionary that establishes the relationship between x and the
sparse vector γ. The reason for the “dictionary” terminology is that any x is
assumed to be (approximately)^{1} expressed as the linear combinations of a few

1The reason for using the term “approximately” is explained in Section 2.4

columns of D. Similarly as the importance of using a high-quality dictionary in linguistics, choice of the dictionary in CS is also crucially important. The quality of D is determined by its ability to express x with a sparse γ while maintaining low coherence between its columns.

In (2.6), Φ compresses the mesurements x = Dγ to achieve y. This matrix is often called as the measurement matrix since it “measures” the environment to achieve the compressed measurements y. Since it directly determines the measurements to be used in the reconstruction, the proper choice of Φ is as important as the proper choice of the sparsifying signal dictionary D. Hence, the first aspect of a succesful CS system is designing D and Φ appropriately. Along with the designs of D and Φ, the other important aspect is the reconstruction algorithm which will be explained following the discussion of D and Φ design.

The RIP provides sufficient conditions for the stable reconstruction of the CS
techniques [1]. The “effective dictionary” ΦD is said to satisfy the RIP with the
restricted isometry constant (RIC) δ_{K} ∈ [0, 1) if it satisfies the following relation
for any non-zero v_{K} with cardinality K:

1 − δ_{K} ≤ kΦDv_{K}k_{2}

kv_{K}k_{2} ≤ 1 + δ_{K}. (2.7)

If Φ can be designed such that it satisfies the RIP for an appropriate D, stable re- constructions are guaranteed provided that the proper reconstruction algorithms are used. However, leaving aside the possibility of a RIP based Φ design, even the verification of the RIP for an arbitrary matrix is NP-hard [38]. On the other hand, random matrices are proven to satisfy the RIP with a high probability [11], and hence they are the most commonly used measurement matrices in the CS literature. However, it is possible to exceed their performance by using mea- surement matrices that are designed based on the alternative design criteria as in [9, 12]. These design criteria are generally based on the mutual coherence, the CRLB, and the mutual information. Furthermore, Φ can be adapted to the prior information on the environment that can provide significant performance improvements. This thesis investigates the adaptive mutual coherence and CRLB based measurement matrix designs in detail.

Figure 2.1: Comparison between the `_{1}and the `_{2} norm regularizations by using a
simple example. Left figure shows the solution when the `_{1}norm of x is minimized
subject to an affine constraint. Right figure shows the solution when the `_{2} norm
of x is minimized subject to the same affine constraint.

As stated previously, apart from the designs of D and Φ, reconstruction is another important aspect of the CS theory. Since the essential assumption is the sparsity of signals, the following approach may be attempted to be solved:

ˆ

γ = arg min

γ

kγk0

subject to y = ΦDγ. (2.8)

Unfortunately, solving (2.8) even for moderate sizes is not practical since it re-
quires an exhaustive combinatorial search over γ’s [39] and it is also numerically
unstable [5]. There are computationally efficient algorithms to approximate the
solution in (2.8) such as the OMP [15], the LASSO [16], and the BPDN ^{2} [17].

While the OMP is a greedy algorithm, the BPDN and the LASSO replace the `_{0}
norm with the `1 norm to obtain a convex relaxation of (2.8) that can be recast
as a linear programming problem [4]. Hence, by replacing the `_{0} norm with the

2For the noiseless signal model, it is actually more accurate to use the term Basis Pursuit (BP) instead of Basis Pursuit Denoising (BPDN).

sparsity promoting `_{1} norm, (2.8) is relaxed to:

ˆ

γ = arg min

γ

kγk_{1}

subject to y = ΦDγ. (2.9)

To visualize the sparsity promoting nature of the `_{1} norm, a simple example
comparing the results achieved by using the `_{1} and the `_{2} norm regularizations is
illustrated in Fig. 2.1. When the `1 norm regularization is used, x1 component of
the solution is 0 while the use of `_{2} norm regularization results in a full solution.

### 2.2 Signal Model for Direction of Arrival Esti- mation

In the DOA estimation by using sensor arrays, as the name implies, the main purpose is to find the directions of impinging signals from sources. DOA estima- tion has many applications including but not limited to radar, sonar, electronic warfare and wireless communications. Certain assumptions are usually made in the DOA estimation literature unless otherwise stated. These assumptions can be listed as follows:

1. Linear, isotropic, and homogenous medium: Medium properties are the same in every direction. Besides, superposition principle can be applied.

2. Narrowband assumption [40]: Source signals are modeled as s(t) =
α(t) cos(2πf_{c}t+β(t)) where f_{c}is a known carrier frequency. Narrowband as-
sumption implies that α(t−τ ) ≈ α(t) and β(t−τ ) = β(τ ) for a propagation
delay τ which is the propagation time across the sensor array.

3. Additive, spatially white Gaussian noise [18]: The dominant noise is as- sumed to be the additive Gaussian receiver (sensor) noise. The noise in- troduced by each sensor is uncorrelated with each other and has the same standard deviation.

Figure 2.2: Illustration of a simple DOA estimation scene

4. Far-field assumption [40]: The sources are located at the far field of the sensor array. Hence, the signal impinging on the array can be closely ap- proximated as the sum of plane waves.

5. Unit-gain assumption: Without loss of generality, the transmitted signal is neither attenuated nor amplified when it propagates through the medium.

As illustrated in Fig. 2.2, the phase front of the narrowband source signal im- pinges on each sensor at phases that show variation across the sensor array. In- deed, this phase difference enables the possibility of DOA estimation. Keeping this in mind, now the signal model for DOA estimation is derived under the listed assumptions. Ignoring the noise for the moment, if the first sensor is determined as the reference sensor, the signal transmitted by the i’th source is received by the first sensor as:

x1i(t) = si(t − τ1i), τ1i = r

c, (2.10)

where r is the distance between the i’th source and the first sensor, c is the speed of the transmitted wave. More generally, for the p’th sensor:

x_{pi}(t) = s_{i}(t − τ_{pi}), τ_{pi}= r + d_{p−1}cos θ_{i}

c = τ_{1i}+d_{p−1}cos θ_{i}

c , (2.11)

where θ_{i} is the DOA of the i’th source and d_{0} ≡ 0. Without loss of generality,
τ_{1i} = 0 can be assumed. Due to the narrowband assumption:

x_{pi}(t) = s_{i}(t − τ_{pi}) ≈ α_{i}(t) cos(2πf_{c}t + β_{i}(t) − 2πf_{c}d_{p−1}cos θ_{i}/c)

= Re{α_{i}(t)e^{jβ}^{i}^{(t)}e^{−j2πf}^{c}^{d}^{p−1}^{cos θ}^{i}^{/c}e^{j2πf}^{c}^{t}}. (2.12)
By defining the complex envelope of s(t) as ˜s_{i}(t) ≡ α_{i}(t)e^{jβ}^{i}^{(t)}, (2.12) can be
rewritten in the phasor domain:

˜

x_{pi}(t) ≈ ˜s_{i}(t)e^{−j2πf}^{c}^{d}^{p−1}^{cos θ}^{i}^{/c}. (2.13)
Assuming there are K sources, the signal received by the p’th sensor can be
written as:

˜
x_{p}(t) =

K

X

i=1

˜

x_{pi}(t) + n_{p}(t) =

K

X

i=1

˜

s_{i}(t)e^{−j2πf}^{c}^{d}^{p−1}^{cos θ}^{i}^{/c}+ n_{p}(t), (2.14)

where n_{p}(t) is the noise introduced by the p’th sensor. For simplicity, let ω_{p}(θ_{i})
be defined as ωp(θi) ≡ 2πfcdp−1cos θi/c and then the received signal for an array
with M sensors can be expressed as:

˜
x_{1}(t)

˜
x_{2}(t)

...

˜
x_{M}(t)

= 1

√M

1 . . . 1

e^{−jω}^{2}^{(θ}^{1}^{)} e^{−jω}^{2}^{(θ}^{2}^{)} . . . e^{−jω}^{2}^{(θ}^{K}^{)}
... . . ...
e^{−jω}^{M}^{(θ}^{1}^{)} e^{−jω}^{M}^{(θ}^{2}^{)} . . . e^{−jω}^{M}^{(θ}^{K}^{)}

˜
s_{1}(t)

˜
s_{2}(t)

...

˜
s_{K}(t)

+

n_{1}(t)
n_{2}(t)

...
n_{M}(t)

,

(2.15) which can be rewritten more compactly as:

x(t) = As(t) + n(t). (2.16)

In (2.16), A ∈ C^{M ×K} is often called as the array steering matrix with the columns
named as the steering vectors a(θ) = [1 e^{−jω}^{1}^{(θ)} ... e^{−jω}^{M}^{(θ)}]^{T}/√

M where the relationship between ω(θ) and θ is determined by the characteristics of the sensor

array. For example, for an isotropic, uniform linear array (ULA) with inter-sensor
spacing d, a(θ) = [1 e^{−jω(θ)} ... e−j(M −1)ω(θ)]^{T}/√

M and ω(θ) = 2πd cos(θ)/λ where λ is the wavelength of the source signal. Note that (2.16) shows the signal model for a continuous system. In real DOA estimation systems, the received signal x(t) is sampled with a uniform sampling interval T . To emphasize the difference between the exact time and the sample index, the term “snapshot” is used. If k denotes the snapshot index, t = kT . Hence, (2.16) can be rewritten as:

x(k) = As(k) + n(k). (2.17)

Finally, the real and the imaginary entries of n are i.i.d. Gaussian with zero mean and σ/√

2M standard deviation, which implies E[n] = 0, E[nn^{H}] = σ^{2}I_{M}, and
E[nn^{T}] = 0, collectively denoted as n ∼ NC(0, σ^{2}I_{M}).

In many applications, a consecutive set of x(k) is used to estimate the second order statistics and for the signal-to-noise ratio (SNR) gain. For example, the well-known DOA estimation methods, namely the Capon’s beamformer and the MUSIC require more than one snapshot to achieve a sample covariance matrix estimate. Therefore, measurements are taken at different k’s and a matrix of measurements is formed by a set of consecutive x(k) vectors as the columns of the matrix X. Then the matrix form of the measurements can be written as:

X = AS + N , (2.18)

where X ∈ C^{M ×N}^{s}, S ∈ C^{K×N}^{s}, and N_{s} is used to denote the number of snap-
shots. Note that for the multi-snapshot case, A is assumed to be the same during
N_{s} snapshots which is not realistic in the dynamic source environments.

It is not obvious to identify the DOA estimation signal model in (2.17) as an application area of the CS theory. First, the relationship between the unknown DOAs (θ’s or ω’s) and the received signal x is not linear. Besides, s in (2.17) is also unknown. Furthermore, the steering vectors in (2.17) are not sparse.

To clarify how the CS based techniques can be used in DOA estimation, it is beneficial to introduce the “grid” concept which is also used in the classical DOA estimation techniques. For gridding, two common approaches are followed. In the first approach, the domain of ω, which is [−π, π), is uniformly discretized in

L points as [ω(θ_{1}), ω(θ_{2}), ... , ω(θ_{L})]. In other words, |ω(θ_{i+1}) − ω(θ_{i})| = 2π/L
for i ∈ {1, 2, ... , L − 1}. The second approach is the uniform discretization of
the θ domain, which is [0, 180^{o}). Either way, the dictionary matrix D can be
formed as D = [a(θ_{1}), a(θ_{2}), ... , a(θ_{L})] where a(θ_{i}) denotes the steering vector
corresponding to the grid point θi. Hence, x in (2.17) can be approximated as the
linear combinations of the columns of D, where the coefficients are determined by
a vector, say ¯s, with the output D¯s. In DOA estimation problems, typically there
are a few targets and many grid points (K << L); therefore, the reconstructed

¯

s is expected to be sparse. Hence, as it is detailed in Section 2.4, this approach converts the DOA signal model such that the techniques of the CS theory become applicable.

### 2.2.1 Wideband DOA Estimation

In some applications, the source signals do not conform to the narrowband as- sumption [41, 42]. In other words, the source signals are not concentrated around a single carrier frequency and they extend over a wide frequency range. Var- ious techniques have been proposed to estimate the DOAs of wideband source signals [41–45].

In wideband DOA estimation, commonly the wideband source signals are de- composed into many frequency bins such that the narrowband assumption can separately be applied to the source signals corresponding to these frequency bins.

The incoherent signal subspace method (ISSM) applies narrowband DOA esti- mation techniques to those signals and averages the results achieved from all frequency bins to get the final estimation [43]. This technique performs poorly when the SNRs at some frequency bins are low, which causes significant degra- dation in the final estimation. In [44], the coherent signal subspace method (CSSM) is introduced in which covariance matrices from all the frequency bins are transformed at a single focusing frequency by using focusing matrices. To de- sign focusing matrices, initial DOA estimates must be known, which is a critical disadvantage of the CSSM. This issue is addressed by the test of orthogonality of

projected subspaces (TOPS) technique combining the ideas of both coherent and incoherent methods [45]. However, all of these techniques use covariance matrix estimates and their performance degrades when there exist coherent sources and the number of snapshots is relatively few.

Most of the wideband DOA estimation algorithms use narrowband DOA es- timation techniques since they decompose the wideband signal into narrowband parts and integrate those parts coherently or incoherently for the final estimation.

Hence, the techniques proposed in this thesis can be applied to wideband DOA estimation in principle when they are combined with such integration methods.

Furthermore, the use of CS eliminates the problem of handling coherent sources or low number of snapshots, which brings the potential of having an important advantage over various state-of-the-art wideband DOA estimation techniques [46].

### 2.2.2 Coherent Source Scenarios

In many practical scenarios, some of the source signals are coherent (fully corre-
lated) with each other possibly due to multipath effects. Assuming that Source
1 and Source 2 are coherent, then s_{2}(k) = cs_{1}(k) where c is a complex constant.

c establishes the gain-phase relationship between these two source signals. Note
that when Source 1 and Source 2 are coherent with the given relationship, the
signal model can be expressed as s = [s_{1} s_{3} ... s_{K}]^{T} with the steering matrix
A = [a(θ_{1}) + ca(θ_{2}) a(θ_{3}) ... a(θ_{K})]. The modified problem model can be inter-
preted as a signal model for K − 1 uncorrelated sources where one of the sources
have the DOA corresponding to the steering vector a(θ_{1}) + ca(θ_{2}), i.e., neither
θ_{1} nor θ_{2} [24]. Hence, the uncorrelatedness assumption may ruin the DOA esti-
mation of both sources.

As it is detailed in Section 2.3, the MUSIC and the Capon’s beamformer require the uncorrelatedness assumption and hence perform poorly in multipath environments. Coherent source scenarios are also numerically investigated in Section 2.5.2, and it is shown that the CS based techniques are typically not affected by the multipath effects.

### 2.2.3 The Method of Performance Evaluation

In DOA estimation, the most commonly used performance evaluation metric is the root mean square error (RMSE) which is defined as:

RMSE ≡ v u u t

1 N K

N

X

n=1 K

X

k=1

(ˆθnk − θnk)^{2}, (2.19)

where θ_{nk} and ˆθ_{nk} denote the actual and the estimated DOAs (in degrees) for the
k’th target and the n’th Monte Carlo iteration among the total of N . Sometimes,
θ_{nk} and ˆθ_{nk} are replaced by ω_{nk} and ˆω_{nk} which is not an important difference since
finding ω is also often referred to as the DOA estimation.

Using the RMSE metric for error evaluation has many advantages. First, the minimum achievable RMSE has a theoretical limit derived from the CRLB [47].

The CRLB and its implications for the CS based DOA estimation will be detailed in Chapter 4. Second, the RMSE reflects the performance of the superresolving algorithms well, i.e., it differentiates between the algorithms with high and low angular resolutions.

### 2.3 The Classical Direction of Arrival Estima- tion Techniques

The most well-known classical DOA estimation techniques can be classified broadly into two main categories: the beamforming techniques and the subspace based techniques. The maximum likelihood (ML) based approaches can also be used for DOA estimation. As it will be detailed in Chapter 4, the ML based approaches are highly useful to derive analytical performance bounds, namely, the CRLB. However, when the ML based techniques are applied on DOA esti- mation, they typically require multidimensional search increasing their computa- tional cost [18]. Computationally more efficient approaches can be achieved by using sub-optimal algorithms that deteriorates the algorithm performance [48],

or the assumption of ULA geometries due to their special steering vector struc- tures [49].

In the beamforming techniques [19], the following beamformer output is com- puted with the beamformer coefficients w:

x_{bf} = w^{H}(θ)x, (2.20)

whose expected power P (θ) = E[|xbf|^{2}] is maximized over possible DOAs:

θ = arg maxˆ

θ

P (θ) = arg max

θ

w^{H}(θ)E[xx^{H}]w(θ). (2.21)
The beamforming techniques differ from each other in their choice for w(θ). As it
is shown in this chapter, the Bartlett beamformer and the Capon’s beamformer
have highly different approaches. In the subspace based techniques, the covari-
ance matrix of the received signal is separated into its signal and noise subspaces,
where the MUSIC is the most popular technique.

In this section, the aforomentioned classical techniques, namely the Bartlett beamformer, the Capon’s beamformer, and the MUSIC are briefly introduced.

Apart from them, the MMSE criterion in DOA estimation is presented which is shown to be closely related with the Capon’s beamformer. However, it is not feasible to directly use the derived MMSE based technique since it needs S in (2.18) which is often unknown.

### 2.3.1 The Bartlett Beamformer

In the Bartlett beamformer, the beamformer coefficients w is chosen as w = a(θ).

In other words, when x is multiplied by the pre-determined steering vector, the
beamformer output x_{bf} = a^{H}(θ)x is achieved for the grid point θ. This metric
denotes the correlation between a(θ) and x. Therefore, the purpose is to find the

“most correlated” steering vector parametrized by θ with the received signal x.

Hence, the beamformer output has the following expected power:

P (θ) = E[|xbf|^{2}] = a^{H}(θ)E[xx^{H}]a(θ). (2.22)

Note that P (θ) depends on the second order statistics of x, i.e., R_{xx} ≡ E[xx^{H}].

The Bartlett beamformer aims to maximize this power expression:

θ = arg maxˆ

θ

a^{H}(θ)R_{xx}a(θ). (2.23)

R_{xx} can explicitly be written as:

R_{xx} = AE[ss^{H}]A^{H} + E[nn^{H}]. (2.24)
Although E[nn^{H}] can be measured in most applications, R_{ss} ≡ E[ss^{H}] is typ-
ically unknown. Hence, the exact R_{xx} expression is not available in general;

therefore, the sample covariance matrix ˆR_{xx} is used as the surrogate of R_{xx}:
Rˆ_{xx} ≡ 1

N_{s}

Ns

X

k=1

x(k)x^{H}(k). (2.25)

The fundamental limitation of the Bartlett beamformer is that its angular reso-
lution is limited by the array beamwidth [18]. Note that although there is R_{xx}
in (2.23), the Bartlett beamformer is not a covariance matrix based technique
unlike the MUSIC and the Capon’s beamformer since the Bartlett beamformer
does not use any special structure of R_{xx} except for the SNR gain.

### 2.3.2 The Capon’s Beamformer

The Capon’s beamformer chooses the beamforming coefficients as the solution to the following optimization problem:

ˆ

w = arg min

w

w^{H}Rxxw

subject to w^{H}a(θ) = 1. (2.26)

The purpose of (2.26) is to minimize the expected power at a certain direc-
tion such that the interference and the noise effects are minimized while keep-
ing the signal gain constant. Since the constraint of w^{H}a(θ) = 1 is affine and

∂^{2}w^{H}R_{xx}w

∂w^{2} = 2Rxx 0, (2.26) is a convex problem. Furthermore, Slater’s con-
dition for (2.26) is the primal feasibility, i.e., ˆw^{H}a(θ) = 1. Hence, strong duality

holds if a feasible solution is achievable which implies that the dual problem can be solved to find the optimal ˆw. Then, the Lagrangian of (2.26) is written as follows:

L(w, λ) = w^{H}R_{xx}w + λ(1 − w^{H}a(θ)). (2.27)
Stationarity condition of Karush-Kuhn-Tucker (KKT) conditions leads to:

∂L(w, λ)

∂w = 0 ⇒ ˆw = λ

2R^{−1}_{xx}a(θ). (2.28)
Furthermore, the primal feasibility implies:

ˆ

w^{H}a(θ) = 1 ⇒ λ = 2

a^{H}(θ)R^{−1}_{xx}a(θ) ⇒ ˆw = R^{−1}_{xx}a(θ)

a^{H}(θ)R^{−1}_{xx}a(θ). (2.29)
Hence, the expected power of the filter output is given by:

P (θ) = ˆw^{H}R_{xx}w =ˆ 1

a^{H}(θ)R^{−1}_{xx}a(θ), (2.30)
leading to the following DOA estimates:

θ = arg maxˆ

θ

1

a^{H}(θ)R^{−1}_{xx}a(θ). (2.31)

The beamformer coefficients achieved in (2.29) has the term R^{−1}_{xx} which is
undesirable for the estimation robustness due to the condition number of R_{xx}.
To prevent that issue, diagonal loading and eigenvalue thresholding techniques are
used in the literature [22]. When the diagonal loading is used, (2.29) is replaced
by:

ˆ

w = (R_{xx}+ µI_{M})^{−1}a(θ)

a^{H}(θ)(R_{xx}+ µI_{M})^{−1}a(θ), (2.32)
where µ is the regularization parameter. In the eigenvalue thresholding technique,
eigenvalue decomposition of R_{xx} is written and the eigenvalues that are smaller
than a threshold are fixed to a constant. As in the Bartlett beamformer, R_{xx} is
generally replaced by ˆR_{xx} since R_{xx} is unknown.

From a geometrical point of view, the cost function of (2.26), i.e., w^{H}R_{xx}w =
C where C denotes the varying cost, is a family of ellipsoids that are centered
at the origin. The constraint w^{H}a(θ) is an affine hyperplane in M dimensional

space. The Capon’s beamformer aims to find the vector ˆw such that the ellipsoid
from the given family of ellipsoids is tangent to the affine hyperplane. A simple
illustration is shown in Fig. 2.3 for a two-dimensional case. When (R_{xx}+ µI_{M})
for µ > 0 is used instead of R_{xx}, the shape of the ellipsoid and hence the tangent
point is changed. As µ gets larger, the ellipsoid gets a more spherical shape.

-1.5 -1 -0.5 0 0.5 1 1.5 2

w1 -1

-0.5 0 0.5 1 1.5

w 2

Cost Function Affine Constraint

Figure 2.3: Geometrical interpretation of the Capon’s beamformer for the two- dimensional case

### 2.3.3 The Multiple Signal Classification (MUSIC)

The MUSIC is a subspace-based technique that exploits the eigenvalue decom-
position of R_{xx} and separate those eigenvalues that reside in noise and signal
subspaces. After explicitly writing R_{xx} as in (2.24), it is observed that K eigen-
values of R_{xx} belong to the signal subspace while the remaining M − K belong
to the noise subspace. The main assumption of the MUSIC is that the noise
subspace is perpendiular to the signal subspace. Naming the eigenvalues of Rxx

as λ_{i}, 1 ≤ i ≤ M , and assuming that they are sorted in a descending manner
such that λ_{1} ≥ λ_{2} ≥ ... ≥ λ_{M} with the corresponding eigenvectors v_{i}’s:

AR_{ss}A^{H}v_{i} = 0, K < i ≤ M, (2.33)

which implies {a(θ_{1}), a(θ_{2}), ..., a(θ_{K})} ⊥ {v_{K+1}, v_{K+2}, ..., v_{M}}. Defining the mat-
rix V_{N} ≡ [v_{K+1} v_{K+2} ... v_{M}] which includes all eigenvectors that belong to the
noise subspace, the following power spectrum is formed:

P (θ) = 1

a^{H}(θ)V_{N}V^{H}_{N}a(θ). (2.34)
When a source exists at θ, a^{H}(θ)V_{N}V^{H}_{N}a(θ) is small due to the perpendicularity
relationship given in (2.33). Hence, P (θ) is expected to take large values when a
source is present at θ, leading to the following problem giving the desired DOAs:

θ = arg maxˆ

θ

1

a^{H}(θ)V_{N}V^{H}_{N}a(θ). (2.35)
Although they follow highly different approaches, the power spectrums of the
Capon’s beamformer (2.31) and the MUSIC (2.35) have a mathematical rela-
tion demonstrated in [50]. As in the aforomentioned techniques, the eigenvalue
decomposition of ˆR_{xx}, not R_{xx}, is written to find V_{N}.

The MUSIC and the Capon’s beamformer, unlike the Bartlett beamformer, can
provide higher resolutions than the array beamwidth. However, these techniques
are covariance matrix based techniques. Apart from the associated problems
that are explained before, reliable estimation of R_{xx} also requires uncorrelated
sources to prevent the rank deficiency. This is a serious problem in a multipath
environment.

The MUSIC is often considered to be the most effective classical DOA estima- tion technique [51]. However, as it is demonstrated in this chapter, the CS based techniques can outperform the MUSIC and the other classical techniques.

### 2.3.4 Minimum Mean Square Error Criterion in DOA Es- timation

Being motivated by the MMSE beamformer [19, 52], the following problem can be posed:

ˆ

w = arg min

w E[|w^{H}x − ˆs|^{2}], (2.36)