Residual Generation and Fault Diagnosis of
Rechargeable Lead-Acid Batteries
by
Sena Ergüllü
Submitted to the Graduate School of Sabancı University in partial fulfillment of the requirements for the degree of
Master of Science
Sabancı University January, 2011
Residual Generation and Fault Diagnosis of Rechargeable
Lead-Acid Batteries
APPROVED BY:
Assist. Prof. Dr. Ahmet Onat
(Thesis Advisor) ...
Assoc. Prof. Dr. Berrin Yanıkoğlu ...
Assoc. Prof. Dr. Serhat Yeşilyurt ...
Assist. Prof. Dr. Cemal Yılmaz ...
Assist. Prof. Dr. Hüsnü Yenigün ...
c
° Sena Ergüllü 2011
This thesis was supported in part by The Ministry of Commerce of Turkey and Temsa Global, under the grand provided by SanTez project "‘Ticari Araçlarda Uzaktan Arıza Tanıma ve Müdahale Sistemi"’ ("‘Remote Fault Diagnosis for Commercial Vehicles"’), code 00241.STZ.2008-1.
Residual Generation and Fault Diagnosis of Rechargeable
Lead-Acid Batteries
Sena ERGÜLLÜ ME, Master’s Thesis, 2011
Thesis Supervisor: Assist. Prof. Dr. Ahmet Onat
Keywords: Fault Diagnosis, Residual Generation, Modeling of Nonlinear Dynamic Systems, Artificial Intelligence Methods
Abstract
In many process and manufacturing industries, early detection of faults has great practical importance. Since it saves time and cost involved in the repairing of the equipment.
Qualitative methods such as neural networks and fuzzy logic are popular tools in model based fault detection and classification of nonlinear dynamic systems. Since it is difficult to accurately model these kind of systems. In the first part of this work, neural network and adaptive neuro-fuzzy logic methods are used in the modeling of a water-tank system to produce residu-als for fault classification. This study shows that neural networks have better performance but longer training time compared to the adaptive neuro-fuzzy logic. The second part of this research investigates the classification tree and Fisher Discriminant Analysis (FDA) approaches in fault classification of nonlinear dynamic systems. Comparing the performance of these approaches indicates that FDA method results in longer computational time but lower tree size for high dimensional training data. The contributions of this thesis are modeling and fault diagnosis of lead-acid battery system using qualita-tive techniques in combination with statistical methods such as classification tree.
Şarj Edilebilen Kurşun-Asit Bataryalarda Rezidu Oluşturma ve
Hata Diyagnozu
Sena Ergüllü ME, Master Tezi, 2011
Tez Danışmanı: Doç. Dr. Ahmet Onat
Anahtar Kelimeler: Hata Diyagnozu, Rezidu Üretme, Lineer Olmayan Dinamik Sistemin Modellenmesi, Yapay Zeka Metodları
Özet
Birçok üretim endüstrisinde hatanın erken tespiti önemli rol oynamak-tadır. Bu zamandan ve maliyetten kazanç sağlayacaktır.
Yapay Sinir Ağları ve Bulanık Mantık gibi kalitatif metodlar, lineer ol-mayan dinamik sistemlerin modele dayalı hata tespit ve sınıflandırılmasında sıkça kullanılan yöntemlerdir. Bunun sebebi, bu sistemlerin doğru mod-ellenmesi çok zordur. Bu çalışmanın ilk kısmında yapay sinir ağları ve adaptif sinir ağı-bulanık mantık metodları ile su tankı sistem modellemesi yapılmıştır. Böylece hata sınıflandırmada kullanılacak artıkların üretilmesi hedeflenmiştir. Bu çalışmadan görülmüştür ki yapay sinir ağları , adaptif sinir ağı-bulanık mantıktan daha iyi sonuç vermektedir, ama eğitim süresi uza-maktadır. Araştırmanın ikinci kısmında lineer olmayan dinamik sistemlerde hatanın sınıflandırması için sınıflandırma ağacı ve Fisher Diskriminant Anal-izi (FDA) yöntemleri kullanılmıştır. Bunların performansları karşılaştırıldığında FDA yöntemiyle büyük boyutlu eğitim verileri için daha uzun sürede ama daha az yapraklı ağaç oluşturulduğu görülmüştür. Bu tezin katkıları Şarj Edilebilir Kurşun-Asit Bataryaların modellenmesi ve hata diyagnozu alanında olmuştur. Bunun için gözlemsel metotlarla istatistik metotlar (sınıflandırma ağacı gibi) birleştirilmiştir.
Acknowledgements
It is a great pleasure to extend my gratitude to my thesis advisor Assist. Prof. Dr. Ahmet Onat for his precious guidance and support. I am greatly indebted to him for his supervision and excellent advises throughout my Masters study. I gratefully thank Assoc. Prof. Dr. Berrin Yanıkoğlu, Assoc. Prof. Dr. Serhat Yeşilyurt, Assist. Prof. Dr. Cemal Yılmaz and Assist. Prof. Dr. Hüsnü Yenigün for their feedback and their valuable time serving as my jury members.
I would like to acknowledge the financial support provided by The Min-istry of Commerce of Turkey and TEMSA; through the project of SANTEZ “Remote Fault Diagnosis of Commercial Vehicles” under the grant 00241.STZ.2008-1.
I would sincerely like to thank to “Remote Fault Diagnosis of Commer-cial Vehicles” project member Yusuf Sipahi, for his pleasant team-work and providing me the necessary motivation during hard times.
Many thanks to Ahmetcan Erdoğan, Can Palaz, Kadir Haspalamutgil, Serhat Dikyar, Alper Mehmet Ergin, Ozan Tokatlı, Duruhan Özçelik, Aykut Cihan Satıcı and to all Mechatronics laboratory members, whom I wish I had the space to acknowledge in person for their great friendship throughout my Masters study.
Finally, I would like to thank my family for all their love and support throughout my life.
Contents
1 Introduction 1
1.1 Fault Detection and Diagnosis . . . 1
1.1.1 Basic Terminology . . . 2
1.1.2 Statement of the Problem . . . 3
1.2 Fault Diagnosis Based on Analytical Models . . . 4
2 System Modeling 6 2.1 General Structure of Faulty Systems . . . 7
2.2 General Structure of Residual Generation . . . 8
2.3 Fault Detectability and Fault Isolability . . . 9
2.4 Quantitative Diagnosis Methods . . . 10
2.4.1 Residual Generation via Parameter Estimation . . . 10
2.4.2 Observer Based Approaches . . . 12
2.4.3 Parity Vector (relation) Methods . . . 14
2.5 Qualitative Diagnosis Methods . . . 16
2.5.1 Fuzzy Model Based Residual Generation . . . 16
2.5.2 Neural Network Model Based Residual Generation . . . 18
2.5.3 Neuro-Fuzzy Model Based Residual Generation . . . . 21
2.6 Conclusion . . . 23
3 Fault Diagnosis Based on Qualitative Methods 24 3.1 Introduction . . . 24
3.2 ANFIS Structure . . . 25
3.2.1 ANFIS LSE Algorithm . . . 30
3.2.2 ANFIS Backpropagation Algorithm . . . 31
3.3 Neural Network Structure . . . 33
3.4.1 Purpose and Method of the Study . . . 36
3.4.2 Process Description . . . 36
3.4.3 Neural Network Process Model . . . 39
3.4.4 Residual Generation Techniques . . . 41
3.4.5 Relationship Between Residuals and Faults . . . 43
3.5 Results . . . 47
3.6 Conclusions . . . 48
4 Fault Diagnosis Based on Classification Tree and Fisher Dis-criminant Analysis 50 4.1 Introduction . . . 50
4.2 Classification Tree Principles . . . 51
4.2.1 Tree Building and Tree Cost . . . 52
4.2.2 Optimal Tree Size Decision using Cross-Validation and Threshold Value . . . 54
4.2.3 Tree pruning . . . 55
4.2.4 Leaf Node Label Decision . . . 56
4.3 Implementation of Classification Tree . . . 56
4.3.1 Decision Boundary Generation . . . 56
4.3.2 Computational Efficiency . . . 57
4.4 Generation of Classification Tree with Fisher Discriminant Analysis . . . 59
4.5 Case Study 1: Rechargeable Lead Acid Battery . . . 61
4.5.1 Lead Acid Battery Principles . . . 61
4.5.2 Lead Acid Battery Characteristics . . . 63
4.5.3 Process Description and Data acquisition . . . 65
4.5.4 Neural Network Process Model . . . 68
4.5.6 Residual Evaluation with Classification Tree . . . 72
4.5.7 Residual Evaluation with Classification Tree and Fisher Discriminant Analysis . . . 75
4.5.8 Results and Discussion . . . 76
4.6 Case Study 2: Nonlinear Mass-Spring-Damper System with Coulomb Friction . . . 79
4.6.1 Neural Network Process Model . . . 81
4.6.2 Residual Generation and Feature Extraction . . . 82
4.6.3 Results and Discussion . . . 83
4.7 Conclusion . . . 85
5 Conclusions 87 A Matlab Codes 89 A.1 Data Acquisition and Normalization of Data for NN . . . 89
A.2 Training of the Neural Network . . . 90
A.3 Generation of Classification Tree . . . 91
A.4 Generation of Classification Tree with FDA . . . 93
List of Figures
1.1 Two main stages in Fault Diagnosis . . . 5
2.1 Fault Diagnosis System Scheme . . . 6
2.2 Parameter Estimation Method Output . . . 12
2.3 Process and State Observer . . . 14
2.4 Output Error Method . . . 15
2.5 Fault Topology of the Monitored System . . . 16
2.6 Neural Network Applications in Fault Diagnosis . . . 21
3.1 ANFIS Model of Sugeno’s fuzzy inference method . . . 26
3.2 Input space partitioning of ANFIS structure . . . 29
3.3 Piecewise Linear Approximation of ANFIS Output . . . 32
3.4 A two layer feed-forward neural network . . . 33
3.5 Water-Tank System Parameters . . . 38
3.6 Water-Tank System Simulation . . . 38
3.7 Water-Tank System in Closed-Loop . . . 39
3.8 Neural Network Model of the Water-Tank System . . . 40
3.9 Neural Network Model Water Level vs Actual Water Level . . 41
3.10 Neural Network Model Flow Rate vs Actual Flow Rate . . . . 42
3.11 Residual Generation Using Model of the System . . . 42
3.12 Membership Functions for residuals . . . 44
3.13 ANFIS Output for Fault Classification . . . 46
3.14 Neural Network Model for Fault Classification . . . 47
3.15 Neural Network Output in Fault Classification . . . 47
4.1 Simple Classification Tree for Illustration . . . 52
4.2 Decision Regions Created By Classification Tree . . . 57
4.3 Simple Decision Boundary for Optimal Tree . . . 58
4.5 Schematic Diagram of the Experimental Setup . . . 66
4.6 Reference SoC vs Actual SoC . . . 68
4.7 Neural Network Output vs Actual Battery Voltage . . . 69
4.8 Lead-Acid Battery Model from the Actual Data . . . 70
4.9 Three Dimensional Residual Space . . . 72
4.10 Classification Tree Before Pruning . . . 73
4.11 Cross Validation Error of Maximum Tree . . . 74
4.12 Classification Tree After Pruning and Cross Validation . . . . 75
4.13 Cost of the Tree with FDA using Cross Validation . . . 76
4.14 Classification Tree in Fault Classification . . . 77
4.15 Classification Tree with FDA in Fault Classification . . . 78
4.16 Schematic Diagram of Mass-Spring-Damper System . . . 79
4.17 Matlab Model of Mass-Spring-Damper System . . . 80
4.18 Reference Input vs Actual Output . . . 81
4.19 Actual Output vs NN Output . . . 82
4.20 Residuals in Three Dimensional Space . . . 83
4.21 Comparison of the Pruned Trees . . . 84
List of Tables
1 Two passes in hybrid learning algorithm for ANFIS . . . 29
2 Water-Tank Parameters . . . 38
3 Fuzzy Logic Training Set . . . 45
4 Comparison of NN with FL . . . 48
5 Comparison of CT and CT with FDA . . . 78
Chapter I
1
Introduction
Fault Diagnosis research deals with real world problems such as plant efficiency, maintainability and reliability. For safety critical systems, such as nuclear applications in plants and aircrafts, the detection of fault occurrence is highly important.
The consequences of the faults could be disastrous in these systems in terms of human mortality and environmental impact. Also in process and manufacturing industries, fault detection is crucial in order to improve the production efficiency, quality of the product and cost of the production.
There are two important approaches for fault diagnosis: hardware
redun-dancy and analytical redunredun-dancy. Hardware redunredun-dancy uses multiplication
of physical devices and a system to detect the occurrence of a fault and its location in the system. The main problem is the significant cost of the extra equipment. Analytical redundancy uses redundant functional relationships between the variables of the system. The main advantage of this approach compared to the hardware redundancy is that no extra equipment is neces-sary. However it requires more processing power.
1.1
Fault Detection and Diagnosis
In the early 1970s fault detection based on analytical methods has begun. Beard [1] designed an observer-based fault detection scheme and Johns [2]
continued his work. Their contribution is named as Beard-Johns Fault Detec-tion Filter. Statistical approaches to fault diagnosis were first used in [3]. Lu-enberger observers were applied for the first time in [4]. Also Mironovsky [5] proposed a residual generation scheme based on consistency checking on the system input and output over a time window.
In 1980s and early 1990s major approaches on quantitative fault diagnosis were developed: observer-based approach, parity relation method, parameter estimation method etc [6]. It must be noticed that these methods are well-established theoretically. Therefore they are called classical or quantitative fault detection methods.
These methods have in common the use of a set of analytical redundancy relationships that represent the model of the system which follows the de-sired performance of the monitored system. The system is monitored for possible digressions that indicate the occurrence of the faults and may assist in isolating the faulty components.
In the last decade the research focused on fault diagnosis for nonlinear dynamic systems. Computational intelligence techniques such as neural net-works, fuzzy systems and genetic algorithms have been successfully applied to the fault diagnosis.
1.1.1 Basic Terminology
These definitions are taken from International Federation of Automatic Control (IFAC) terminology.
Fault Diagnosis, Fault Tolerant Control
A fault represents an unexpected change of system function, although it may not represent a physical failure. Failure indicates a serious breakdown of a system component or function that leads to a significantly deviated
behavior of the whole system. The term fault rather indicates a malfunction that does not affect significantly normal behavior of the system.
An incipient (soft) fault represents a small and slowly increasing fault. At the beginning effects on the system are unnoticeable. A fault is called
hard (abrupt) fault if its effects on the system are longer and bring the system
very close to the limit of unacceptable behavior.
A fault is called intermittent if its effects on the system are hidden for discontinuous periods of time [6]. Although a fault is tolerable at the moment it occurs, it must be diagnosed as early as possible, otherwise it may lead to serious consequences in time.
A fault diagnosis system is a monitoring system that is used to detect faults and diagnose their location and significance. The system performs the following three functions:
Fault Detection: to indicate if a fault occurred or not in the system. Fault Isolation: to determine the location of the fault
Fault Identification: to estimate the size and nature of the fault.
As another concept, a fault tolerant control system is a controlled system that continues to operate normally although there are faults in the system or in the controller. An important aspect of this system is automatic reconfig-uration, once a malfunction is detected and isolated. Fault diagnosis decides how to perform the reconfiguration.
1.1.2 Statement of the Problem
Although technological developments have led to increasingly reliable mechanical, electrical and electronic vehicle systems, these systems still fail. The main goal of fault diagnosis system in a vehicle is to avoid damage to the vehicle and prevent dangerous situations for occupants.
In this research fault diagnosis of commercial vehicles, which requires di-agnosis of all faults in the system, is performed. Fault didi-agnosis in vehicles is essential for low fuel consumption, high safety, efficient service and main-tenance.
Objectives of This Research
• Investigate model based fault detection and diagnosis algorithms for
nonlinear dynamic systems such as water-tank system, lead-acid bat-tery system and mass-spring-damper system.
• Design a reasonable model of these systems and create fault scenarios. • Validate the developed fault diagnosis algorithms on simulation and
real time environment and compare their performances.
1.2
Fault Diagnosis Based on Analytical Models
Model based fault diagnosis is determination of the faults by comparing available system measurements with a priori information represented by the analytical model of the system through generation of residuals and their analysis. A residual is a fault indicator that reflects the faulty condition of the monitored system [7] similar to temperature or blood glucose level measurements of a patient which are used as symptoms to diagnose a disease. Unfortunately an analytical model of the system is rarely accurate due to uncertainties, disturbances and noise. This results in differences between the analytical model output and the system output due to unmodelled dynamics and other uncertainties.
A fault diagnosis task contains two stages: residual generation and
resid-ual evaluation which are shown in Figure 1.1. Residresid-ual generation is a
output information.
Input Plant Output
Residuals Fault Alarm Residual Generation Residual Evaluation N M
Figure 1.1: Two main stages in Fault Diagnosis
Residual generation represents an algorithm which is used to generate residuals. Residual evaluation represents examining residual signals in order to decide if a fault has occurred. It also isolates the fault. In most cases they must themselves be nonlinear dynamic systems. They may be implemented using statistical methods, e.g. likelihood ratio testing or classification tree [6].
Chapter II
2
System Modeling
In this section residual generation structure will be given and analytical conditions for fault detectability and isolability will be discussed. For sim-plicity it will be assumed that a linear model can reproduce system dynamics. In the case of nonlinear dynamics, it is assumed that the model is lin-earized around a few operating points. The transition between different operating regions is performed using qualitative techniques such as fuzzy logic [8]. However nonlinear systems will be considered later in the thesis.
The information used for fault diagnosis and isolation is the measured input to the actuators and the output of the sensors. The measured output
y(t) is also used by feedback control and the controller generates the control
signal u(t) which is shown in Figure 2.1.
Fault time Fault location Residual Generation Decision Function Generator Fault Decision Logic Sensors System Dynamics Actuators Controller fa(t) uR(t)
uc(t) u(t) y(t)
fs(t) yR(t)
Figure 2.1: Fault Diagnosis System Scheme
of the system even if it is in a closed control loop. If the signal is not available, then FDS uses reference command uc(t) as an input. In this case
the controller plays an important role because a robust controller can hide the effects of faults, therefore making fault diagnosis difficult [9].
2.1
General Structure of Faulty Systems
The state space model of the plant shown in Figure 2.1:˙x(t) = Ax(t) + BuR(t)
yR(t) = Cx(t)
(2.1)
where x ∈ Rn is the state vector of the plant, u
R∈ Rr is the input vector to
the actuator and yR ∈ Rm is the output vector of the plant. Under normal
operating conditions,
uR(t) = u(t)
y(t) = yR(t)
(2.2)
A, B, C are known matrices with known dimensions. Faults in the system
could occur due to actuators, system components and sensors. The dynamics of the system can change as follows:
• actuator fault
uR(t) = u(t) + fa(t) (2.3)
where fa∈ Rr is the actuator vector fault.
• system dynamics (component) fault
˙x(t) = Ax(t) + BuR(t) + fc(t) (2.4)
• sensor fault
y(t) = yR(t) + fs(t) (2.5)
where fs∈ Rm is the sensor vector fault.
If the previous three fault categories are considered simultaneously, the time domain representation of the system model changes to:
˙x(t) = Ax(t) + Bu(t) + Bfa(t) + fc(t)
y(t) = Cx(t) + fs(t)
(2.6)
In more general case with all possible faults in the state space model:
˙x(t) = Ax(t) + Bu(t) + R1f (t)
y(t) = Cx(t) + R2f (t)
(2.7) where f (t) ∈ Rg is a fault vector, f
i(t)(i= 1...g) are specific faults and R1
and R2 are fault entry matrices which represents the effect of faults on the
system.
Input-output transfer matrix in frequency domain for the faulty system model is: y(s) = Gu(s)u(s) + Gf(s)f (s) (2.8) where Gu(s) = C(sI − A)−1B Gf(s) = C(sI − A)−1R1+ R2 (2.9)
2.2
General Structure of Residual Generation
Input values of a residual generator are inputs and outputs of the monitored system as expressed by:
where Hu(s) and Hy(s) are transfer matrices realizable using stable linear
systems.
The residual (r(t)) must be designed (in ideal case) to be zero for fault free case and nonzero when a fault occurs which is shown in (2.11).
r(t) = 0 if and only if f (t) = 0 (2.11) Therefore the matrices Hu(s), Hy(s) and Gu(s) (defined in (2.9)) must
satisfy the following constraint condition:
Hu(s) + Hy(s)Gu(s) = 0 (2.12)
Equation (2.12) is a generalized representation of all residual genera-tors [7]. For the aim of residual generation design, one must choose two matrices which satisfy (2.12). Based on the parametrization chosen for Hu(s)
and Hy(s), a different way to generate residuals is obtained.
Assume that J is a function of residual signal r(t). Fault detection is done by comparing the residual evaluation function J(r(t)) with a threshold function T (t) using condition in (2.13). If the residual exceeds the threshold, a fault may be occurred.
J(r(t)) ≤T(t) for f (t) = 0
J(r(t)) > T (t) for f (t) 6= 0 (2.13)
2.3
Fault Detectability and Fault Isolability
For a faulty system the residual vector is defined as:r(s) = Hy(s)Gf(s)f (s) = Grf(s)f (s)
= [Grf(s)]1f1(s) + [Grf(s)]2f2(s)... + [Grf(s)]ifi(s)
(2.14)
The residual-fault relationship is represented by Grf(s) = Hy(s)Gf(s),
where [Grf]i is the ith column matrix of Grf and fi is the ith component of
f (s).
Fault Detectability Condition: If the ith column of G
rf(s) is nonzero, Grf(s)i 6= 0, the fault fi is
de-tectable in the residual r(s). This is called the fault detectability condition of the residual r(s) to the fault fi [7].
Fault Isolability Condition:
A fault is isolable using a residual vector set, if it is distinguishable from other faults using this set. Usually each residual from the considered set is sensitive to a subset of faults and insensitive to the others [10].
2.4
Quantitative Diagnosis Methods
The main point in model based fault diagnosis is residual generation method each of which has its specific technique of computing the residual vector.
Three important methods will be represented in this section. These meth-ods focus on discrete-time dynamic linear models.
2.4.1 Residual Generation via Parameter Estimation
When the process parameters are not known exactly, they can be de-termined with parameter estimation methods by measuring the input and output signals, if the basic structure of the model is known [6] .
It is assumed that the faults are reflected in the physical system parame-ters and these parameparame-ters are estimated online using well-known parameter estimation methods. The results are then compared with the parameters of the reference model obtained under fault-free assumptions. Any discrepancy
would indicate that a fault may have occurred. For the nth order discrete-time estimated model:
Θ = [a1...an, b1...bn]T (2.15)
is the parameter vector where ai and bi (i=1,..,n) represent the coefficients
in A(z) and B(z) transfer matrices.
Output error of the system (or the loss function) is calculated as:
e(t) = y(t) − ˆy(Θ, t) (2.16) where
ˆ
y(Θ, z) = B(z)ˆˆ
A(z)u(z) (2.17)
is the model output in which ˆA(z) and ˆB(z) correspond to the estimates of A(z) and B(z) as depicted in Figure 2.2.
Since e(t) is a nonlinear parameter, direct calculation of Θ is generally not possible. Numerical optimization methods can be used to minimize the loss function (2.16) as (2.17).
If a fault in the process changes one or several parameters by ∆Θ the output changes for small deviations according to
∆y(t) = ψ(t)T∆Θ(t) + ∆ψ(t)TΘ(t) + ∆ψ(t)T∆Θ(t) (2.18)
and the parameter estimator indicates a change ∆Θ.
Generally the process parameters Θ depend on physical process coeffi-cients p(like stiffness,damping factor, resistance...). If L is a function de-pending on p,
u(t)
y(t)
e(t)
u(t)
Parameter
Estimation
B(z)
A(z)
A(z)
B(z)
^y
Figure 2.2: Parameter Estimation Method Output
via nonlinear algebraic equations. If the inversion of the relationship ex-ists [11]:
p = L−1(Θ) (2.20)
where changes (∆p) of the process coefficients, from which the fault alarm is obtained, can be calculated.
2.4.2 Observer Based Approaches
The main idea of the observer based technique is to estimate the outputs of the system from the measurements by using either Luenberger observers in a deterministic setting or Kalman filters in a noisy environment. The output estimation error is used as residual. The advantage of using observer is the flexibility in the selection of its gains which leads to a rich variety of FDS schemes [12].
In order to obtain the structure of an observer the discrete-time, time-invariant and linear dynamic model for the plant in a state space form is considered.
x(t + h) = Ax(t) + Bu(t)
y(t) = Cx(t) (2.21)
where h is the sampling time interval, u(t) ∈ Rr, x(t) ∈ Rn and y(t) ∈ Rm.
Assuming that all matrices A,B and C are perfectly known, an observer is used to reconstruct the system variables based on the measured inputs and outputs u(t) and y(t).
ˆ
x(t + h) = Aˆx(t) + Bu(t) + He(t)
e(t) = y(t) − C ˆx(t) (2.22)
The observer scheme described by (2.22) is drawn in Figure 2.3. For the state estimation error ex(t), it follows from the equations (2.22) as:
ex(t) = x(t) − ˆx(t)
ex(t + h) = (A − HC)ex(t)
(2.23) If the observer is stable, the state error ex(t) vanishes asymptotically.
lim
t→∞ex(t) = 0 (2.24)
This can be achieved by the proper design of the observer feedback matrix
H [7]. Thus, the design of the observer feedback matrix H is important in
residual generation. However if the signals are affected by noise, Kalman filter must be used instead of classical observers [13].
h h x(t+h) = Ax(t)+Bu(t) y(t) = Cx(t) u(t) y(t) r(t) x(t+h) e(t) x(t) y(t) W C A B H z
Figure 2.3: Process and State Observer
2.4.3 Parity Vector (relation) Methods
Parity relations approach provides a proper check of the parity (consis-tency) of measurements acquired from the monitored system. In the early development of fault diagnosis, the parity vector (relation) approach was ap-plied to static or parallel redundancy schemes [14] which may be obtained directly from measurements (hardware redundancy) or from analytical rela-tions (analytical redundancy).
In the first case, two methods can be used to obtain redundant relations which requires several sensors with similar functions to measure the same variable. The second approach consists of dissimilar sensors to measure dif-ferent variables but their outputs being relative to each other.
In case of analytical model based fault detection, the model can be written in the form of Gm(z) = ˆA(z)/ ˆB(z) and to run it in a parallel to the process
described by the transfer function Gp(z) :
Gp(z) =
A(z)
Thereby forming an error vector eP(z): eP(z) = ( A(z) B(z) − ˆ A(z) ˆ B(z))u(z) (2.26)
The methodology described here is shown in Figure 2.4.
r(t)
y(t)
u(t)
A(z)
B(z)
B(z)
A(z)
^y(t)
^
^Figure 2.4: Output Error Method
Assume that Gm(z) = Gp(z) i. e. ˆ A(z) ˆ B(z) = A(z) B(z) (2.27) then the residual becomes
eP(z) =
A(z)
B(z)fu(z) + fy(z) (2.28)
where fu(z) and fy(z) are additive input and output faults which are shown
in Figure 2.5. Moreover the error vector r(z) computed by (2.28) corresponds to the output error of the parameter estimation method which is computed by (2.16).
The residuals generated are called parity equations [10] under the as-sumptions of fault occurrence and of exact agreement between the process and the model.
Actuators u(t) y(t) y*(t) Plant fy(t) fu(t) uR(t) u*(t) Input Sensors Output Sensors
Figure 2.5: Fault Topology of the Monitored System
Therefore (2.28) can be used to implement and design the residual gener-ation system in order to meet fault detection and isolgener-ation specificgener-ations as well [15] .
2.5
Qualitative Diagnosis Methods
2.5.1 Fuzzy Model Based Residual GenerationDuring the last forty years, modeling and control of dynamic systems with fuzzy set techniques have received considerable attention. Many systems are not suitable to conventional modeling techniques due to lack of precise, formal knowledge about the system and due to time varying characteristics [16] .
Fuzzy modeling along with neural networks are powerful tools to facilitate effective development models. One of the reasons for this case is that fuzzy systems are capable of integrating information from different sources such as physical laws, measurements and heuristics.
Fuzzy models can be seen as logical models which use "IF-THEN" rules to establish qualitative relationships among variables in the model. Fuzzy sets serve as smooth interfaces between the qualitative variables involved in the numerical data at the inputs and outputs of the model. The rule-based nature of fuzzy models uses the information expressed in the form of natural language statements.
More specifically, Mamdani [17] and Takagi Sugeno (TS) [8] are two suc-cessful fuzzy system types, since they are able to approximate any continuous function with a desired level of accuracy [18] .
Mamdani Fuzzy System
Fuzzy information is expressed as fuzzy sets and linguistic variables which are called membership functions. Also fuzzy rule base is required for the representation of fuzzy information generally in the form :
Rulel: IF (x1 is Al1) ...(xn is Aln) THEN (y is Bl) (2.29)
The membership functions of fuzzy sets Al
i and Bl are denoted as µAl i and µBl, respectively, where µAl i : X → [0, 1], (i = 1, 2, .., n) µBl : Y → [0, 1], (l = 1, 2, ..., M) (2.30)
In (2.30); n is the number of inputs of the fuzzy system and M is the number of IF-THEN rules. In the Mamdani fuzzy model minimum fuzzy inference system is used. For a given input x∗ = (x∗
1, x∗2, .., x∗n) ∈ X, the
output of the fuzzy inference system µBl(y) is defined as:
µBl(y) = max(1≤l≤M ,y∈Y )[min(y∈Y )µAl
1(x ∗ 1), µAl 2(x ∗ 2), .., µAl n(x ∗ n), µBl(y)] (2.31) where the min operator selects the minimum value among the values of mem-bership functions in the IF proposition of a given input x∗ and the
member-ship function of the THEN proposition of the output universe Y . For the final output of the fuzzy system, a defuzzifier is needed which represents the fuzzy set at the output of the system.
TS Fuzzy System
Unlike Mamdani fuzzy rules, TS rules use piecewise linear functions of the input variables.
Each rule comprises IF-THEN condition and has the following form:
Rulel: IF (x 1is Al1) ...(xnis Aln) THEN yl = n X i=1 kl ixi+ cli) (2.32)
where l refers to the lth rule, n is the number of inputs, Al
i is the fuzzy set in
the input (antecedent) and yl is a crisp first-order polynomial function in the
output (consequent) [19]. Finally ki and ci represent a factor and a constant
of the polynomial defined in the first order TS model, respectively. Output of the fuzzy system with M rules is aggregated as:
y = PM l=1µlyl PM l µl (2.33) where µl is the degree of activation of the rule l:
µl = l Y i=1 µAl i (2.34) where µAl i is defined in (2.30).
Generally with the similar system requirements such as number of rules and membership function TS is more accurate than Mamdani model. Due to the incompleteness of knowledge, the rules and its predicates need to be updated to optimize the system. Also the main relations representing the Mamdani model is not continuous due to the presence of MAX or MIN operator. Therefore the optimization techniques that use derivatives, e.g. gradient descent method can not be applied. This makes the Mamdani model less adaptable to fault diagnosis application [20].
2.5.2 Neural Network Model Based Residual Generation
The potential of neural networks for fault detection and isolation has been better understood recently. Artificial neural network based approach is
especially suitable for processes for which accurate mathematical models are too difficult or too expensive to obtain.
Neural networks try to mimic the computational structures of the mam-mal brains by nonlinear mapping between the input and output that consists of interconnected nodes arranged in layers. The layers are connected such that the signal on the input side can propagate through the network and reach output side. Neural network behaviors are determined by the transfer functions of the units, network topology and connection pattern of layers.
Among all the forms of Neural Networks, the two layer feed-forward neural network has been the most popular. This class of networks consists of two layers of nodes, namely the hidden layer and the output layer. Also there exist two layers of weights serving as connection between the input and the hidden layer, as well as between the hidden layer and the output layer. No connection is allowed with its own layer and the information flows in one direction. Sigmoid functions are usually selected as the transfer function for hidden layer nodes and linear functions for the nodes of the output layer. Equations of the transfer functions are:
f1(x) = x (2.35) f2(x) = 1 1 + e−x (2.36) f3(x) = exp(x) − exp(−x) exp(x) + exp(−x) (2.37)
where the functions are pure linear, log-sigmoid and tan-sigmoid functions respectively.
This class of neural networks can approximate any functional continu-ous mapping from one finite dimensional space to the other arbitrarily well, provided that the number of hidden neurons is sufficiently large. Therefore this class of neural networks and two layers of weights can approximate any
decision boundary to within arbitrary accuracy. This is the reason why two layer functions are applied to process modeling and fault diagnosis by pattern recognition.
During the training time, neural network uses the error in the output values to update the weights connecting layers, until the accuracy is within the tolerance level. The training time for the feed forward neural network using one of the variations of backpropagation is important. For large scale applications, memory and computation time required for training a neural network can exceed the hardware limits. Therefore the performance of neural networks is determined by the available data. It is possible that neural networks will generate unpredictable outputs when presented with an input out of the range of training data. So retraining of neural network may be required.
Neural networks can be applied to fault detection and diagnosis as a process model or a pattern classifier. For this purpose neural networks can be summarized in three categories which is shown in Figure 2.6.
In the first one, neural network is used to differentiate various faulty out-put patterns from normal operating conditions. According to the different measured process output data. Training of the neural network can be per-formed offline or online. In the second figure, neural networks are used as classifiers to isolate faults represented by process model-generated residuals. The process model is the mathematical model of the process based on fault diagnosis structure which uses the mechanism provided by the model. When the mathematical models are not available, a neural network process model can be employed to generate residuals; another network is then used to isolate faults.
Neural Network Model Neural Network Classifier Neural Network Classifier Neural Network Classifier Process Process Process Neural Network Model Math Model Faults Faults Faults Output Output Output Input Input Input Residual Residual Model Output Model Output (a) (c) (b)
Figure 2.6: Neural Network Applications in Fault Diagnosis
2.5.3 Neuro-Fuzzy Model Based Residual Generation
The main drawback of neural networks as stated in the previous section is their "black box" nature, while the disadvantage of fuzzy systems is repre-sented by difficult and time consuming process of knowledge acquisition. The advantage of neural network over fuzzy systems is learning and adaptation capabilities, while the advantage of fuzzy system is the human understand-able form of knowledge representation. Neural networks use an implicit way of knowledge representation while neuro-fuzzy systems represent knowledge in an explicit form, such as rules.
The combination of neural networks and fuzzy systems can be done in two ways:
Neural networks implemented using fuzzy logic
These hybrid systems are mainly neural networks equipped with abil-ities of processing fuzzy information. These systems are usually termed as Fuzzy Neural Networks and they are networks where the inputs, outputs and weights are fuzzy sets, and they consist of a special type of neurons called fuzzy neurons.
Fuzzy logic implemented using neural networks
These systems can be viewed as fuzzy systems augmented with neural net-work facilities, such as learning, adaptation and parallelism. These systems are usually called Neuro-Fuzzy Systems. Neuro-Fuzzy Systems can be always interpreted as a set of fuzzy rules and can be represented as a feed-forward network architecture [21].
In addition to these two approaches, there is another way of hybridization of neural networks and fuzzy systems, where each method maintains its own identity and the hybrid neuro-fuzzy system consists of modules cooperating in solving the problem. These kind of neuro-fuzzy systems are combinations of hybrid systems. Detailed explanations and related equations are given in section 3.2.2.
In some approaches a neural network (such as self organizing map) can preprocess the input data for fuzzy system. However in fault diagnosis ap-plications, fuzzy system is used as a pre-processor for a neural network.
A Neuro-fuzzy (NF) system is a neural network which is topologically equivalent to the structure of a fuzzy system. Network inputs/outputs and weights are real numbers but the network operations are specific to fuzzy systems: fuzzification, fuzzy operations (conjunction , disjunction), defuzzi-fication. Therefore NF systems can be used to identify fuzzy models directly from input-output relationships, but they can be used to optimize an initial
fuzzy model acquired from human expert, using additional data.
2.6
Conclusion
In this section the importance of fault diagnosis for industrial systems is explained, a literature overview is done and basic terminology about fault detection is given.
The ways of designing residuals are discussed. The most commonly used residual generation techniques are introduced and applicability of analytical model based fault diagnosis are discussed.
Other Fault Diagnosis methods such as fuzzy logic, neural networks and qualitative modeling have been discussed. In the next chapter this method will be implemented on a water-tank system and the residual will be obtained using the analytical model. Then the residual will be classified using the same methodology and faults will be identified clearly.
Chapter III
3
Fault Diagnosis Based on Qualitative
Meth-ods
3.1
Introduction
The main objective of fault diagnosis is early warning for the operators to take appropriate measures and prevent the system from breaking down after the occurrence of faults. This will improve the reliability and safety of the system [22]. Since the systems are becoming more complex, automated fault monitoring schemes are developed in case of human operators.
For fault diagnosis of nonlinear plants, computer intelligence based meth-ods such as neural networks, fuzzy logic and genetic algorithms are often used [23] [24]. Among these techniques, neural networks are important for their ability to approximate nonlinear functions and their online learning ability. Also they can be used as a model to generate residuals for classifying and isolating the faults [25]. However their disadvantage is the difficulty in isolating the faults due to their black box nature. Another approach is fuzzy reasoning which allows symbolic generalization of numerical data by fuzzy rules and expert knowledge integrated into the fault diagnosis procedures to achieve better diagnosis [26]. On the other hand adaptive neuro-fuzzy systems have the ability of neural networks which can approximate nonlin-ear functions with arbitrary accuracy and they are able to incorporate fuzzy
rules which allows expert knowledge in linguistic form to be included. In this section, a brief overview is given about Adaptive Neuro Fuzzy Sys-tem (ANFIS) structure and neural network structure. The proposed scheme is to be illustrated by a simulation example of a water-tank system. Per-formances of these two approaches are compared. Finally a conclusion is drawn.
3.2
ANFIS Structure
A Fuzzy Logic System (FLS) is a nonlinear mapping from an input space to an output space. The mapping is based on the conversion of the inputs from crisp numerical domain to fuzzy domain using fuzzy sets and fuzzifiers, and then applying fuzzy rules and fuzzy inference engine to perform the nec-essary operations in the fuzzy domain. In the end, the result is converted back to the crisp numerical domain using defuzzifiers. Hence, a FLS con-tains five main components: fuzzy sets, fuzzifiers, fuzzy rules, an inference engine and defuzzifiers [27]. Adaptive neuro-fuzzy networks are enhanced FLSs with learning, generalization and adaptation capabilities. These net-works encode the fuzzy if-then rules into a neural network-like structure and then use appropriate learning algorithms to minimize the output error based on training/validation datasets.
ANFIS is a Fuzzy-Sugeno model of integration where the final fuzzy infer-ence system is optimized via neural network training [8]. It maps the inputs through the input membership functions and parameters, then through the output membership to the outputs. It will be explained next.
fuzzy inference system which is expressed with four rules as follows: Rule 1 : If x is A1 and y is B1 then f1 = p1x + q1y + r1
Rule 2 : If x is A2 and y is B2 then f2 = p2x + q2y + r2
Rule 3 : If x is A1 and y is B2 then f3 = p3x + q3y + r3
Rule 4 : If x is A2 and y is B1 then f4 = p4x + q4y + r4
(3.1)
where x and y are the inputs, Ai and Bi are the fuzzy sets and fi (i=1,2,3,4)
are the membership functions (fuzzy region specified by the fuzzy rules) and
pi,qi and ri are the design parameters. "If x is A1 and y is B1" part is called
the premise part of a rule, and "then f1 = p1x + q1y + r1" is called the
consequent part of a rule. Using fi (i=1,2,3,4) the output function for this
model is expressed as,
f = w1f1+w2f2+w3f3+w4f4 w1+w2+w3+w4
= ¯w1f1+ ¯w2f2+ ¯w3f3 + ¯w4f4
(3.2)
where wi (i=1,2,3,4) is explained in (3.6). ANFIS architecture for these four
rules is illustrated in Figure 3.1.
Figure 3.1: ANFIS Model of Sugeno’s fuzzy inference method
This structure has five layers and the node functions in each of these layers are explained below:
The fuzzification process is taken here where the membership functions transform the input x to the output O1
i. The output of every node i in the
first layer is defined with a node function O1
i where the superscript denotes
the layer number:
O1
i = µAi(x), (for i=1,2) (3.3)
O1
i = µBi−2(x), (for i=3,4) (3.4)
where x is the input to node i, µAi(x) is the membership function defining
linguistic label (small, large, etc.) Ai (or Bi). Generally the µAi(x) is chosen
as bell-shaped, which is between 0 and 1, with the following formula:
µAi(x) = 1 1 + ¯ ¯ ¯x−ci ai ¯ ¯ ¯2bi (3.5)
where ai, bi, ci are referred to the premise parameter set of this layer. Other
continuous and piecewise differentiable functions such as trapezoidal or triangular-shaped membership functions can also represent the node functions in this layer.
Layer 2: Firing Strength of Rule
Each node output represents a firing strength of a rule. The T-norm (product, fuzzy-AND..) operators perform the node function in this layer. These nodes multiply the incoming signals and send the product out. For instance,
O2
i = wi = µAi(x) × µBi(y), (i = 1,2) (3.6)
where x is the T-norm implemented as product.
Layer 3: Normalized Firing Strengths
The normalization process is performed in this layer. The ith node
cal-culates the ratio of the ith rule’s firing strength to the sum of all rules’ firing
strengths: O3 i = ¯wi = wi w1+ w2 + w3+ w4 , (i = 1,2,3,4) (3.7)
Layer 4: Consequent Parameters
Every node in this layer is shown with a node function:
O4
i = ¯wifi = ¯wi(pix + qiy + ri) (3.8)
where ¯wiis the node output of layer 3 and pi, qi, ri is the parameter set which
is called as consequent parameters. The output of this layer forms Takagi-Sugeno outputs.
Layer 5: Overall Output
The single node in this layer sums all the incoming signals and computes the overall output. Output is linear in terms of the consequent parameters as seen in the last equation of (3.9):
O15 =X i ¯ wifi = P iwifi P iwi = w1 w1+ w2+ w3+ w4 f1+ w2 w1+ w2 + w3+ w4 f2+ ... = ¯w1(p1x + q1y + r1) + ¯w2(p2x + q2y + r2) + ... = ( ¯w1x)p1 + ( ¯w1y)q1+ ( ¯w1)r1 + ( ¯w2x)p2+ ( ¯w2y)q2+ ( ¯w2)r2+ ... (3.9)
Input space partitioning of the two-input ANFIS structure is shown in Figure 3.2. Grey area shows the undetermined region whereas dark area is the membership function combinations for these inputs. Two membership functions are associated with each input. The premise part of a rule (defined in 3.1) linearizes the fuzzy subspace and the consequent part (defined in 3.1) specifies the output within this fuzzy subspace. Therefore ANFIS uses two set of parameters which are called as S1 and S2 where S1 is the set of premise parameters (ai,bi,ci for i=1,2,3,4 )and S2 is the set of consequent parameters
A 1 A2 B 2 B 1 A 1 A 2 B 1 B2
Figure 3.2: Input space partitioning of ANFIS structure
ANFIS learning algorithm is a two-pass hybrid learning algorithm con-sisting of forward pass and backward pass. In the forward pass, functional signals go forward up to the layer 4 and S2 parameters are computed using least squared error (LSE) algorithm on layer 4. In the backward pass, the error rates are propagated backward and S1 parameters are computed using a gradient descent algorithm (usually backpropagation) to be explained next. In Table 1 the signals and parameters for each pass is represented.
Table 1: Two passes in hybrid learning algorithm for ANFIS Forward Pass Backward Pass Premise Parameters fixed gradient descent Consequent Parameters least-squares fixed Signals Node Outputs Error Signals
3.2.1 ANFIS LSE Algorithm
In the forward pass of hybrid learning algorithm, consequent parameters are identified by least squares estimate. Assume that S is the total parameter set which is the combination of S1 and S2 sets.
S = S1 ∪ S2 and S1 ∩ S2 = φ (3.10) Then the output becomes:
Output = F ( ¯I, S) (3.11) where ¯I is the input vector and
H(Output) = HoF ( ¯I, S) (3.12) where H is a function of output and H o F is linear in terms of S2.
For the given values of S1, using P training data 3.12 can be transformed
into the equation:
B = AX (3.13)
where X is the unknown vector containing the elements of S2.
Generally no exact solution is found for this equation. Therefore LSE minimizes the error kAX − Bk2 by approximating X with X∗ (least squares
estimate of X). The estimate of X, X∗ can be defined as:
X∗ = (ATA)−1ATB (3.14)
where AT is the transpose of A, and (ATA)−1AT is the pseudo-inverse of A
if ATA is nonsingular.
It is difficult to compute the LSE of X∗ because P is large. Therefore X
is often solved iteratively using the formulas [28]:
Si+1= Si−
Sia(i + 1)a(i + 1)TSi
1 + a(i + 1)TS
ia(i + 1)
Xi+1= Xi + S(i + 1)a(i + 1)(b(i + 1)T − a(i + 1)TXi) (3.16)
for i = 0, 1, ..., P − 1 where X0 = 0, S0 = γI (γ is a large number and I is
identity matrix), aT
i is the ith row of matrix A, bTi is ith element of vector B,
X∗ is X
P (Xi+1 value for i = P − 1).
3.2.2 ANFIS Backpropagation Algorithm
For a training data set with P entries, the error measure or energy function can be defined as:
Ep = N (L)X m=1
(Tm,p− OLm,p)2 (3.17)
where (1 ≤ p ≤ P ); N(L) is the number of nodes in layer L; Tm,p is the
mth component of pth target output vector and OL
m,p is the mth component
of actual output vector. The overall error measure is:
E =
P
X
p=1
Ep (3.18)
Next, the error rate is calculated for the gradient descent in E over the pa-rameter space. The error rate for the output node at (L, i) can be calculated as:
∂Ep
∂OL i,p
= −2(Ti,p− OLi,p) (3.19)
For the internal node at (k, i) the error rate is defined by the chain rule:
∂Ep ∂Ok i,p = N (k+1)X m=1 ∂Ep ∂Ok+1 m,p ∂Ok+1 m,p ∂Ok i,p (3.20) where (1 ≤ k ≤ L − 1).
Generalization of this equation for the α parameter is:
∂Ep ∂α = X O∗∈S ∂Ep ∂O∗ ∂O∗ ∂α , (3.21)
where S is the set of nodes whose outputs depend on α. The derivative of the overall measure E with respect to α is:
∂Ep ∂α = P X p=1 ∂Ep ∂α (3.22)
For the generic parameter α the updated formula is:
∆α = −η∂E
∂α (3.23)
where η = k
qP
α(∂E∂α)2
is the learning rate, k is the step size and ∂E
∂α is the
ordered derivative.
Two-pass training is much faster than the gradient descent algorithm since it decomposes the parameter set as S1 and S2. It is possible if the member-ship function of each rule is replaced by a piecewise linear approximation with two consequent parameters. As seen in Figure 3.3, the consequent parame-ters constitute set S2 and the hybrid learning rule can be applied directly.
0
10
-10
0
10
-10
0
0
2
2
4
4
6
6
8
8
X
X
Y
Y
3.3
Neural Network Structure
Feed-forward neural network is a nonlinear mapping between input and output that consists of interconnected nodes arranged in layers. The layers are connected such that signal on the input side propagates through the network and reaches the output side without feedback loops.
Consider a multilayer feed-forward neural network with one hidden layer shown in Figure 3.4. The input signals to the ni input layer nodes are
denoted by x1,x2...,xni; the output signals of the nO output layer nodes are
denoted by y1,y2...,ynO; and the output signals of the nh hidden layer nodes
are denoted by h1,h2...,hnh. The nonshaded nodes are bias nodes with inputs
set equal to unity. Connection between nodes of different layers of network are weights and biases which correspond to the dotted line connections in Figure 3.4. No connection is allowed back to its own layer and the information flow is only one directional.
Figure 3.4: A two layer feed-forward neural network
Consider an initial forward feed of the neural network structure. For a specific input pattern (set of input values) output of jth hidden layer is given
by, hj = f ( ni X i=1 w0 jixi+ b0j) (3.24)
where f is the activation function, w0
ji is the strength of connection from the
ith input to the jth hidden layer node and b0
j is the bias value for the jth
hidden layer node.
The output of the kth output node is given by,
yk = f ( nh
X
j=1
wkjhj+ bk) (3.25)
where bk is the bias for the kth output node and wkj is the strength of the
connection from the jth hidden layer node to the kth output node.
Backpropagation, one of the popular training algorithms, uses gradient descent algorithm to update the weights and therefore the activation func-tions must be differentiable. Some of the activation funcfunc-tions for the neural networks are given in section 2.5.2. The result of the feed-forward process is the output pattern y1, y2, ...ynO.
In the training stage, the neural network uses input/output training sets to learn the functional mapping of the inputs to the outputs. Output training data is referred to the target output of the neural network. The goal is to train the network until the output of the neural network is close to the target output [29].
Training process goes until the output pattern is suitably close to the tar-get pattern which is achieved by minimizing the sum-of-squares error (SSE) with respect to weight vector w, given of the form;
E(w) = 1 2 N X n=1 c X k=1 (yk(xn; w) − tnk)2 (3.26) where
N= the number of training patterns c= the number of outputs
xn=input vector
tn
k=target value for output node k when the input vector is xn
Neural Network is trained by updating the weights using a backpropa-gation learning rule. Change in weight (w0
ji) is in the mth iteration is given
by:
w(m)ji = wji(m−1)+ ∆w(m)ji (3.27)
where
wji(m): the weight between the jth node of the output layer and the ith
node of the hidden layer in the mth training iteration.
wji(m−1): the weight between the jth node of the output layer and the ith
node of the hidden layer in the (m − 1)th training iteration.
∆w(m)ji : the weight adjustment Weight adjustment is given by
∆wji(m)= ηδj(m)o(m)i + α∆wji(m−1) (3.28)
η: learning rate,
δj(m): error signal of the jth node in the mth training iteration
o(m)i : output value of the ith node of the hidden layer in the mth iteration
α: momentum term, 0 < α < 1
If j is an output layer node, δj(m) is:
δj(m) = (t(m)j − y(m)j )˜g0(X
i
wji(m)o(m)i + w(m)jo ) (3.29)
where t(m)j : target value for output layer node j
yj(m): network output value of node j
wjo(m): weight between the oth node of the hidden layer and the jth node
of the output layer in the mth training iteration.
If j is a hidden layer node,then we have:
δj(m) = g0(X i
w(m)ji o(m)i + w(m)jo )X
k
δ(m)k w(m)kj (3.30) where g0 is the derivative of the hidden layer transfer function.
Using error backpropagation algorithm error for each node is calculated and the weights of all nodes are recursively updated starting from the output layer to the hidden layer.
Usually, the nodes in the hidden layer and in the output layer employ the same transfer function, for instance log-sigmoid function for the hidden layer nodes and linear function for the output layer nodes.
3.4
Case Study: Water-Tank System
3.4.1 Purpose and Method of the StudyGenerally, in closed-loop controlled systems, it is difficult or not possible to observe fault effect on the system, since the controller tolerates the faulty situation and attempts to bring the system to the desired operating point.
The main purpose of this study is to detect and identify the predefined faulty situations in the system. For this purpose the tank system is modeled using the qualitative techniques. Then the residuals obtained from the nor-mal mode and the faulty modes are classified using neural network classifiers.
3.4.2 Process Description
The process under investigation is a water-tank system obtained from MATLAB/SIMULINK environment. The aim of this process is to model the
water-tank system and simulate it in closed-loop such that water level is at the desired position indicated by a reference. For this reason a PID controlled valve is used which allows the entrance of water into the tank (Figure 3.7). Also there is another valve at the bottom of the tank to deplete the water in the tank.
Consider an open water tank with cross-sectional area A (see Figure 3.5). Water is pumped into the tank through the valve at the top, at rate of flow of qin cubic meters per second. Water is flowing out of the tank through a
hole in the bottom of the tank of area a. The rate of flow of water through the hole is according to the Bernoulli equation given by,
qout = a
p
2gh (3.31)
where h is level of tank and g is the acceleration of gravity. Conservation of mass yields,
Adh
dt = qin− qout= qin− a
p
2gh (3.32)
This relation shows the nonlinear behavior with dynamic characteristics of the system depending on the operating point. It depends on the direction of the valve position changes (opening or closing). Valve is driven by the PID controller which controls the difference between the reference flow rate and the actual flow rate. The system has two outputs which are water level and instantaneous flow rate. The system is shown in Figure 3.6. Also the dimensions of the tank are given in Table 2. Initial water level in the tank is 0.5 meter and the reference point is changed randomly with a sample time of 50 seconds.
The problem is to detect and diagnose the faults in closed-loop operation of the system. The investigated cases are :
qin
qout H
h
Figure 3.5: Water-Tank System Parameters
water level overflow flag flow out flow in Outlet Area sqrt(2gh) 1/area tank volume sum overflow sensor
Figure 3.6: Water-Tank System Simulation
Table 2: Water-Tank Parameters
Height 2 m
Bottom Area 1 m2
Outpipe Cross Section 0.01 m2
F0: Normal operating condition
F1: Restriction at the output valve of the tank
F2: Leakage on the wall of the tank at a specific height from the bottom. The faults affect the system in different ways. Generally the remarkable difference between the outputs of normal operating condition and neural network model which are called residuals indicate the fault alarm in the system.
3.4.3 Neural Network Process Model
As it is said in the previous section, residuals are necessary for the fault diagnosis process. Therefore, it is important to determine the current value of water level for normal operating conditions. A neural network can be used for this purpose. Analytically modeling is not preferred since the aim of this thesis is modeling of the automotive systems. Although these kind of systems are not complicated, their structure is commercially reserved.
The simulink model of the tank and controller is shown in Figure 3.7.
Random Number Saturation error Controller Tank Max Inflow Tank water level flow rate overflow Water Tank 1 2 3 Valve PID 0.5
Figure 3.7: Water-Tank System in Closed-Loop
Modeling is done by using the Neural Network Toolbox of MATLAB. Since the actual system has two outputs, model of the actual system will also have two outputs which are water level and water flow rate from the
output valve. Therefore two neural network models are designed for this sys-tem both of which are three-input, one-output including three layers: input layer, hidden layer and output layer. In the input layer three neurons and in the output layer one neuron is used. In the hidden layer 25 neurons are used for both of the systems. Knowledge of the actual system, extensive training of different combinations of input variables and network topologies are utilized to identify the input to the neural network. The inputs represent a trade off between performance of the neural network under normal and faulty conditions. Input and output variables for training are;
Inputs = u(i), y(i − 1), dydt|(i−1);
Output=y(i);
where i is the current discrete time value, (i-1) refers to the previous value; u is instantaneous entering water flow rate, y(i-1) and dy/dt|i−1 are
one sample time delayed water level and output flow rate of the water. The sampling interval is taken as 0.1 second.
One of the neural networks will model the water level whereas the second one will model the flow rate of the water. The scheme of the neural network model is given in Figure 3.8.
1 z 1 z water level water flow-rate Neural Network u(t) water level(t-1)
water flow rate(t-1)
Neural Network 3
3
3
The neural network is trained in a batch mode (offline) using experimen-tal data obtained as the system operates in normal mode. The system is simulated for 800 seconds. Therefore, the training data set with 8000 in-put/output data is used until the average error for each training pattern is approximately 10−4.
A tan-sigmoid activation function (2.37) is used for the hidden layer and a pure linear activation function (2.36) is used for the output layer. Matlab Neural Network Toolbox is utilized in the training process. The actual output vs neural network output of the system are plotted as a function of time in Figure 3.9 and Figure 3.10 .
w a ter lev el time actual output NN output 0 50 100 150 200 250 300 350 400 450 500 0 0.2 0.4 0.6 0.8 1 1.2 1.4
Figure 3.9: Neural Network Model Water Level vs Actual Water Level
3.4.4 Residual Generation Techniques
As it is mentioned, neural network model for the residual generation can be used for fault diagnosis purposes. Residuals are based on the comparison of features from the process with the nominal ones realized from the model.
time 0 50 100 150 200 250 300 350 400 450 500 w a ter flo w r a te 0 0.05 0.1 0.15 -0.1 -0.15 -0.2 -0.05 actual output NN output
Figure 3.10: Neural Network Model Flow Rate vs Actual Flow Rate
Model Plant Model MCL PCL Controller Controller y1 ^ y2 ^ y u u ^ W
Figure 3.11: Residual Generation Using Model of the System
Simulation is performed with model in closed loop (MCL) control which runs in paralel to the process in closed-loop (PCL) using the same reference signal (Figure 3.11). For this case, two types of residuals are generated:
1. Output Based Residual
These types of residuals can be derived in both closed-loop and open-loop operation, and they do not require process excitation. It is the output error
between the process and the model within a time window of appropriate length l: Sr = 1 l l X i=1 | ˆy1(k − i) − y(k − i)| (3.33)
For processes with multiple outputs, the residuals are decoupled from out-puts [22].
2. Signal Based Residual
The performance of the controller in constant operation regions and dur-ing the set point changes affects the system behavior. Therefore the symp-toms can be derived by defining different performance indices (CPI). In this case the difference between the control reference signal W (k) and the con-trolled variable y(k):
SCP I = ICP I − ˆICP I (3.34) = 1 l l X i=1 (W (k − i) − y(k − i))2−1 l l X i=1 (W (k − i) − ˆy2(k − i))2 (3.35)
In the constant operating regions SCP I is also affected from noise and
dis-turbances. Therefore, for comparable residuals these effects must be similar. The next step of fault diagnosis is defining the fault-residual relationships. This can be solved by prior knowledge or from experiments.
3.4.5 Relationship Between Residuals and Faults
In the system fuzzy logic and neural network methods are used to relate the faults to the residuals.
Application of Adaptive Neuro-Fuzzy Logic as Residual Evaluators
For the purpose of relating the residuals with faults fuzzy logic method is used. In this method, the membership functions for the residual are built and for every faulty condition a chain of rules is chosen. By the way a database
