View of Prediction of COVID Patients using Gated Recurrent Unit Neural Networks

(1)

Prediction of COVID Patients using Gated Recurrent Unit Neural Networks

Gyana Ranjan Patraa_{, and Mihir Narayan Mohanty}b

.

a _{Assistant Professor, Department of ECE, ITER, Siksha ‘O Anusandhan Deemed to be University, Bhubaneswar,} Odisha, India

b_{Professor Department of ECE, ITER, Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, Odisha, India}

Article History: Received: 10 January 2021; Revised: 12 February 2021; Accepted: 27 March 2021; Published online: 20 April 2021

__________________________________________________________________________________________________ Abstract: Since January 2020 the world has witnessed a new pandemic that has put the humanity at a grave risk. The COVID-19 disease originated from People's Republic of China has affected 215 nations and has put the life and economy to a standstill. In this scenario, it is very important to predict the patients affected by COVID, so that the administration and the health professionals can take suitable decisions regarding enforcing lockdown, creating isolation and medical facilities and other tasks. In this paper deep learning is utilized as a method of prediction of patients in five countries. Day wise prediction is performed for a week for evaluation that can be extended for more time. It is a type of time series prediction method using deep learning approach. Three different methods are used for prediction and have been found excellent result using the Gated Recurrent Units (GRUs). The results obtained by GRUs are very accurate and establish its supremacy over other networks. Thus, it can be used as a tool for prediction of number of COVID-19 patients by the administrators and health officials.

Keywords: Deep learning, time series, gated recurrent units, COVID-19, Long short-term memory

1. Introduction

The spread of the novel corona virus (SARS-CoV-2) has affected the lives and the economies of the world and has caused deaths of hundreds of thousands of people. This disease which is termed as the COVID-19, has been declared as a pandemic by the World Health Organization (WHO) (Wang, Li-sheng, 2020). The COVID-19 virus finds its origin in Wuhan, which is the capital of the Hubei providence in the People’s Republic of China during November 2019. Since then it has spread to 215 countries in the world and till the end of July 2020, the number of patients has risen to 17,188,007 with 670,207 deaths with the number of daily new cases rising to almost 300,000 (WHO, 2020; Worldometers, 2020). The top five worse affected countries are the United States of America (USA), Brazil, India, Russia and United Kingdom.

In this situation it is essential to predict the number of new patients on a daily or weekly basis. The proper prediction of this figure will allow various government officials to plan and prepare proper facilities and policies to curb the infection from large scale spreading.

Many studies for prediction of COVID-19 patients have been attempted in literature. A study has been made using linear regression, Multi-Layer Perceptron (MLP) and Vector autoregression which achieves results with 95% confidence interval (CI) (Sujath, R., 2020). A model using Machine Learning (ML) and cloud computing has been implemented using data driven techniques to achieve the results (Tuli, Shreshth, 2020). A model using discrete wavelet decomposition and autoregressive integrated moving average (ARIMA) has been implemented for forecasting casualties up to one month (Singh, Sarbjit, 2020). An improved time series model using the two-piece scale mixtures of normal (TP-SMN) with the auto regressive model has been implemented for the prediction of the spread and death rate (Maleki, Mohsen, 2020). A Generalized Additive Model (GAM)finds its application for the prediction of the transmission in Mainland China by taking temperature as well as humidity data in to account (Qi, Hongchao 2020). Forecasting of the COVID-19 has been attempted for predicting the number of patients for Indiausing Genetic Programming (Salgotra, 2020) which has a prediction window of 10 days. Tomar& Gupta (Tomar, Anuradha, 2020; Chimmula, 2020) and Reddy & Zhang (working independently) have used the Long Short-Term Memory (LSTM) network for effective prediction for India and Canada respectively. The maximum error achieved through the LSTM network is reported at about 8% . Similarly, the classification of infection probability of corona virus has been attempted in (Mohapatra, S. K., 2020).

In this paper, it is aimed to accurately predict the total number of patients in the top five worse affected countries, namely, USA, Brazil, India, Russia and United Kingdom. A comparison of three popular networks namely long short-term memory (LSTM), Bidirectional LSTM (BiLSTM) and Gated recurrent units (GRUs) have been made in conjunction with the prediction of COVID-19 data available in the public domain. The above-mentioned networks have some attractive qualities like good handling of temporal dependencies of time series data, learning models that are distribution free and the inherent flexibility of encompassing the nonlinear features. These models have been implemented using the data available from WHO website.

(2)

Turkish Journal of Computer and Mathematics Education Vol.12 No.9 (2021), 2333– 2340

Research Article

2. Deep Learning Models

Deep learning (DL) methods have been successfully used in many applications where they have demonstrated marked improvement on various performance measures. Some of the recent applications include arrhythmia classification and speech enhancement. In this section, the working of three popular DL methods namely LSTM, BiLSTM and the GRU are explained. These networks will be later used for the prediction of the total number of patients of five different countries. The Fig.1 shows the structure of the three DL models used in this paper.

2.1 LSTM models

The regular Artificial Neural Networks suffer from time-dependent learning problemsand to avoid the same recurrent neural networks (RNNs) have been developed (Hochreiter, S., 2016). But the RNNs are associated with the vanishing gradient problems which results in noticeable drop in the efficiency. The LSTM network is a sophisticated network associated with a gated memory unit. The LSTM contains three gates known as the input, forget, and output gates which control the flow of information. These gates contain logistic functions of weighted sums, where the weights are updated by the backpropagation algorithm. The input and the output gates are used to manage the cell state. The output gate which is also referred as the hidden state and represents the memory directed to use. Such a mechanism allows the LSTM model to memorize information for a long time which is missing in the simple RNN. This mechanism helps the LSTM to achieve the desirable characteristics for capturing long-term dependencies which makes it a formidable network to handle time-series data.

The LSTM network can be characterized by the following equations. Given the input time-series 𝑋𝑡 and the number of units as ℎ, the gates have the following equations.

• Input gate : 𝐼𝑡= 𝜎(𝑥𝑡𝑊𝑥𝑖+ 𝐻𝑡−1𝑊ℎ𝑖+ 𝑏𝑖) _{Eqn. (1)}

• Forget Gate : 𝐹𝑡= 𝜎(𝑥𝑡𝑊𝑥𝑓+ 𝐻𝑡−1𝑊ℎ𝑓+ 𝑏𝑓) _{Eqn. (2)}

• Output Gate : 𝑂𝑡= 𝜎(𝑥𝑡𝑊𝑥𝑜+ 𝐻𝑡−1𝑊ℎ𝑜+ 𝑏𝑜) _{Eqn. (3)}

• Intermediate Cell State : 𝐶̃𝑡= tanh(𝑥𝑡𝑊𝑥𝑐+ 𝐻𝑡−1𝑊ℎ𝑐+ 𝑏𝑐) _{Eqn. (4)} • Cell State (next memory input) : 𝐶𝑡= 𝐹𝑡∘ 𝐶𝑡−1∘ 𝐶̃𝑡 Eqn. (5)

• New State : 𝐻𝑡= 𝑂𝑡∘ tanh(𝐶𝑡) Eqn. (6)

(3)

(c)

Fig.1. Schematic representation of (a) LSTM architecture (b) Bidirectional LSTM architecture and (c) GRU architecture (Image courtesey [15])

2.2 Bidirectional LSTM

The LSTM reconstructs the current state by taking the backward context only. In many applications the forward context also has some effect on the current state. In the BiLSTM model has been constructed by considering this. The BiLSTM is a manifestation of the bi-directional RNN (Schuster, M., 1997) and the LSTM networks. This is achieved by combining the two hidden states and thereby allowing information flow from both forward and backward layers (Zeroual, 2018). The BiLSTM networks have been used in many applications where it has shown good performances.

2.3 GRU models

GRU models have been proposed as an alternative and improved version of LSTM (Cho, K., 2014). The GRU is a more efficient network as it contains a smaller number of parameters and a simpler design than the LSTM model. The GRU model contains a single gate called as an update gate which is derived by merging the input gateand the forget gateof the LSTM. Thus, the GRU model contains only two gates namely the update gate and the reset gate and this gives the model the ability to calculate the hidden states in RNN models. The update gate delivers the quantity of memory that has been previously stored and the reset gate makes it possible to provide way ofblending between the new and the previously stored memory units. The GRU model exhibits the following mathematical relationships.

Update gate

:

𝑍

𝑡

= 𝜎(𝑥

𝑡

𝑊

𝑥𝑧

+ 𝐻

𝑡−1

𝑊

ℎ𝑧

+ 𝑏

𝑧

)

_{Eqn. (7)}

Reset Gate

:

𝑅

𝑡

= 𝜎(𝑥

𝑡

𝑊

𝑥𝑟

+ 𝐻

𝑡−1

𝑊

ℎ𝑟

+ 𝑏

𝑟

)

_{Eqn. (8)}

Cell state

:

𝐻

̃

𝑡

= tanh(𝑥

𝑡

𝑊

𝑥ℎ

+ (𝑅

𝑡

∘ 𝐻

𝑡−1

)𝑊

ℎℎ

+ 𝑏

ℎ

)

_{Eqn. (9)}

New state

:

𝐻

𝑡

= 𝑍

𝑡

∘ 𝐻

𝑡−1

+ (1 − 𝑍

𝑡

) ∘ 𝐻

̃

𝑡

_{Eqn. (10)}

where,

𝑊

𝑥𝑟

,

𝑊

𝑥𝑧

,

𝑊

ℎ𝑟

𝑊

𝑥ℎ

, 𝑊

ℎℎ

and

𝑏

𝑟

, 𝑏

𝑧

are weight and bias parameters respectively for

different gates. In this network for the

𝑡

𝑡ℎ

_{time step, the current update gate}

_𝑍

𝑡

is combines the

previous hidden state 𝐻

𝑡−1

and current candidate hidden state 𝐻

̃

𝑡

to generate the new state𝐻

𝑡

.

3 Results and Observations 3.1 Data description

The COVID-19 data has been obtained from the online database maintained by WHO (https://covid19.who.int/). The main objective here is to achieve correct predictionof the total number of reported cases in the five countries with a forecast window of 7 days. The chosen countries are USA, Brazil, India, Russia and United Kingdom. The data is presented with Day 0 being the date of detection of the first patient in that country. The 7-day forecast window refers to the days from 1st_{to 7}th_{February 2021 for all the countries.}

(4)

Research Article

3.2 Evaluation metrics

Three performance measure parameters were used to test the DL forecasting models. They are the 𝑅-Value, Normalized Root Mean Square Error (𝑁𝑅𝑀𝑆𝐸), mean absolute percentage error (𝑀𝐴𝑃𝐸).

𝑅 𝑣𝑎𝑙𝑢𝑒 =𝑐𝑜𝑣(𝑦𝑡, 𝑦̂ )𝑡 𝜎𝑦_𝑡− 𝜎𝑦̂_𝑡 Eqn. (11) 𝑁𝑅𝑀𝑆𝐸 = √1 𝑛∑ ( (𝑦𝑡− 𝑦𝑡̂ )2 max(𝑦𝑡)) 𝑛 𝑡=1 Eqn. (12) 𝑀𝐴𝑃𝐸 =100 𝑛 ∑ ( (𝑦𝑡− 𝑦𝑡̂ )2 𝑦𝑡 ) 𝑛 𝑡=1 % Eqn. (13)

For better results the 𝑅 value should be very close to 1 and 𝑁𝑅𝑀𝑆𝐸 as well as 𝑀𝐴𝑃𝐸 values should be small positive numbers closing on zero.

3.3 Experimental setup

For all the countries the dataset contains different number of entries as the first data corresponds to the date of the detection of the first patient. In all cases the last 7 entries were taken as test data and the rest was taken as training data. The simulation was done in MATLAB 2020a running on a Core i3 machine with 8 GB RAM without any GPU. The different characteristics of the networks are provided in the Table 1. The three networks were trained and tested for 30 runs with 250 epochs for each run. The training and test results obtained are tabulated in the Table 2.

Table 1. Parameter setup for the networks

Parameters GRU BiLSTM LSTM

Stages 1. Sequence Input Layer 2. GRU Layer 3. Fully connected layer 4. Regression layer 1. Sequence Input Layer 2. BiLSTM Layer 3. Fully connected layer 4. Regression layer 1. Sequence Input Layer 2. LSTM Layer 3. Fully connected layer 4. Regression layer Number of hidden units 400 400 400 Training Algorithm

ADAM ADAM ADAM

Max Epochs 250 250 250

Number of runs

30 30 30

3.4 Observations

From the Table 2, we observe that the GRU achieves an𝑅 value of close to 1 in all five cases followed by the LSTM and the BiLSTM networks. The 𝑁𝑅𝑀𝑆𝐸 and𝑀𝐴𝑃𝐸 values of the GRU model is the lowest amongst the three networks considered in this paper. The 𝑀𝐴𝑃𝐸 values range within 1.31% to 2.5% for GRU model and its values range within 4.00 % to 4.88% for BiLSTM and 2.76 % to 12.00% for the LSTM networks. The best values

(5)

Research Article

A comparison of the average training time taken by the models is presented in Table 3. It can be observed that the GRU is the fastest amongst the other two networks. The GRU can be trained in approximately half the time taken by LSTM and one fourth of the time taken by BiLSTM.

The prediction for the 7-day window has been presented in the Table 4. It can be observed that the GRU model closely follows the actual number of cases than the BiLSTM and the LSTM networks. The maximum error shown by the GRU model is about 3.27% whereas the errors in BiLSTM model vary between 2.00% to 18.84% and the same for the LSTM network varies between 1.05% to 18.15%. In all possible cases the GRU produces less error of prediction than the other two models

These results prove that the GRU is effectively better in prediction the total number of COVID-19 patients than the other two algorithms.

Table 2. Performance measures for different networks

Country Performance Measures GRU BiLSTM LSTM

USA

R Value 0.9997 0.9972 0.9982

Normalized RMS Error 0.1539 0.5161 0.4109

Mean Absolute Percentage Error 1.3127 4.8895 4.0650

Brazil

R Value 0.9998 0.9978 0.9987

India

R Value 0.9998 0.9886 0.9825

Russia

R Value 0.9999 0.9990 0.9951

United Kingdom

R Value 0.9993 0.9971 0.9981

Table 3. Average time taken for training the different models (250 epochs in each case)

Country

Average training time

GRU

BiLSTM

LSTM

USA

1min 45sec

8min 23sec

4min 06sec

Brazil

1 min 30sec

6min 31sec

3min 10sec

India

1 min 58sec

8min 11sec

3min 49sec

Russia

1 min 27sec

7min 36sec

3min 36sec

United

(6)

Research Article

Day Actual GRU %ge

Error BiLSTM %ge Error LSTM %ge Error 1 25817939 25793180 0.0959 25145304 2.6053 2508329 0 2.8455 2 25930068 25813564 0.4493 25077591 3.2876 2507564 6 3.2951 3 26055512 25806265 0.9566 25066679 3.7951 2514627 9 3.4896 4 26172274 25804737 1.4043 24979918 4.5558 2515514 1 3.8863 5 26293150 25836569 1.7365 24833170 5.5527 2512633 9 4.4377 6 26418016 25865695 2.0907 24665154 6.6351 2510168 6 4.9827 7 26547977 25896012 2.4558 24478589 7.7949 2508300 7 5.5182 BRAZIL

Error BiLSTM %ge Error LSTM %ge Error 1 9176975 9147627 0.3198 8992747 2.0075 9080167 1.0549 2 9204731 9122956 0.8884 9007050 2.1476 9121649 0.9026 3 9229322 9081607 1.6005 9038847 2.0638 9159373 0.7579 4 9283418 9162195 1.3058 9010903 2.9355 9139869 1.5463 5 9339420 9320312 0.2046 8882657 4.8907 9011466 3.5115 6 9396293 9274470 1.2965 8796997 6.378 8917495 5.0956 7 9447165 9241859 2.1732 8725827 7.6355 8833779 6.4928 INDIA

Error BiLSTM %ge Error LSTM %ge Error 1 10757610 10703209 0.5057 10424802 3.0937 1007232 9 6.3702 2 10766245 10725075 0.3824 10327187 4.0781 9933627 7.7336 3 10777284 10746310 0.2874 10608986 1.5616 9748754 9.5435 4 10790183 10710746 0.7362 10416238 3.4656 9510877 11.8562 5 10802591 10657750 1.3408 9815461 9.1379 9273020 14.1593 6 10814304 10620652 1.7907 9263273 14.3424 9059034 16.2311 7 10826363 10597429 2.1146 8786481 18.8418 8860869 18.1547 RUSSIA

(7)

Table 4. Prediction of different models in the 7-day forecast window 4. Conclusion

In this study the GRU model has been used to effective prediction of the number of COVID-19

patients in the top five worse affected countries with a 7-day window. The results obtained from the

GRU model has outshined the BiLSTM and LSTM models in terms of 𝑅 value, 𝑁𝑅𝑀𝑆𝐸 and 𝑀𝐴𝑃𝐸

values. GRU model is seen to be approximately two times faster than the LSTM and four times faster

than the BiLSTM models. Thus, it can be concluded that the GRU model can be used as an effective

prediction tool for the forecasting of the number of COVID-19 patients.

References

1. Aradhyula, T.V., Bian, D., Reddy, A.B., Jeng, Y.R., Chavali, M., Sadiku, E.R. and Malkapuram, R., 2020. Compounding and the mechanical properties of catla fish scales reinforced-polypropylene composite–from biowaste to biomaterial. Advanced Composite Materials, 29(2), pp.115-128.

2. Arunkarthikeyan K., Balamurugan K. & Rao P.M.V (2020) Studies on cryogenically treated WC-Co insert at different soaking conditions, Materials and Manufacturing Processes, 35:5, 545-555, DOI: 10.1080/10426914.2020.1726945

3. Chimmula, Vinay Kumar Reddy, Zhang, Lei, (2020)Time series forecasting of COVID-19 transmission in

Canada using LSTM networks, Chaos, Solitons & Fractals, Volume 135

4. Cho, K., Merriënboer, B. Van, Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., et al. (2014)Learning

phrase representations using RNN encoder-decoder for statistical machine translation, arXiv:14061078. 5. Garikipati, P. and Balamurugan, K., 2021. Abrasive Water Jet Machining Studies on AlSi 7+ 63% SiC

Hybrid Composite. In Advances in Industrial Automation and Smart Manufacturing (pp. 743-751). Springer, Singapore.

6. Graves, A., Schmidhuber, J., Framewise phoneme. (2005) classification with bidirectional LSTM and

other neural

network architectures, Neural Netw, 18 (5–6), pp.

7. Hochreiter, S., Schmidhuber, J.,(1997)Long short-term memory, Neural Comput, 9 (8), pp. 1735-1780 8. Ram, R., &Mohanty, M. N. (2019). Deep Neural Network based Speech Enhancement. Cognitive

Informatics and

Soft Computing (pp. 281-287). Springer, Singapore.

2 3884730 3869918 0.3813 3805353 2.0433 3686104 5.1131 3 3901204 3879701 0.5512 3797986 2.6458 3664779 6.0603 4 3917918 3890453 0.7011 3796239 3.1057 3646512 6.9273 5 3934606 3900749 0.8605 3798252 3.4655 3632212 7.6855 6 3951233 3910480 1.0314 3798921 3.8548 3620163 8.3789 7 3967281 3919765 1.1977 3798362 4.2578 3608762 9.0369 UNITED KINGDOM

Error BiLSTM %ge Error LSTM %ge Error 1 3817180 3812706 0.1172 3749242 1.7798 3731072 2.2558 2 3835787 3785124 1.3208 3761285 1.9423 3758484 2.0153 3 3852627 3744561 2.8050 3775582 1.9998 3783241 1.8010 4 3871829 3758597 2.9245 3734584 3.5447 3755697 2.9994 5 3892463 3773661 3.0521 3689284 5.2198 3726216 4.2710 6 3911577 3795759 2.9609 3630417 7.1879 3684080 5.8160 7 3929839 3801204 3.2733 3582182 8.8466 3652726 7.0515

(8)

Research Article

9. Maleki, Mohsen, Mahmoudi, Mohammad Reza, Heydari, Mohammad Hossein, Pho, Kim-Hung,(2020)Modeling and forecasting the spread and death rate of coronavirus (COVID-19) in the world using time series models, Chaos, Solitons& Fractals, Volume 140.

10. Qi, Hongchao, Xiao, Shuang, Shi, Runye, Ward, Michael P., Chen, Yue, Tu, Wei, Su, Qing, Wang,

Wenge, Wang,

Xinyi, Zhang, Zhijie, (2020) “COVID-19 transmission in Mainland China is associated with temperature and

humidity: A time-series analysis”, Science of The Total Environment, Volume 728.

11. Schuster, M., Paliwal, K. K.,(1997)Bidirectional recurrent neural networks, IEEE Trans Signal Process, 45

(11),pp. 2673-2681.

12. Sujath, R., Chatterjee, J.M. &Hassanien, A.E. A machine learning forecasting model for COVID-19

pandemic in

India. Stoch Environ Res Risk Assess 34, 959–972 (2020).

13. Singh, Sarbjit, Parmar, Kulwinder Singh, Kumar, Jatinder, Makkhan, Sidhu Jitendra Singh,(2020)Development of new hybrid model of discrete wavelet decomposition and autoregressive integrated moving average (ARIMA) modelin application to one month forecast the casualties cases of COVID-19, Chaos, Solitons & Fractals, Volume 135.

14. Srivatsav, V., Ravishankar, C., Ramakarishna, M., Jyothi, Y. and Bhanuparakash, T.N., 2018, August. Mechanical and thermal properties of chicken feather reinforced epoxy composite. In AIP Conference

Proceedings (Vol. 1992, No. 1, p. 040034). AIP Publishing LLC.

15. Tomar, Anuradha, Gupta, Neeraj, (2020) “Prediction for the spread of COVID-19 in India and effectiveness of preventivemeasures, Science of the Total Environment 728”.

16. Tuli, Shreshth, Tuli, Shikhar, Tuli, Rakesh, Gill, Sukhpal Singh, (2020) Predicting the growth and trend of COVID-19pandemic using machine learning and cloud computing, Internet of Things, Volume 11, 17. Wang, Li-sheng, Wang, Yi-ru, Ye, Da-wei, Liu, Qing-quan, 2020b.(2020)”A review of the 2019 Novel

Coronavirus(COVID-19) based on current evidence”. Int. J. Antimicrob. Agents, Volume 55, Issue 6. 18. WHO, 2020 https://www.who.int/emergencies/diseases/novel-coronavirus-2019/advice-for-public.

(Accessed on 30/07/2020)

19. Worldometers, 2020. https://www.worldometers.info/coronavirus/. (Accessed on 30/07/2020)

20. Yookesh, T.L., Boobalan, E.D. and Latchoumi, T.P., 2020, March. Variational Iteration Method to Deal with Time Delay Differential Equations under Uncertainty Conditions. In 2020 International Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 252-256). IEEE

21. Zeroual, Abdelhafid, Harrou, Fouzi, Dairi, Abdelkader, Sun, Ying,(2020)”Deep learning methods for forecasting COVID-19 time-Series data: A comparative study,Chaos”, Solitons & Fractals,Volume 140.