• Users Online: 5906
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2021  |  Volume : 6  |  Issue : 5  |  Page : 76-82

Automated atrial fibrillation prediction using a hybrid long short-term memory network with enhanced whale optimization algorithm on electrocardiogram datasets


1 CEO, Vee Technologies; Vice-Chairman, Sona Group of Institutions, Bengaluru, Karnataka, India
2 Department of Computer Science and Engineering, Sona College of Technology, Salem, Tamil Nadu, India
3 Department of Knowledge Transfer and Valourization, Sona College of Technology, Salem, Tamil Nadu, India

Date of Submission30-Jul-2021
Date of Decision07-Sep-2021
Date of Acceptance24-Sep-2021
Date of Web Publication19-Nov-2021

Correspondence Address:
Revathi Thavamani Kalyanasundaram
Department of Computer Science and Engineering, Sona College of Technology, Salem- 636 005, Tamil Nadu
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2468-8827.330654

Rights and Permissions
  Abstract 


Background: Cardiac arrhythmias are one of the leading causes of heart failure. In particular, atrial fibrillation (AFib) is a kind of arrhythmia that can lead to heart stroke and myocardial infarction. It is very important and crucial to predict AFib at an early stage to prevent heart disease. Electrocardiogram is one of the premium diagnostic tools which is used by most of the researchers for predicting irregular heartbeats. There are many works carried out in finding heart disease using machine learning classifiers. Aims and Objectives: Deep learning based hybrid Long Short Term Memory (LSTM) network is hybridized with Enhanced Whale Optimization (EWO) to minimize the network optimization and configuration issues faced in the existing models and proposed to increases the accuracy of predicting AFib. Materials and Methods: The proposed LSTM network is hybridized with a EWO technique for predicting AFib. This study uses a hybrid LSTM EWO network for classifying the various output labels of heart disease. EWO is used to predict the most relevant features from the raw dataset. Then, the LSTM model is used to predict the AFib of a patient from normal ECG data. Results: The DL based LSTM EWO achieves better results in all the performance metrics by analyzing the optimized features in feature space, training, and testing phase and successfully obtains better performance in an effective manner. LSTM improves the accuracy by reducing the number of units in the hidden layer which optimizes the network configuration. The proposed model achieves 96.12% accuracy which is 12.81% higher than RF, 15.01% higher than GB, 28.04% higher than CART, and 16.92% higher than SVM. Conclusion: The proposed model hybrid LSTM network integrated EWO for predicting the AFib. The EWO is applied for selecting the most appropriate features needed for the model to learn and produce improvised performance. The optimization and network configuration problems faced in the existing studies are avoided by choosing the suitable number of LSTM units and the size of the time window. This has been implemented through LSTM units and their window size. In addition, we made a statistical examination to prove the importance of proposed work against other models. It is observed that the experimental results attained with 96% of accuracy, better than conventional models.

Keywords: Arrhythmia, atrial fibrillation, deep learning, electrocardiogram, long short-term memory network, whale optimization


How to cite this article:
Valliappa C, Kalyanasundaram RT, Balasubramaniam S, Sennan S, Sampath Kumar NK. Automated atrial fibrillation prediction using a hybrid long short-term memory network with enhanced whale optimization algorithm on electrocardiogram datasets. Int J Non-Commun Dis 2021;6, Suppl S1:76-82

How to cite this URL:
Valliappa C, Kalyanasundaram RT, Balasubramaniam S, Sennan S, Sampath Kumar NK. Automated atrial fibrillation prediction using a hybrid long short-term memory network with enhanced whale optimization algorithm on electrocardiogram datasets. Int J Non-Commun Dis [serial online] 2021 [cited 2023 Jan 27];6, Suppl S1:76-82. Available from: https://www.ijncd.org/text.asp?2021/6/5/76/330654




  Introduction Top


The World Health Organization says in a report that cardiovascular disease is the leading cause of death worldwide, with an estimated 17.9 million people dying every year.[1] Irregular heartbeat is the primary cause of heart disease. There are many kinds of irregular beats that can cause heart disease. Atrial fibrillation (AFib) is an irregular heart rhythm and is a type of arrhythmia.[2] It can interrupt the normal flow of blood in the body and increase the risk of blood clots and stroke. AFib is the most vigorous arrhythmia connected closely with death rate. It is the primary cause of affecting initial cardio strokes, thromboembolic events, and even leads to myocardial infarction.[3] In general, heart disease affects males, who are highly vulnerable compared to females.[4] It is predicted to affect 40–50 million around the globe, at the end of the year 2050.[5]

Electrocardiogram (ECG) signal is the primary tool used by most of the researchers for predicting heart disease. The classification of ECG signal into different arrhythmia categories is very difficult since there are issues with the pattern recognition task. The examination of ECG signals is the best way to predict various arrhythmias diagnosing heart disease. Automatic ECG arrhythmia prediction can provide improved accuracy and provides a better solution for cardiac abnormalities mass screening at an affordable cost.[6],[7]

Machine learning (ML) techniques are largely used by researchers since they provide effective results after training the model with the right dataset.[8],[9],[11] Although ML methods are effective in producing results, integrating the selected features with the suitable ML algorithm is a difficult process according to researchers.[12] In addition, the performance of the system gets degraded due to missing values and uncertainty.[13],[14] Scalable methods are needed to process the enormous amount of gathered health-care data.[15],[16]

Deep learning (DL) is a subset of ML and it has been widely used in several fields for successful analysis. Furthermore, DL techniques are playing a major role in the medical health-care field for data discovery and are used mainly for classifying diseases such as diabetes, heart disease, and brain-related diseases.[17],[18] In this study, we propose a DL-based long short-term memory (LSTM) network hybridized with an enhanced whale optimization (EWO) technique for predicting AFib. LSTM has been chosen because it memorizes the dependencies for a longer period so that network configuration issues are eliminated and scalability of the model in diagnosing AFib is enhanced.

Related work

AFib is the most frequently encountered arrhythmia in clinical practice. The global prevalence rate of AFib cases has increased in recent times due to lifestyle practices. AFib is associated with an increase in morbidity. AFib is an adverse event which causes heart disease. This risk factor is generally identified through ECG signals which may have noises present in them. It is important to remove them from raw signals, as only then, the prediction will become accurate. Fourier transform (FT) is a transformation function that is used to divide the function based on time into the function based on temporal frequencies. FT is not capable of analyzing ECG signals, as it fails to provide useful information relevant to the time of event of frequency components. FT is not effective in decomposing short interval signals in ECG. The wavelet transform (WT) overcomes this issue by decomposing shorter waves into various sets of wavelets.[19]

Raj et al. used WT for longer windows and shorter windows in the ECG signal analysis. The RR interval is a kind of dynamic feature which is calculated and integrated to the morphological features to establish the final feature set. These features have been exploited in the model to classify the ECG signals with support vector machine (SVM).[20]

Hosseini et al. have proposed a two-stage feedforward neural network. They have used two network architectures based on single-stage and two-stage feedforward to recognize abnormalities. It uses compressed component methods for effective ECG signal classification.[21] Wang and Chang used SVM for the classification of ECG data and they have compared decision tree classifier, genetic algorithm, and DL model and diagnosed congestive heart failure from ECG signals.[22] Singular value decomposition is a method used to predict the best outcome by finding the optimal set of factors. It is used to extract the features of physiological signals and SVM has focused on diagnosing different physiological states of the patients. They used only relevant and significant features to classify physiological states which produced better results.

Khorrami and Moavenian have extracted RR intervals using discrete wavelet transform for feature selection. Multilayer perceptron neural network (MLPNN) and SVM methodologies are used for classifying the features which resulted in the prediction of heart disease.[23] Training performance is improved at least four times because of MLPNN and SVM. They have selected the best feature extraction method which has resulted in less computation time, training, and testing performance.

Vishwa et al. presented an automated artificial neural network for cardiac arrhythmia using multi-channel ECG recordings. They designed a better diagnostic decision support system based on arrhythmia classification output. These networks models are trained and tested using MIT-BIH arrhythmia dataset.[24] Classifier performance is evaluated using sensitivity, specificity, classification accuracy, mean squared error, receiver operating characteristics, and area under the curve.

From the research literature, it is observed that the models designed were not focused on solving network optimization problems and failed to memorize the dependencies for a long period. In addition, the models proposed by the researchers are unsuccessful in providing the reliability to assess sensitive medical data. In this perspective, we would be interested in designing an efficient DL model in implementing cardiology-related problems and diagnosing risk factors of arrhythmia embedded with an optimization approach. This paper presents an effective automated LSTM-EWO which has been tested and compared with the most common traditional ECG analyzers on an appropriate database.


  Materials and Methods Top


In this section, we describe how our proposed LSTM network is hybridized with an EWO technique for predicting AFib. This study uses a hybrid LSTM-EWO network for classifying the various output labels of heart disease. EWO is used to predict the most relevant features from the raw dataset. Then, the LSTM model is used to predict the AFib of a patient from normal ECG data.

Enhanced whale optimization algorithm

This is the first step of the proposed study. This algorithm is used to extract the most relevant features used for predicting the AFib. The general whale optimization algorithm was introduced in the year 2016 by Mirjalili and Lewis. It is a kind of meta-heuristic algorithm. It works on the principle of simulating the hunting behavior of whales.[25] From the hunting actions of humpback whales, the EWO algorithm can be portrayed conceptually as linking three types of strategies. They are:

  1. Encircling prey
  2. Bubble net attack technique and
  3. Prey searching.


Encircling prey

Every humpback whale symbolizes a search agent, and locations of the whales the search space in the two-dimensional work unit. Updating the position of the prey in the search space decides the optimal solution. The mathematical model of this behavior is shown below





Where,

it represents the current iteration

represents vector position of the current iteration

indicates the position details of the optimized solution

and represent the coefficient vectors.







Where,

represents convergence vector. This vector reduces linearly from 2 to 0 and improves the iterations count and is denoted as a random vector with the range (0,1).

Bubble net attack technique

This is categorized into two parts. First, the shrinking and encircling process of the prey is defined. Second, the curved growing encirclement and suppression (of the prey by the whale) phase is applied. To complete the shrinking and encircling processes, the value of must be reduced sequentially. When the value of coefficient vector becomes lesser than 1, i.e. ≤1 then, the agent's new location is updated using equation 2. The spiral position resembles the actions of whale moving toward its prey in the spiral motion and is as follows,



where, denotes the distance between the agent to the present optimal solution. s is a variable used to describe the shape of the spiral and n indicates number randomly from (−1,1). In general, a whale hunts its prey with the help of both above-said mechanisms at the same time. To replicate this concurrent behavior, EWO fixes the same chances to decide the above-mentioned two mechanisms. This helps in updating the locations of the whale during the optimization process. It can be described mathematically as follows:



Prey searching

In this optimization algorithm, it is significant to balance both the utilization and investigation steps. In this EWO algorithm, by using the size of vector balancing exploration and exploitation is done. When , it estimates the optimum position and removes the local minima. It is performed by selecting the opted search agent randomly. This is the way in which whale searches the prey to update its position. The prey searching is as follows.





Where, denoted as position's random vector. The EWO Algorithm is given below:

Algorithm 1: Enhanced whale optimization

Whale's Population Initialization: Pi (i = 1, 2, 3, 4…., n)

Calculation of fitness Function: For each agent, compute the fitness value

denotes the optimum agent

while (it < iterations )

for all agents

control parameters have to be updated

if parameter <0.5



Current whale location must be updated by,

Choose the whale randomly called,

Current whale location must be updated by,

endif

elsif parameter ≥ 0.5

Current whale location must be updated by,

endif

Endfor

verify any whale goes out of the search area and modify it

Compute the fitness function value of every new whale

Update P* when the best optimal solution found

it=it+1

End while

Return the value of P*

Long short-term memory networks

The selected and optimized features are fed into the proposed LSTM model. LSTM is a variety of recursive neural networks (RNN) and it is applicable to design networks to analyze the data flow and is used to organize the hidden layers for learning the details about previous input layer. However, RNN has an issue in preserving the details for a prolonged period. To overcome this issue, LSTM is applied in recognizing and preserving the dependencies among input time-series data and nonlinear data to be in dynamic nature for a prolonged period. LSTM network consists of memory cell, an input, an output, and forget gates. All these gates are multiplicative in nature. There must be regular associations among cells and the gates. LSTM involves operations with the cells constantly. The cell conveys state value over a random time gap. All the gates conduct three different operations such as read, write, and reset for the cells.[26],[25],[26],[27],[28] The functioning of an LSTM is illustrated with the following equations:













The forget gate is used in removing redundant data from the cell state. It accepts two inputs, the hidden state of the previous cell, hstatet-1 and the present cell's input xc. These input values are multiplied with weight matrix,Wm. This result is added with the bias factor biasf. This is done using the equation (10). Next, we use a sigmoid function which gives the vector of values range from 0 to 1 and it is given to ot. The cell state is assigned with these calculated values.

The purpose of using this particular function is to decide the suitable values retained in the cell state and irrelevant values to be removed from the cell state. If the cell state is assigned with the value 0, then the input gate will make the network to forget the cell. In the same way if initialized with value 1, then the gate requests for learning the cell state. This process is done in equation (10) for the whole output. To add any new data to the cell state, we have to use the input gate. It is also getting the same two inputs as arguments hstatet-1 and xc . There are two phases through which we can add new data into the cell state.

First phase: Both the arguments mentioned above are multiplied with Wn a weighted matrix and then added with biasn a new kind of bias factor. Consecutively, σ is applied to compute the output of the input's gate which is done in the equation (11).

Second Phase: The arguments hstatet-1 are multiplied with Wi and added with biasi. The tanh function is applied which produces the result values between the range − 1 and 1. This phase is done in equation (12) and forms cellt. This possesses the entire candidate's new value that is ready to be added in the cell.

In the same way, the output of the output's gate ogate is computed using the equation (14) and the value of hstate is also computed using the equation (15). The outcome of this LSTM is to remember and recognize the dependencies for a prolonged period.

The proposed LSTM-EWOA contains an LSTM layer, 1 flatter unit for creating one-dimensional output, 1 fully connected unit, 1 drop out unit, and finally, 1 output layer. The output layer is classified multi-class for each ECG input beat. This study uses the ReLU function that is to be applied to the fully connected unit and the softmax function is applied to the output unit layer. In general, softmax function is preferred over the multi-class classification.





Where, zv is a vector which contains r real numbers as values. A 10-fold cross-validation is applied to assess the model. The batch size of the network is allotted to be 128 and the rate of learning is set to 0.001. [Figure 1] below depicts the workflow of the proposed model.
Figure 1: Process flow diagram of long short-term memory with enhanced whale optimization

Click here to view



  Experimental results Top


Dataset

In this research, we applied the proposed LSTM-EWO model with ECG signals dataset gathered from the MIT-BIH AFib. This dataset consists of only the features of AFib of patients with 23 long-term and 2 led ECG recordings. All these signals are sampled up to the frequency of 250Hz. It also provides additional details of annotated files about 4 different types of ECG rhythms. These annotations are Named Normal beat (N), Atrial Fibrillation (AFib), Atrial Flutter (AFL), and AtrioVentricular junctional rhythm (AV junctional rhythm), as shown in [Table 1].
Table 1: Total number of beats in each class

Click here to view



  Results and Discussion Top


The hybrid LSTM-EWO, a kind of DL network, is designed and implemented for the automated prediction of AFib in a Python 3.6 environment. It uses Keras, a DL tool for execution. TensorFlow 2.0 supports the Keras library in its backend. All experiments are carried out in a desktop computer configured with an Intel Core i5-9600 CPU. A 16 GB RAM memory is used for storage and an 8GB NVIDIA GeForce RTX 2070 GPU for processing the input data.

Performance metrics for model evaluation

To effectively assess the proposed network's performance, various indices are applied, which include accuracy, precision, F1 score, and sensitivity. Accuracy is represented as the rate of right classifications. Precision is calculated to find the ratio between right positive predictions and total number of positive outcomes predicted. In the same way, sensitivity is meant for determining the proportion of the number of exact true positives to the total number of actual positive cases. Then, F1-score is for finding the average among precision and sensitivity. The mathematical notations of these metrics are as follows:











Where,

TP represents true positive

TN represents true negative

FP denotes false positive

FN denotes false negative.

Performance analysis

In this section, the experimental outcomes and the predicted accuracy attained by other models and methods used by other researchers are compared with the proposed LSTM-EWO. In the proposed work, the experiments are conducted to assess the performance effectiveness of the proposed model. [Table 2] shows the performance of various other related models with the proposed LSTM-EWO model.
Table 2: Performance comparison of the proposed model with various other related models

Click here to view


[Figure 2] depicts the overall performance comparison of the proposed LSTM-EWO with SVM, CART, GB, and RF. The DL-based LSTM-EWO achieves better results in all the performance metrics by analyzing the optimized features in feature space, training, and testing phase and successfully obtains better performance in an effective manner. LSTM improves the accuracy by reducing the number of units in the hidden layer which optimizes the network configuration. The proposed model achieves 96.12% accuracy which is 12.81% higher than RF, 15.01% higher than GB, 28.04% higher than CART, and 16.92% higher than SVM.
Figure 2: Performance comparison of the proposed long short-term memory with enhanced whale optimization model with other models

Click here to view



  Conclusion Top


We proposed a new model based on a hybrid LSTM network integrated EWO for predicting the AFib. The EWO is applied for selecting the most appropriate features needed for the model to learn and produce improvised performance. The optimization and network configuration problems faced in the existing studies are avoided by choosing the suitable number of LSTM units and the size of the time window. This has been implemented through LSTM units and their window size. We performed the evaluation study of LSTM-EWO with various other classification algorithms and results from existing studies. In addition, we made a statistical examination to prove the importance of proposed work against other models. It is observed that the experimental results attained with 96% of accuracy, better than conventional models.

In future, we wish to incorporate the suitable outlier detection technique in the LSTM-EWO model. Internet of things-based decision support systems with security and privacy protections will be used to protect the interests of individuals and provide valuable benefits to society.

Ethical approval statement

This article does not contain studies with human participants or animals performed by any one of the authors.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Kelly BB and Fuster V. eds., . Promoting cardiovascular health in the developing world: a critical challenge to achieve global health, National Academies Press 2010, Washington DC, USA. 2010.  Back to cited text no. 1
    
2.
Aarabi G, Schnabel RB, Heydecke G, Seedorf U. Potential impact of oral inflammations on cardiac functions and atrial fibrillation. Biomolecules 2018;8:E66.  Back to cited text no. 2
    
3.
Chebbout R, Heywood EG, Drake TM, Wild JR, Lee J, Wilson M, et al. A systematic review of the incidence of and risk factors for postoperative atrial fibrillation following general surgery. Anaesthesia 2018;73:490-8.  Back to cited text no. 3
    
4.
Anselmino M, Battaglia A, Gallo C, Gili S, Matta M, Castagno D, et al. Atrial fibrillation and female sex. J Cardiovasc Med (Hagerstown) 2015;16:795-801.  Back to cited text no. 4
    
5.
Naccarelli GV, Varker H, Lin J, Schulman KL. Increasing prevalence of atrial fibrillation and flutter in the United States. Am J Cardiol 2009;104:1534-9.  Back to cited text no. 5
    
6.
Jain S, Bajaj V, Kumar A. Efficient algorithm for classification of electrocardiogram beats based on artificial bee colony-based least-squares support vector machines classifier. Electron Lett 2016;52:1198-200.  Back to cited text no. 6
    
7.
Edla S, Kovvali N, Papandreou-Suppappola A. Electrocardiogram signal modeling with adaptive parameter estimation using sequential Bayesian methods. IEEE Trans Signal Process 2014;62:2667-80.  Back to cited text no. 7
    
8.
Mishra J, Tarar S. Chronic Disease Prediction Using Deep Learning. Springer, Singapore: International Conference on Advances in Computing and Data Sciences; 2020. p. 201-11.  Back to cited text no. 8
    
9.
Wang H, Huang Z, Zhang D, Arief J, Lyu T, Tian J. Integrating co-clustering and interpretable machine learning for the prediction of intravenous immunoglobulin resistance in kawasaki disease. IEEE Access 2020;8:97064-71.  Back to cited text no. 9
    
10.
Mienye ID, Sun Y, Wang Z. An improved ensemble learning approach for the prediction of heart disease risk. Informatics in Medicine Unlocked. 2020;20:100402.  Back to cited text no. 10
    
11.
Tama BA, Im S, Lee S. Improving an intelligent detection system for coronary heart disease using a two-tier classifier ensemble. BioMed Research International. 2020;2020:9816142.  Back to cited text no. 11
    
12.
Shouman M, Turner T, Stocker R. Integrating clustering with different data mining techniques in the diagnosis of heart disease. J Comput Sci Eng 2013;20:1.  Back to cited text no. 12
    
13.
Yang S, Guo JZ, Jin JW. An improved Id3 algorithm for medical data classification. Comp Electr Eng 2018;65:474-87.  Back to cited text no. 13
    
14.
Seera M, Lim CP. A hybrid intelligent system for medical data classification. Expert Syst Appl 2014;41:2239-49.  Back to cited text no. 14
    
15.
Nguyen T, Khosravi A, Creighton D, Nahavandi S. Medical data classification using interval type-2 fuzzy logic system and wavelets. Appl Soft Comp 2015;30:812-22.  Back to cited text no. 15
    
16.
Karim AM, Güzel MS, Tolun MR, Kaya H, Celebi FV. A new framework using deep auto-encoder and energy spectral density for medical waveform data classification and processing. Biocybern Biomed Eng 2019;39:148-59.  Back to cited text no. 16
    
17.
Shickel B, Tighe PJ, Bihorac A, Rashidi P. Deep EHR: A survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J Biomed Health Inform 2018;22:1589-604.  Back to cited text no. 17
    
18.
Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: Review, opportunities and challenges. Brief Bioinform 2018;19:1236-46.  Back to cited text no. 18
    
19.
Stockwell RG. A basis for efficient representation of the S-transform. Dig Signal Process 2007;17:371-93.  Back to cited text no. 19
    
20.
Raj S, Ray KC. ECG signal analysis using DCT-based DOST and PSO optimized SVM. IEEE Trans Instrum Meas 2017;66:470-8.  Back to cited text no. 20
    
21.
Hosseini HG, Luo D, Reynolds KJ. The comparison of different feed forward neural network architectures for ECG signal diagnosis. Med Eng Phys 2006;28:372-8.  Back to cited text no. 21
    
22.
Wang CC, Chang CD. SVD and SVM based approach for congestive heart failure detection from ECG signal. In The 40th International Conference on Computers & Industrial Engineering . IEEE. Awaji City, Japan 2010. P. 1-5.  Back to cited text no. 22
    
23.
Khorrami H, Moavenian M. A comparative study of DWT, CWT and DCT transformations in ECG arrhythmias classification. Expert Sys Appl 2010;37:5751-7.  Back to cited text no. 23
    
24.
Vishwa A, Lal MK, Dixit S, Vardwaj P. Clasification of arrhythmic ECG data using machine learning techniques. IJIMAI 2011;1:67-70.  Back to cited text no. 24
    
25.
Mirjalili S, Lewis A. The whale optimization algorithm. Adv Eng Softw 2016;95:51-67.  Back to cited text no. 25
    
26.
Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016.  Back to cited text no. 26
    
27.
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9:1735-80.  Back to cited text no. 27
    
28.
Gers FA, Schmidhuber J, Cummins F. Learning to forget: Continual prediction with LSTM. Neural Comput 2000;12:2451-71.  Back to cited text no. 28
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
   Abstract
  Introduction
   Materials and Me...
  Experimental results
   Results and Disc...
  Conclusion
   References
   Article Figures
   Article Tables

 Article Access Statistics
    Viewed944    
    Printed66    
    Emailed0    
    PDF Downloaded51    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]