Diseases Detection On Chest X-RaysWith Deep Learning

Abstract

In this paper, we compared several classic neural networks that can detect 14 different thoracic diseases from chest X-rays as well as tried to improve their performance by adding squeeze and excitation blocks which won first place in the 2017 ImageNet competition. We re-implemented the ChexNet onChestX-ray14 dataset to validate the accuracy shown in the paper and employed several classic neural networks with the squeeze and excitation blocks which might improve the accuracy. The experiment results indicated that the pre-trained DenseNet121 outperforms other models.

Introduction

In the United States, more than 1 million adults are hospitalized for pneumonia, and about 50,000 people die each year from the disease (CDC, 2017). Chest X-rays are currently the best method for diagnosing pneumonia (WHO, 2001). However, detecting pneumonia in chest X-rays is a challenging task that relies on the availability of expert radiologists. In 2017, Stanford ML Group published a paper that they could automatically detect chest X-ray pneumonia from levels beyond the practicing radiologist with deep learning. Inspired by this, we Implement and train a model(based on DenseNet) and tried to improve the performance by adding squeeze and excitation blocks.

Research contributions

In this paper, we compared the performances of different classic neural networks including DenseNet 121, DenseNet 169, and ResNet50, and tried to add squeeze and excitation blocks that won first place in the 2017 ImageNet competition into convolutions layers of DenseNet121 and ResNet50 to see whether it can improve the performance. Among these models, the SE-DenseNet121 was implemented by ourselves.

Related Work

According to xxx, the CheXNet is a 121-layer convolutional neural network. The network architecture was not given by this paper, but there are many implementations on Github. According to the code provided by arnoweng, we can get the architecture of the CheXNet (shown in Table 1).

Table 1. Sizes of convolutional kernels for DenseNets121 architectures. Note that each conv layer shown in the table corresponds to the sequence BN-ReLU-Conv.

Rajpurkar et al. also mentioned that they pre-trained the DenseNet121 on the ImageNet dataset. In order to improve the performance of DenseNet121, we considered adding the squeeze and excitation module that proposed and won first place in the ImageNet competition in 2017 into the existing DenseNet 121. Figure 1 shows the schema of the original Inception module (left) and the SE-Inception module (right). The purpose of the SE block is to weigh the importance of the channel of filters so that the network will enhance more useful channels and ignore the less useful ones.

Proposed Method

We employed 5 neural networks to solve this problem and CAM(Class activation mapping) to generate a heatmap on top of the X-Ray scan to interpret the neural networks’ output. Referring to the architecture of ResNet50 and SEResNet50 shown in the paper. We added the SE(squeeze and excitation) modules into the Dense blocks in DenseNet121, the architecture is described in table 2.

Figure 1. The schema of the original Inception module (left) and the SE-Inception module (right).

Table 2. Sizes of convolutional kernels for SE-DenseNets121 architectures. Note that each conv layer shown in the table corresponds to the sequence BN-ReLU-Conv.

Experiment

Data Description

We use the ChestX-ray14 dataset released by NIH clinic center which contains 112,120 frontal-view X-ray images of 30,805 unique patients (78468, 22433, and 11219 in training, test, and validation set respectively). Each image is with up to 14 different thoracic pathology labels. Figure 2 shows the thoracic pathology labels distribution in the dataset.

Figure 2. The disease distribution in the ChestX-ray14 dataset.

Considering the time of training, we downsize the dataset on the condition of dropping duplication with the same patient ID and labels. 59,891 X-ray images of 30,805 unique patients remained (41754, 12151, and 5986 images remained in training, test, and validation sets respectively ). Figure 3 shows the thoracic pathology labels distribution in the downsized dataset.

Figure 3. The disease distribution in downsized ChestX-ray14 dataset

Hyperparameter Settings

According to the paper, The weights of the network are initialized with weights from a model pre-trained on ImageNet. The network is trained end-to-end using Adam with standard parameters (1 = 0.9 and 2 = 0.999). The mini- batches are set as 16. The initial learning rate is 0.0001 (in Stanford’s paper, the initial learning rate is set as 0.001 ) that is decayed by a factor of 10 each time the validation loss plateaus after an epoch, and pick the model with the lowest validation loss. Fixed the above hyper parameters, we trained 4 pre-trained models (DenseNet121, DenseNet169, ResNet50 and SEResNet50) and 2 non-pre-trained models (DenseNet121 and SEDenseNet121) on the downsized dataset. The pre-trained models were trained on the ImageNet dataset.

Results and Discussion

Table 3 indicates that our implementation outperforms the published results, which might be caused by the difference in the initial learning rate. Table 4 gives the results of DenseNet121 trained on our downsized dataset. The performance is slightly worse than the results got by the full-sized dataset. Therefore, we think it is reasonable to reduce the time of training in this way.

Table 3. Our implementation outperforms the CheXNet published results on 14 pathologies’ mean AUROC in the original ChestXray14 dataset.

Table 4 shows that among the pre-trained models, DenseNet121 still outperforms other models. Compared with pre-trained DenseNet169, the increase of depth did not bring an increase in AUROC. Besides, compared with ResNet50, the performance of SEResNet50 is more close to the performance of DenseNet121. It means the SE blocks did improve the performance of ResNet50. In this case, We considered that weather SE blocks would increase the performance of DenseNet121. Therefore, we added the SE block into each Dense block in DenseNet121 like what we described in Table 2. However, there is no existing SEDenseNet121 pre-trained on ImageNet. We compared its performance with non-pre-trained DenseNet121. Table 5 shows the results.

Table 4. pre-trained DenseNet121 outperforms the other pre-trained models’ results on 14 pathologies’ mean AUROC in the downsized ChestX-ray14 dataset.

Unfortunately, without the pre-trained model, DenseNet121 performs much better than our SEDenseNet121. We suppose the reason is that the SE block might only be robust to the ImageNet dataset and images in the ChestX-ray14 dataset are in black and white. Therefore, without the pre-trained model, SEDenseNet121 cannot perform better than DenseNet121.

Table 5. DenseNet121 without pre-train outperforms SEDenseNet121 non-pretrained models’ results on 14 pathologies’ mean AUROC in the downsized ChestX-ray14 dataset.

We also generated two heatmap pictures (Figures 4 and 5) with CAM as an example to interpreted the outcome of deep learning models.

Figure 4. Patient with congestive heart failure and cardiomegaly (enlarged heart).

Figure 5. Patient with a large right pleural effusion (fluid in the pleural space).

Conclusions and Future Work

In this case, we compared the performance of several existing models (DenseNet121, DenseNet169, ResNet50, and SEResNet50) on solving the chest X-ray detection problem and proposed a new model (SEDenseNet121) to beat the results of DenseNet121. However, the performance of DenseNet121 is still the best. In the future, if we have enough computational resources, we will pre-train the SEDenseNet121 on ImageNet first and then train it on the ChestX-ray14 dataset to validate whether this model can perform better than DenseNet121 in this problem.

Github Repo for this project: https://github.com/wmjpillow/DiseasesDetection-On-ChestXRays-With-DeepLearning