Medicine through the lens of a computer

The diagnosis of many different diseases ranging from different cancers to infectious diseases like pneumonia rely heavily on imaging techniques such as x-rays, CT scans, MRIs, ultrasounds and biopsy imaging. The images generated by these techniques are generally read by a medical professional which then makes a diagnosis based on what is seen in the image. In the era of technology, the idea that of developing clinical decision support tools that can analyze large volumes of image data (and other data formats as well) and suggest next steps for treatment and flag potential problems which in turn would enhance healthcare efficiency and reduce human error in diagnosis is a very attractive one. But will it work? As a proof of principle, I used chest x-rays images to develop a machine learning model to diagnose pneumonia.

Pneumonia is an inflammatory condition of the lung usually caused by infection with viruses or bacteria. Typically, symptoms include some combination of productive or dry cough, chest pain, fever, and trouble breathing. Severity is variable, but in severe cases or cases that go undiagnosed pneumonia can cause death. In fact, each year, pneumonia affects about 450 million people globally (7% of the population) and results in about 4 million deaths. However, cases, if the disease is diagnosed and treated in its early stages, pneumonia is largely treatable. Therefore, it is critical to develop better tools that can streamline and improve the diagnosis of pneumonia.

The dataset used in this analysis/model is available on Kaggle (https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia) and contains a total number of 5863 chest x-ray images from either healthy (normal) or patients diagnosed with pneumonia (pneumonia). The dataset is comprised of three sub datasets:

· Training dataset (contains 5216 chest x-ray images)

· Test dataset (contains 629 chest x-ray images)

· Validation dataset (contains 16 chest x-ray images)

Here is a count of the number of chest x-ray images ascribed to each diagnosis included in the training dataset:

Before diving into building a complex model capable of accurately predicting pneumonia, it is important to understand how pneumonia is diagnosed. The most frequently used method to diagnose pneumonia is a chest x-ray. Here are how the chest x-ray images of healthy lungs looks like:

And here is how chest x-rays of pneumonia patients looks like:

When interpreting the x-ray, doctors look for white spots in the lungs (called infiltrates) that identify the infection. Looking at the x-ray images in our datasets we can see that chest x-rays of pneumonia patients show the presence of white spots and the general architecture of the lung is different than of healthy people. Thus, analysis of x-ray images is one of the most reliable ways to effectively diagnose pneumonia.

Neural networks algorithms are made out of layers of “neurons” connected in a way that the input of one layer is the output of the previous layer. This type of architecture allows neural networks the flexibility to represent well the underlying relationships between the tasks input and output. To develop a model capable of taking chest x-ray images and predict if a patient has pneumonia or is healthy, I used the convolution neural networks (CNN) algorithm. This is based on traditional neural networks but employs a specific architecture type that exploits special properties of an image. CNN first reduces the size of images using convolutional layers and pooling layers to then feed the reduced data to fully connected layers. Thus, to build a CNN we need to compile the model by providing the instructions for each of the hyperparameters within each layer.

To get the x-ray images ready for modeling, I have use ImageDataGenerator to load the data and prepare the data for modeling. This is very useful when handling large datasets as it allows for the images to be progressively loaded in batches from file, retrieving just enough data for what is needed immediately. The constructor for the ImageDataGenerator contains many arguments to specify how to manipulate the image data after it is loaded, including pixel scaling and data augmentation. Thus, also allowing to not configurate the specific details about each image as it is loaded but also expand the training dataset with new, plausible examples. Using this class, I scaled the array of pixels in the original images of all my datasets to pixel values between 0 and 1, which a range preferred for neural networks modeling. I also used data augmentation methods including rotation, width and height shifts and zoom to create variations of the training set images that are likely to be seen by the model.

As seen in the graph with the image counts, there is a great imbalance in the number of images in each class having about 3 times more pneumonia x-rays than normal x-rays. Imbalances in the number of images in each class will invariably affect how the model performs as it biases the model towards the more abundant class. To balance the data, I first calculated weight of each class of x-ray images by dividing the number of x-rays images in each class by the total number of images. This show how much one class is represented versus the other. Then I assigned the weight of the pneumonia class to the value of weight that I calculated the normal class to have and vice-versa to equally penalize under or over-represented classes in the training set.

Now let’s see if my CNN model can actually predict pneumonia from chest x-rays!

This model is pretty good, having an accuracy of 87%, which means that 87% of the time it can accurately decide whether the x-ray image is of a normal healthy lung or a lung from a person with pneumonia. However, it still has a 19% false positive rate of diagnosing a normal lung as pneumonia. To get a sense into what features of the x-ray images were contributing for the improper diagnosis of normal x-rays as pneumonia, I used the Lime package to depict the features that the model found most important in making its predictions.

The images shown display the features that the model found the most important in green and the ones found to be less important in red. From this image we can see that the CNN model picked up a large portion of the diaphragm and the sternum as key features in making its incorrect prediction, suggesting that the high intensity pixels coming from these areas may be adding noise to the model. Taking this into consideration, additional pre-processing methods that can remove or suppress the intensity of pixels coming from the diaphragm and sternum areas could also be valuable in improving this model for future clinical use.

Check out my GitHub to look at how I developed this model!