On this page
Trip down memory lane… 💭
One of the biggest stories of the last decade has been about how COVID-19 has taken the world by storm; it has caused many unprecedented challenges in healthcare, economy, and social dynamics. As a result of these challenges, many critical care facilities have collapsed, even in established and industrialized nations. Despite all the medical efforts being put forward, there was a high false detection rate which made it extremely difficult to identify infected people. Moreover, there has been an inefficiency and scarcity of current COVID-19 tests which has resulted in many initiatives to explore other testing methodologies.
One of the most crucial aspects of controlling the spread of the virus is early detection, which has the possibility to improve the chances of successful treatment and prevent further transmission.
One of the methods to identify COVID-19 infections involves using radiological imaging techniques such as X-rays or computed tomography (CT). In fact, chest CT scans have emerged as a potential game-changer for detecting and quantifying instances of COVID-19, according to researchers. But what if we could supercharge the diagnostic capabilities of CT scans using artificial intelligence ? That’s precisely what researchers worldwide are striving to do, harnessing AI as a powerful tool to develop quick and affordable diagnostic methods to halt the pandemic’s spread.
The objectives of these efforts are manifold — from curbing transmission to early detection, creating efficient treatments, and comprehending the economic impacts of the virus. Compared to standard head X-rays, CT scans provide more precise information about tissue and structure, enabling doctors to diagnose a wide range of traumas and disorders. Therefore, it’s no surprise that X-ray and CT scans are the two most frequently used imaging methods to examine COVID-19 patients.
Although deep learning methods have made significant strides in various medical areas, they still face a significant hurdle — dependence on vast quantities of data. To tackle this issue of data scarcity, researchers have turned to a range of strategies, including data augmentation and generative adversarial networks (GAN). However, these strategies come with their own challenges. For instance, different data augmentation techniques can lead to overfitting issues, while GAN image generation techniques may not be able to simulate actual patient data, introducing unintended bias during experimental validation. In a bid to find a solution, several studies have employed transfer learning approaches using pre-trained Convolutional Neural Network (CNN) models. However, these models were trained on a large non-medical dataset (ImageNet), and fine-tuning them for promising diagnostic findings often involves lengthy training times.
The answer in all this lies in few-shot learning, also known as FSL.
Understanding Few-Shot Learning + Meta-Learning☄️
All machines are taught to learn from experience, but meta-learning basically takes that to the next level. FSL is an instantiation of meta-learning. In simple words, meta-learning is a machine learning technique that involves learning a learning algorithm; this allows the model to quickly adapt to new tasks and environments. It refers to the ability of an AI model to learn how to solve new problems more efficiently by leveraging the experience it has accumulated from solving similar problems in the past. Essentially, meta-learning enables machines to develop a kind of “intuition” about problem-solving, which is similar to the way humans learn from experience.
Meta-learning has several advantages over traditional machine learning techniques. Firstly, it can help in addressing the problem of insufficient training data, which is often a major bottleneck in AI research. By leveraging past learning experiences, the AI model can adapt to new tasks more quickly and accurately. Additionally, meta-learning can help in developing transferable knowledge, which is the ability to apply what has been learned in one task to solve a completely different but related task. This can significantly reduce the training time and computational resources needed to solve new problems, thus making the process more efficient.
Few Shot Learning
Few-shot learning is a machine learning technique that allows an AI model to learn new concepts or classes with only a few examples, often as few as one or two.
This is achieved by leveraging prior knowledge or experience that the model has acquired during training on a larger dataset. Few-shot learning is particularly useful in scenarios where obtaining large amounts of training data is impractical or expensive, such as in medical diagnosis or cybersecurity. By enabling the AI model to generalize to new, unseen examples with minimal training data, few-shot learning opens up new possibilities for intelligent automation and decision-making in various industries. Additionally, it has been gaining attention in recent years due to its potential to reduce the amount of data required for AI models to learn and adapt to new tasks, making it a promising area of research in machine learning.
The Triplet Siamese Network
A new network called the Siamese network has found success in this field. The Siamese network is a meta-learning model that has been successful in using FSL in various fields. This network uses a ‘network architecture that is often used to create and evaluate feature vectors for each input’. Two or more subnetworks, configured using identical parameters and weights, form the Siamese network. By examining the feature vectors of this network and training a similarity function among labeled data points, it becomes feasible to evaluate the likeness of inputs. Extrapolating the applications of this network into the field of COVID-19; to categorize the patient with few training CT images with 3 identical subnetworks.
What’s been done in the past?
In the past, researchers have developed deep learning techniques to detect COVID-19 using medical images such as CXR and CT scans in order to control the spread of the epidemic. To speed up the identification of COVID-19 patients, a VGG-19 transfer learning method that leverages CT-Scan and chest X-ray data was proposed. They employed the COVID-19 chest X-ray dataset, which includes 673 radiological images from 342 individuals. Furthermore, the SARS-COV-2 CT scan dataset contains 1230 CT scans from individuals who tested negative for COVID-19 and 1252 CT images from those who tested positive for COVID-19.
The proposed model has an accuracy of 95.61% in identifying COVID-19 cases. Based on their methodology, patients diagnosed with pneumonia are more likely to test positive for COVID-19, resulting in a false positive. Another fully automated technique to detect COVID-19 from chest CT results of patients was also proposed. The ResNet50V2 model was used as the framework for their experiment. Their dataset consisted of 48,260 CT scans, including 282 healthy individuals and 15,589 images from 95 COVID-19 patients. The proposed model achieved 98.49% accuracy in binary image classification for 7,996 test images.
Understanding the Specifics of the Triplet Siamese Model
The proposed model consists of many stages and processes and can be seen in the image below:
The raw CT scan images underwent a preprocessing pipeline that included several steps, such as data scaling, shuffles, noise removal, image sharpening, normalization, and adjustments to brightness and contrast. The preprocessed data was then separated into two datasets, a training dataset, and a testing dataset. The transfer-learning-based models were trained on one set, while the proposed Triplet Siamese Network was trained on the other. The system’s performance was evaluated using various metrics such as accuracy, ROC AUC, specificity, sensitivity, F1-score, and the confusion matrix. The accuracy and loss were recorded after each epoch, while validation accuracy and loss were calculated using 10-fold cross-validation. Now that we’ve gotten some insight about the model, let’s jump into the specifics about the dataset and how this model was made!
The Dataset
The COVID-19 dataset is a combination of seven open datasets, featuring a total of 2618 images of community-acquired pneumonia, 7593 COVID-19 images, and 6893 normal images. These datasets have been extensively used in COVID-19 diagnostic literature, demonstrating their effectiveness in deep learning applications. By merging these resources, the combined dataset is anticipated to enhance the classification performance of deep learning techniques. The table below shows the properties of this dataset:
There were 2 datasets that were created, one for the CNN models and the other one for the FSL model. There were a total of 12344 CT scans used for training and 3390 CT scans for evaluating the model. Here is the split of that data:
Pre-Processing the Data
The data that was collected for the proposed model came from a variety of different sources. All of these images featured noise, and if we were to train the model with the raw data, the model wouldn’t be as accurate. For context, noise in an image is caused by random signal fluctuations which corrupts the data that is used to create an image. To get around this, unsharp masking was used. Unsharp masking is an image sharpening technique; the blurry exposure is subtracted from the original image. This allows the image to only be represented with edges only. The resulting image is added to the original image to get a better and enhanced version of the image. To implement this, a gaussian filter is used and the unsharp mask increases the high frequency components of the image.
On top of the unsharp masking, background removal must also be implemented. To do this, a simple function can be implemented to normalize the pixel values to eliminate the background. K-Means were used to establish the ‘threshold for differentiating between the foreground (soft tissue/bone) and background (lung/air). In order to refine the mask, a second foreground mask was initiated and a bitwise AND operation was performed to obtain the intersection of the two foreground masks. The enclosed mask was then filled to eliminate any holes, and subsequently applied it to the image. The background and its associated noise were removed by cropping the masked image.
Algorithm-Based Details of the Proposed System
The proposed model (Triplet Siamese Network) has 3 identical subnetworks. The model is fed 3 images, 2 images which are similar (anchor and positive samples), while the 3rd one is different (a negative example). For the sake of visual purposes, here’s what it looks like:
The overarching goal of this model was to learn the similarities between different types of images. The proposed model receives triplet images, hence the name of the model. These triplet images are used to generate embeddings and it outputs the distance between the anchor and the positive & negative embeddings. In order to compute the distance, the Pairwise Distance method was used. The pairwise distances were used to measure the similarity between each vector. This is the formula used to calculate the Pairwise Distance:
As with all Machine Learning models, a loss function is necessary and for the proposed model, the Marking Ranking Loss was used as the loss function. The loss function used predicts the relative distances between inputs, whereas other common loss functions (cross-entropy loss) goal is to learn to predict directly a label. The Margin Ranking Loss function is a powerful tool for enhancing model performance. However, to leverage its full potential, a similarity score between data points is required. The anchor image’s similarity with the positive image should be high, while the anchor image’s dissimilarity with the negative image should be low. To achieve this, the ensemble model extracted features from all three input data points, creating embedded representations.
To compute the similarity score, the Pairwise Distance method was used. By applying the computed gradients to the Siamese network’s weights and biases, we fine-tuned the feature extractors, enabling them to produce comparable representations for similar inputs and distant representations for dissimilar ones. As mentioned previously, the model will be evaluated with 8 unique metrics: Accuracy, Precision, Recall, F1-score, Specificity, and AUC ROC score. Before jumping into the results, here is what the visual of the model looks like:
The Results of the Triplet Siamese Network 📑
In order to train the transfer-learning-based models, the dataset was split into two portions: an 80% training dataset and a 20% test dataset. Similarly, for our proposed Triplet Siamese Network dataset (referred to as the meta-dataset), was partitioned into 600 images for training and 10,152 images for testing. To ensure that the dataset was partitioned in a patient-aware manner, there was no CT scan slice from the same patient used in both training and testing. To gauge the efficacy of the few-shot learning model in detecting COVID-19 cases, the accuracy and F1 score was compared with those of several pre-trained CNN models.
The suggested model in Table 4 exhibits superior performance compared to the implemented pre-trained models, achieving a substantially better score. Moreover, this score was obtained from a much larger dataset of 10152 images, whereas the other models were only tested on 3390 images 🤯. Moreover, the proposed model demonstrates more stable training and validation loss when compared to pre-trained CNN models (see the graphs below). To prevent overfitting to the training data, the L2 dropout regularization strategy was implemented, and early stopping with a learning schedule was employed using the ReduceLROnPlateau callback from Pytorch.
The suggested model produces excellent Recall (98.719%) and Precision (98.719%) values, which are crucial performance predictions for medical applications. The model was trained on only 600 CT images, indicating promising results. Moreover, the few-shot learning model comprises an ensemble of pre-trained CNN models, increasing the network’s robustness and accuracy. Ensemble models are more trustworthy and robust than basic deep learning techniques, highlighting the advantages of few-shot learning over other cutting-edge models for COVID-19 patient categorization. The CovidExpert model outperforms cutting-edge CNN models, with an accuracy of 98.719%, a specificity of 99.36%, a sensitivity of 98.719%, and an F1-score of 98.719%. The CovidExpert model displayed superior performance for all performance indicators.
Future Thoughts… 💭
In conclusion, the proposed few-shot learning model, CovidExpert, has shown promising results in detecting COVID-19 cases in CT images. It outperformed several pre-trained CNN models, with a high accuracy, specificity, sensitivity, and F1 score. CovidExpert also demonstrated superior performance on a class-by-class basis, making it a reliable tool for medical applications. The success of this model opens up possibilities for further exploration and improvement in the field of few-shot learning. Future work could focus on expanding the dataset and exploring other ensemble techniques to enhance the model’s robustness and accuracy. Additionally, the model’s applicability can be extended to other medical imaging tasks beyond COVID-19 detection. Overall, CovidExpert has proven to be a promising tool in the fight against the COVID-19 pandemic and has the potential to make a significant impact in the medical field.
If you enjoyed this article, feel free to give it claps and share it! You can catch me on LinkedIn and if you want to check out some of my other work, here’s my personal website :)