On this page
We often take things for granted, there’s always the famous saying
“You don’t know what you’ve got till it’s gone”
That has been the case since the beginning of the pandemic. Over the last 2 years, our lives have changed drastically because of Covid-19. What was once considered normal is something that we can only wish to get back to. With Covid-19 heavily impacting our lives, we now have to adhere to many different safety protocols. Now it’s mandatory for us to wear masks 😷. But it’s important that everyone is adhering to these safety measures, so I trained a Convolutional Neural Network to detect face masks 😁
Wait… computers can see!?
The simple answer is yes, computers do have eyes, multiple AI processes are carried through to allow this to happen. Let’s take a closer look at how these computers are actually able to see.
So wait… how do computers see? 👀
Well, it’s not as simple as having a computer look through a camera, the computer can’t make sense of what is in the frame by simply just looking at it, there are many layers to this process. This may seem complicated, like how does a computer even put all this together? This entire process is coined under the term, computer vision. It’s a field of Artificial Intelligence (AI) that allows computers to see 👁
Object Detection 📸
One of the most AI processes that must be followed through with is object detection. Object detection isn’t as complicated as it sounds, it’s split into 2 key components, image classification and image localization.
Image classification is literally what it sounds like, it classifies the image, it essentially classifies what objects are in the image, whether that be a traffic signal, person or another car. Image localization finds the exact location of these classified objects. Take a look at the image below, that is what the computer will see when these 2 processes are combined together.
But wait, is it that simple?
Well, not exactly, within image classification and localization, there are a couple of other steps that must be taken. Let’s start with step 1, image classification.
Image Classification 🎥
Image classification is broken up into more steps, the first one is training. In order for computers to identify images, neural networks are used. 🧠
Neural networks are used to identify patterns or trends within data. As for image classification, a specific type of neural network is used, the ones that are used are called convolutional neural networks. These CNNS are trained to identify a diverse range of objects, whether that be face masks or any other objects
Wait..what?
This might be getting really complicated, but it isn’t as complicated as it sounds. CNNs are trained with a certain set of images and they learn how to identify these images. There are multiple layers to a CNN, with every layer, it helps convert the input data into something the computer can understand. The initial layer would be used to identify something like edges and lines. As the layers get deeper, the more specific the information would get. An extremely deep layer might be able to identify limps and faces.
However, as amazing as this sounds, there are limitations to using CNNs. The first one being that it can only identify objects that take up a significant portion of the image. So when incorporating this with face mask detection, it would be really difficult for the neural network to identify if someone is wearing a mask or not if they’re really far away. To get around this limitation, sliding windows can be used.
Sliding Windows
As seen in the gif on the left, the window is slid over the image, and the data extracted from the sliding window is run through the trained CNN to see if it matches the objects that the CNN has been trained with. The CNN would be trained with people that are wearing masks or not. If the sliding window sees something else other than the masks, the CNN would return false, but if it’s a person wearing a mask or not wearing a mask, it would return true.
However, if the object is smaller or bigger than the sliding window, it wouldn’t be detected at all. You might think, what!? But there’s a whole other algorithm that is run to help get around this limitation, this algorithm is called: YOLO
Introducing YOLO 👋
Some of you reading this might be thinking, what does you only live once have to do with identifying images. Let me explain, YOLO in this case stands for, you only look once. The reason it’s called YOLO is because the image only has to go through the CNN once. Hence the saying, you only look once. This process is how these classified objects are located. We’re aware of what the objects are via the CNN, but what if there’s a huge crowd of people, it can’t do much knowing that 5 people are wearing masks and 3 aren’t, it needs to know where these people are in the frame.
Let’s break this complicated process down, the initial image is first resized to fit the input layer, it’s almost like fixing the size of an image to make it fit on a google slide for example. Once this is done, it’s run through the trained CNN, when it gets through all the layers, a bounding box appears around each object with a probability of what it may be.
Before getting to that stage, there’s a process that must be done. Let’s break this down. Below is an image of YOLO in action, the image is first split into a
S x S grid. In the image below, it’s a 13 x 13 grid.
Within these initial grid cells, the YOLO algorithm will predict 3 things, size, location and confidence score. For these initial grid cells, they are further split into 5 bounding boxes with object probability. These 5 bounding boxes consist of 5 things, the x coordinate, the y coordinate, width, height and the confidence score.
What’s the confidence score?
It’s literally in the name, it’s essentially how confident the algorithm is about the classified object being in the box. To measure the confidence score, a simple calculation is done, it’s called Intersection over Union (IoU).
How complicated can this possibly get?😅
This question might be crossing your mind, but hear me out, this is the exciting part, it’s where everything is put together.
Intersection over union is utilized in object detection as it compares the ground truth box with the detected box. It divides the area of overlap with the total area of the union. The greater the overlapping region, the higher the confidence score. Higher confidence scores also result in higher accuracy when detecting the object. The IoU metric is excellent in determining confidence.
If we go back to the previous example, There are a total of 845 bounding boxes in the image below (13 x 13 x 5). There are a lot of bounding boxes, the ones with a certain confidence score are kept since the other ones don’t serve many purposes (there aren’t any objects). Usually, the confidence scores above 0.5 are kept. This allows for it to eventually return a final image as seen below.
Training my CNN to detect Face Masks
Using all this knowledge, I trained my own Convolutional Neural Network to detect face masks based on a video or image input of a crowd. Here’s how I did it 😉
Downloading Datasets
I went onto Kaggle and I downloaded a huge dataset of people wearing masks, not wearing masks and wearing masks improperly. To ensure that my CNN knows how to identify a person that is wearing a mask from someone who isn’t, I graphically annotated these images. I used LabelImg to annotate this dataset. For this CNN, I only used 3 labels, “without_mask”, “with_mask” and “mask_weared_incorrectly”.
Using Google Colab
Once I finished annotating this dataset, I went onto Google Colab to train my CNN. I cloned the YOLO v5 repo onto my google colab file and installed the required dependencies to run this face mask detection model. I created a custom dataset folder within Google Colab and uploaded the images and the labelled dataset.
Training my CNN
Once I uploaded my dataset, I created a custom YAML file that provided a path to train/validate the data and the number of classes.
After all that was done, I ran the command to start training my dataset. This process took about 2–3 hours, training a CNN took longer than I initially anticipated 😅
Testing my Model
After I finished training my CNN, I decided to upload an image and see whether or not my neural network was accurate in detecting face masks. I imported the Python Imaging Library and did all this. The final output looked something like this
I recorded a demo on showing the testing process live, check it out here
What’s Next 🧐
Overall, YOLO and CNNs are used to help classify and locate objects, whether that be in a video or image. This algorithm can be incorporated in many areas, not just for face mask detection. I’m looking forward to seeing which industries it impacts next. Neural Networks are the future of data analysis and image recognition. With more iterations, testing and data, these Neural Networks will soon reach 100% accuracy.
If you enjoyed this article, feel free to connect on LinkedIn 😁