Categories
tech

Building Real-Time Vehicle Detection System

Author profile picture

@sabinaSabina

Customer Success AI Engineer at Xailient

From automobile counting and sensible parking programs to Autonomous Driving Assistant Systems, the call for for detecting vehicles, buses, and motorbikes is expanding and shortly can be as commonplace of an application as face detection.

And after all, they want to run real-time to be usable in maximum real-world packages, as a result of who will depend on an Autonomous Driving Assistant Systems if it can not hit upon vehicles in entrance folks whilst riding. 

In this put up, I will be able to display you the way you’ll be able to put in force your personal automotive detector the usage of pre-trained fashions which can be to be had for download: MobileNet SSD and Xailient Car Detector.

Before diving deep into the implementation, let’s will get somewhat acquainted and learn about those fashions. But be happy to skip to the code and effects if you want. 

MobileNet SSD

MobileNet is a lightweight deep neural community structure designed for mobiles and embedded imaginative and prescient packages.

In many real-world packages similar to a self-driving automotive, the popularity duties want to be performed in a well timed model on a computationally restricted software. To fulfil this requirement, MobileNet was once evolved in 2020. 

The core layers of MobileNet is constructed on depth-wise separable filters. The first layer, which is a complete convolution, is an exception. 

To be told additional about MobileNet, please check with the paper.

Around the similar time (2020), SSD: Single Shot detector was once additionally evolved by way of Google Research group to cater the will for fashions that may run real-time on embedded units and not using a important trade-off in accuracy. 

Single Shot object detection or SSD takes one unmarried shot to hit upon a couple of gadgets inside the symbol. The SSD means is according to a feed-forward convolutional community that produces a fixed-size number of bounding packing containers and rankings for the presence of object elegance cases in the ones packing containers.

It’s composed of two portions:

  1. Extract function maps, and
  2. Apply convolution filter out to hit upon gadgets

SSD is designed to be unbiased of the bottom community, and so it may run on best of any base networks similar to VGG, YOLO, MobileNet. In the unique paper, Wei Liu and group used VGG-16 community as the bottom to extract function maps.

To be told additional about SSD, please check with the paper.

To additional take on the sensible barriers of operating top useful resource and power-consuming neural networks on low-end units in real-time packages, MobileNet was once built-in into the SSD framework. So, when MobileNet is used as the bottom community within the SSD, it become MobileNet SSD.

The MobileNet SSD means was once first educated at the COCO dataset and was once then fine-tuned on PASCAL VOC achieving 72.7% mAP (imply moderate precision).

MobileSSD for Real-time Car Detection

Step 1: Download pre-trained MobileNetSSD Caffe fashion and prototxt.

We’ll use a MobileNet pre-trained downloaded from https://github.com/chuanqi305/MobileNet-SSD/ that was once educated in Caffe-SSD framework.

Download the pre-trained MobileNet SSD fashion and prototxt from right here.
MobileNetSSD_deploy.caffemodel

MobileNetSSD_deploy.prototxt

Step 2: Implement Code to make use of MobileNet SSD

import time
import cv2 as cv
import numpy as np
import math # load our serialized fashion from disk
print("Load MobileNetSSD fashion") prototxt_path = "MobileNetSSD_deploy.prototxt"
model_path = "MobileNetSSD_deploy.caffemodel" # initialize the checklist of sophistication labels MobileNet SSD was once educated to hit upon
CLASSES = ["background", "aeroplane", "bicycle", "fowl", "boat", "bottle", "bus", "automotive", "cat", "chair", "cow", "diningtable", "canine", "horse", "motorcycle", "particular person", "pottedplant", "sheep", "settee", "educate", "tvmonitor"] internet = cv.dnn.readNetFromCaffe(prototxt_path, model_path) def process_frame_MobileNetSSD(next_frame): rgb = cv.cvtColor(next_frame, cv.COLOR_BGR2RGB) (H, W) = next_frame.form[:2] # convert the body to a blob and move the blob throughout the # community and acquire the detections blob = cv.dnn.blobFromSymbol(next_frame, length=(300, 300), ddepth=cv.CV_8U) internet.setInput(blob, scalefactor=1.0/127.5, imply=[127.5, 127.5, 127.5]) detections = internet.ahead() # loop over the detections for i in np.arange(0, detections.form[2]): # extract the boldness (i.e., chance) related # with the prediction self belief = detections[0, 0, i, 2] # filter vulnerable detections by way of making sure the `self belief` # is larger than the minimal self belief if self belief > 0.7: # extract the index of the category label from the # detections checklist idx = int(detections[0, 0, i, 1]) # if the category label isn't a automotive, forget about it if CLASSES[idx] != "automotive": proceed # compute the (x, y)-coordinates of the bounding field # for the article field = detections[0, 0, i, 3:7] * np.array([W, H, W, H]) (startX, startY, endX, endY) = field.astype("int") cv.rectangle(next_frame, (startX, startY), (endX, endY), (0, 255, 0), 3) go back next_frame def VehicheDetection_UsingMobileNetSSD(filename): cap = cv.VideoCapture(filename) # Write output document frame_width = int(cap.get(cv.CAP_PROP_FRAME_WIDTH)) frame_height = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT)) # Define the codec and create VideoWriter object fps = 20 length = (int(frame_width),int(frame_height)) fourcc = cv.VideoWriter_fourcc('m','p','4','v') out = cv.VideoWriter() luck = out.open('output_mobilenetssd.mov', fourcc, fps, length, True) frame_count = 0 # get started timer t1 = time.time() whilst True: ret, next_frame = cap.learn() # Reads the following video body into reminiscence if ret == False: ruin frame_count += 1 next_frame = process_frame_MobileNetSSD(next_frame) # write body out.write(next_frame) key = cv.waitKey(50) if key == 27: # Hit ESC key to forestall ruin # finish timer t2 = time.time() # calculate FPS fps = str( waft(frame_count / waft(t2 - t1))) + ' FPS' print("/MobileNetSSD Car Detector") print("Frames processed: {}".layout(frame_count)) print("Elapsed time: {:.2f}".layout(waft(t2 - t1))) print("FPS: {}".layout(fps)) cap.unlock() cv.destroyAllWindows() out.unlock()

(Parts of this code is encouraged from PyImageSearch weblog.)

Experiments:
I ran the above code on two other units:

  1. On my dev device, which is Lenovo Yoga 920 with Ubuntu18.04 working machine.
  2. On cheap, resource-constrained software, which is Raspberry Pi 3B+ with Raspbian Buster working machine.

Results:

On my dev device, Lenovo Yoga, with MobileNet SSD, I were given an inference pace of 23.3 FPS and once I ran RaspberryPi 3B+, the inference pace was once 0.9 FPS, the usage of all 4 cores.Pretty dramatic. This experiment presentations that you probably have a formidable software to run the MobileNetSSD, it plays smartly and can serve the real-time requirement.  But in case your application is focused to be deployed on a computationally restricted IoT/embedded software such because the Raspberry Pi, this doesn’t appear to be a excellent are compatible for a real-time application.

Xailient

Xailient fashion makes use of selective consideration option to carry out detection. It is encouraged by way of the running mechanism of the human eye.Xailient fashions are optimized to run on low persistent units which can be reminiscence and resource-constrained. 

Now let’s see how Xailient Pre-trained Car detector plays.

Xailient Car Detector for Real-time Car Detection

Step-1: Download pre-trained Car Detector fashion.
We’ll use a Xailient’s pre-trained automotive detector fashion downloaded from console.xailient.com.

Step 2: Implement Code to make use of Xailient Car detector mode

import time
import cv2 as cv
import numpy as np
import math
from xailient import dnn # initialize Xailient fashion
print("Initialize Xailient fashion")
THRESHOLD = 0.6 # Value between 0 and 1 for self belief ranking
detectum = dnn.Detector() def process_frame_xailient(next_frame): _, bboxes = detectum.process_frame(next_frame, THRESHOLD) # Extract bbox coords # Loop thru checklist (if empty this can be skipped) and overlay inexperienced bboxes # Format of bboxes is: xmin, ymin (best left), xmax, ymax (backside proper) for i in bboxes: cv.rectangle(next_frame, (i[0], i[1]), (i[2], i[3]), (0, 255, 0), 3) go back next_frame
def VehicheDetection_UsingXailient(filename): cap = cv.VideoCapture(filename) # Write output document frame_width = int(cap.get(cv.CAP_PROP_FRAME_WIDTH)) frame_height = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT)) # Define the codec and create VideoWriter object fps = 20 length = (int(frame_width),int(frame_height)) fourcc = cv.VideoWriter_fourcc('m','p','4','v') out = cv.VideoWriter() luck = out.open('output_xailient.mov', fourcc, fps, length, True) frame_count = 0 # get started timer t1 = time.time() whilst True: ret, next_frame = cap.learn() # Reads the following video body into reminiscence if ret == False: ruin frame_count += 1 next_frame = process_frame_xailient(next_frame) # write body out.write(next_frame) key = cv.waitKey(50) if key == 27: # Hit ESC key to forestall ruin # finish timer t2 = time.time() # calculate FPS fps = str( waft(frame_count / waft(t2 - t1))) + ' FPS' print("/nXailient Car Detector") print("Frames processed: {}".layout(frame_count)) print("Elapsed time: {:.2f}".layout(waft(t2 - t1))) print("FPS: {}".layout(fps)) cap.unlock() cv.destroyAllWindows() out.unlock()

Experiments:
I ran the above code the similar two units of units:

  1. On my dev device, which is Lenovo Yoga 920 with Ubuntu18.04 working machine.
  2. On cheap, useful resource constrained software, which is Raspberry Pi 3B+ with Raspbian Buster working machine.

Results:

On dev device, there’s a slight development on inference pace when the usage of Xailient Car Detector even if most effective 1 core is used. On Raspberry Pi, on the other hand, Xailient processes 8x extra frames according to 2d with a unmarried core. 

Summarizing the result of each fashions:

The video I used for this experiment was once downloaded from Pexels.com

In this put up, we seemed the will for real-time detection fashions, in brief presented MobileNet, SSD, MobileNetSSD and Xailient, all of which have been evolved to unravel the similar problem: to run detection fashions on low-powered, resource-constrained IoT/embedded units with a proper stability of pace and accuracy. We used pre-trained MobileNetSSD and Xailient automotive detector fashions and carried out experiments on two separate units: dev device and a cheap IoT software. Results display a slight development in pace of Xailient Car detector over MobileNetSSD, within the dev device and an important development within the cheap IoT software, even if most effective 1 core was once used.

Originally revealed in xailient.com/weblog

If you wish to have to increase your automotive detection application to automotive monitoring and pace estimation, right here’s an excellent weblog by way of PyImageSearch.

References

  1. Howard, Andrew G., et al. “Mobilenets: Efficient convolutional neural networks for cellular imaginative and prescient packages.” arXiv preprint arXiv:1704.04861 (2020).
  2. Liu, Wei, et al. “Ssd: Single shot multibox detector.” European convention on pc imaginative and prescient. Springer, Cham, 2020.
  3. https://www.pyimagesearch.com/2020/12/02/opencv-vehicle-detection-tracking-and-speed-estimation/
  4. https://honingds.com/weblog/ssd-single-shot-object-detection-mobilenet-opencv/
  5. https://github.com/chuanqi305/MobileNet-SSD
  6. https://mc.ai/object-detection-with-ssd-and-mobilenet/
  7. https://machinethink.internet/weblog/mobilenet-v2/#:~:textual content=SSD%20is%20designed%20to%20be,detection%20portion%20of%20the%20community.

Previously revealed at https://www.xailient.com/put up/real-time-vehicle-detection-with-mobilenet-ssd-and-xailient

Tags

The Noonification banner

Subscribe to get your day-to-day round-up of best tech tales!