Object detection w/ Raspberry PLC & AI

How to detect objects using a pre-trained AI model and a Raspberry PLC, in a live video
March 21, 2024 by
Object detection w/ Raspberry PLC & AI
Boot & Work Corp. S.L., DAVID CORONADO TRAVASET

Introducing AI-powered object detection with Raspberry PLC

Object detection is the ability to identify and locate specific objects in an image or video. AI can do this job better due to its ability to learn patterns and characteristics of objects, allowing it to make accurate and efficient detections in large data sets in a rapid and automated manner.

Example of computer vision with artificial intelligence

A PLC with access to this technology is a very useful tool to execute tasks of:

  • QA
  • Security
  • Process automation
  • Logistics and materials handling (for example: tracking and sorting products in warehouses and distribution centers)
  • Production optimization
  • Predictive maintenance


Below, we show you how to implement this technology with a simple example, with the help of our Raspberry PLC and a model called MobileNet and SSD (Single Shot MultiBox Detector).

Requirements

- Raspberry PLC
- USB camera
- AI Model: MobileNetSSD. We will have to download the files: 

MobileNetSSD_deploy.caffemodel:  

MobileNetSSD_deploy.prototxt.txt: 


How does it work ?

The concept is simple: the USB-connected camera to the Raspberry Pi captures video frames, which our script reads and feeds into the MobileNetSSD model to identify patterns in the image corresponding to objects it has been trained to detect. Once the object is detected, it returns its position, the name of the object, and the confidence level or probability of it being the identified object. Leveraging this, we'll generate a box in the frame displaying all this information. This process will be repeated for each video frame.

what FPS are frames per second

The MobileNetSSD model is already pre-trained to detect a series of objects:

0: 'background', 1: 'aeroplane', 2: 'bicycle', 3: 'bird', 4: 'boat', 5: 'bottle',
6: 'bus', 7: 'car', 8: 'cat', 9: 'chair', 10: 'cow', 11: 'diningtable', 12: 'dog',
13: 'horse', 14: 'motorbike', 15: 'person', 16: 'pottedplant', 17: 'sheep',
18: 'sofa', 19: 'train', 20: 'monitor'

These are the objects that we can detect.


Next, we are going to connect the USB camera to one of the USB inputs of the Raspberry PLC, by default it will be found in /dev/video0, make sure it is connected with the command:


lsusb


or, with:


ls /dev/video*


Now that we are sure that we have connected the camera to the Raspberry PLC, let's move on to the programming.

Programming time

We will use Python to generate the script and the OpenCV library.

First we will create a folder and put the files corresponding to the MobileNetSSD model in it, then we will create the Python file.


But first, we will create a virtual environment with the virtualenv tool to be able to install the necessary libraries, have good isolation of dependencies and avoid conflicts between projects.


Inside the folder that we have created, we execute the commands:


  • Install virtualevn


sudo apt install virtualenv -y


  • Create the environment:


virtualenv <virtual environment name>


  • Activate the environment:

source <virtual environment name>/bin/activate


  • Disable the environment:

deactivate


In our case, We will use the name venv:


virtualenv venv
source venv/bin/activate


When activated, it appears in the command line prompt with the name of the environment:

Not activated: pi@raspberrypi:~

Activated: (venv) pi@raspberrypi:~


Now we can install the necessary dependencies with pip: such as opencv-python, numpy, among others.

This code will show the detection of objects through the camera in a window that we can close by pressing the letter q:

main.py:


This code works on a system with a graphical interface, or on our Touchberry panel.


Without user interfice - Raspberry PLC

In order to see this object detection on the Raspberry PLC we are going to use the Flask library to generate a route within our local network, which provides us with a website where we can see the content captured by the camera.

In this case, the folder must have:

  • The two AI model files
  • The ​app.py (where the specified path will be provided)
  • A directory called "templates" and inside an html file (example: /templates/index.html)
  • The directory that will be generated when the virtual environment is created.

app.py:


index.html:



With these codes, the AI files, and the installed dependencies we will be able to see in the browser, in the path:

<raspberry ip>:5000/


Here is an example to execute it:

Execution-example

Now, from the browser of another computer connected to the same network as the raspberry plc, and searching in my case is: http://10.42.0.10:5000

we can see the result.

Here we will look at some casual examples that we have captured:

AI object detection test image 1

AI object detection test image 2

Comparison with PLC based on Raspberry Pi 4 vs Raspberry Pi 5


We have done several tests to compare performance with the different versions of the Raspberry. And testing with the PLC based on Raspberry Pi 4 we have achieved an average of 7 FPS with web streaming, and with the Raspberry Pi 5 an average of 15 FPS, double that of the Raspberry Pi 4.


FPS can be improved by using image quality rescaling and compressions at the time of data sending to achieve smoother streaming.

There are many types of pre-trained models for any kind of target, already created models can be trained or re-trained, in order to improve the efficiency and accuracy of detections.

​Search in our Blog

Object detection w/ Raspberry PLC & AI
Boot & Work Corp. S.L., DAVID CORONADO TRAVASET March 21, 2024
Share this post

Looking for your ideal Programmable Logic Controller?

Take a look at this product comparison with other industrial controllers Arduino-based. 

We are comparing inputs, outputs, communications and other features with the ones of the relevant brands.


Industrial PLC comparison >>>