ENG) Crosswalk auxiliary machine with Jetson Nano

Github Repo

https://github.com/minvamos/Jetson_Detection_System

GitHub - minvamos/Jetson_Detection_System: Real-time object detection using Jetson Nano 2GB and Yolov5 custom models

Why?

Before proceeding with the project with Jetson Nano, I thought it was important to take advantage of Jetson Nano's strengths. If so, I wanted to take advantage of the following features.

1. Small size
2. GPU-equipped

Therefore, using gpu, it was determined that a portable device using a deep learning model would be good.

The final project to be carried out is a crosswalk use assistance device mounted on the visually impaired.

 

Idea

The Jetson Nano is equipped with a sound card and camera and functions as a portable device using a mobile battery.
When crosswalks (green light + safe crosswalks) are recognized, the user is informed of the crosswalk status through the earphone.

 

Algorithm

 

It operates in an infinite loop by default, and reads the frame of the image every cycle. In the frame, class 0 (Green light), class 1 (Safety crosswalk) is detected, indicating the traversable state.

 

Custom model

A custom model was created using labeled photos based on YOLOv5.

YOLO was chosen because YOLO thought it would be advantageous for real-time detection because it processes images at once, and also expects to detect crosswalks and traffic lights well even in situations caused by various variables because of high accuracy in new data.

 

Training

Labeling was performed on 3201 images using LabelIMG. Classes are as follows.
0 : Green Light
1 : Safety crosswalk
2 : Dangerous crosswalk
3 : Red Light
2642 images were used for training and 659 images for validation.

Optimization

For optimization, the following procedure was carried out.

 

Memory Swap : 5GB
Camera resolution: 3264x2448 > 640x480
Light Model Selection: YOLOv5 Nano
Converting Yolo-based models to tensorRT engines

 

Using the TensorRT engine, you can expect performance improvements using the NVIDIA GPU acceleration on the Jetson Nano.

Performance Improvement

In terms of FPS, there was a performance improvement of about 8 times or more.

 

Operating Video

https://youtu.be/FFOofk3XE1s?si=QAbxF3lAwSqtwNZk