Point, Shoot, and Detect: Object Detection with PyTorch Mobile

In this article we will set up a Mobile Application for Object Detection. We will use PyTorch Mobile to optimize and run the YOLOv5 on an Android app.

AI mobile app development Challenges

Developing AI applications in mobile is difficult due to slow loading speeds and scalability problems. They heavily rely on cloud infrastructure, In addition, certain AI frameworks or designs can't be used on mobile devices, limiting their access and usefulness.

Larger AI systems need cloud connections, making them costly. Developers need to make algorithms compatible with mobile devices. We should be able to execute our AI locally on the phone instead of using the cloud as a Service. This will contribute to making AI more accessible and efficient in mobile applications. In this article, we will introduce you to PyTorch Mobile and setup a basic app for testing out this framework.

PyTorch

PyTorch is a machine learning framework based on the Torch library. People use PyTorch's tools to make neural networks and machine learning algorithms. PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on a CUDA-capable NVIDIA GPUs.

About PyTorch mobile

Running machine learning algorithms on mobile devices for prediction is crucial in today's tech landscape. The move to edge computing is not about speed. It's also about protecting user privacy and enabling new interactive experiences. PyTorch Mobile helps connect training and deployment in the PyTorch environment. This platform streamlines the journey from model inference in mobile applications. PyTorch Mobile is in beta and gaining popularity for widespread production use. This software has APIs that will make mobile based machine learning better in the future.

Next, we will focus on trying out an Android app to test PyTorch Mobile capabilities.

Setting up the Demo App

YOLOv5 Vision AI

YOLO is one of the popular open-source vision AI. YOLOv5 is a great object detection algorithm that uses Deep Convolution Neural Networks(DCNN). This helps to predict object classes instantaneously.YOLOv5 can detect people, vehicles, and animals in various environments. Its full form is "You only look once". This indicates that it takes the input photo and spits it into grids. Then it assigns bounding boxes and probabilities to each grid. The current v5 works on a complex architecture called the EfficientDet network, which improves its capability to detect objects smoothly.

These are the steps we take to setup YOLO for mobile:

Optimizing the YOLO for the Android App

Clone repo and install requirements.txt in a Python>=3.8.0 environment

git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt

Create an optimized model for mobile

python export.py --weights yolov5s.pt --include torchscript --optimize

You will get a yolov5s.torchscript which is a mobile-optimized version of the model. Then we move it to the app directory and import it at ObjectDetection/app/src/main/assets/yolov5s.torchscript.

Setting up the Android App

We will use an app from the PyTorch example repository to demonstrate PyTorch Mobile.

This is the repository here

For setup download the latest Android Studio from here

Setup the Android Studio SDK from the SDK tools menu:

Connect to a phone using wireless debugging.

Run the gradle script with the --stacktrace option to show more debug information.

./gradlew installDebug --stacktrace

This will create a prompt on your phone to install the app.

Results

Conclusion

This concludes our exploration of PyTorch Mobile. We got to know how to run algorithms in a mobile Android app and set up basic object detection. Hope this article was useful for you.