What can deep learning do for you?
Mobile phones and tablets are now powerful enough to take advantage of the awesome opportunities offered by machine learning and artificial intelligence.
Here are some of the things your app can do with deep learning:
Computer vision
- detect objects, faces & landmarks on photos, videos, or live camera
- recognize handwriting and printed text within images
- search using pictures
- track motion and poses
- recognize gestures
- understand emotional cues in photos and videos
- enhance images and remove imperfections
- automatically tag and categorize visual content
- add special effects and filters
- detect explicit content
- create 3D models of interior spaces
- augmented reality
Natural language processing
- extract meaning from text
- chatbots and conversational UI
- translate text into other languages
- intelligent spelling correction
- summarize texts
- detect topics and sentiment
Speech and audio
- speech recognition and synthesis
- translate speech in real-time
- automatic subtitling of videos
- recommend music based on your mood or surroundings
- intelligently filtering audio
- sound / music recognition
- language or dialect detection
- identify speakers by voice
And much more… There’s no limit to what’s possible when you combine the raw power of deep learning with the phone’s camera and its many other sensors.
Deep learning turns your mobile into a truly smart assistant.
Deep learning on the device
Companies such as Google, Amazon, and Microsoft offer deep learning services in the cloud — but there are many advantages to doing deep learning directly on the user’s device, for a more seamless user experience.
Getting deep learning to work well on mobile comes with its own set of challenges:
- It needs to be fast — which means using optimized compute kernels on the GPU. Building and deploying those compute kernels takes special skills.
- The model needs to be small enough to fit in memory while still providing great performance. Compressing models to remove unnecessary parts is as much an art as a science.
- Training on the device is currently impractical for most applications. To keep the model up-to-date, you need to have some way to retrain your model and then distribute the new weights to the user’s device.
- To train on user data, you need some way to collect data from their devices and gather it in the cloud — without violating the user’s privacy. Doing this right requires care and consideration.
Making deep learning models work well on iOS devices is what I specialize in. Let’s talk if you need advice on how to add deep learning to your app.
iOS Machine Learning Books
Machine Learning by Tutorials explains how to get started with machine learning for people who are already familiar with iOS development.
Core ML Survival Guide is for developers who are running into problems getting their models to work with Core ML — or who want to do advanced things that are not well documented elsewhere.
MobileNet source code library
The MobileNet neural network architecture is designed to run efficiently on mobile devices. It’s a fast, accurate, and powerful feature extractor. I recommend using it over larger and slower architectures such as VGG-16, ResNet, and Inception.
Because MobileNet-based models are becoming ever more popular, I’ve created a source code library for iOS and macOS that has Metal-accelerated implementations of MobileNet V1 and V2.
This library makes it easy to add MobileNet-based neural networks into your apps, for tasks such as:
- image classification
- real-time object detection — with SSD or SSDLite
- semantic image segmentation — with DeepLabv3+
- as a feature extractor that is part of a custom model
Because this library is written to take advantage of Metal, it is much faster than Core ML and TensorFlow Lite!
If you’re interested in using MobileNet in your app or as the backbone for a larger model, this library is the best way to get started. Click to learn more
Who am I?
Matthijs Hollemans
Programming is the most fun for me when it’s difficult. That’s why I specialize in low-level coding, GPU optimization, and algorithms. My current research interest is deep learning on iOS, and I’d love to help you add deep learning into your app. Send me an email