This is a comprehensive set of notes and resources.
- Introduction
- Why On-Device
- Deploying Segmentation Models On-Device
- Preparing for On-Device Deployment
- Quantizing Models
- Device Integration
- Conclusion
Resources: FFNet Paper
Resources:
- Expore more compiler options here (Compiling PyTorch to TensorFlow Lite).
Research Papers (from the course):
- FFNet (Fuss Free Network):
- Introducing Apple’s On-Device and Server Foundation Models
- Deep Learning On Mobile @ Ganju - 2020 Embedded Vision Summit Slides Final
- Model Compression: A Survey of Techniques, Tools, and Libraries
- 5 Types of ML Accelerators
- Introducing OpenVINO™ 2024.4
- Joint Pruning, Quantization and Distillation for Efficient Inference of Transformers
- https://www.ai-for-devs.com/blog/llama-3-2
- Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
- Qualcomm partners with Meta to support Llama 3.2. Why this is a big deal for on-device AI
- ExecuTorch
- ONNX Explained: A New Paradigm in AI Interoperability
- Edge AI: A Comprehensive Guide to its Engineering Principles and Applications
- Google AI Edge - Deploy AI across mobile, web, and embedded applications
- hat Is Edge AI? How It Works, Applications, Challenges
- ST Edge AI Suite - Tools: Create, optimize, and deploy your machine learning algorithms.
- AWS Machine Learning Blog - Demystifying machine learning at the edge through real use cases
- Model Compression: A Critical Step Towards Efficient Machine Learning
- Edge TPU inferencing overview
- Large Language Models On-Device with MediaPipe and TensorFlow Lite
Online Books: