Edge AI Optimization Stack¶
Wang et al. "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies", arXiv pp, arXiv:2501.03265v1, 2025
After identifying performance bottlenecks, you can apply Edge AI SOTA optimization techniques across the data pipeline, model, and system architecture (as shown in diagram above) to meet your application requirements.
For example, if you decide to use quantization, reducing the model precision from FP32 to INT8, it can reduce the memory footprint and speed up computation. You can use the following template to compare your model performance before and after applying optimization.
| Model | Bit-Width (W/A) | Size (MB) | Params (MB) | Top-1% Accuracy | Storage (MB) | Latency (ms) | Throughput (inf/s) | Energy (W) | Compute (OPS/GFlops) | Platform | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CPU | GPU | NPU | -- | |||||||||
| ResNet-50 | ||||||||||||
| FP32 (Full Precision) | - | - | - | - | - | - | - | - | - | - | MCU, Pi, Jetson Nano | |
| INT8 (Quantized) | - | - | - | - | - | - | - | - | - | - | ||
Data Optimization (Still to Come)¶
Model Optimization: A Full Process¶
System Optimization (Still to Come)¶
- HW
- SW
- Power