bharathsudharsan / TinyML-CAM
- четверг, 3 ноября 2022 г. в 00:35:08
Code for MobiCom'22 paper 'TinyML-CAM: 80 FPS Image Recognition in 1 Kb RAM'
ESP32 classifying Raspberry Pi Pico, Portenta H7, Wio Terminal from image frames
Following can be observed from the video:
Time. For image frames, the digital signal processing (DSP) based features extraction time is ≈ 12 ms, while classification time is ≈ < 20 𝜇𝑠 (1/1000th of DSP).
FPS. It is 1000/12 ms = 83.3 FPS, which is the time taken by the TinyML-CAM system for HOG features extraction (using DSP) plus classification. Since the ESP32 has a 30 FPS frame rate, just to capture frames, it takes 1000/30 = 33 ms. Since the DSP plus classification time is only ≈ 12 ms, the image recognition happens in real-time between two consecutive frames, thus not altering the ESP32 camera's FPS.
Accuracy. As expected during Pairplot analysis, Portenta and Pi (features overlapped) are mislabelled quite often, which can be rectified by improving dataset quality.
Memory. Consumes only 1 kB of RAM - difference between the RAM calculated by Arduino IDE before and after adding the TinyML-CAM image recognition system.
https://dl.acm.org/doi/pdf/10.1145/3495243.3558264
To lower the DSP time (currently 12 ms) by implementing mathematical approximation methods, which will boost the frame rate - i.e., if reduced to 6 ms, then 1000/6 ms = 166.6 FPS.
Similar to the TinyML benchmark, we plan to test the pipeline on a range of datasets, ML algorithms, and IoT boards.