Adding Vision to ESP32 with Edge Impulse

Reporting from Shanghai, China
Sep 29, 2021

You only need a few lines of additional code to run powerful ML algorithms on ESP32, especially if you also use the Arduino framework and Edge Impulse.

ESP32 is renowned for combining affordability, powerful wireless capabilities and energy efficiency. These characteristics have made ESP32 the SoC of choice in a wide range of AIoT solutions. Moreover, if you use the Arduino framework and Edge Impulse, ESP32 can run powerful machine-learning algorithms with only a few lines of additional code.

Edge Impulse has been using ESP32 for a while, with multiple different camera modules, to run image recognition ML models on-device. These models are capable of a variety of complex tasks: e.g. detecting whether a person is in frame, evaluating crop growth, or even estimating the weight of an object; all just from a photo!

With the free Edge Impulse platform for embedded machine learning, you can run any of these projects on ESP32, or create your own by following the "Adding Sight to Your Sensors" tutorial by Edge Impulse. The model can be deployed as an open-source Arduino library with everything you need to run your trained impulse, and this library can be added to an existing ESP32 Arduino project, with a single click.

Interested in trying this out yourself?

All you need to get started is an ESP32 dev kit and either an Arducam Mini 2MP Plus or an ESP32-CAM. The latter uses the same sensor as the Arducam, but includes first-party drivers to directly interface the sensor with the ESP32-CAM board. The “ESP32-CAM + Edge Impulse” demo identifies different plant species and is complete with a web interface, camera preview, and on-chip image resizing. Check out the code on GitHub or follow one of the ESP32-CAM walkthroughs linked below:

What’s more, these examples can run any trained image classification model just by replacing the Edge Impulse library. This works without any changes to the ESP32 firmware itself, meaning you can rapidly prototype and test your machine-learning algorithms in the real world.

Finally, Edge Impulse also supports processing any type of sensor data on ESP32, not just images! A minimal code example for running any standalone Edge Impulse model on ESP32 can be found here. From this starting point, you can add any Arduino compatible sensor drivers, and use the Edge Impulse data forwarder to collect training data.