Integrating Resource-Constrained Devices into MANOLO Benchmarking

MANOLO aims to develop a comprehensive benchmarking framework that accommodates both edge and cloud computing resources. In this context, resource-constrained devices like microcontrollers and Fraunhofer’s ADELIA processors require a different approach compared to high-performance cloud resources that utilize powerful GPUs and CPUs. Consequently, the framework must be designed to seamlessly integrate these diverse computing platforms.

Extensive discussions among MANOLO partners, including ATOS, FRAUNHOFER IIS, and Demokritos “NCSR”, were essential to explore viable implementation strategies. The insights from these discussions have been incorporated into the MANOLO framework, enabling effective benchmarking across various hardware platforms. This collaborative approach allows for accurate performance estimations of AI models on the specified devices, empowering MANOLO users to consider a wide range of hardware options in their evaluations.

Part of the agreed strategy includes local benchmarking experiments in the laboratories of Fraunhofer for different microcontrollers and Fraunhofer’s ADELIA processor. Based on these benchmarking experiments, estimation functions will be derived which will allow an end user to get a performance estimation (including latency and energy efficiency) for future AI workloads for these resource constrained devices.

In conclusion, Fraunhofer will contribute to the mentioned benchmarking experiments in their laboratories, and it will integrate the derived estimation functions in the MANOLO Framework. These activities will contribute to the project’s goal to provide a comprehensive benchmarking framework. Also, this will bring the EU community forward by enabling the benchmarking of a wide set of hardware platforms ranging from cloud resources to edge devices such as microcontrollers and Fraunhofer’s ADELIA processor.