Share

N2D2: design and implementation of neural networks optimized for embedded systems

This complete deep learning platform was developed for the design of deep neural networks deployed in constrained operating environments. Industrial companies can use N2D2 to integrate advanced data processing functions into their embedded systems.

How N2D2is used

Embedding AI into constrained systems

N2D2 (Neural Network Design and Deployment) addresses the need to integrate image recognition (object classification and detection), signal processing, and other AI functions into systems constrained by power consumption or dissipation, latency, form factor (dimensions, size, etc.), and/or cost criteria.

CEA-List embedded systems engineers developed N2D2 for the design of deep neural networks optimized specifically for embedded systems. N2D2 has functions to reduce the computational complexity of models and their memory requirements, most often using quantization (during or after training) and topological optimization techniques. Comparative analyses are performed to select the best hardware target (MCU, GPU, or FPGA) according to the constraints involved and then generate an optimized implementation using the most appropriate programming model (C++, TensorRT, OpenCL, RTL, etc.).

N2D2 is available on GitHub under the CeCILL-C open-source license (equivalent to LGPL, a non-contaminating license). N2D2 is open to contributions and improvements from its users.

Portability and interoperability

N2D2 has many advantages:

  • an integrated approach encompassing the entire design flow, from application development to deployment: data formatting, neural network exploration, learning, testing, and optimized code generation;
  • compatibility with a wide range of commercially available hardware targets, offering optimized implementations in C++ for MCUs and DSPs (STM32 and ARM cores in particular), TensorRT for GPUs, OpenCL for DSPs, GPUs, and FPGAs;
  • optimized operation with the specialized hardware accelerators developed at CEA: PNeuro (an SIMD DSP for ASICs, currently being scaled up by one of our partners) and DNeuro (data flow architecture optimized for FPGAs);
  • a high degree of interoperability, with support for the ONNX standard and integration with the PyTorch and Keras platforms;
  • a modular design and simple abstraction layer, so features can be added and modified with ease, including the low-level implementation of calculation functions depending on the specific characteristics of the hardware being deployed (approximate calculation, specific saturated arithmetic, etc.);
  • sovereignty and control of the code, as N2D2 is totally independent of other deep learning platforms;
  • a multiparadigm approach that integrates the simulation of neuromorphic neural network models into the same platform. These models use impulse information coding to mimic how biological neurons work and could support even more efficient implementations of neural networks on new non-volatile memory technologies.

We decided to work with N2D2 because of the flexibility of porting applications to different platforms—N2D2’s primary advantage.

Gabriel FRICOUT

R&D Measurement and Control Team Leader — ArcelorMittal

Security, manufacturing, marketing

Use cases:

  • security and defense: intrusion detection and access control in constrained environments, segmentation of aerial and space images, real-time target re-identification;
  • manufacturing: real-time in-line quality control using image recognition and anomaly prediction or detection;
  • transportation: real-time environmental perception for autonomous vehicles using image segmentation and object detection;
  • smart marketing: behavioral analysis and personalized targeting.
Use case

Machine vision

ArcelorMittal used N2D2 to deploy machine vision systems to improve its surface inspection processes and detect product defects in real time. Because the code is portable, their teams were able to retrieve C executables, run them on their machines, and then deploy their developments to GPUs and FPGA/DSP platforms.

With the core of N2D2 now open source, ArcelorMittal has confidence that the platform is here to stay and that new functions will be added over time.

See also

press releases

July 19, 2021 | Dolphin Design and CEA-List

Dolphin Design and CEA-List have formed a new joint R&D lab of embedded systems. Their goal is to achieve the best tradeoff for Edge AI devices between SW flexibility, energy efficiency, and peak perf...
Read more