ST AEKD-AICAR1 Manual de usuario
Introduction
The AEKD-AICAR1 is a versatile deep learning system based on a long-short term memory (LSTM) recurrent neural network
(RNN). It is able to classify the car state:
•car parked
• car driving on a normal condition road
• car driving on a bumpy road
• car skidding or swerving
The main idea is to define an AI-car sensing node ECU with an embedded artificial intelligence processing.
The system hosts an SPC58EC Chorus 4M microcontroller, which is able to acquire discrete acceleration variations on a
three-axis reference system.
The AIS2DW12 motion sensor mounted on the AEK-CON-SENSOR1 board retrieves inertial data. The acquired data are
transmitted to the LSTM RNN, which classifies the car state. The classification result is shown on the AEK-LCD-DT028V1 LCD
touch display.
The LSTM RNN has been implemented and trained using the TensorFlow 2.4.0 framework (Keras) in the Google Colab
environment. The AI-SPC5Studio plug-in has been used to convert the resulting trained neural network into an optimized C
code library, which can run on an MCU with limited power computing resources.
Figure 1. AEKD-AICAR1 evaluation kit
The LSTM RNN training has been performed with several time-series acceleration waveforms recorded on a real vehicle in
motion. The resulting prediction accuracy, calculated by the confusion matrix, is about 93%. Field tests carried-out under all road
conditions with a sedan confirm the adherence of the computed results compared with the real road conditions.
Note: The AEKD-AICAR1 is an evaluation tool for R&D laboratory use only. It is not destined for use inside a vehicle.
Getting started with the AEKD-AICAR1 evaluation kit for car state classification
UM3053
User manual
UM3053 - Rev 1 - September 2022
For further information contact your local STMicroelectronics sales office. www.st.com
1Neural network basic principles
1.1 Artificial neural network
The artificial neural network works as the human brain neural network. Data are transferred to the neuron through
the inputs. Then, they are sent as an output after processing.
Figure 2. Artificial neural network
Artificial neural networks are built on layers of different neural units. Each unit consists of three parts:
• an input part that receives the data
• a hidden part that uses the neuron weight to calculate the result
• an output part that receives the calculation results and applies an eventual bias
The weight associated with each neuron determines its firing probability. The bias is the measure of assumption
made by the form of the output.
This architecture is typical for deep learning processes like data classification and pattern recognition neural
networks.
1.2 Long short-term memory recurrent neural network (LSTM RNN)
Traditional artificial neural networks keep no memory of what happened in the past. They take their decision only
on the data provided instant by instant. These architectures are well suited for data classification and pattern
recognition.
For other applications, like speech recognition, the proper classification requires a memory of the context, that is,
the prior words in the speech recognition application.
A recurrent neural network (RNN) is a class of artificial neural network that includes neurons connected in a loop.
UM3053
Neural network basic principles
UM3053 - Rev 1 page 2/39
Figure 3. Recurrent neural network
In most cases, the output values of an upper layer are used as the input for a lower level one. This
interconnection enables the use of one of the layers as the state memory and allows modeling a dynamic time
behavior dependent on the information previously received.
A typical recurrent neural network able to keep the information received is the long short-term memory (LSTM)
recurrent network. Unlike traditional artificial neural networks, LSTM has feedback connections. It can process not
only single data points but also entire sequences of data. For this reason, the LSTM neural network architecture is
the best candidate to model a deep learning system for an AI-car sensing node, using a collection of time-series
based on acceleration values.
UM3053
Long short-term memory recurrent neural network (LSTM RNN)
UM3053 - Rev 1 page 3/39
2Designing an AI-car sensing node
2.1 Tool-set introduction
An AI-car sensing node is an AI-deep learning system based on an LSTM recurring neural network, which can
provide a car state classification: parking, driving on a normal condition road, driving on a bumpy road, and car
skidding or swerving.
Figure 4. AI-car sensing node: car state classification
The LSTM RNN has been modeled with TensorFlow (Keras framework). This is an open-source software library
for machine learning, which provides optimized modules to implement AI algorithms related to the classification
problem.
A significant amount of computing power is required to implement sufficiently robust and efficient models. For
initial tests, we can rely on a standard machine. As the dataset size increases, the execution of complex training
algorithms becomes rapidly prohibitive.
To address this issue, there are several cloud services that offer computing power. Google Colab is an alternative
platform, which allows running the code directly on the cloud, even if with some limitations.
2.2 Creating a Google Colab notebook
To use the Google Colab platform, you need a Google account to login. After logging in, create a new “Colab
notebook” project file.
UM3053
Designing an AI-car sensing node
UM3053 - Rev 1 page 4/39
Figure 5. Project file
The document that you are creating is not a static web page. It is an interactive environment that lets you write
and execute your Python code to implement an LSTM RNN by using the TensorFlow framework.
2.3 Colab notebook setup and package importing
To implement and train an LSTM RNN, install the Tensorflow 2.4.0 framework on your Google Colab notebook by
using the packet index package (PIP) command.
%pip install tensorflow==2.4.0
Note: PIP is already installed on your Google Colab notebook.
Using the Python commands, import the following packages:
•Pandas
Open-source software library for data manipulation and analysis.
import pandas as pd
•Numpy
Open-source software library, adding support for large, multidimensional arrays and matrices, along with a
large collection of high-level mathematical functions to operate on these arrays.
import numpy as np
•Tensorflow
Open-source software library for machine learning, which provides optimized modules to implement artificial
intelligence (AI) algorithms.
import tensorflow as tf
•Sklearn
A package providing several common utility functions and transformer classes to change raw feature vectors
into a representation that is more suitable for the downstream estimators.
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.model_selection import train_test_split , StratifiedShuffleSplit
from sklearn.metrics import confusion_matrix
•OS
The Python language OS module has several useful functions to make the program interact with the
computer operating system.
import os
UM3053
Colab notebook setup and package importing
UM3053 - Rev 1 page 5/39
•Random
This module implements pseudo-random number generators for various distributions.
import random
•Seaborn
Seaborn is a Python data visualization library based on Matplotlib. It provides a high-level interface to draw
attractive and informative statistical graphics.
import seaborn as sn
•Matplot
Matplotlib is a comprehensive library to create static, animated, and interactive visualizations in Python.
import matplotlib.pyplot as plt
2.4 AI-car sensing node life cycle
The following steps define the life cycle of the AI-car sensing node implementation as a deep learning model for
classification:
1. Model definition
2. Model training
3. Model fitting and compilation
4. Model evaluation
2.4.1 Model definition
To define the model, we have chosen the topology of the LSTM network. The AI-car sensing node network is
based on the same network architecture of a speech recognition system. It consists of a neural convolution kernel
as an input layer, able to elaborate a convolution function with the input vector over a temporal dimension.
The input vector is a time sequence of TIMESERIES_LEN number of samples, which consist of discrete
acceleration variations on a three-axis (x, y, z) reference system: Δax, Δay, and Δaz.
TIMESERIES_LEN represents the minimum size of the temporal window for the car state classification. With an
acquisition sampling time equal to 100 msec, the value of TIMESERIES_LEN has been fixed to 50 samples (5
seconds per acquisition).
A dense function implements the output layer. This function can provide an output shape of four-dimensional
vectors (one for each expected status: parking, normal, bumpy, skid). The Softmax function activates this layer
and converts the output vector values to a probability distribution.
From an implementation perspective, this involves a model layer architecture built with a connection topology into
a cohesive model:
UM3053
AI-car sensing node life cycle
UM3053 - Rev 1 page 6/39
## Conv1D based model
model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(filters=16, kernel_size=3, activation='relu',
input_shape=(TIMESERIES_LEN, 3)),
tf.keras.layers.Conv1D(filters=8, kernel_size=3, activation='relu'),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dense(4, activation='softmax')
])
_________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d (Conv1D) (None, 48, 16) 160
conv1d_1 (Conv1D) (None, 46, 8) 392
dropout (Dropout) (None, 46, 8) 0
flatten (Flatten) (None, 368) 0
dense (Dense) (None, 64) 23616
dense_1 (Dense) (None, 4) 260
=================================================================
Total params: 24,428
Trainable params: 24,428
Non-trainable params: 0
As described in the above code block table, 24,428 trainable parameters define the AI-car sensing node network
model. These parameters are related to the LSTM RNN topology chosen.
2.4.2 Model training
The AI-car sensing node network training has been performed by acquiring datasets of acceleration variations in
the time domain:
•over a three-axis reference system (Δax, Δay, Δaz)
• with a frequency equal to 10 Hz
• a sample time of 100 msec
Each dataset targets a specific car state: parking, normal driving, bumpy driving or skidding.
The use of acceleration variations instead of raw acceleration data avoids misinterpretations of the car state. If
the car is parked on a road with a negative (or positive) slope (as shown in the figure below), the IMU registers
a nonzero X-axis acceleration that would not lead to the parking state. Instead, if we consider an acceleration
variation in the time domain, the result would be 0, leading to the correct parking state.
UM3053
AI-car sensing node life cycle
UM3053 - Rev 1 page 7/39
Figure 6. Acceleration variations
Car speed Vx = 0 Km/sec;
Car acceleration ax! = 0;
Temporal variation of acceleration Δax = 0
2.4.2.1 Training dataset
The acquisition of training datasets can be performed using a specific configuration of the AI-car sensing node
mock-up (acquisition mode). A proper source code #define enables the acquisition mode. In this mode, the
application sends each acceleration sample read from the sensor to a serial port with a baud rate set to 38400.
The serial data stream can be read through a common serial client (Tera Term or Putty). The data acquisition and
parsing in a CSV file are managed via a specific Python script (see Section Appendix C ).
The following figure shows an example of a time-based dataset generated with the Python script.
UM3053
AI-car sensing node life cycle
UM3053 - Rev 1 page 8/39
Figure 7. Time-based dataset example
The time column contains the recorded acceleration time samples. The other three columns contain the
acceleration values measured on the X, Y, and Z axis, respectively (without the gravity offset).
To prepare the file for the network training, you have to:
• Compute the acceleration variations (excel cell operation: B3 = B3–B2 for X axis).
UM3053
AI-car sensing node life cycle
UM3053 - Rev 1 page 9/39
• Add a status column for the car state for each specific acquisition (P = parking; N = normal; B = bumpy; S=
skid).
Figure 8. Adding the status column
• Create a CSV file for each car state through a specific acquisition task.
Figure 9. Creating CSV files
• Merge all CSV files by coping and pasting each row without changing the time column values.
• Add a new dummy row at the end of the file with the time value equal to 100.
• Add random values for Ax, Ay, and Ax status.
Note: A ready-to-use dataset (Diff_profile.csv) has been created to train the network for the car state classification.
2.4.2.2 Neural network training with Google Colab
Import the training dataset contained in the CSV file into the Colab notebook environment. Then, run a parser
function on the imported data to build an input vector compliant with the LSTM RNN.
The following code block shows the Google Colab script, which imports and loads the CSV file.
from google.colab import files
uploaded = files.upload()
db = pd.read_csv('Diff_profile.csv',sep=',')
Parse each column of the CSV file (status, Acc_x, Acc_y, Acc_z, time).
UM3053
AI-car sensing node life cycle
UM3053 - Rev 1 page 10/39
Tabla de contenidos
Otros manuales de Placa madre de ST
ST
ST STEVAL-USBPD45H Manual de usuario
ST
ST BlueNRG-LP Manual de usuario
ST
ST FDA803S Manual de usuario
ST
ST ST72F651 Manual de usuario
ST
ST STM32G081B-EVAL Manual de usuario
ST
ST SPC56B-DIS Manual de usuario
ST
ST UM1932 Manual de usuario
ST
ST STM32100B-EVAL Manual de usuario
ST
ST SPC563M64CAL144 Manual de usuario
ST
ST STEVAL-IDB007V Series Manual de usuario
ST
ST STM32H7 Nucleo-144 Manual de usuario
ST
ST STM32L4 Series Manual de usuario
ST
ST SPC58NHADPT386S Manual de usuario
ST
ST EVAL-L99H02QF Manual de usuario
ST
ST STEVAL-ILL059V1 Manual de usuario
ST
ST STM3221G-EVAL Manual de usuario
ST
ST EVALSP1310CPU Manual de usuario
ST
ST STM32H735G-DK Manual de usuario
ST
ST STM3210C-EVAL Manual de usuario
ST
ST STEVAL-PCC008V1 Manual de usuario
Manuales populares de Placa madre de otras marcas
Telit Wireless Solutions
Telit Wireless Solutions SL869-3DR Manual de usuario
Gigabyte
Gigabyte GA-9IVDT Manual de usuario
Texas Instruments
Texas Instruments ADS8372EVM Manual de usuario
Commell
Commell MS-C73 Manual de usuario
IBT Technologies
IBT Technologies MB860 Manual de usuario
Nvidia
Nvidia TEGRA DG-04927-001_V01 Manual de usuario
