> ## Documentation Index
> Fetch the complete documentation index at: https://docs.edgeimpulse.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Predictive Maintenance Using Audio Classification - Arduino Nano 33 BLE Sense

> Use audio classification on an Arduino Nano 33 BLE Sense to detect machine anomalies for predictive maintenance.

Created By: Shebin Jose Jacob

Public Project Link: [https://studio.edgeimpulse.com/public/162492/latest](https://studio.edgeimpulse.com/public/162492/latest)

## Intro

Every manufacturing environment is equipped with machines. For a better-performing manufacturing unit, the health of machines plays a major role and hence maintenance of the machines is important. We have three strategies of maintenance namely - Preventive maintenance, Corrective maintenance, and Predictive maintenance.

If you want to find the best balance between preventing failures and avoiding over-maintenance, Predictive Maintenance (PdM) is the way to go. Equip your factory with relatively affordable sensors to track temperature, vibrations, and motion data, use predictive techniques to schedule maintenance when a failure is about to occur, and you'll see a nice reduction in operating costs.

In the newest era of technology, teaching computers to make sense of the acoustic world is now a hot research topic. So in this project, we use sound to do some predictive maintenance using an Arduino Nano 33 BLE Sense.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/intro.jpeg?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=eb02af4252367c9a73facaf3d8cf6b9b" alt="Arduino Nano 33 BLE Sense predictive maintenance device in a blue 3D-printed enclosure" width="1500" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/intro.jpeg" />
</Frame>

## How Does It Work?

We use the Nano 33 BLE Sense to listen to the machine continuously. The MCU runs an ML model which is trained on two sets of acoustic anomalies and a normal operation mode. When the ML model identifies an anomaly, the operator is immediately notified and the machine may be shut down for maintenance after proper inspection. Thus, we can reduce the possible damage caused and can reduce the downtime.

## Hardware Requirements

* Nano 33 BLE Sense
* LED

## Software Requirements

* Edge Impulse
* Arduino IDE

## Hardware Setup

The hardware setup consists of a Nano 33 BLE Sense, which is placed beside an old AC motor.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/motor.jpeg?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=e5bc31fae2455ef7e13e5d639f530453" alt="An old AC motor used as the audio source for anomaly data" width="740" height="493" data-path=".assets/images/predictive-maintenance-with-sound/motor.jpeg" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/nano.jpeg?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=3b91a4ff2b5be612e19c829919e6d7d4" alt="Arduino Nano 33 BLE Sense positioned beside the motor for sound capture" width="1568" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/nano.jpeg" />
</Frame>

## Software Setup

If you haven't connected the device to Edge Impulse dashboard, follow [this tutorial](/hardware/boards/arduino-nano-33-ble-sense) to get it connected. After a successful connection, it should be present in the **Devices** tab.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/devices.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=cd37e13e77e2567edcaecd3d6021ba0d" alt="Edge Impulse Devices page listing the connected Nano 33 BLE Sense" width="1600" height="923" data-path=".assets/images/predictive-maintenance-with-sound/devices.png" />
</Frame>

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. Follow [this tutorial](https://www.edgeimpulse.com/blog/collect-sensor-data-straight-from-your-web-browser) to learn more about it.

## TinyML Model Generation

### 1. Data Collection

Clean data is the most important requirement to train a well-performing model. In our case, we have collected 3 classes of sound - two classes of anomalies, one normal operation class, and a noise class. Each sample is 2 seconds long. The raw data of these classes is visualised below.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/data-1.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=a736239dd6afbf1917e07db62669bd74" alt="Raw audio waveform for the normal operation class" width="1359" height="880" data-path=".assets/images/predictive-maintenance-with-sound/data-1.png" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/data-2.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=6a028563f49c921212e9c634f67f9409" alt="Raw audio waveform for the first anomaly class" width="1367" height="885" data-path=".assets/images/predictive-maintenance-with-sound/data-2.png" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/data-3.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=6429ce30f94676acf01caa2d453b0e92" alt="Raw audio waveform for the second anomaly class" width="1370" height="881" data-path=".assets/images/predictive-maintenance-with-sound/data-3.png" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/data-4.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=d6217c4a12531e663fcee12546d8cb07" alt="Raw audio waveform for the noise class" width="1375" height="882" data-path=".assets/images/predictive-maintenance-with-sound/data-4.png" />
</Frame>

If the data is not split into training and testing datasets, split the dataset into training and testing datasets in the ratio 80:20, which forms a good dataset for model training.

### 2. Impulse Design

An Impulse is the machine learning pipeline that takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/impulse.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=090510edec580e16f7556f57df60d341" alt="Impulse design with time series input, MFE processing, and classification blocks" width="1600" height="924" data-path=".assets/images/predictive-maintenance-with-sound/impulse.png" />
</Frame>

Here we are using the **Time Series data** as the input block. Now, we have two choices for the processing block - MFCC and MFE. As we are dealing with non-vocal audio and MFE performs well with non-vocal audio, we have chosen **MFE** as our processing block. We have used **Classification** as our learning block since we have to learn patterns and apply them to new data to categorize the audio into one of the given 4 classes.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/parameters.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=8a4ed77f8765a2d52f9c5776b9908fd9" alt="MFE parameter settings for the audio classification impulse" width="1600" height="926" data-path=".assets/images/predictive-maintenance-with-sound/parameters.png" />
</Frame>

In the **MFE** tab, you can tweak the parameters if you're good with audio handling, else leave the settings as it is and generate features.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/features.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=09154e4f69b374bd2a5bca0b63098994" alt="Feature explorer showing generated MFE clusters for the audio dataset" width="1600" height="925" data-path=".assets/images/predictive-maintenance-with-sound/features.png" />
</Frame>

### 3. Model Training and Testing

Now that we have our Impulse designed, let's proceed to train the model. The settings we employed for model training are depicted in the picture. You can play about with the model training settings so that the trained model exhibits a higher level of accuracy, but be cautious of overfitting.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/training.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=6df1cadf3a7a5555d4cd65546fcff4d0" alt="Classifier training settings for the acoustic anomaly model" width="939" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/training.png" />
</Frame>

A whopping 94.7% accuracy is achieved by the trained model.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/yRS4dkKDI2LDSlCA/.assets/images/predictive-maintenance-with-sound/accuracy.png?fit=max&auto=format&n=yRS4dkKDI2LDSlCA&q=85&s=61dec3802b94038b8ae3bf54ac42ea54" alt="Training results showing the audio classifier accuracy and confusion matrix" width="1600" height="926" data-path=".assets/images/predictive-maintenance-with-sound/accuracy.png" />
</Frame>

Let's now use some unknown data to test the model's functionality. To assess the model's performance, move on to **Model Testing** and **Classify All**.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/model-testing.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=387478374076af12dd8e60ba90eac6c4" alt="Model testing page with classify all results for unseen audio samples" width="1600" height="925" data-path=".assets/images/predictive-maintenance-with-sound/model-testing.png" />
</Frame>

We have got 95.07% testing accuracy, which is pretty awesome. Now let's test the model with some real-world data. Navigate to **Live Classification** and collect some data from the connected device.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/yRS4dkKDI2LDSlCA/.assets/images/predictive-maintenance-with-sound/classification-1.jpeg?fit=max&auto=format&n=yRS4dkKDI2LDSlCA&q=85&s=5a28b0a1ba641d52dae92b5410ede7e7" alt="Live classification result for normal operation audio" width="1600" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/classification-1.jpeg" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/yRS4dkKDI2LDSlCA/.assets/images/predictive-maintenance-with-sound/classification-2.jpeg?fit=max&auto=format&n=yRS4dkKDI2LDSlCA&q=85&s=a2bb594f8b211711f72ae5100a65c8db" alt="Live classification result for the first acoustic anomaly" width="1600" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/classification-2.jpeg" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/yRS4dkKDI2LDSlCA/.assets/images/predictive-maintenance-with-sound/classification-3.jpeg?fit=max&auto=format&n=yRS4dkKDI2LDSlCA&q=85&s=7b9c500612e3312d726b70d07bbf30f9" alt="Live classification result for the second acoustic anomaly" width="1600" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/classification-3.jpeg" />
</Frame>

We have collected some real-world data of **Normal Operation Mode**, **Anomaly 1**, and **Anomaly 2** respectively, and all of them are correctly classified. So our model is ready for deployment.

## Deployment

For deployment, navigate to the **Deployment** tab, select **Arduino Library** and build the library. It will output a zip library, which can be added to **Arduino IDE**.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/deployment.png?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=d7e88667e8c9ccc7180e6775bb35b2dd" alt="Deployment page configured to build an Arduino library" width="1164" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/deployment.png" />
</Frame>

## Final Product

Nano 33 BLE Sense along with an LED is enclosed in a 3D printed case, which is our final product. The device is capable of identifying acoustic anomalies in a machine and alerts the user using the alert LED.

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/product.jpeg?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=ef84491b52ef5fb150c6cb0b5b05492a" alt="The final Nano 33 BLE Sense enclosure with an external alert LED" width="1500" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/product.jpeg" />
</Frame>

<br />

<Frame>
  <img src="https://mintcdn.com/edgeimpulse/iTt6mNOXOy0YNrkb/.assets/images/predictive-maintenance-with-sound/intro.jpeg?fit=max&auto=format&n=iTt6mNOXOy0YNrkb&q=85&s=eb02af4252367c9a73facaf3d8cf6b9b" alt="The completed acoustic anomaly detector enclosure with the Nano board inside" width="1500" height="1000" data-path=".assets/images/predictive-maintenance-with-sound/intro.jpeg" />
</Frame>
