Smart maintenance based on vehicle CAN bus data from scratch in Python
In this article we prototype an algorithm that automatically scores engine health based on vehicle CAN bus data.
- Summary
- Fetch the data
- Load libraries
- Load vehicle data
- Turn time series data into tables
- Monitoring engine health
- Anomaly detection algorithm: learn what observations to expect to recognize anomalies
Summary
Equipment that behaves anomalously or breaks down unexpectedly is a major cost driver in manufacturing, logistics, public transport, and any other sector that relies on complex machinery.
A big promise of data analytics and machine learning in this space is to detect anomalies in machinery automatically and to alert their user of occurring faults. As an extension, the prediction of machinery faults and breakdowns is an important field of application.
Automated detection and prediction of machinery breakdown is a key algorithmic approach behind smart and predictive maintenance.
In this article we showcase a simple algorithmic approach for anomaly detection in the space of automated engine health detection.
Our approach here can be an interesting starting point for the development of smart telematics solutions for automated and predictive vehicle breakdown detection.
Fetch the data
We'll make use of an open data set of vehicle CAN bus data, called Automotive CAN bus data: An Example Dataset from the AEGIS Big Data Project.
A CAN bus is a local network of sensors and actuators in modern vehicles that provides a stream of data for all important signals of a vehicle - such as its present velocity, interior temperature, and potentially hundreds of other signals.
This data set encompasses time series data (traces) of various vehicles driven by different drivers.
Let's go ahead and download a data set for driver 1 and a data set for driver 2:
!wget --quiet https://zenodo.org/record/3267184/files/20181113_Driver1_Trip1.hdf
!wget --quiet https://zenodo.org/record/3267184/files/20181114_Driver2_Trip3.hdf
import h5py
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.mixture import GaussianMixture
plt.rcParams['figure.figsize'] = (10,10)
sns.set(style="darkgrid")
driver_1 = h5py.File('20181113_Driver1_Trip1.hdf', 'r')
driver_2 = h5py.File('20181114_Driver2_Trip3.hdf', 'r')
Both files contain multiple subgroups of data, one of which is the aformentioned CAN bus:
list(driver_1.keys())
list(driver_2.keys())
data_driver_1 = {}
data_driver_2 = {}
for channel_name, channel_data in driver_1['CAN'].items():
data_driver_1[channel_name] = channel_data[:, 0]
table_driver_1 = pd.DataFrame(
data=data_driver_1,
index=channel_data[:, 1]
)
table_driver_1 = table_driver_1.loc[:, table_driver_1.nunique() > 1]
for channel_name, channel_data in driver_2['CAN'].items():
data_driver_2[channel_name] = channel_data[:, 0]
table_driver_2 = pd.DataFrame(
data=data_driver_2,
index=channel_data[:, 1]
)
table_driver_2 = table_driver_2.loc[:, table_driver_2.nunique() > 1]
The tabular data for driver 1 looks as follows - it holds 158,659 measured time points in 28 channels that we deem relevant:
table_driver_1
The tabular data for driver 2 looks as follows - it holds 136,154 measured time points in 29 channels that we deem relevant:
table_driver_2
Monitoring engine health
One use case of automated anomaly detection lies in checking the health status of a vehicle's engine.
Here we'll look at engine oil temperature as a function of velocity: As you'll notice in the below plots, oil temperature goes up with higher velocity and increased operating duration.
Engine 1 - healthy
Let's look at the engine of the vehicle of driver 1 where engine oil temperature appears normal as it keeps to within a certain band:
temperature_1 = table_driver_1[['VehicleSpeed', 'OilTemperature1']].copy()
sns.relplot(
x='index',
y='value',
hue='channel',
kind='line',
data=temperature_1.reset_index().melt(id_vars='index', var_name='channel')
);
Plotting engine oil temperature against vehicle velocity makes the correlation between the two metrics more apparent:
plot = sns.scatterplot(
x='VehicleSpeed',
y='OilTemperature1',
data=temperature_1,
color='blue',
alpha=.1
);
plot.axes.set_ylim(0., 110.);
temperature_2 = table_driver_2[['VehicleSpeed', 'OilTemperature1']].copy()
temperature_2.loc[5000:5500, 'OilTemperature1'] *= 1.15
sns.relplot(
x='index',
y='value',
hue='channel',
kind='line',
data=temperature_2.reset_index().melt(id_vars='index', var_name='channel')
);
plot = sns.scatterplot(
x='VehicleSpeed',
y='OilTemperature1',
data=temperature_2,
color='blue',
alpha=.1
);
plot.axes.set_ylim(0., 140.);
temperature_1['vehicle'] = 'vehicle 1'
temperature_2['vehicle'] = 'vehicle 2'
combined = pd.concat([temperature_1, temperature_2], axis=0, sort=True)
plot = sns.scatterplot(
x='VehicleSpeed',
y='OilTemperature1',
data=combined,
hue='vehicle',
alpha=.1
);
plot.axes.set_ylim(0., 140.);
You'll notice that the engine oil temperature of vehicle 2 tends to be higher than of vehicle 1 and vehicle 2 shows an island of high engine oil temperature separated from the bulk of data points.
We can use this by modelling the distribution of value pairs velocity and oil temperature that we would usually expect to observe.
Let's model the expected distribution of data points with a model called Gaussian mixture models. This model fits a set of Gaussian distributions to the distribution of value pairs we observed for vehicle 1 - thus defining what a healthy distribution of values looks like.
model = GaussianMixture(n_components=4)
model.fit(temperature_1[['VehicleSpeed', 'OilTemperature1']])
After training our model on the observations for vehicle 1, let's score the likelihood of observing each observation we have on vehicle 2:
temperature_2['health_score'] = model.score_samples(temperature_2[['VehicleSpeed', 'OilTemperature1']])
sns.relplot(
x='index',
y='value',
hue='metric',
kind='line',
data=temperature_2[['OilTemperature1', 'health_score']].reset_index().melt(id_vars='index', var_name='metric')
);
To clarify what we did here: We learned what a healthy distribution of vehicle velocity and engine oil temperature looks like based on data from vehicle 1 and applied this model to data from vehicle 2.
Looking at the health score we compute for vehicle 2 we notice that our health score drops markdely between 5000 and 5500 seconds into our CAN bus trace - exactly where oil temperature spikes unhealthily.
While we would need to do a lot more validation and model calibration before we could use this approach in a live environment, we can already see the potential of this approach.
With this approach we can start devising data-driven products and services for smart telematics and smart maintenance applications.