Skip to content
Search
Generic filters
Exact matches only

Display Deep Learning Model Training History in Keras

Last Updated on October 3, 2019

You can learn a lot about neural networks and deep learning models by observing their performance over time during training.

Keras is a powerful library in Python that provides a clean interface for creating deep learning models and wraps the more technical TensorFlow and Theano backends.

In this post you will discover how you can review and visualize the performance of deep learning models over time during training in Python with Keras.

Discover how to develop deep learning models for a range of predictive modeling problems with just a few lines of code in my new book, with 18 step-by-step tutorials and 9 projects.

Let’s get started.

  • Update Mar/2017: Updated for Keras 2.0.2, TensorFlow 1.0.1 and Theano 0.9.0.
  • Update Mar/2018: Added alternate link to download the dataset.
  • Update Sep/2019: Updated for Keras 2.2.5 API.
  • Update Oct/2019: Updated for Keras 2.3.0 API.

Display Deep Learning Model Training History in Keras

Display Deep Learning Model Training History in Keras
Photo by Gordon Robertson, some rights reserved.

Access Model Training History in Keras

Keras provides the capability to register callbacks when training a deep learning model.

One of the default callbacks that is registered when training all deep learning models is the History callback. It records training metrics for each epoch. This includes the loss and the accuracy (for classification problems) as well as the loss and accuracy for the validation dataset, if one is set.

The history object is returned from calls to the fit() function used to train the model. Metrics are stored in a dictionary in the history member of the object returned.

For example, you can list the metrics collected in a history object using the following snippet of code after a model is trained:


# list all data in history
print(history.history.keys())

# list all data in history

print(history.history.keys())

For example, for a model trained on a classification problem with a validation dataset, this might produce the following listing:

[‘accuracy’, ‘loss’, ‘val_accuracy’, ‘val_loss’]

[‘accuracy’, ‘loss’, ‘val_accuracy’, ‘val_loss’]

We can use the data collected in the history object to create plots.

The plots can provide an indication of useful things about the training of the model, such as:

  • It’s speed of convergence over epochs (slope).
  • Whether the model may have already converged (plateau of the line).
  • Whether the mode may be over-learning the training data (inflection for validation line).

And more.

Need help with Deep Learning in Python?

Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with code).

Click to sign-up now and also get a free PDF Ebook version of the course.

Start Your FREE Mini-Course Now!

Visualize Model Training History in Keras

We can create plots from the collected history data.

In the example below we create a small network to model the Pima Indians onset of diabetes binary classification problem. This is a small dataset available from the UCI Machine Learning Repository. You can download the dataset and save it as pima-indians-diabetes.csv in your current working directory (update: download from here).

The example collects the history, returned from training the model and creates two charts:

  1. A plot of accuracy on the training and validation datasets over training epochs.
  2. A plot of loss on the training and validation datasets over training epochs.

# Visualize training history
from keras.models import Sequential
from keras.layers import Dense
import matplotlib.pyplot as plt
import numpy
# load pima indians dataset
dataset = numpy.loadtxt(“pima-indians-diabetes.csv”, delimiter=”,”)
# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]
# create model
model = Sequential()
model.add(Dense(12, input_dim=8, activation=’relu’))
model.add(Dense(8, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))
# Compile model
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
# Fit the model
history = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)
# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history[‘accuracy’])
plt.plot(history.history[‘val_accuracy’])
plt.title(‘model accuracy’)
plt.ylabel(‘accuracy’)
plt.xlabel(‘epoch’)
plt.legend([‘train’, ‘test’], loc=’upper left’)
plt.show()
# summarize history for loss
plt.plot(history.history[‘loss’])
plt.plot(history.history[‘val_loss’])
plt.title(‘model loss’)
plt.ylabel(‘loss’)
plt.xlabel(‘epoch’)
plt.legend([‘train’, ‘test’], loc=’upper left’)
plt.show()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

# Visualize training history

from keras.models import Sequential

from keras.layers import Dense

import matplotlib.pyplot as plt

import numpy

# load pima indians dataset

dataset = numpy.loadtxt(“pima-indians-diabetes.csv”, delimiter=”,”)

# split into input (X) and output (Y) variables

X = dataset[:,0:8]

Y = dataset[:,8]

# create model

model = Sequential()

model.add(Dense(12, input_dim=8, activation=’relu’))

model.add(Dense(8, activation=’relu’))

model.add(Dense(1, activation=’sigmoid’))

# Compile model

model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])

# Fit the model

history = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)

# list all data in history

print(history.history.keys())

# summarize history for accuracy

plt.plot(history.history[‘accuracy’])

plt.plot(history.history[‘val_accuracy’])

plt.title(‘model accuracy’)

plt.ylabel(‘accuracy’)

plt.xlabel(‘epoch’)

plt.legend([‘train’, ‘test’], loc=’upper left’)

plt.show()

# summarize history for loss

plt.plot(history.history[‘loss’])

plt.plot(history.history[‘val_loss’])

plt.title(‘model loss’)

plt.ylabel(‘loss’)

plt.xlabel(‘epoch’)

plt.legend([‘train’, ‘test’], loc=’upper left’)

plt.show()

The plots are provided below. The history for the validation dataset is labeled test by convention as it is indeed a test dataset for the model.

From the plot of accuracy we can see that the model could probably be trained a little more as the trend for accuracy on both datasets is still rising for the last few epochs. We can also see that the model has not yet over-learned the training dataset, showing comparable skill on both datasets.

Plot of Model Accuracy on Train and Validation Datasets

Plot of Model Accuracy on Train and Validation Datasets

From the plot of loss, we can see that the model has comparable performance on both train and validation datasets (labeled test). If these parallel plots start to depart consistently, it might be a sign to stop training at an earlier epoch.

Plot of Model Loss on Training and Validation Datasets

Plot of Model Loss on Training and Validation Datasets

Summary

In this post you discovered the importance of collecting and reviewing metrics during the training of your deep learning models.

You learned about the History callback in Keras and how it is always returned from calls to the fit() function to train your models. You learned how to create plots from the history data collected during training.

Do you have any questions about model training history or about this post? Ask your question in the comments and I will do my best to answer.

Develop Deep Learning Projects with Python!

Deep Learning with Python

 What If You Could Develop A Network in Minutes

…with just a few lines of Python

Discover how in my new Ebook:
Deep Learning With Python

It covers end-to-end projects on topics like:
Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and more…

Finally Bring Deep Learning To
Your Own Projects

Skip the Academics. Just Results.

See What’s Inside

error: Content is protected !!