list/repeat the three ingredients to a feed forward network: input, hidden layers, output
classify/categorize parts of a feed forward network when presented a network architecture (as from keras.model.summary())
describe a convolutional layer
describe a max pooling layer
calculate the output data shape of an image when transformed by a fixed convolutional layer
interpret errors with convolutional layers
execute a 3 layer network on the MNIST data (or similar)
differentiate a dense layer and a convolutional layer
experiment with values of dense layer and a convolutional layer
select a layer type depending on the input data
develop a 5 layer network that comprises both layer types
This lesson will be shared as a video.
for learners: a stub notebook to get you started can be obtained from the lesson03 repo.
for instructors: the video script is available here.
This lesson is based on the wonderful keras introduction by Walter de Back - https://gitlab.com/wdeback/dl-keras-tutorial/-/blob/master/notebooks/2-convolution.ipynb - https://gitlab.com/wdeback/dl-keras-tutorial/-/blob/master/notebooks/3-cnn-mnist.ipynb
Question 1
Fill in the blanks to produce a CNN for classification!
from tensorflow import keras from keras.layers import Input, Dense, Dropout, Flatten, ____2D, _________2D
#load the data
#define the network conv1 = Conv2D(16, kernel_size=(3,3), activation=’______’, input_shape=X_train.shape[1:]) conv2 = ______(32, kernel_size=(3,3), activation=’relu’) mpool = _________(pool_size=(2,2))
## MLP layers flat = Flatten() dense1 = Dense(128, _____________) dense2 = Dense(num_classes, __________)
#compile and train
x_inputs = Input(shape=X_train.shape[1:])
x = conv1(_______) x = ______(x) x = ______(x)
x = flat(x) x = dense1(x) output_yhat = dense2(x)
model = keras.Model(inputs = _______, outputs = _______, name=”hello-world-cnn”)
Question 2
The Flatten operation rearranges an input image (or feature map) into a sequence of numbers. How does it perform this?
the pixel intensities are averaged per row and concatenated
all rows of the input are added and provided as a result
all columns of the input are concatenated (from top to bottom)
all rows of the input are concatenated (from top to bottom)
Question 3
For an input image shape of 28x28 what is the shape of the feature map after running the image through a single 5x5 convolutional filter?
24x28
20x28
26x26
24x24
Question 4
For an MNIST input image, how many parameters does a Conv2D layer require when being defined to produce 16 feature maps as output and a 3x3 neighborhood. How many parameters does a Dense layer with 16 outputs have? Compute the two parameter counts!
run the lesson notebook using the fashionMNIST dataset. Use the code below to load it. Try to reach a similar accuracy as you saw with the orginal MNIST. Does the same architecture work exactly the same with fashionMNIST?
from keras.datasets import fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
using the MNIST dataset, we saw that overfitting can be problematic. When reducing the size of the network in terms of number of parameters, the downside of this is that our network looses capacity to learn features. Cut the dense layer from 128 neurons to 64 and 32. What do you observe with these network sizes? Compare the accuracy of the trimmed networks with a network that uses a drop-out layer which masks 25% of the neurons.