3.7. MNISTΒΆ

In [1]:
from conx import Network, Layer, Conv2DLayer, MaxPool2DLayer, FlattenLayer
Using TensorFlow backend.
conx, version 3.5.17
In [2]:
net = Network("MNIST")
net.add(Layer("input", shape=784, vshape=(28, 28), colormap="hot", minmax=(0,1)))
net.add(Layer("hidden1", shape=512, vshape=(16,32), activation='relu', dropout=0.2))
net.add(Layer("hidden2", shape=512, vshape=(16,32), activation='relu', dropout=0.2))
net.add(Layer("output", shape=10, activation='softmax'))

net.connect('input', 'hidden1')
net.connect('hidden1', 'hidden2')
net.connect('hidden2', 'output')
In [3]:
net.compile(error='mean_squared_error', optimizer='sgd')
In [4]:
dataset = net.dataset
dataset.get('mnist')
#dataset.slice(0, 100)
dataset.inputs.reshape(0, 784)
dataset.shuffle()
dataset.summary()
Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 3s 0us/step

Dataset name: MNIST

Original source: http://yann.lecun.com/exdb/mnist/

The MNIST database of handwritten digits, available from this page, has 70,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image. It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.

Dataset Split: * training : 70000 * testing : 0 * total : 70000

Input Summary: * shape : [(784,)] * range : [(0.0, 1.0)]

Target Summary: * shape : [(10,)] * range : [(0.0, 1.0)]

In [5]:
widget = net.dashboard()
In [6]:
widget
In [7]:
net
Out[7]:
MNISTLayer: output (output) output range: (0, 1) shape = (10,) Keras class = Dense activation = softmaxoutputWeights from hidden2 to output output/kernel:0 has shape (512, 10) output/bias:0 has shape (10,)Layer: hidden2 (hidden) output range: (0, +Infinity) shape = (512,) dropout = 0.2 Keras class = Dense activation = reluhidden2Weights from hidden1 to hidden2 hidden2/kernel:0 has shape (512, 512) hidden2/bias:0 has shape (512,)Layer: hidden1 (hidden) output range: (0, +Infinity) shape = (512,) dropout = 0.2 Keras class = Dense activation = reluhidden1Weights from input to hidden1 hidden1/kernel:0 has shape (784, 512) hidden1/bias:0 has shape (512,)Layer: input (input) output range: (0, 1) shape = (784,) Keras class = Inputinput
In [8]:
net.snapshot(height="500px")
Out[8]:
MNISTLayer: output (output) output range: (0, 1) shape = (10,) Keras class = Dense activation = softmaxoutputWeights from hidden2 to output output/kernel:0 has shape (512, 10) output/bias:0 has shape (10,)Layer: hidden2 (hidden) output range: (0, +Infinity) shape = (512,) dropout = 0.2 Keras class = Dense activation = reluhidden2Weights from hidden1 to hidden2 hidden2/kernel:0 has shape (512, 512) hidden2/bias:0 has shape (512,)Layer: hidden1 (hidden) output range: (0, +Infinity) shape = (512,) dropout = 0.2 Keras class = Dense activation = reluhidden1Weights from input to hidden1 hidden1/kernel:0 has shape (784, 512) hidden1/bias:0 has shape (512,)Layer: input (input) output range: (0, 1) shape = (784,) Keras class = Inputinput
In [9]:
net.config["show_targets"] = True
In [11]:
net.train(1, batch_size=128, verbose=1)
net.test()
Training...
       |  Training |  Training
Epochs |     Error |  Accuracy
------ | --------- | ---------
#    1 |   0.08899 |   0.00000
#    2 |   0.08578 |   0.00000
========================================================================
#    2 |   0.08578 |   0.00000
========================================================
Testing validation dataset with tolerance 0.1...
Total count: 70000
      correct: 0
      incorrect: 70000
Total percentage correct: 0.0
In [12]:
%%time
for pattern in net.dataset.inputs[:100]:
    net.propagate(pattern)
CPU times: user 163 ms, sys: 13 ms, total: 176 ms
Wall time: 145 ms
In [13]:
from conx import Network, Layer, Conv2DLayer, MaxPool2DLayer, FlattenLayer
In [14]:
net = Network("MNIST-CNN")
net.add(Layer("input", (28,28,1), colormap="hot", minmax=(0,1)))
net.add(Conv2DLayer("Conv2D-1", 16, (5,5), activation="relu"))
net.add(MaxPool2DLayer("maxpool1", (2,2)))
net.add(Conv2DLayer("Conv2D-2", 132, (5,5), activation="relu"))
net.add(MaxPool2DLayer("maxpool2", (2,2)))
net.add(FlattenLayer("flatten", visible=True))
net.add(Layer("hidden", 1000, activation='relu', vshape=(8,125), image_maxdim=500))
net.add(Layer("output", 10, activation='softmax'))

net.connect()
In [15]:
net.compile(error="binary_crossentropy", optimizer="rmsprop")
In [16]:
ds = net.dataset
ds.get('mnist')
In [17]:
ds.slice(100)
In [18]:
net.train()
Evaluating initial training metrics...
Training...
       |  Training |  Training
Epochs |     Error |  Accuracy
------ | --------- | ---------
#    0 |   0.32450 |   0.00000
#    1 |   0.35377 |   0.00000
========================================================================
#    1 |   0.35377 |   0.00000
In [19]:
net.dashboard()
In [21]:
net.snapshot(height="500px")
Out[21]:
MNIST-CNNLayer: output (output) shape = (10,) Keras class = Dense activation = softmaxoutputWeights from hidden to output output/kernel has shape (1000, 10) output/bias has shape (10,)Layer: hidden (hidden) shape = (1000,) Keras class = Dense activation = reluhiddenWeights from flatten to hidden hidden/kernel has shape (2112, 1000) hidden/bias has shape (1000,)Layer: flatten (hidden) Keras class = FlattenflattenWeights from maxpool2 to flattenLayer: maxpool2 (hidden) Keras class = MaxPooling2Dmaxpool21320Weights from Conv2D-2 to maxpool2Layer: Conv2D-2 (hidden) Keras class = Conv2D activation = reluConv2D-21320Weights from maxpool1 to Conv2D-2 Conv2D-2/kernel has shape (5, 5, 16, 132) Conv2D-2/bias has shape (132,)Layer: maxpool1 (hidden) Keras class = MaxPooling2Dmaxpool1160Weights from Conv2D-1 to maxpool1Layer: Conv2D-1 (hidden) Keras class = Conv2D activation = reluConv2D-1160Weights from input to Conv2D-1 Conv2D-1/kernel has shape (5, 5, 1, 16) Conv2D-1/bias has shape (16,)Layer: input (input) shape = (28, 28, 1) Keras class = Inputinput10
In [19]:
net
Out[19]:
MNIST-CNNLayer: output (output) shape = (10,) Keras class = Dense activation = softmaxoutputWeights from hidden to output output/kernel has shape (1000, 10) output/bias has shape (10,)Layer: hidden (hidden) shape = (1000,) Keras class = Dense activation = reluhiddenWeights from flatten to hidden hidden/kernel has shape (2112, 1000) hidden/bias has shape (1000,)Layer: flatten (hidden) Keras class = FlattenflattenWeights from maxpool2 to flattenLayer: maxpool2 (hidden) Keras class = MaxPooling2Dmaxpool21320Weights from Conv2D-2 to maxpool2Layer: Conv2D-2 (hidden) Keras class = Conv2D activation = reluConv2D-21320Weights from maxpool1 to Conv2D-2 Conv2D-2/kernel has shape (5, 5, 16, 132) Conv2D-2/bias has shape (132,)Layer: maxpool1 (hidden) Keras class = MaxPooling2Dmaxpool1160Weights from Conv2D-1 to maxpool1Layer: Conv2D-1 (hidden) Keras class = Conv2D activation = reluConv2D-1160Weights from input to Conv2D-1 Conv2D-1/kernel has shape (5, 5, 1, 16) Conv2D-1/bias has shape (16,)Layer: input (input) shape = (28, 28, 1) Keras class = Inputinput10
In [22]:
for pattern in net.dataset.inputs[:10]:
    net.propagate(pattern)
In [21]:
net.train(epochs=100, accuracy=.7)
Training...
Epoch #    2 | train error 0.30195 | train accuracy 0.91000 | validate% 0.00000
Epoch #    3 | train error 0.29779 | train accuracy 0.90000 | validate% 0.00000
Epoch #    4 | train error 0.22481 | train accuracy 0.90000 | validate% 0.15000
Epoch #    5 | train error 0.14948 | train accuracy 0.95200 | validate% 0.06000
Epoch #    6 | train error 0.32091 | train accuracy 0.90300 | validate% 0.00000
Epoch #    7 | train error 0.18543 | train accuracy 0.93000 | validate% 0.20000
Epoch #    8 | train error 0.10438 | train accuracy 0.96200 | validate% 0.35000
Epoch #    9 | train error 0.06161 | train accuracy 0.98600 | validate% 0.54000
Epoch #   10 | train error 0.04596 | train accuracy 0.98900 | validate% 0.47000
Epoch #   11 | train error 0.08075 | train accuracy 0.96700 | validate% 0.49000
Epoch #   12 | train error 0.12112 | train accuracy 0.95800 | validate% 0.51000
Epoch #   13 | train error 0.08936 | train accuracy 0.96900 | validate% 0.50000
Epoch #   14 | train error 0.04674 | train accuracy 0.98900 | validate% 0.69000
Epoch #   15 | train error 0.02372 | train accuracy 0.99600 | validate% 0.77000
========================================================================
Epoch #   15 | train error 0.02372 | train accuracy 0.99600 | validate% 0.77000