3.11. Autoencoding¶

3.11.1. One-hot Encoder¶

[1]:

import conx as cx

Using TensorFlow backend.
ConX, version 3.7.5

[2]:

size = 5

[3]:

net = cx.Network("Autoencoder")
net.add(cx.Layer("input", size),
        cx.Layer("hidden", 5, activation="relu"),
        cx.Layer("output", size, activation="sigmoid"))

[3]:

'output'

[4]:

net.connect()

[5]:

net.compile(error='mse', optimizer="adam")

[6]:

net.picture()

[6]:

[7]:

patterns = [cx.onehot(i, size) for i in range(size)]

[8]:

patterns[0]

[8]:

[1, 0, 0, 0, 0]

[9]:

net.dataset.load([(p,p) for p in patterns])

[10]:

net.picture()

[10]:

[11]:

net.dataset.info()

Dataset: Dataset for Autoencoder

Information: * name : None * length : 5

Input Summary: * shape : (5,) * range : (0.0, 1.0)

Target Summary: * shape : (5,) * range : (0.0, 1.0)

[12]:

net.reset()
net.train(accuracy=1, epochs=10000, report_rate=200, tolerance=0.4, plot=True)

========================================================
       |  Training |  Training
Epochs |     Error |  Accuracy
------ | --------- | ---------
#  913 |   0.03599 |   1.00000

[13]:

net.propagate(net.dataset.inputs[0])

[13]:

[0.767602264881134,
03251620754599571,
05481736734509468,
10548711568117142,
14955520629882812]

[14]:

net.evaluate(tolerance=0.4, show=True)

========================================================
Testing validation dataset with tolerance 0.4...
# | inputs | targets | outputs | result
---------------------------------------
0 | [[1.00, 0.00, 0.00, 0.00, 0.00]] | [[1.00, 0.00, 0.00, 0.00, 0.00]] | [0.77, 0.03, 0.05, 0.11, 0.15] | correct
1 | [[0.00, 1.00, 0.00, 0.00, 0.00]] | [[0.00, 1.00, 0.00, 0.00, 0.00]] | [0.04, 0.85, 0.06, 0.07, 0.07] | correct
2 | [[0.00, 0.00, 1.00, 0.00, 0.00]] | [[0.00, 0.00, 1.00, 0.00, 0.00]] | [0.04, 0.04, 0.67, 0.03, 0.25] | correct
3 | [[0.00, 0.00, 0.00, 1.00, 0.00]] | [[0.00, 0.00, 0.00, 1.00, 0.00]] | [0.21, 0.13, 0.23, 0.70, 0.05] | correct
4 | [[0.00, 0.00, 0.00, 0.00, 1.00]] | [[0.00, 0.00, 0.00, 0.00, 1.00]] | [0.22, 0.19, 0.37, 0.06, 0.60] | correct
Total count: 5
      correct: 5
      incorrect: 0
Total percentage correct: 1.0

[15]:

net.dashboard()

3.11.2. MNIST Autoencoding¶

[16]:

net = cx.Network("MNIST-Autoencoder")
net.add(cx.ImageLayer("input", (28,28), 1),
        cx.Conv2DLayer("conv", 3, (5,5), activation="relu"),
        cx.MaxPool2DLayer("pool", pool_size=(2,2)),
        cx.FlattenLayer("flatten"),
        cx.Layer("hidden3", 25, activation="relu"),
        cx.Layer("output", (28,28,1), activation="sigmoid"))
net.connect()
net.compile(error="mse", optimizer="adam")

[17]:

net.picture()

[17]:

[18]:

net.get_dataset('mnist')
net.dataset.set_targets_from_inputs()
net.dataset.targets.reshape(0, (28 * 28))
net.dataset.info()

WARNING: network 'MNIST-Autoencoder' target bank #0 has a multi-dimensional shape; is this correct?

Dataset: MNIST

Original source: http://yann.lecun.com/exdb/mnist/

The MNIST dataset contains 70,000 images of handwritten digits (zero to nine) that have been size-normalized and centered in a square grid of pixels. Each image is a 28 × 28 × 1 array of floating-point numbers representing grayscale intensities ranging from 0 (black) to 1 (white). The target data consists of one-hot binary vectors of size 10, corresponding to the digit classification categories zero through nine. Some example MNIST images are shown below:

MNIST Images

Information: * name : MNIST * length : 70000

Input Summary: * shape : (28, 28, 1) * range : (0.0, 1.0)

Target Summary: * shape : (784,) * range : (0.0, 1.0)

[19]:

net.dashboard()

[20]:

net.propagate_to_features("pool", net.dataset.inputs[0], cols=3)

[20]:

Feature 0

Feature 1

Feature 2

[21]:

image = net.dataset.inputs[0]
output = net.propagate_to_image("output", image)
output.size

[21]:

(28, 28)

[22]:

net.propagate_to("hidden3", image)

[22]:

[0.0,
0,
0,
0,
2590802311897278,
2684532105922699,
0,
12310053408145905,
0,
4014770984649658,
0,
0,
0,
14835278689861298,
0,
0,
0,
0,
390286386013031,
0,
0,
34615498781204224,
0,
0,
029673978686332703]

[31]:

net.dataset.slice(10)

[32]:

net.train(accuracy=0.5, epochs=1000, report_rate=100, tolerance=.4, plot=True)

========================================================
       |  Training |  Training
Epochs |     Error |  Accuracy
------ | --------- | ---------
# 1472 |   0.00070 |   0.50000

[33]:

net.evaluate(show_inputs=False, show_outputs=False, show=True)

========================================================
Testing validation dataset with tolerance 0.4...
# | result
---------------------------------------
0 | X
1 | X
2 | X
3 | correct
4 | correct
5 | X
6 | correct
7 | correct
8 | correct
9 | X
Total count: 10
      correct: 5
      incorrect: 5
Total percentage correct: 0.5

[ ]: