XOR Multiple Inputs/Targets

Testing...

In [1]:
from conx import Network, Layer, SGD
Using Theano backend.
In [2]:
net = Network("XOR Network", 2, 4, 1, activation="sigmoid")
dataset = [
    ([0, 0], [0]),
    ([0, 1], [1]),
    ([1, 0], [1]),
    ([1, 1], [0])
]
In [3]:
net.set_dataset(dataset)
net.dataset.summary()
Input Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : (2,)
   range  : (0.0, 1.0)
Target Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : (1,)
   range  : (0.0, 1.0)
In [4]:
net.dataset.targets[0]
Out[4]:
[0.0]
In [5]:
net.dataset._inputs.shape
Out[5]:
(4, 2)
In [6]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [7]:
net.dashboard()
In [8]:
net.propagate([-1, 1])
Out[8]:
[0.66179907]
In [9]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
Epoch #   25 | train error 0.25232 | train accuracy 0.50000 | validate% 0.00000
Epoch #   50 | train error 0.24992 | train accuracy 0.50000 | validate% 0.00000
Epoch #   75 | train error 0.24946 | train accuracy 0.50000 | validate% 0.00000
Epoch #  100 | train error 0.24887 | train accuracy 0.50000 | validate% 0.00000
Epoch #  125 | train error 0.24778 | train accuracy 0.50000 | validate% 0.00000
Epoch #  150 | train error 0.24551 | train accuracy 0.50000 | validate% 0.00000
Epoch #  175 | train error 0.24048 | train accuracy 0.50000 | validate% 0.00000
Epoch #  200 | train error 0.22956 | train accuracy 0.50000 | validate% 0.00000
Epoch #  225 | train error 0.20854 | train accuracy 0.75000 | validate% 0.00000
Epoch #  250 | train error 0.17474 | train accuracy 0.75000 | validate% 0.00000
Epoch #  275 | train error 0.12637 | train accuracy 1.00000 | validate% 0.00000
Epoch #  300 | train error 0.07126 | train accuracy 1.00000 | validate% 0.00000
Epoch #  325 | train error 0.03632 | train accuracy 1.00000 | validate% 0.00000
Epoch #  350 | train error 0.02109 | train accuracy 1.00000 | validate% 0.25000
Epoch #  375 | train error 0.01418 | train accuracy 1.00000 | validate% 0.25000
Epoch #  400 | train error 0.01051 | train accuracy 1.00000 | validate% 0.50000
Epoch #  425 | train error 0.00828 | train accuracy 1.00000 | validate% 0.75000
Epoch #  450 | train error 0.00680 | train accuracy 1.00000 | validate% 0.75000
========================================================================
Epoch #  462 | train error 0.00625 | train accuracy 1.00000 | validate% 1.00000
In [10]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [0.0, 0.0] | [0.0] | [0.0] | correct
1 | [0.0, 1.0] | [1.0] | [0.9] | correct
2 | [1.0, 0.0] | [1.0] | [0.9] | correct
3 | [1.0, 1.0] | [0.0] | [0.1] | correct
Total count: 4
Total percentage correct: 1.0
In [11]:
net.propagate_to("input", [0, 1])
Out[11]:
[0.0, 1.0]
In [12]:
net.propagate([0.5, 0.5])
Out[12]:
[0.15998174]
In [13]:
net.propagate_to("hidden", [1, 0])
Out[13]:
[0.46510639786720276,
 0.9962420463562012,
 0.8901682496070862,
 0.9435310363769531]
In [14]:
net.propagate_to("output", [0.25, 0.75])
Out[14]:
[0.5570621490478516]
In [15]:
net.propagate_to("input", [0.25, 0.25])
Out[15]:
[0.25, 0.25]
In [16]:
net.propagate_from("input", [1.0, 1.0])
Out[16]:
[0.17901607]
In [17]:
net.propagate_from("hidden", [1.0, 0.0, 1.0, -1.0])
Out[17]:
[0.79928958]
In [18]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [0.0, 0.0] | [0.0] | [0.0] | correct
1 | [0.0, 1.0] | [1.0] | [0.9] | correct
2 | [1.0, 0.0] | [1.0] | [0.9] | correct
3 | [1.0, 1.0] | [0.0] | [0.1] | correct
Total count: 4
Total percentage correct: 1.0
In [19]:
from conx import Network, Layer, SGD
In [20]:
net = Network("XOR2 Network")
net.add(Layer("input1", 1))
net.add(Layer("input2", 1))
net.add(Layer("hidden1", 10, activation="sigmoid"))
net.add(Layer("hidden2", 10, activation="sigmoid"))
net.add(Layer("shared-hidden", 5, activation="sigmoid"))
net.add(Layer("output1", 1, activation="sigmoid"))
net.add(Layer("output2", 1, activation="sigmoid"))
In [21]:
net
Out[21]:
<Network name='XOR2 Network' (uncompiled)>
In [22]:
net.connect("input1", "hidden1")
net.connect("input2", "hidden2")
net.connect("hidden1", "shared-hidden")
net.connect("hidden2", "shared-hidden")
net.connect("shared-hidden", "output1")
net.connect("shared-hidden", "output2")
In [23]:
net.layers[2].incoming_connections
Out[23]:
[<Layer name='input1', shape=(1,), act='None'>]
In [24]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [25]:
net.config["hspace"] = 200
net.dashboard()
In [26]:
dataset = [
    ([[0],[0]], [[0],[0]]),
    ([[0],[1]], [[1],[1]]),
    ([[1],[0]], [[1],[1]]),
    ([[1],[1]], [[0],[0]])
]
In [27]:
net.set_dataset(dataset)
In [28]:
net.get_weights("hidden2")
Out[28]:
[[[0.21540981531143188,
   0.04578733071684837,
   -0.18966764211654663,
   -0.016449660062789917,
   0.2383221834897995,
   -0.4972211420536041,
   -0.024766184389591217,
   0.4420282244682312,
   -0.6260616183280945,
   -0.0624220110476017]],
 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]
In [29]:
net.propagate_to("output1", [[1], [0]])
Out[29]:
[0.7853870987892151]
In [30]:
for i in range(20):
    (epoch_count, loss, acc, val_acc) = net.train(epochs=100, verbose=0)
    for index in range(4):
        net.propagate(dataset[index][0])
In [31]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
Epoch # 2025 | train error 0.01677 | train accuracy 2.00000 | validate% 0.00000
========================================================================
Epoch # 2047 | train error 0.01183 | train accuracy 2.00000 | validate% 1.00000
In [32]:
net.propagate_from("input1", [0.0])
Out[32]:
[[0.5], [0.5]]
In [33]:
net.propagate_from("shared-hidden", [0.0] * 5)
Out[33]:
[[0.5], [0.5]]
In [34]:
net.propagate_to("hidden1", [[1], [1]])
Out[34]:
[0.8598753213882446,
 0.7098890542984009,
 0.6287320852279663,
 0.26783427596092224,
 0.8600228428840637,
 0.7995230555534363,
 0.14518599212169647,
 0.23925507068634033,
 0.8647616505622864,
 0.18686796724796295]
In [35]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [[0.0], [0.0]] | [[0.0], [0.0]] | [[0.1], [0.1]] | correct
1 | [[0.0], [1.0]] | [[1.0], [1.0]] | [[0.9], [1.0]] | correct
2 | [[1.0], [0.0]] | [[1.0], [1.0]] | [[0.9], [0.9]] | correct
3 | [[1.0], [1.0]] | [[0.0], [0.0]] | [[0.1], [0.1]] | correct
Total count: 4
Total percentage correct: 1.0
In [36]:
net.dataset.slice(2)
In [37]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
========================================================================
Epoch # 2048 | train error 0.00859 | train accuracy 2.00000 | validate% 1.00000

Conx model is a Keras Model

In [38]:
from keras.utils.vis_utils import model_to_dot
from IPython.display import HTML
In [39]:
dot = model_to_dot(net.model, rankdir="BT")
In [40]:
HTML(dot.create_svg().decode())
Out[40]:
G 140455298006824 input1: InputLayer 140455245822328 hidden1: Dense 140455298006824->140455245822328 140455246122848 input2: InputLayer 140455298006488 hidden2: Dense 140455246122848->140455298006488 140455243405632 concatenate_1: Concatenate 140455245822328->140455243405632 140455298006488->140455243405632 140455245822776 shared-hidden: Dense 140455243405632->140455245822776 140455243129856 output1: Dense 140455245822776->140455243129856 140455243405128 output2: Dense 140455245822776->140455243405128