XOR Multiple Inputs/Targets

Testing...

In [4]:
from conx import Network, Layer, SGD
In [5]:
net = Network("XOR Network", 2, 4, 1, activation="sigmoid")
In [6]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [7]:
dataset = [
    ([0, 0], [0]),
    ([0, 1], [1]),
    ([1, 0], [1]),
    ([1, 1], [0])
]
net["output"].minmax = (0, 1)
In [8]:
net.dataset.load(dataset)
net.dataset.summary()
Input Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : (2,)
   range  : (0.0, 1.0)
Target Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : (1,)
   range  : (0.0, 1.0)
In [9]:
net.dataset.targets[0]
Out[9]:
[0.0]
In [11]:
net.dataset.inputs.shape()
Out[11]:
(2,)
In [12]:
net.dashboard()
In [13]:
net.propagate([0, 0])
Out[13]:
[0.6550639867782593]
In [14]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
Epoch #   25 | train error 0.25244 | train accuracy 0.50000 | validate% 0.00000
Epoch #   50 | train error 0.25053 | train accuracy 0.50000 | validate% 0.00000
Epoch #   75 | train error 0.25023 | train accuracy 0.50000 | validate% 0.00000
Epoch #  100 | train error 0.25012 | train accuracy 0.50000 | validate% 0.00000
Epoch #  125 | train error 0.25004 | train accuracy 0.75000 | validate% 0.00000
Epoch #  150 | train error 0.24997 | train accuracy 0.75000 | validate% 0.00000
Epoch #  175 | train error 0.24989 | train accuracy 0.50000 | validate% 0.00000
Epoch #  200 | train error 0.24978 | train accuracy 0.50000 | validate% 0.00000
Epoch #  225 | train error 0.24961 | train accuracy 0.50000 | validate% 0.00000
Epoch #  250 | train error 0.24932 | train accuracy 0.50000 | validate% 0.00000
Epoch #  275 | train error 0.24876 | train accuracy 0.50000 | validate% 0.00000
Epoch #  300 | train error 0.24758 | train accuracy 0.50000 | validate% 0.00000
Epoch #  325 | train error 0.24484 | train accuracy 0.50000 | validate% 0.00000
Epoch #  350 | train error 0.23839 | train accuracy 0.50000 | validate% 0.00000
Epoch #  375 | train error 0.22438 | train accuracy 0.50000 | validate% 0.00000
Epoch #  400 | train error 0.19824 | train accuracy 0.50000 | validate% 0.00000
Epoch #  425 | train error 0.16033 | train accuracy 0.75000 | validate% 0.00000
Epoch #  450 | train error 0.11312 | train accuracy 1.00000 | validate% 0.00000
Epoch #  475 | train error 0.06428 | train accuracy 1.00000 | validate% 0.00000
Epoch #  500 | train error 0.03400 | train accuracy 1.00000 | validate% 0.00000
Epoch #  525 | train error 0.02009 | train accuracy 1.00000 | validate% 0.00000
Epoch #  550 | train error 0.01349 | train accuracy 1.00000 | validate% 0.25000
Epoch #  575 | train error 0.00992 | train accuracy 1.00000 | validate% 0.50000
Epoch #  600 | train error 0.00776 | train accuracy 1.00000 | validate% 0.75000
========================================================================
Epoch #  618 | train error 0.00667 | train accuracy 1.00000 | validate% 1.00000
In [15]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [0.00,0.00] | [0.00] | [0.06] | correct
1 | [0.00,1.00] | [1.00] | [0.92] | correct
2 | [1.00,0.00] | [1.00] | [0.92] | correct
3 | [1.00,1.00] | [0.00] | [0.10] | correct
Total count: 4
Total percentage correct: 1.0
In [16]:
net.propagate_to("input", [0, 1])
Out[16]:
[0.0, 1.0]
In [17]:
net.propagate([0.5, 0.5])
Out[17]:
[0.2228575497865677]
In [18]:
net.propagate_to("hidden", [1, 0])
Out[18]:
[0.9699694514274597,
 0.9991346001625061,
 0.7620226740837097,
 0.7633542418479919]
In [19]:
net.propagate_to("output", [1, 1])
Out[19]:
[0.09997931122779846]
In [20]:
net.propagate_to("input", [0.25, 0.25])
Out[20]:
[0.25, 0.25]
In [21]:
net.propagate_from("input", [1.0, 1.0])
Out[21]:
[0.87671763]
In [22]:
net.propagate_from("hidden", [1.0, 0.0, 1.0, -1.0])
Out[22]:
[0.49268335]
In [23]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [0.00,0.00] | [0.00] | [0.06] | correct
1 | [0.00,1.00] | [1.00] | [0.92] | correct
2 | [1.00,0.00] | [1.00] | [0.92] | correct
3 | [1.00,1.00] | [0.00] | [0.10] | correct
Total count: 4
Total percentage correct: 1.0
In [24]:
from conx import Network, Layer, SGD
In [25]:
net = Network("XOR2 Network")
net.add(Layer("input1", 1))
net.add(Layer("input2", 1))
net.add(Layer("hidden1", 10, activation="sigmoid"))
net.add(Layer("hidden2", 10, activation="sigmoid"))
net.add(Layer("shared-hidden", 5, activation="sigmoid"))
net.add(Layer("output1", 1, activation="sigmoid", minmax=(-1,1)))
net.add(Layer("output2", 1, activation="sigmoid", minmax=(-1,1)))
In [26]:
net
Out[26]:
<Network name='XOR2 Network' (uncompiled)>
In [27]:
net.connect("input1", "hidden1")
net.connect("input2", "hidden2")
net.connect("hidden1", "shared-hidden")
net.connect("hidden2", "shared-hidden")
net.connect("shared-hidden", "output1")
net.connect("shared-hidden", "output2")
In [28]:
net.layers[2].incoming_connections
Out[28]:
[<Layer name='input1', shape=(1,), act='None'>]
In [29]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [30]:
net.config["hspace"] = 200
net.dashboard()
In [31]:
net.propagate([[0], [0]])
Out[31]:
[[0.6482962965965271], [0.48509156703948975]]
In [32]:
dataset = [
    ([[0],[0]], [[0],[0]]),
    ([[0],[1]], [[1],[1]]),
    ([[1],[0]], [[1],[1]]),
    ([[1],[1]], [[0],[0]])
]
In [33]:
net.dataset.load(dataset)
In [34]:
net.get_weights("hidden2")
Out[34]:
[[[-0.6610402464866638,
   0.01789213716983795,
   0.09658310562372208,
   -0.5663502216339111,
   -0.6618386507034302,
   0.014390095137059689,
   0.020138438791036606,
   -0.4793321490287781,
   -0.5139104723930359,
   0.4509921371936798]],
 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]
In [35]:
net.model.layers[-1].output
Out[35]:
sigmoid.0
In [36]:
import numpy as np
In [37]:
net.model.predict([np.array([[1]]), np.array([[1]])])
Out[37]:
[array([[ 0.63918394]], dtype=float32), array([[ 0.47949073]], dtype=float32)]
In [38]:
net.propagate([[1], [1]])
Out[38]:
[[0.6391839385032654], [0.479490727186203]]
In [39]:
for i in range(20):
    (epoch_count, loss, acc, val_acc) = net.train(epochs=100, verbose=0)
    for index in range(4):
        net.propagate(dataset[index][0])
In [40]:
net.reset()
In [41]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
Epoch #   25 | train error 0.50005 | train accuracy 1.00000 | validate% 0.00000
Epoch #   50 | train error 0.50005 | train accuracy 1.00000 | validate% 0.00000
Epoch #   75 | train error 0.50003 | train accuracy 1.00000 | validate% 0.00000
Epoch #  100 | train error 0.50003 | train accuracy 1.00000 | validate% 0.00000
Epoch #  125 | train error 0.50002 | train accuracy 1.00000 | validate% 0.00000
Epoch #  150 | train error 0.50001 | train accuracy 1.00000 | validate% 0.00000
Epoch #  175 | train error 0.50000 | train accuracy 1.00000 | validate% 0.00000
Epoch #  200 | train error 0.50000 | train accuracy 1.00000 | validate% 0.00000
Epoch #  225 | train error 0.49999 | train accuracy 1.00000 | validate% 0.00000
Epoch #  250 | train error 0.49998 | train accuracy 1.00000 | validate% 0.00000
Epoch #  275 | train error 0.49997 | train accuracy 1.00000 | validate% 0.00000
Epoch #  300 | train error 0.49996 | train accuracy 1.00000 | validate% 0.00000
Epoch #  325 | train error 0.49995 | train accuracy 1.00000 | validate% 0.00000
Epoch #  350 | train error 0.49993 | train accuracy 1.00000 | validate% 0.00000
Epoch #  375 | train error 0.49992 | train accuracy 1.00000 | validate% 0.00000
Epoch #  400 | train error 0.49990 | train accuracy 1.00000 | validate% 0.00000
Epoch #  425 | train error 0.49988 | train accuracy 1.00000 | validate% 0.00000
Epoch #  450 | train error 0.49986 | train accuracy 1.00000 | validate% 0.00000
Epoch #  475 | train error 0.49983 | train accuracy 1.00000 | validate% 0.00000
Epoch #  500 | train error 0.49980 | train accuracy 1.00000 | validate% 0.00000
Epoch #  525 | train error 0.49976 | train accuracy 1.00000 | validate% 0.00000
Epoch #  550 | train error 0.49970 | train accuracy 1.00000 | validate% 0.00000
Epoch #  575 | train error 0.49964 | train accuracy 1.00000 | validate% 0.00000
Epoch #  600 | train error 0.49955 | train accuracy 1.00000 | validate% 0.00000
Epoch #  625 | train error 0.49944 | train accuracy 1.00000 | validate% 0.00000
Epoch #  650 | train error 0.49929 | train accuracy 1.00000 | validate% 0.00000
Epoch #  675 | train error 0.49908 | train accuracy 1.00000 | validate% 0.00000
Epoch #  700 | train error 0.49877 | train accuracy 1.00000 | validate% 0.00000
Epoch #  725 | train error 0.49830 | train accuracy 1.00000 | validate% 0.00000
Epoch #  750 | train error 0.49754 | train accuracy 1.00000 | validate% 0.00000
Epoch #  775 | train error 0.49620 | train accuracy 1.00000 | validate% 0.00000
Epoch #  800 | train error 0.49361 | train accuracy 1.25000 | validate% 0.00000
Epoch #  825 | train error 0.48783 | train accuracy 1.50000 | validate% 0.00000
Epoch #  850 | train error 0.47283 | train accuracy 1.50000 | validate% 0.00000
Epoch #  875 | train error 0.43332 | train accuracy 1.50000 | validate% 0.00000
Epoch #  900 | train error 0.36876 | train accuracy 1.50000 | validate% 0.00000
Epoch #  925 | train error 0.28413 | train accuracy 1.50000 | validate% 0.00000
Epoch #  950 | train error 0.10794 | train accuracy 2.00000 | validate% 0.00000
Epoch #  975 | train error 0.03188 | train accuracy 2.00000 | validate% 0.00000
Epoch # 1000 | train error 0.01718 | train accuracy 2.00000 | validate% 0.50000
========================================================================
Epoch # 1002 | train error 0.01659 | train accuracy 2.00000 | validate% 1.00000
In [42]:
net.propagate_from("input1", [0.0])
Out[42]:
[[0.5], [0.5]]
In [43]:
net.propagate_from("shared-hidden", [0.0] * 5)
Out[43]:
[[0.5], [0.5]]
In [44]:
net.propagate_to("hidden1", [[1], [1]])
Out[44]:
[0.7277137637138367,
 0.2255076766014099,
 0.7404088973999023,
 0.13241004943847656,
 0.8331649303436279,
 0.8214220404624939,
 0.8211300373077393,
 0.6355288028717041,
 0.19784897565841675,
 0.27547237277030945]
In [45]:
net.test()
Testing on training dataset...
# | inputs | targets | outputs | result
---------------------------------------
0 | [[0.00],[0.00]] | [[0.00],[0.00]] | [[0.10],[0.09]] | correct
1 | [[0.00],[1.00]] | [[1.00],[1.00]] | [[0.91],[0.91]] | correct
2 | [[1.00],[0.00]] | [[1.00],[1.00]] | [[0.91],[0.91]] | correct
3 | [[1.00],[1.00]] | [[0.00],[0.00]] | [[0.09],[0.08]] | correct
Total count: 4
Total percentage correct: 1.0
In [46]:
net.dataset.slice(2)
In [47]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
Training...
========================================================================
Epoch # 1003 | train error 0.01748 | train accuracy 2.00000 | validate% 1.00000

Conx model is a Keras Model

In [48]:
from keras.utils.vis_utils import model_to_dot
from IPython.display import HTML
In [49]:
dot = model_to_dot(net.model, rankdir="BT")
In [50]:
HTML(dot.create_svg().decode())
Out[50]:
G 140418684597080 input1: InputLayer 140418687666608 hidden1: Dense 140418684597080->140418687666608 140418970212896 input2: InputLayer 140418970210544 hidden2: Dense 140418970212896->140418970210544 140418670903928 concatenate_2: Concatenate 140418687666608->140418670903928 140418970210544->140418670903928 140418687665824 shared-hidden: Dense 140418670903928->140418687665824 140418670625736 output1: Dense 140418687665824->140418670625736 140418670904376 output2: Dense 140418687665824->140418670904376