XOR Multiple Inputs/Targets

Testing...

In [1]:
from conx import Network, Layer, SGD
conx, version 3.4.0
Using Theano backend.
In [2]:
net = Network("XOR Network", 2, 4, 1, activation="sigmoid")
In [3]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [4]:
dataset = [
    ([0, 0], [0]),
    ([0, 1], [1]),
    ([1, 0], [1]),
    ([1, 1], [0])
]
net["output"].minmax = (0, 1)
In [5]:
net.dataset.load(dataset)
net.dataset.summary()
Input Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : [(2,)]
   range  : (0.0, 1.0)
Target Summary:
   count  : 4 (4 for training, 0 for testing)
   shape  : [(1,)]
   range  : (0.0, 1.0)
In [6]:
net.dataset.targets[0]
Out[6]:
[0.0]
In [7]:
net.dataset.inputs.shape
Out[7]:
[(2,)]
In [8]:
net.dashboard()
In [9]:
net.propagate([0, 0])
Out[9]:
[0.5188825726509094]
In [10]:
net.reset()
net.train(epochs=2000, accuracy=1.0, report_rate=25, plot=True)
_images/XOR_10_0.svg
========================================================================
       |  Training |  Training
Epochs |     Error |  Accuracy
------ | --------- | ---------
#  380 |   0.00652 |   1.00000
In [11]:
net.test()
========================================================
Testing train dataset with tolerance 0.1000...
Total count: 1
      correct: 1
      incorrect: 0
Total percentage correct: 1.0
In [12]:
net.propagate_to("input", [0, 1])
Out[12]:
[0.0, 1.0]
In [13]:
net.propagate([0.5, 0.5])
Out[13]:
[0.37886133790016174]
In [14]:
net.propagate_to("hidden", [1, 0])
Out[14]:
[0.19043518602848053,
 0.9795732498168945,
 0.9983406066894531,
 0.7423139810562134]
In [15]:
net.propagate_to("output", [1, 1])
Out[15]:
[0.09938135743141174]
In [16]:
net.propagate_to("input", [0.25, 0.25])
Out[16]:
[0.25, 0.25]
In [17]:
net.propagate_from("input", [1.0, 1.0])
Out[17]:
[0.099381357]
In [18]:
net.propagate_from("hidden", [1.0, 0.0, 1.0, -1.0])
Out[18]:
[1.312265e-06]
In [19]:
net.test()
========================================================
Testing train dataset with tolerance 0.1000...
Total count: 1
      correct: 1
      incorrect: 0
Total percentage correct: 1.0
In [20]:
from conx import Network, Layer, SGD
In [21]:
net = Network("XOR2 Network")
net.add(Layer("input1", 1))
net.add(Layer("input2", 1))
net.add(Layer("hidden1", 10, activation="sigmoid"))
net.add(Layer("hidden2", 10, activation="sigmoid"))
net.add(Layer("shared-hidden", 5, activation="sigmoid"))
net.add(Layer("output1", 1, activation="sigmoid", minmax=(-1,1)))
net.add(Layer("output2", 1, activation="sigmoid", minmax=(-1,1)))
In [22]:
net
Out[22]:
<Network name='XOR2 Network' (uncompiled)>
In [23]:
net.connect("input1", "hidden1")
net.connect("input2", "hidden2")
net.connect("hidden1", "shared-hidden")
net.connect("hidden2", "shared-hidden")
net.connect("shared-hidden", "output1")
net.connect("shared-hidden", "output2")
In [24]:
net.layers[2].incoming_connections
Out[24]:
[<Layer name='input1', shape=(1,), act='None'>]
In [25]:
net.compile(loss='mean_squared_error',
            optimizer=SGD(lr=0.3, momentum=0.9))
In [26]:
net.config["hspace"] = 200
net.dashboard()
In [27]:
net.propagate_to_features("hidden1", [[[1], [1]]])
In [28]:
net.propagate([[1], [1]])
Out[28]:
[[0.43671050667762756], [0.6797348260879517]]
In [29]:
dataset = [
    ([[0],[0]], [[0],[0]]),
    ([[0],[1]], [[1],[1]]),
    ([[1],[0]], [[1],[1]]),
    ([[1],[1]], [[0],[0]])
]
In [30]:
net.dataset.load(dataset)
In [31]:
net.get_weights("hidden2")
Out[31]:
[[[0.4540221095085144,
   0.08793536573648453,
   -0.1828816831111908,
   -0.5521659851074219,
   0.32966530323028564,
   -0.686873197555542,
   0.3806053698062897,
   -0.25110286474227905,
   0.21008996665477753,
   0.04664979130029678]],
 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]
In [32]:
net.propagate([[1], [1]])
Out[32]:
[[0.43671050667762756], [0.6797348260879517]]
In [33]:
net
Out[33]:
XOR2 NetworkLayer: output1 (output) shape = (1,) Keras class = Dense activation = sigmoidoutput1Layer: output2 (output) shape = (1,) Keras class = Dense activation = sigmoidoutput2Weights from shared-hidden to output1 output1/kernel has shape (5, 1) output1/bias has shape (1,)Weights from shared-hidden to output2 output2/kernel has shape (5, 1) output2/bias has shape (1,)Layer: shared-hidden (hidden) shape = (5,) Keras class = Dense activation = sigmoidshared-hiddenWeights from hidden1 to shared-hidden shared-hidden/kernel has shape (20, 5) shared-hidden/bias has shape (5,)Layer: hidden1 (hidden) shape = (10,) Keras class = Dense activation = sigmoidhidden1Weights from hidden2 to shared-hidden shared-hidden/kernel has shape (20, 5) shared-hidden/bias has shape (5,)Layer: hidden2 (hidden) shape = (10,) Keras class = Dense activation = sigmoidhidden2Weights from input1 to hidden1 hidden1/kernel has shape (1, 10) hidden1/bias has shape (10,)Layer: input1 (input) shape = (1,) Keras class = Inputinput1Weights from input2 to hidden2 hidden2/kernel has shape (1, 10) hidden2/bias has shape (10,)Layer: input2 (input) shape = (1,) Keras class = Inputinput2
In [34]:
for i in range(20):
    (epoch_count, results) = net.train(epochs=100, verbose=0, report_rate=25, plot=True)
    for index in range(4):
        net.propagate(dataset[index][0])
_images/XOR_34_0.svg
In [35]:
net.reset()
In [36]:
net.train(epochs=2000, accuracy=1.0, report_rate=25, plot=True)
_images/XOR_36_0.svg
========================================================================
       |  Training |   output1 |   output2
Epochs |     Error |       acc |       acc
------ | --------- | --------- | ---------
#  881 |   0.01815 |   1.00000 |   1.00000
In [37]:
net.propagate_from("shared-hidden", [0.0] * 5)
Out[37]:
[[0.8253845], [0.82618642]]
In [38]:
net.propagate_to("hidden1", [[1], [1]])
Out[38]:
[0.73835289478302,
 0.8181569576263428,
 0.7488211393356323,
 0.21614700555801392,
 0.8158043026924133,
 0.13216380774974823,
 0.19218505918979645,
 0.8437826037406921,
 0.7830644249916077,
 0.8318384289741516]
In [39]:
net.test()
========================================================
Testing train dataset with tolerance 0.1000...
Total count: 4
      correct: 4
      incorrect: 0
Total percentage correct: 1.0
In [40]:
net.dataset.slice(2)
In [41]:
net.train(epochs=2000, accuracy=1.0, report_rate=25)
No training required: accuracy already to desired value
Training dataset status:
       |  Training |   output1 |   output2
Epochs |     Error |       acc |       acc
------ | --------- | --------- | ---------
#  881 |   0.01815 |   1.00000 |   1.00000

Conx model is a Keras Model

Requires:

  • pip install pydot-ng
In [42]:
from keras.utils.vis_utils import model_to_dot
from IPython.display import HTML
In [43]:
dot = model_to_dot(net.model, rankdir="BT")
In [44]:
HTML(dot.create_svg().decode())
Out[44]:
G 140204328768512 input1: InputLayer 140204322822576 hidden1: Dense 140204328768512->140204322822576 140204324048344 input2: InputLayer 140204328767560 hidden2: Dense 140204324048344->140204328767560 140204320915240 concatenate_2: Concatenate 140204322822576->140204320915240 140204328767560->140204320915240 140204322822016 shared-hidden: Dense 140204320915240->140204322822016 140204328110400 output1: Dense 140204322822016->140204328110400 140204324048008 output2: Dense 140204322822016->140204324048008