Could someone explain this neural network machine learning code?

Question

import numpy as np

def nonlin(x, deriv=False):
    if (deriv == True):
        return (x * (1 - x))
    return 1 / (1 + np.exp(-x))

X = np.array([[1,1,1],
              [3,3,3],
              [2,2,2]
              [2,2,2]])

y = np.array([[1],
              [1],
              [0],
              [1]])

np.random.seed(1)

syn0 = 2 * np.random.random((3, 4)) - 1
syn1 = 2 * np.random.random((4, 1)) - 1

for j in xrange(100000):
    l0 = X
    l1 = nonlin(np.dot(l0, syn0))
    l2 = nonlin(np.dot(l1, syn1))
    l2_error = y - l2
    if (j % 10000) == 0:
        print "Error: " + str(np.mean(np.abs(l2_error)))

    l2_delta = l2_error * nonlin(l2, deriv=True)

    l1_error = l2_delta.dot(syn1.T)

    l1_delta = l1_error * nonlin(l1, deriv=True)

    syn1 += l1.T.dot(l2_delta)
    syn0 += l0.T.dot(l1_delta)

print "Output after training"
print l2

Could someone explain what it is printing out and why it is important. This code seems to make no sense to me. The for loop is supposed to optimize the neurons in the network for the given dataset. But how does it do this? What is the nonlin function for? What does this do?:

syn0 = 2 * np.random.random((3, 4)) - 1
syn1 = 2 * np.random.random((4, 1)) - 1

Answer 1

This is a simple code to train a neural network:

This is the activation of a neuron, the function returns the activation function or its derivative (it is a sigmoid function).

def nonlin(x, deriv=False):
    if (deriv == True):
        return (x * (1 - x))
    return 1 / (1 + np.exp(-x))

In this picture you can see a neuron, this function appears as step function in the picture:

https://blog.dbrgn.ch/images/2013/3/26/perceptron.png

These are the raining data, four data with 3 features

X = np.array([[1,1,1],
          [3,3,3],
          [2,2,2]
          [2,2,2]])

The label of every data (binary classification task)

y = np.array([[1],
          [1],
          [0],
          [1]])

This code initialize the weights of a neural network(it is a neural network with two layers of weights):

np.random.seed(1)

syn0 = 2 * np.random.random((3, 4)) - 1
syn1 = 2 * np.random.random((4, 1)) - 1

In this picture the weights are vnm and wkn and are matrices (the value of every link).

https://www.google.es/search?q=multilayer+perceptron&espv=2&biw=1920&bih=964&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjtjP6jr_jPAhUHORoKHbiACdsQ_AUIBigB#imgrc=5npsALleN2cFQM%3A

In your case you have 3 input neurons, four hidden neurons and one output neuron. The value of every link are storaged in syn0 and syn1.

This code trains the neural network, it passes every data, evaluates the error and updates the weights using back propagation:

for j in xrange(100000):
    l0 = X
    l1 = nonlin(np.dot(l0, syn0))
    l2 = nonlin(np.dot(l1, syn1))
    l2_error = y - l2
    if (j % 10000) == 0:
        print "Error: " + str(np.mean(np.abs(l2_error)))

    l2_delta = l2_error * nonlin(l2, deriv=True)

    l1_error = l2_delta.dot(syn1.T)

    l1_delta = l1_error * nonlin(l1, deriv=True)

    syn1 += l1.T.dot(l2_delta)
    syn0 += l0.T.dot(l1_delta)