Note: Before you going, you can see the previous post from here

Introduction

To provide a useful but relatively simple example of machine learning with TensorFlow, you will create a neural network with only an input and output layer that learns the relationship between two variables, x and y. The neural network will predict the value of y for a given value of x once you train the network. x and y could represent any numerical value, such as the heart rate and cholesterol level, or age and income level. For this Lab Step, the data used for training is randomly generated by perturbing values around a known linear function between x and y. The known linear function can be written as y = a*x + b where a is the slope of the line and b is the intercept. The following image depicts the function:

In the example, a=0.5 and b=2. After training the neural network, the predicted values should closely match the values of the known function.

Start a Jupyter notebook server

Step 1

In your SSH shell, copy the Jupyter notebook URL output by the list command, and paste it into your browser:

Recall that the tunnel is open to port 8000 on your local machine and not 8888.

Step 2

Replace the port 8888 with 8000 in the URL and navigate to the site:

Step 3

Click on the New button above the file listing table, and select Environment (conda_tensorflow_p36):

The new notebook opens in a new browser tab:

Step 4

Paste the following Python script into the cell and read through the comments and code:

'''Single neuron neural network'''
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
# Draw plots inline in the notebook
%matplotlib inline
# Set up sample points perturbed away from the ideal linear relationship 
# y = 0.5*x + 2.5
num_examples = 60
points = np.array([np.linspace(-1, 5, num_examples),
  np.linspace(2, 5, num_examples)])
points += np.random.randn(2, num_examples)
x, y = points
# Include a 1 to use as the bias input for neurons
x_with_bias = np.array([(1., d) for d in x]).astype(np.float32)
# Training parameters
training_steps = 100
learning_rate = 0.001
losses = []
with tf.Session():
  # Set up all the tensors, variables, and operations.
  input = tf.constant(x_with_bias)
  target = tf.constant(np.transpose([y]).astype(np.float32))
  # Initialize weights with small random values
  weights = tf.Variable(tf.random_normal([2, 1], 0, 0.1))
  tf.global_variables_initializer().run()
  # Calculate the current prediction error
  y_predicted = tf.matmul(input, weights)
  y_error = tf.subtract(y_predicted, target)
  # Compute the L2 loss function of the error
  loss = tf.nn.l2_loss(y_error)
  # Train the network using an optimizer that minimizes the loss function
  update_weights = tf.train.GradientDescentOptimizer(
    learning_rate).minimize(loss)
  for _ in range(training_steps):
    # Repeatedly run the operations, updating the TensorFlow variable.
    update_weights.run()
    losses.append(loss.eval())
  # Training is done, get the final values for the graphs
  w = weights.eval()
  y_predicted = y_predicted.eval()
# Show the fit and the loss over time.
fig, (ax1, ax2) = plt.subplots(1, 2)
plt.subplots_adjust(wspace=.3)
fig.set_size_inches(11, 4)
# Plot the perturbed points in blue dots
ax1.scatter(x, y, c="b", alpha=.6)
# Plot the predicted values in green dots
ax1.scatter(x, np.transpose(y_predicted)[0], c="g", alpha=0.6)
line_x_range = (-3, 7)
# Plot the predicted line in green
ax1.plot(line_x_range, [w[1] * x + w[0]
                       for x in line_x_range], "g", alpha=0.6)
# Plot the noise-free line (0.5*x + 2.5) in red
ax1.plot(line_x_range, [0.5 * x + 2.5
                        for x in line_x_range], "r", alpha=0.6)
ax1.set_xlabel("x")
ax1.set_ylabel("y")
ax2.plot(range(0, training_steps), losses)
ax2.set_xlabel("Training step")
ax2.set_ylabel("Loss")
plt.show()

• The code concerning TensorFlow is all inside the with tf.Session(): block.
• There are a few functions that you haven't seen before but their names accurately describe what they do.
• TensorFlow provides several loss functions and training algorithms.
• The tf.nn.l2_loss is a common choice, while the tf.train.GradientDescentOptimizer is also popular but usually not the most efficient training algorithm.
• It is what more advanced algorithms are based upon.

Step 5

Click the Run button to start running the experiment:

• The code will start running.
• It should take less than 30 seconds to see the plots that are generated after training completes.
• The first time is much slower than following times because libraries need to be loaded.
• If you run the script again it should finish in one second or less.

Step 6

Take a moment to analyze the plots, your plots will be different due to the random nature of the code but they should resemble the following:

• The predicted relationship (green) matches the noise-free relationship (red) quite closely.
• The plot of the loss at each step shows that the training makes slow progress after around the 50th step.
• In the example above, the lowest possible loss is around 40 when modeling the relationship with a straight line.

Step 7

Close the TensorFlow Python 3.6 notebook by closing the browser tab.

Step 8

In the file list, select the Running notebook:

Step 9

To stop your notebook from running, in the top-left, click Shutdown:

Warning: Notebooks use the EC2 instance's resources to perform operations. If you leave one running, subsequent lab steps may fail to execute properly due to being unable to use the GPU at the same time as an existing running notebook.

GitHub
LinkedIn
Facebook
Medium