In the previous article, ML: Simple Math, we discussed the linear equation -
y = mx + b
where -
m = slope
b = y-intercept (point at which the line intersects the Y-axis)
By convention in ML, the equation is written as -
y = b + wx
where -
y = label (output)
x = feature (input)
w = weight (the slope, m)
b = bias (the y-intercept)
Reconsider the previous example. We calculated the slope m and the y-intercept b as m = -1 and b = 5.
(0, 5)
(2, 3)
(6, -1)
(4, ?)
We were able to simply substitute a few given values and find m and b. For complex data with multiple features (inputs) it is very difficult to find the weights and biases. A model needs to be trained extensively to compute the weights and biases.
Time to look at the actual code.
Imports.
import torch
import torch.nn as nn
import torch.optim as optim
import copy
import numpy as np
Define training data based on given inputs.
X_train = torch.tensor([0.0, 2.0, 6.0])
y_train = torch.tensor([5.0, 3.0, -1.0])
Get a few more points on the graph as test data to verify the model's predictions.
X_test = torch.tensor([ 5.0, 1.0, 3.0])
y_test = torch.tensor([0.0, 4.0, 2.0])
Define a simple model.
model = nn.Linear(1, 1)
The next step is to find slope m (weight w) and b the y-intercept (bias). You can ignore all the complexities in the below function for now and understand that its job is to find weight and bias.
Here we train the model with the given input and compute the weight w and bias b.
def find_weight_and_bias():
loss_fn = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.02)
n_epochs = 1000
batch_size = 1
batch_start = torch.arange(0, len(X_train), batch_size)
best_mse = np.inf
best_weights = None
history = []
for epoch in range(n_epochs):
model.train()
for start in batch_start:
X_batch = X_train[start:start+batch_size]
y_batch = y_train[start:start+batch_size]
y_pred = model(torch.reshape(X_batch, (-1, 1)))
loss = loss_fn(y_pred, torch.reshape(y_batch, (-1, 1)))
optimizer.zero_grad()
loss.backward()
optimizer.step()
model.eval()
y_pred = model(torch.reshape(X_test, (-1, 1)))
mse = loss_fn(y_pred, torch.reshape(y_test, (-1, 1)))
mse = float(mse)
history.append(mse)
if mse < best_mse:
best_mse = mse
best_weights = copy.deepcopy(model.state_dict())
return best_weights
Now, let's find weight and bias.
weight_and_bias = find_weight_and_bias();
print(f'Weight and Bias: {weight_and_bias}')
Predicting the value for x = 4 -
model.load_state_dict(weight_and_bias)
y = model(torch.tensor([4.0]))
print(f'Predicting value, x = 4')
print(f'y: {y}')
Output -
Weight and Bias:
OrderedDict([('weight', tensor([[-1.0000]])), ('bias', tensor([5.0000]))])
Predicting value, x = 4
y: tensor([1.0000], grad_fn=<AddBackward0>)
Observe that w = -1, b = 5and the value of y for x = 4 is 1.
See all the code in the colab here.
