Don't Learn here. I think Everything here is exact opposite of what you do in a real job. I am not a profession AI guy. I am a noob.
Forward pass was simple.
I need to pass the input value of one layer to another layers.
But Backpropagation has multiple steps.
I think backpropagation is when you make your network aware of loss. By Aware I mean updating value based on loss of network. I just realized this today(17-03-2024). So Below is the methods that I wrote to train and get loss of network.
def train(self,train_X,train_Y):
# pass the train_X to Network to get loss of network or the error in prediction.
self.inputL_out = self.inputL.forward(train_X)
for l in range(0,len(self.neuronsLayers)):
if l == 0:
self.neuronsLayers[l].forward(self.inputL_out)
else:
self.neuronsLayers[l].forward(self.neuronsLayers[l-1].f_out)
if l == len(self.neuronsLayers) -1:
self.outL_f_out = self.outL.forward(self.neuronsLayers[l].f_out) #self.outL_f_out is output of network.
# now calculate the loss.
# I am calculating two type for losses. 1. loss on actual answer 2. Loss on wrong answer.
losses = []
for i in range(0,self.out_dim):
if train_Y != i:
if abs(self.outL_f_out[i]) > 0.6:
losses.append(abs(self.outL_f_out[i])-0.6) #prediction for other digits other then actual answer should be less then 0.6
else:
losses.append(0)
else:
losses.append(0.99 - abs(self.outL_f_out[train_Y])) #loss for actual answer.
print(losses)
print(self.outL_f_out)
Until today I was stuck on derivative and how to update weights.
After wasting two days(2 hours per day) I realized I need to update weight based on loss.
But still how do I update weight & by how much.
And why my network in predicting negative probability.(wrong choice of activation function I think.)
I am weak at math
Top comments (0)