After having calculated the loss using the Loss functions, we now need to adapt the weights and biases for training. How this is done is defined via the Optimizer.

The two main methods are:

• Stochastic gradient descent:

  optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
  

• Adam:

  optimizer = optim.Adam([var1, var2], lr=0.0001)