Local and Global Minima

A local minimum is a point in the parameter space in the vicinity of which the loss function reaches no smaller value. A global minimum is a point in the parameter space where the loss function reaches the smallest possible value. A global minimum is also a local minimum, but not vice-versa. Most optimization algorithms are only able to find local minima.