Grid Search is a hyper-parameter optimization technique which consists of selecting the best performing model from the set of all models corresponding to hyper-parameters placed in a finite grid of the hyper-parameter space. For example: in a multi-layer perceptron model with a single hidden layer, we can perform a 2D grid search for the optimal combination of the number of hidden units and the learning rate, given a finite set of options for each.