The least square method is not an algorithm, but a model. There are many methods to solve this kind of model, such as QR decomposition method of linear least squares model, LSQR algorithm, SVD method of total least squares model, Gauss-Newton method of nonlinear least squares model and so on.

Besides the least square model based on 2- norm, there are other models based on other norms, such as the best uniform approximation model based on infinite norm. The best uniform approximation problem of a variable can be solved by Remez algorithm.

Some models can be solved by general optimization algorithm (random gradient descent method is one of them), and sometimes they can be solved by algebraic method, for example, some problems can be solved by nonlinear eigenvalue problems.

Generally speaking, both modeling and solving need to be considered, but they cannot be completely confused.