Statistics is a discipline that studies uncertainty, and uncertainty permeates the whole discipline, so that mathematics and statistics are fundamentally different ways of thinking.
In statistics, it is very common to define things by intuition and examples, that is, "what you see is what you get", which is rarely black and white as in mathematics. This is due to an inevitable reason: statisticians deal with real data, which are often confusing and difficult to sort out and study from strict definitions.
The non-statistical method proves its correctness by theory.
For example, we can prove by induction that Dijkstra algorithm always returns the shortest path in the graph, or that the quick sorting method always arranges the arrays in the sorted order. In order to compare the running time, we use the big O symbol, which is a mathematical structure used to strictly control the running time of the program, and it depicts the behavior of the running time when the input of the program tends to infinity.