For hypothesis testing, we observe the population through samples to support or deny our assumptions about a certain feature of the population. Our sample data may be imperfect, so inferring the population from the data may face two risks.

If our result is our positive result (generally speaking, we all want to refuse to get the so-called significant difference), then the relationship between the result and the fact may be as follows:

Among them, false positives are the so-called first-class errors, and false negatives are the risk of second-class errors. In mathematical concepts, they are called "discarding truth" and "taking falsehood" respectively.

In addition, there are some definitions to understand:

It may not fit our hypothesis. For example, this may just be our own decision. Therefore, for this point, the probability that our samples fall into the rejection domain is different between different values.

At this time, we need a function to calculate this probability for its different values to evaluate what our optimal hypothesis should be under the current sample. This function is a power function. Used for:

Is a denied domain of significance level.

The definition of efficacy is a little different from efficacy function, which is closer to the original meaning of exclusion domain. Refers to the probability that the sample falls into the rejection domain when it is false, that is, the efficacy of alternative hypothesis.

Neither of these mistakes is what we want to see. At first glance, this sentence is quite right. After all, the probability of their occurrence adds up to 1.

(This picture is from Baidu Encyclopedia _ Efficacy Function)

But if you think about it carefully, these two kinds of errors come from conditional probabilities in different situations, and they should have little to do with each other. Why are the two related?

My understanding is that the most difficult thing for us is to know the absolute truth of the overall situation, and what we have is only a limited sample. We can only calculate the efficacy function according to these limited samples, significance levels and our assumptions, but we don't know the real situation of the whole. So, from the perspective of the imprecise world, all we have are these two conditional probabilities, these two risks, and these two probability of making mistakes. We expand the sample, change the significance level, and so on. They change at the same time for them, so we can't reduce them together.

Power consumption analysis can help us design experiments. The analysis involves the following four quantities, and the fourth one can be calculated by knowing three.

Specific curative effect analysis can use proportional test, t test, chi-square test, variance analysis, correlation analysis, linear model analysis and so on.

To be continued. . .

1. Advanced Engineering Mathematics, edited by Ng Man Tat Bing, Science Press, P 150.

2. actual combat in R language (2nd edition), Robert I. Kabacoff, China Gong Xin Publishing Group &; Analysis of the efficacy of people's post and telecommunications publishing house 10 chapter

3. Baidu Encyclopedia _ efficacy function