2.  means that we reject  in favor of 
The significance level is usually a matter of choice for the researcher, and is selected in advance of the test.
Choosing a significance level  is not important unless the researcher must make a decision on the basis of the test.
Terminology
If the p-value falls below the predetermined significance level, we say the results are “statistically significant”.
If the p-value does not fall below the significance level, we say the results are “not statistically significant”.
Example: Let suppose that the math SAT scores are distributed with mean 500 and standard deviation 40. Suppose that an “SAT school” proclaims “Our students have higher math SAT scores.” A random sample of 3600 “alumni” from the SAT school has a sample mean of 502.
Test the schools claim at significance level 0.05.
Solution: Let  denote the true mean for all students from SAT school.
Here we test  vs 
(This is a RTA test)
The test statistic comes out to 
The p-value is 
Since the p-value is less than 0.05, the result is statistically significant and we reject the null hypothesis.
Therefore, we conclude that the school has a benefit to the students.
Observe that the result is highly statistically significant, but of not much practical significance. We say that the observed gain of 2 points is statistically significant, but not of much practical significance.
Statistical Significance is not equal to practical significance.
(See the above example)
Share with your friends: |
