Student’s T-Statistic

views updated

Student’s T-Statistic

In measuring social and economic progression, statistical methods that take into account various magnitudes of uncertainties are used for inference and decision-making. One of the tasks in statistical inference is to estimate population mean from a sample of observations. For example, one may want to estimate the mean value of the retail price (μ ) of a gallon of unleaded gasoline based on prices from a few gas stations in a large city. A natural estimate of μ is the arithmetic sample mean . The estimator X ̂ has many desirable statistical properties, especially when are independent and identically distributed as a normal (Gaussian) random variable with mean μ and variance σ². From properties of normal distribution, the following Z -statistic

is a standard normal random variable with mean μ = 0 and variance σ² = 1. This Z -statistic has been used to make statistical inferences and decisions when variance σ² is known or the number of observations n is large. However, in most applications, the variance σ² is usually unknown and the number of observations n may be small. William Sealy Gosset (1876–1937), a chemist at the brewery of Arthur Guinness Sons and Co. (Boland 1984), studied the small sample property of

where is the sample variance. Gosset’s work resulted in the birth of Student’s T -statistic. Gosset published his work in Biometrika in 1908 under the pseudonym “Student” from his 1904 report to the company titled “The Application of the Law of Error to Work of the Brewery.” Gosset’s employer was against the work done for the company being made public but allowed him to publish it under a pseudonym. Gosset’s original statistic was

The Student’s T -statistic used in the current formulation in expression (2) is due to English statistician Ronald Aylmer Fisher (1925).

Student’s T statistic was one of the important breakthroughs in statistical sciences in the twentieth century (Kotz and Johnson 1992). The density function of the Student’s T -statistic with v (integer) degrees of freedom is

where Γ(a) is the Gamma function defined as

Density functions of the Z -statistic and Student’s T -statistic are bell-shaped curves with Student’s T having a heavier tail. The density function of Student’s T -statistic relative to the density function of the standard normal is shown in Figure 1.

The distribution of the Student’s T -statistic is the Student’s T -distribution and statistical tests based on the

Student’s T -statistic form various Student’s T -tests. The expectation (mean) of the Student’s T -statistic is 0, and the variance is v /(v – 2) for v ≥ 2. When v = 1, the Student’s T -statistic becomes the Cauchy random variable, which does not have a variance. The Student’s T -statistic has infinite variance when v = 2. The degrees of freedom of the Student’s T -statistic defined in expression (2) are n – 1.

Student’s T -statistic has been used for estimation and decision-making in many fields of science ranging from agriculture, biology, economics, public health, and zoology. As a simple illustration, to estimate the average unleaded retail gasoline price per gallon in a large city, the author of this entry conducted an informal random sample survey on his way (seventeen miles) home at the end of April in 2006. He observed prices of $2.999, $2.879, $2.959, $2.839, $2.899, $3.019, and $2.919 without including the duplicates. The sample mean (X̄) of these seven observations was $2.930. Under normality assumption of the gasoline price, the sample mean is usually a good point estimator of the underlying mean value μ. Because of the randomness of the sample, a better approach for statistical inference is to construct an interval estimator. A classic method is to find the 100(1 – α)% confidence interval for the underlying mean gasoline price based on the Student’s T -statistic:

where t _{1 - α/2, n - 1} is the critical value that satisfies

p (|T |≥t _{1-α/2, n-1}) = α

and α is the probability that the confidence interval does not cover the true underlying mean value μ. The critical values of Student’s T -statistic for various values of α and degrees of freedom are available in all statistical software packages and in most basic statistical books. A commonly used confidence is 95% (α = 0.05) for interval estimation. From our sample observations, the 95% confidence interval for μ is

which is (2.870, 2.990).

In addition to estimation, Students T -statistic is alsouseful for hypothesis testing. The hypothesis to be testedis the null hypothesis. For example, the null hypothesis may be H ₀: μ = μ₀ and the alternative hypothesis can be set as H _α: μ≠μ₀ (two-sided test). One may commit twotypes of errors in testing statistical hypotheses. Rejectingthe null hypothesis when it is true is the Type I error andaccepting the null hypothesis when it is false is the Type IIerror. The probability of making Type I error is denoted by α which is the same value used above for constructinga confidence interval. The probability of making Type IIerror is denoted by β. Type I error and Type II error areinversely related. The power of the test is 1 – β, which is the probability of rejecting the null hypothesis when it is false. The upper bound of α is the size of the test, and thecritical (rejection) region for a given Student’s T -test of size α with v degrees of freedom is {|T |≥t _1-α/2-v}. The p value is the probability of the test statistic as contradictory to the null hypothesis as the observed Student’s T -statistic. Using the above seven observations, we can conduct a two-sided Student’s T -test on the null hypothesis H ₀: μ = 3.000 vs μ ≠ 3.000 by constructing the test statistic:

The absolute value of the observed Student’s T -statistic ǀTǀ > 2.447 = t _{1 – 0.05/2,6}. That is, the observed Student’s T -statistic (–2.8493) is in the critical region, which leads to rejecting the null hypothesis. A very popular way of performing statistical hypothesis testing is by computing the p -value. The two-sided p -value from the above example is 2P (T ≤ –2.8493) = 0.0292, which is less than the commonly used size of 0.05. Therefore, one would reject the null hypothesis of μ = 3.000 and conclude that the underlying mean retail price for unleaded gasoline in that city was different from $3.000 per gallon.

The above simple example on estimation and hypothesis testing is formulated based on a two-sided Student’s T -statistic. Similar estimation and hypothesis testing can be done for a one-sided test. In planning a scientific investigation, scientists need to decide how many samples are needed in order to control both Type I and Type II errors. Student’s T -statistic plays a fundamental role in designing scientific investigations. Alan Agresti and Barbara Finlay (1997) provide an introduction to statistical estimation and hypothesis testing.

Student’s T -statistic can be extended in many directions. For example, if X_i denotes the difference of the gasoline prices of a gas station at two different occasions, one can conduct a paired Student’s T -test for quantifying the changes. If one wants to make a statistical comparison of the gasoline prices in two cities, one may construct a two-sample Student’s T -statistic

where n ₁, n ₂ are the number of observations from sample (city) one and two, respectively, , , . If the two samples have equal variances, one can form the pooled Student’s t -statistic as

where

is the pooled sample variance. Under the null hypothesis of equal means of and (H ₀: μ₁ = μ₂), the two-sample Student’s T -statistic in expression (7) follows the Student’s T -distribution with n ₁ + n ₂ – 2 degrees of freedom if and are independent and identically normally distributed.

For more advanced statistical inferences, such as in correlation analysis, linear regression, and generalized linear models, the estimators of the parameters are approximately of Student’s T -distribution. Hence, one can perform a statistical analysis on the estimated parameters based on Student’s T -statistic.

As the sample size increases, the degrees of freedom increase and the Student’s T -statistic converges to the standard normal random variable (Z ). Asymptotically, one may use the Z -test to replace the Student’s T -test when degrees of freedom are large (n ≥ 30). Student’s T -statistic is widely used for small sample analysis. One of the fundamental assumptions in deriving the distribution of Student’s T -statistic is the normality of . This assumption is not easy to check for a small sample size. Many nonparametric (distribution-free) procedures have been proposed for conducting statistical inference without assuming normality of (Hollander and Wolfe 1999). For multiple sample inferences, Fisher (1925) extended the two-sample Student’s T -test to analysis of variance (ANOVA) when are normally distributed. Without assuming normality, many rank-based techniques were developed for nonparametric versions of the two-sample Student’s T -test and Fisher’s ANOVA (Hollander and Wolfe 1999). A great many statistical methods have been invented in the twentieth century. Student’s T -statistic is one of the most widely used statistical tools not only by professional statisticians but also by all scientists involved in data analysis and decision making.