Determining a sufficient statistic directly from the definition can be a tedious process. The following result can simplify this process by allowing one to spot a sufficient statistic directly from the functional form of the density or mass function. Fisher-Neyman Factorization Theorem Let f θ x be the density or mass function for the random vector x , parametrized by the vector θ . The statistic t T x is sufficient for θ if and only if there exist functions a x (not depending on θ ) and b θ t such that f θ x a x b θ t for all possible values of x . In an earlier example we computed a sufficient statistic for a binary communication source (independent Bernoulli trials) from the definition. Using the above result, this task becomes substantially easier.

Bernoulli Trials Revisited Suppose x n Bernoulli θ are IID, n n 1 , … , N . Denote x x 1 … x n . Then f θ x n 1 N θ x n 1 θ 1 x n θ k 1 θ N k a x b θ k where k n 1 N x n , a x 1 , and b θ k θ k 1 θ N k . By the Fisher-Neyman factorization theorem, k is sufficient for θ .

The next example illustrates the appliction of the theorem to a continuous random variable.

Normal Data with Unknown Mean Consider a normally distributed random sample x 1 , … , x N θ 1 , IID, where θ is unknown. The joint pdf of x x 1 … x n is f θ x n 1 N f θ x n 1 2 N 2 -1 2 n 1 N x n θ 2 We would like to rewrite f θ x is the form of a x b θ t , where dim t N . At this point we require a trick-one that is commonly used when manipulating normal densities, and worth remembering. Define x 1 N n 1 N x n , the sample mean. Then f θ x 1 2 N 2 -1 2 n 1 N x n x x θ 2 1 2 N 2 -1 2 n 1 N x n x 2 2 x n x x θ x θ 2 Now observe n 1 N x n x x θ x θ n 1 N x n x x θ x x 0 so the middle term vanishes. We are left with f θ x 1 2 N 2 -1 2 n 1 N x n x 2 -1 2 n 1 N x θ 2 where a x 1 2 N 2 -1 2 n 1 N x n x 2 , b θ t -1 2 n 1 N x θ 2 , and t x . Thus, the sample mean is a one-dimensional sufficient statistic for the mean.

Proof of Theorem First, suppose t T x is sufficient for θ . By definition, f θ T x t x is independent of θ . Let f θ x t denote the joint density or mass function for ( X , T ( X ) ) . Observe f θ x f θ x t . Then f θ x f θ x t f θ t x f θ t a x b θ t where a x f θ t x and b θ t f θ t . We prove the reverse implication for the discrete case only. The continuous case follows a similar argument, but requires a bit more technical work (Scharf, pp.82; Kay, pp.127). Suppose the probability mass function for x can be written f θ x a x b θ x where t T x . The probability mass function for t is obtained by summing f θ x t over all x such that T x t : f θ t x T x t f θ x t x T x t f θ x x T x t a x b θ t Therefore, the conditional mass function of x, given t, is f θ t x f θ x t f θ t f θ x f θ t a x x T x t a x This last expression does not depend on θ, so t is a sufficient statistic for θ. This completes the proof. From the proof, the Fisher-Neyman factorization gives us a formula for the conditional probability of x given t. In the discrete case we have f t x a x x T x t a x An analogous formula holds for continuous random variables (Scharf, pp.82).

Further Examples The following exercises provide additional examples where the Fisher-Neyman factorization may be used to identify sufficient statistics. Uniform Measurements Suppose x 1 , … , x N are independent and uniformly distributed on the interval θ 1 θ 2 . Find a sufficient statistic for θ θ 1 θ 2 . Express the likelihood f θ x in terms of indicator functions. Poisson Suppose x 1 , … , x N are independent measurements of a Poisson random variable with intensity parameter θ: x x 0 , 1 , 2 , … f θ x θ θ x x

Find a sufficient statistic t for θ.

What is the conditional probability mass function of x, given t, where x x 1 … x N ?

Normal with Unknown Mean and Variance Consider x 1 , … , x N μ σ 2 , IID, where θ 1 μ and θ 2 σ 2 are both unknown. Find a sufficient statistic for θ θ 1 θ 2 . Use the same trick as in .