Infinitely Divisible Distributions.

In the study of limit theorems for sums of independent random variables infinitely divisible distributions play a very important role.

Definition 3.1. A distribution μ is said to be infinitely divisible if for every positive integer n, μ can be written as the n-fold convolution (λ_{n}∗)^{n} of some other probability distribution λ_{n}.

Exercise 3.19. Show that the normal distribution with density

p(x) = \frac{1}{\sqrt{2π}} exp[−\frac{x^{2}}{2} ]

is infinitely divisible.

Exercise 3.20. Show that the Poisson distribution with

p(n) = \frac{e^{−n}}{n!} for n ≥ 0

is infinitely divisible.

Exercise 3.21. Show that any probabilty distribution supported on a finite set {x_{1}, …, x_{k}} with

μ[{x_{j}}] = p_{j}

and p_{j} ≥ 0, ∑_{j=1}^{k} p_{j} = 1 is infinitely divisible if and only if it is degenrate, i.e. μ[{x}] = 1 for some x.

Exercise 3.22. Show that for any nonnegative finite measure α with total mass a, the distribution

e(F) = e^{−a} ∑_{j=1}^{∞} \frac{(α∗)^{j}}{j!}

with characteristic function

ê(F)(t) = exp[∫ (e^{itx} − 1)dα]

is an infinitely divisible distribution.

Exercise 3.23. Show that the convolution of any two infinitely divisible distributions is again infinitely divisible. In particular if μ is infinitely divisible so is any translate μ ∗ δ_{a} for any real a.

We saw in the last section that the asymptotic behavior of μ_{n} ∗ δ_{A_{n}} can be investigated by means of the asymptotic behavior of λ_{n} ∗ δ_{A_{n}} and the characteristic function λ̂_{n} of λ_{n} has a very special form

λ̂_{n} = ∏_{j=1}^{k_{n}} exp[ β̂_{n,j}(t) − 1 + i t a_{n,j}]

= exp [ ∑_{j=1}^{k_{n}} ∫ [ e^{i t x} − 1 ] dβ_{n,j} + i t ∑_{j=1}^{k_{n}} a_{n,j}]
= exp [ ∫ [ e^{i t x} − 1 ] dM_{n} + i t a_{n}]
= exp [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{n} + i t [ ∫ θ(x) dM_{n} + a_{n}]]
= exp [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{n} + i t b_{n}]  (3.11)

We can make any reasonable choice for θ(·) and we will need it to be a bounded continuous function with

|θ(x) − x| ≤ C|x|^{3}

near 0. Possible choices are θ(x) = \frac{x}{1+x^{2}} , or θ(x) = x for |x| ≤ 1 and sign (x) for |x| ≥ 1. We now investigate when such things will have a weak limit. Convoluting with δ_{A_{n}} only changes b_{n} to b_{n} + A_{n}.

First we note that

μ̂(t) = exp [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM + i t a]

is a characteristic function for any measure M with finite total mass. In fact it is the characteristic function of an infinitely divisible probability distribution. It is not necessary that M be a finite measure for μ to make sense. M could be infinite, but in such a way that it is finite on {x : |x| ≥ δ} for every δ > 0, and near 0 it integrates x^{2} i.e.,

M[x : |x| ≥ δ] < ∞ for all δ > o  (3.12)
∫_{|x|≤1} x^{2}dM < ∞  (3.13)

To see this we remark that

μ̂_{δ}(t) = exp [ ∫_{|x|≥δ}[ e^{i t x} − 1 − i t θ(x) ] dM + i t a]

is a characteristic function for each δ > 0 and because

|e^{i t x} − 1 − i t x | ≤ C_{T} x^{2}

for |t| ≤ T, μ̂_{δ}(t) → μ̂(t) uniformly on bounded intervals where μ̂(t) is given by the integral

μ̂(t) = exp [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM + i t a]

which converges absolutely and defines a characteristic function. Let us call measures that satisfy 3.12 and 3.13, that can be expressed in the form

∫ \frac{x^{2}}{1 + x^{2}} dM < ∞  (3.14)

admissible Levy measures. Since the same argument applies to \frac{M}{n} and \frac{a}{n} instead of M and a, for any admissible Levy measure M and real number a, μ̂(t) is in fact an infinitely divisible characteristic function. As the normal distribution is also an infinitely divisible probability distribution, we arrive at the following

Theorem 3.20. For every admissible Levy measure M, σ^{2} > 0 and real a

μ̂(t) = exp [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM + i t a − \frac{σ^{2}t^{2}}{2} ]

is the characteristic function of an infinitely divisible distribution μ.

We will denote this distribution μ by μ = e (M, σ^{2}, a). The main theorem of this section is

Theorem 3.21. In order that μ_{n} = e (M_{n}, σ_{n}^{2}, a_{n}) may converge to a limit μ it is necessary and sufficient that μ = e (M, σ^{2}, a) and the following three conditions 3.15 3.16 and 3.17 are satisfied.

For every bounded continuous function f that vanishes in some neighborhood of 0,

lim_{n→∞} ∫ f(x)dM_{n} = ∫ f(x)dM  (3.15)

For some ( and therefore for every) ℓ > 0 such that ± ℓ are continuity points for M, i.e., M{± ℓ} = 0

lim_{n→∞} [σ_{n}^{2} + ∫_{−ℓ}^{ℓ} x^{2}dM_{n}] = [σ^{2} + ∫_{−ℓ}^{ℓ} x^{2}dM]  (3.16)
a_{n} → a as n → ∞.  (3.17)

Proof. Let us prove the suficiency first. Condition 3.15 implies that for every ℓ such that ± ℓ are continuity points of M

lim_{n→∞} ∫_{|x|≥ℓ}[ e^{i t x} − 1 − i t θ(x) ] dM_{n} = ∫_{|x|≥ℓ}[ e^{i t x} − 1 − i t θ(x) ] dM

and because of condition 3.16, it is enough to show that

lim_{ℓ→0} lim sup_{n→∞} | ∫_{−ℓ}^{ℓ}[ e^{i t x} − 1 − i t θ(x) + \frac{t^{2}x^{2}}{2} ] dM_{n}
− ∫_{ℓ}^{ℓ}[ e^{i t x} − 1 − i t θ(x) + \frac{t^{2}x^{2}}{2} ] dM|
= 0

in order to conclude that

lim_{n→∞} [ − \frac{σ_{n}^{2}t^{2}}{2} + ∫ [ e^{i t x} − 1 − i t θ(x)] dM_{n}]
= [ − \frac{σ^{2}t^{2}}{2} + ∫ [ e^{i t x} − 1 − i t θ(x)] dM]

This follows from the estimates

| e^{i t x} − 1 − i t θ(x) + \frac{t^{2}x^{2}}{2} | ≤ C_{T} |x|^{3}

and

∫_{−ℓ}^{ℓ} |x|^{3}dM_{n} ≤ ℓ ∫_{−ℓ}^{ℓ} |x|^{2}dM_{n}

Condition 3.17 takes care of the a_{n} terms.

We now turn to proving the necessity. If μ_{n} has a weak limit μ then the absolute values of the characteristic functions |μ̂_{n}(t)| are all uniformly close to 1 near 0. Since

|μ̂_{n}(t)| = exp [ − ∫ (1 − cos t x) dM_{n} − \frac{σ_{n}^{2}t^{2}}{2} ]

taking logarithms we conclude that

lim_{t→0} sup_{n} [\frac{σ_{n}t^{2}}{2} + ∫ (1 − cos t x) dM_{n}] = 0

This implies 3.18, 3.19 and 3.20 below.

For each ℓ > 0,

sup_{n} M_{n}{x : |x| ≥ ℓ} < ∞  (3.18)

lim_{A→∞} sup_{n} M_{n}{x : |x| ≥ A} = 0  (3.19)

For every 0 ≤ ℓ < ∞,

sup_{n} [σ_{n}^{2} + ∫_{−ℓ}^{ℓ} |x|^{2}dM_{n}] < ∞  (3.20)

We can choose a subsequence of M_{n} (which we will denote by M_{n} as well ) that ‘converges’ in the sense that it satisfies conditions 3.15 and 3.16 of the Theorem. Then e (M_{n}, σ_{n}^{2}, 0) converges weakly to e (M, σ^{2}, 0). It is not hard to see that for any sequence of probability distributions α_{n} if both α_{n} and α_{n} ∗ δ_{a_{n}} converge to limits α and β respectively, then necessarily β = α ∗ δ_{a} for some a and a_{n} → a as n → ∞. In order complete the proof of necessity we need only establish the uniqueness of the representation. Lemma 3.22. (Uniqueness). Suppose μ = e (M_{1}, σ_{1}^{2}, a_{1}) = e (M_{2}, σ_{2}^{2}, a_{2}), then M_{1} = M_{2}, σ_{1}^{2} = σ_{2}^{2} and a_{1} = a_{2}.

Proof. Since μ̂(t) never vanishes by taking logarithms we have

ψ(t) = [ − \frac{σ_{1}^{2}t^{2}}{2} + ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{1} + i t a_{1}]
= [ − \frac{σ_{2}^{2}t^{2}}{2} + ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{2} + i t a_{2}] (3.21)

We can verify that for any admissible Levy measure M

lim_{t→∞} \frac{1}{t^{2}} ∫ [ e^{i t x} − 1 − i t θ(x) ] dM = 0

Consequently

lim_{t→∞} \frac{ψ(t)}{t^{2}} = σ_{1}^{2} = σ_{2}^{2}

leaving us with

ψ(t) = [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{1} + i t a_{1}]
= [ ∫ [ e^{i t x} − 1 − i t θ(x) ] dM_{2} + i t a_{2}]

for a different ψ. If we calculate

H(s, t) = \frac{ψ(t + s) + ψ(t − s)}{2} − ψ(t)

we get

∫ e^{i t x} (1 − cos s x)dM_{1} = ∫ e^{i t x} (1 − cos s x)dM_{2}

for all t and s. Since we can and do assume that M{0} = 0 for any admissible Levy measure M we have M_{1} = M_{2}. If we know that σ_{1}^{2} = σ_{2}^{2} and M_{1} = M_{2} it is easy to see that a_{1} must equal a_{2}. Finally

Corollary 3.23. If μ is any infinitely divisible distribution, then for some admissible Levy measure M, σ^{2} > 0 and real number a, μ = e (M, σ^{2}, a) .

Proof. We can write μ = μ_{n}∗^{n} = μ_{n} ∗ μ_{n} ∗ ⋯ ∗ μ_{n} with n terms. If we show that μ_{n} ⇒ δ_{0} then the sequence is uniformly infinitesimal and by the earlier theorem on accompanying laws μ will be the limit of some λ_{n} = e (M_{n}, 0, a_{n}) and therefore has to be of the form e (M, σ^{2}, a) for some choice of admissible Levy measure M, σ^{2} > 0 and real a. In a neighborhood around 0, μ̂(t) is close to 1 and it is easy to check that

μ̂_{n}(t) = [μ̂(t)]^{\frac{1}{n}} → 1

as n → ∞ in that neighborhood. As we saw before this implies that μ_{n} ⇒ δ_{0}.

Applications.

1. Convergence to the Poisson Distribution. Let {X_{n,j} : 1 ≤ j ≤ k_{n}} be k_{n} independent random variables taking the values 0 or 1 with probabilities 1 − p_{n,j} and p_{n,j} respectively. We assume that

lim_{n→∞} sup_{1≤j≤k_{n}} p_{n,j} = 0

which is the uniform infinitesimality condition. We are interested in the limiting distribution of S_{n} = ∑_{j=1}^{k_{n}} X_{n,j} as n → ∞. Since we have to center by the mean we can pick any level say \frac{1}{2} for truncation. Then the truncated means are all 0. The accompanying laws are given by e (M_{n}, 0, a_{n}) with M_{n} = (∑ p_{n,j})δ_{1} and a_{n} = (∑ p_{n,j}) θ(1). It is clear that a limit exists if and only if λ_{n} = ∑ p_{n,j} has a limit λ as n → ∞ and the limit in such a case is the Poisson distribution with parameter λ.

2. Convergence to the normal distribution. If the limit of S_{n} = ∑_{j} X_{n,j} of k_{n} uniformly infinitesimal mutually independent random variables exists, then the limit is Normal if and only if M ≡ 0. If a_{n,j} is the centering needed, this is equivalent to

lim_{n→∞} ∑_{j} P [|X_{n,j} − a_{n,j}| ≥ ϵ] = 0

for all ϵ > 0. Since lim_{n→∞} sup_{j} |a_{n,j}| = 0, this is equivalent to

lim_{n→∞} ∑_{j} P [|X_{n,j}| ≥ ϵ] = 0

for each ϵ > 0.

3. The limiting variance and the mean are given by

σ^{2} = lim_{n→∞} ∑_{j} E{[X_{n,j} − a_{n,j}]^{2} : |X_{n,j} − a_{n,j}| ≤ 1}

and

a = lim_{n→∞} ∑_{j} a_{n,j}

where

a_{n,j} = ∫_{|x|≤1} x dα_{n,j}

Suppose that E[X_{n,j}] = 0 for all 1 ≤ j ≤ k_{n} and n. Assume that σ_{n}^{2} = ∑_{j} E{[X_{n,j}]^{2}} and σ^{2} = lim_{n→∞} σ_{n}^{2} exists. What do we need in order to make sure that the limiting distribution is normal with mean 0 and variance σ^{2}? Let α_{n,j} be the distribution of X_{n,j}.

|a_{n,j}|^{2} = | ∫_{|x|≤1} x dα_{n,j}|^{2} = | ∫_{|x|>1} x dα_{n,j}|^{2} ≤ α_{n,j}[ |x| > 1 ] ∫ |x|^{2} dα_{n,j}

and

∑_{j} |a_{n,j}|^{2} ≤ { ∑_{1≤j≤k_{n}} ∫ |x|^{2} dα_{n,j}}{ sup_{1≤j≤k_{n}} α_{n,j}[ |x| > 1 ]}
≤ σ_{n}^{2}{ sup_{1≤j≤k_{n}} α_{n,j}[ |x| > 1 ]}
→ 0

Because ∑_{j} |a_{n,j}|^{2} → 0 as n → ∞ we must have

σ^{2} = lim_{n→∞} ∑ ∫_{|x|≤ℓ} |x|^{2} dα_{n,j}

for every ℓ > 0 or equivalently

lim_{n→∞} ∑ ∫_{|x|>ℓ} |x|^{2} dα_{n,j} = 0

for every ℓ establishing the necessity as well as sufficiency in Lindeberg’s Theorem. A simple calculation shows that

∑_{j} |a_{n,j}| ≤ ∑_{j} ∫_{|x|>1} |x| dα_{n,j} ≤ ∑_{j} ∫_{|x|>1} |x|^{2} dα_{n,j} = 0

establishing that the limiting Normal distribution has mean 0.

Exercise 3.24. What happens in the Poisson limit theorem (application 1) if λ_{n} = ∑_{j} p_{n,j} → ∞ as n → ∞? Can you show that the distribution of \frac{S_{n}−λ_{n}}{\sqrt{λ}_{n}} converges to the standard Normal distribution?

Laws of the iterated logarithm.

When we are dealing with a sequence of independent identically distributed random variables X_{1}, ⋯, X_{n}, ⋯ with mean 0 and variance 1, we have a strong law of large numbers asserting that

P \{ lim_{n→∞} \frac{X_{1} + ⋯ + X_{n}}{n} = 0\} = 1

and a central limit theorem asserting that

P \{\frac{X_{+} ⋯ + X_{n}}{\sqrt{n}} ≤ a\} → ∫_{−∞}^{a} \frac{1}{\sqrt{2π}} exp[−\frac{x^{2}}{2} ] dx

It is a reasonable question to ask if the random variables \frac{X_{1}+···+X_{n}}{\sqrt{n}} themselves converge to some limiting random variable Y that is distributed according to the the standard normal distribution. The answer is no and is not hard to show. Lemma 3.24. For any sequence n_{j} of numbers → ∞,

P \{ lim sup_{j→∞} \frac{X_{1} + ⋯ + X_{n_{j}}}{\sqrt{n_{j}}} = +∞\} = 1

Proof. Let us define

Z = lim sup_{j→∞} \frac{X_{1} + ⋯ + X_{n_{j}}}{\sqrt{n_{j}}}

which can be +∞. Because the normal distribution has an infinitely long tail, i.e the probability of exceeding any given value is positive, we must have

P [Z ≥ a] > 0

for any a. But Z is a random variable that does not depend on the particular values of X_{1}, ⋯, X_{n} and is therefore a set in the tail σ-field. By Kolmogorov’s zero-one law P [Z ≥ a] must be either 0 or 1. Since it cannot be 0 it must be 1. The lemma is proved.

Since we know that \frac{X_{1}+···+X_{n}}{n} → 0 with probability 1 as n → ∞, the question arises as to the rate at which this happens. The law of the iterated logarithm provides an answer. Theorem 3.25. For any sequence X_{1}, ⋯, X_{n}, ⋯ of independent identically distributed random variables with mean 0 and Variance 1,

P \{ lim sup_{n→∞} \frac{X_{1} + ⋯ + X_{n}}{\sqrt{n log log n}} = \sqrt{2}\} = 1

We will not prove this theorem in the most general case which assumes only the existence of two moments. We will assume instead that E[|X|^{2+α}] < ∞ for some α > 0. We shall first reduce the proof to an estimate on the tail behavior of the distributions of \frac{S_{n}}{\sqrt{n}} by a careful application of the Borel-Cantelli Lemma. This estimate is obvious if X_{1}, ⋯, X_{n}, ⋯ are themselves normally distributed and we will show how to extend it to a large class of distributions that satisfy the additional moment condition. It is clearthat we are interested in showing that for λ > \sqrt{2},

P \{S_{n} ≥ λ\sqrt{n log log n} infinitely often \} = 0

It would be sufficient because of Borel-Cantelli lemma to show that for any λ > \sqrt{2},

∑_{n} P \{S_{n} ≥ λ\sqrt{n log log n}\} < ∞

This however is too strong. The condition of the Borel-Cantelli lemma is not necessary in this context because of the strong dependence between the partial sums S_{n}. The function ϕ(n) = \sqrt{n log log n} is clearly well defined and non-decreasing for n ≥ 3 and it is sufficient for our purposes to show that for any λ > \sqrt{2} we can find some sequence k_{n} ↑ ∞ of integers such that

∑_{n} P { sup_{k_{n−1}≤j≤k_{n}} S_{j} ≥ λ ϕ(k_{n−1})} < ∞  (3.22)

This will establish that with probability 1,

lim sup_{n→∞} \frac{sup_{k_{n−1}≤j≤k_{n}} S_{j}}{ϕ(k_{n−1})} ≤ λ

or by the monotonicity of ϕ,

lim sup_{n→∞} \frac{S_{n}}{ϕ(n)} ≤ λ

with probability 1. Since λ > \sqrt{2} is arbitrary the upper bound in the law of the iterated logarithm will follow. Each term in the sum of 3.22 can be estimated as in Levy’s inequality,

P { sup_{k_{n−1}≤j≤k_{n}} S_{j} ≥ λ ϕ(k_{n−1})} ≤ 2 P {S_{k_{n}} ≥ (λ − σ) ϕ(k_{n−1})}

with 0 < σ < λ, provided

sup_{1≤j≤k_{n}−k_{n−1}} P \{|S_{j}| ≥ σϕ(k_{n−1})\} ≤ \frac{1}{2}

Our choice of k_{n} will be k_{n} = [ρ^{n}] for some ρ > 1 and therefore

lim_{n→∞} \frac{ϕ(k_{n−1})}{\sqrt{k}_{n}} = ∞

and by Chebychev’s inequality, for any fixed σ > 0,

sup_{1≤j≤k_{n}} P \{|S_{j}| ≥ σϕ(k_{n−1})\} ≤ \frac{E[S_{n}^{2}]}{[σϕ(k_{n−1})]^{2}}
= \frac{k_{n}}{[σϕ(k_{n−1})]^{2}}
= \frac{k_{n}}{σ^{2}k_{n−1} log log k_{n−1}}
= o(1) as n → ∞.  (3.23)

By choosing σ small enough so that λ − σ > \sqrt{2} it is sufficient to show that for any λ′ > \sqrt{2},

∑_{n} P {S_{k_{n}} ≥ λ′ ϕ(k_{n−1})} < ∞

By picking ρ sufficiently close to 1, ( so that λ′\sqrt{ρ} > \sqrt{2}), because \frac{ϕ(k_{n−1})}{ϕ(k_{n})} = \frac{1}{\sqrt{ρ}} we can reduce this to the convergence of

∑_{n} P {S_{k_{n}} ≥ λ ϕ(k_{n})} < ∞  (3.24)

for all λ > \sqrt{2}.

If we use the estimate P [X ≥ a] ≤ exp[−\frac{a^{2}}{2} ] that is valid for the standard normal distribution, we can verify 3.24.

∑_{n} exp [ − \frac{λ^{2}(ϕ(k_{n}))^{2}}{2 k_{n}}] < ∞

for any λ > \sqrt{2}.

To prove the lower bound we select again a subsequece, k_{n} = [ρ^{n}] with some ρ > 1, and look at Y_{n} = S_{k_{n+1}} −S_{k_{n}}, which are now independent random variables. The tail probability of the Normal distribution has the lower bound

P [X ≥ a] = \frac{1}{\sqrt{2π}} ∫_{a}^{∞} exp[−\frac{x^{2}}{2} ]dx
≥ \frac{1}{\sqrt{2π}} ∫_{a}^{∞} exp[−\frac{x^{2}}{2} − x](x + 1)dx
≥ \frac{1}{\sqrt{2π}} exp[−\frac{(a + 1)^{2}}{2} ]

If we assume Normal like tail probabilities we can conclude that

∑_{n} P \{Y_{n} ≥ λϕ(k_{n+1})\} ≥ ∑_{n} exp [ − \frac{1}{2}[1 + \frac{λϕ(k_{n+1})}{\sqrt{(ρ^{n+1} − ρ^{n})}}]^{2}] = +∞

provided \frac{λ^{2}ρ}{2(ρ−1)} < 1 and conclude by the Borel-Cantelli lemma, that Y_{n} = S_{k_{n+1}} − S_{k_{n}} exceeds λϕ(k_{n+1}) infinitely often for such λ. On the other hand from the upper bound we already have (replacing X_{i} by −X_{i})

P \{ lim sup_{n} \frac{−S_{k_{n}}}{ϕ(k_{n+1})} ≤ \begin{matrix}√ & \overline{2} \\ √ & \overline{ρ} \end{matrix}\} = 1

Consequently

P \{ lim sup_{n} \frac{S_{k_{n+1}}}{ϕ(k_{n+1})} ≥ \sqrt{\frac{2(ρ − 1)}{ρ}} − \begin{matrix}√ & \overline{2} \\ √ & \overline{ρ} \end{matrix}\} = 1

and therefore,

P \{ lim sup_{n} \frac{S_{n}}{ϕ(n)} ≥ \sqrt{\frac{2(ρ − 1)}{ρ}} − \begin{matrix}√ & \overline{2} \\ √ & \overline{ρ} \end{matrix}\} = 1

We now take ρ arbitrarily large and we are done.

We saw that the law of the iterated logarithm depends on two things. (i). For any a > 0 and p < \frac{a^{2}}{2} an upper bound for the probability

P [S_{n} ≥ a\sqrt{n log log n}] ≤ C_{p}[log n]^{−p}

with some constant C_{p} (ii). For any a > 0 and p > \frac{a^{2}}{2} a lower bound for the probability

P [S_{n} ≥ a\sqrt{n log log n}] ≥ C_{p}[log n]^{−p}

with some, possibly different, constant C_{p}.

Both inequalities can be obtained from a uniform rate of convergence in the central limit theorem.

sup_{a} |P \{\frac{S_{n}}{\sqrt{n}} ≥ a\} − ∫_{a}^{∞} \frac{1}{\sqrt{2π}} exp[−\frac{x^{2}}{2} ] dx| ≤ Cn^{−δ}  (∗∗)

for some δ > 0 in the central limit theorem. Such an error estimate is provided in the following theorem Theorem 3.26.(Berry-Esseen). Assume that the i.i.d. sequence {X_{j}} with mean zero and variance one satisfies an additional moment condition E|X|^{2+α} < ∞ for some α > 0. Then for some δ > 0 the estimate (∗∗) holds.

Proof. The proof will be carried out in several lemmas. Lemma 3.27. Let −∞ < a < b < ∞ be given and 0 < h < \frac{b−a}{2} be a small positive number. Consider the function f_{a,b,h}(x) defined as

f_{a,b,h}(x) = \{ \begin{matrix}0  for − ∞ < x ≤ a − h \\ \frac{x−a+h}{2h}  for a − h ≤ x ≤ a + h \\ 1  for a + h ≤ x ≤ b − h \\ 1 − \frac{x−b+h}{2h} for b − h ≤ x ≤ b + h \\ 0  for b + h ≤ x < ∞ \end{matrix}

For any probability distribution μ with characteristic function μ̂(t)

∫_{−∞}^{∞} f_{a,b,h}(x) dμ(x) = \frac{1}{2π} ∫_{−∞}^{∞} μ̂(y)\frac{e^{−i a y} − e^{−i b y}}{i y} \frac{sin h y}{h y} dy

Proof. This is essentially the Fourier inversion formula. Note that

f_{a,b,h}(x) = \frac{1}{2π} ∫_{−∞}^{∞} e^{i x y} \frac{e^{−i a y} − e^{−i b y}}{i y} \frac{sin h y}{h y} dy

We can start with the double integral

\frac{1}{2π} ∫_{−∞}^{∞} ∫_{−∞}^{∞} e^{i x y} \frac{e^{−i a y} − e^{−i b y}}{i y} \frac{sin h y}{h y} dy dμ(x)

and apply Fubini’s theorem to obtain the lemma. Lemma 3.28. If λ, μ are two probability measures with zero mean having λ̂(·), μ̂(·) for respective characteristic functions. Then

∫_{−∞}^{∞} f_{a,h}(x) d(λ − μ)(x) = \frac{1}{2π} ∫_{−∞}^{∞}[λ̂(y) − μ̂(y)]\frac{e^{−i a y}}{i y} \frac{sin h y}{h y} dy

where f_{a,h}(x) = f_{a,∞,h}(x), is given by

f_{a,h}(x) = \{ \begin{matrix}0  for − ∞ < x ≤ a − h \\ \frac{x−a+h}{2h}  for a − h ≤ x ≤ a + h \\ 1  for a + h ≤ x < ∞ \end{matrix}

Proof. We just let b → ∞ in the previous lemma. Since |λ̂(y)−μ̂(y)| = o(|y|), there is no problem in applying the Riemann-Lebesgue Lemma. We now proceed with the proof of the theorem.

λ[[a, ∞)] ≤ ∫ f_{a−h,h}(x) dλ(x) ≤ λ[[a − 2h, ∞)]

and

μ[[a, ∞)] ≤ ∫ f_{a−h,h}(x) dμ(x) ≤ μ[[a − 2h, ∞)]

Therefore if we assume that μ has a density bounded by C,

λ[[a, ∞)] − μ[[a, ∞)] ≤ 2hC + ∫ f_{a−h,h}(x) d(λ − μ)(x)

Since we get a similar bound in the other direction as well,

sup_{a} |λ[[a, ∞)] − μ[[a, ∞)]| ≤ sup_{a} | ∫ f_{a−h,h}(x) d(λ − μ)(x)|
+2hC
≤ \frac{1}{2π} ∫_{−∞}^{∞} |λ̂(y) − μ̂(y)|\frac{| sin h y |}{h y^{2}} dy
+2hC  (3.25)

We take λ to be the distribution of \frac{S_{n}}{\sqrt{n}} having as its characteristic function λ̂_{n}(y) = [ϕ(\frac{y}{\sqrt{n}})]^{n} where ϕ(y) is the characteristic function of the common distribution of the {X_{i}} and has the expansion

ϕ(y) = 1 − \frac{y^{2}}{2} + O(|y|^{2+α})

for some α > 0. We therefore get, for some choice of α > 0,

|λ̂_{n}(y) − exp[−\frac{y^{2}}{2} ]| ≤ C \frac{|y|^{2+α}}{n^{α}} if |y| ≤ n^{\frac{α}{2+α}}

Therefore for θ = \frac{α}{2+α}

∫_{−∞}^{∞} |λ̂_{n}(y) − exp[−\frac{y^{2}}{2} ]|\frac{| sin h y |}{h y^{2}} dy

= ∫_{|y|≤n^{θ}} |λ̂_{n}(y) − exp[−\frac{y^{2}}{2} ]|\frac{| sin h y |}{h y^{2}} dy

+ ∫_{|y|≥n^{θ}} |λ̂_{n}(y) − exp[−\frac{y^{2}}{2} ]|\frac{| sin h y |}{h y^{2}} dy

≤ \frac{C}{h} \{ ∫_{|y|≤n^{θ}} \frac{|y|^{α}}{n^{α}} dy + ∫_{|y|≥n^{θ}} \frac{dy}{|y|^{2}} \}
≤ C \frac{n^{(α+1)θ−α} + n^{−θ}}{h}
= \frac{C}{hn^{\frac{α}{α+2}}}

Substituting this bound in 3.25 we get

sup_{a} |λ_{n}[[a, ∞)] − μ[[a, ∞)]| ≤ C_{1} h + \frac{C}{h n^{\frac{α}{2+α}}}

By picking h = n^{− \frac{α}{2(2+α)}} we get

sup_{a} |λ_{n}[[a, ∞)] − μ[[a, ∞)]| ≤ C n^{− \frac{α}{2(2+α)}}

and we are done.