17.1 Appendix b to applied probability: some mathematical aids

Series

• 1. Geometric series From the expression $(1-r)(1+r+r^2+...+r^n)=1-r^{n+1}$ , we obtain
  $$\sum _{k=0}^n{r}^k=\frac{1-r^{n+1}}{1-r}\text{for}r\ne 1$$
  For $\left|r\right|<1$ , these sums converge to the geometric series ${\displaystyle \sum _{k=0}^{\infty }{r}^k=\frac{1}{1-r}}$
  Differentiation yields the following two useful series:
  $$\sum _{k=1}^{\infty }k{r}^{k-1}=\frac{1}{{(1-r)}^2}\text{for}\left|r\right|<1\text{and}\sum _{k=2}^{\infty }k(k-1)r^{k-2}=\frac{2}{{(1-r)}^3}\text{for}\left|r\right|<1$$
  For the finite sum, differentiation and algebraic manipulation yields
  $$\sum _{k=0}^{n}k{r}^{k-1}=\frac{1-r^n[1+n(1-r)]}{{(1-r)}^2}\text{which}\text{converges}\text{to}\frac{1}{{(1-r)}^2}\text{for}\left|r\right|<1$$
• 2. Exponential series . ${\displaystyle e^x=\sum _{k=0}^{\infty }\frac{x^k}{k!}\text{and}{e}^{-x}=\sum _{k=0}^{\infty }{(-1)}^k\frac{x^k}{k!}\text{for}\text{any}x}$
  Simple algebraic manipulation yields the following equalities usefulfor the Poisson distribution:
  $$\sum _{k=n}^{\infty }k\frac{x^k}{k!}=x\sum _{k=n-1}^{\infty }\frac{x^k}{k!}\text{and}\sum _{k=n}^{\infty }k(k-1)\frac{x^k}{k!}=x^2\sum _{k=n-2}^{\infty }\frac{x^k}{k!}$$
• 3. Sums of powers of integers ${\displaystyle \sum _{i=1}^{n}i=\frac{n(n+1)}{2}\sum _{i=1}^n{i}^2=\frac{n(n+1)(2n+1)}{6}}$

Some useful integrals

• 1. The gamma function ${\displaystyle \Gamma \left(r\right)={\int }_0^{\infty }{t}^{r-1}{e}^{-t}dt\text{for}r>0}$
  Integration by parts shows $\Gamma \left(r\right)=(r-1)\Gamma (r-1)\text{for}r>1$
  By induction $\Gamma \left(r\right)=(r-1)(r-2)\cdots (r-k)\Gamma (r-k)\text{for}r>k$
  For a positive integer $n,\Gamma \left(n\right)=(n-1)!\text{with}\Gamma \left(1\right)=0!=1$
• 2. By a change of variable in the gamma integral, we obtain
  $${\int }_0^{\infty }{t}^r{e}^{-\lambda t}dt=\frac{\Gamma (r+1)}{{\lambda }^{r+1}}r>-1,\lambda >0$$
• 3. A well known indefinite integral gives
  $${\int }_a^{\infty }t{e}^{-\lambda t}dt=\frac{1}{{\lambda }^2}{e}^{-\lambda a}(1+\lambda a)\text{and}{\int }_a^{\infty }{t}^2{e}^{-\lambda at}dt=\frac{1}{{\lambda }^3}{e}^{-\lambda a}[1+\lambda a+{\left(\lambda a\right)}^2/2]$$
  For any positive integer m ,
  $${\int }_a^{\infty }{t}^m{e}^{-\lambda t}dt=\frac{m!}{{\lambda }^{m+1}}{e}^{-\lambda a}\left[1,+,\lambda ,a,+,\frac{{\left(\lambda a\right)}^2}{2!},+,\cdots ,+,\frac{{\left(\lambda a\right)}^m}{m!}\right]$$
• 4. The following integrals are important for the Beta distribution.
  $${\int }_0^1{u}^r{(1-u)}^{s}du=\frac{\Gamma (r+1)\Gamma (s+1)}{\Gamma (r+s+2)}r>-1,s>-1$$
  For nonnegative integers ${\displaystyle m,n{\int }_0^1{u}^m{(1-u)}^{n}du=\frac{m!n!}{(m+n+1)!}}$

Some basic counting problems

We consider three basic counting problems, which are used repeatedly as components of more complex problems. The first two, arrangements and occupancy are equivalent. The third is a basic matching problem.

1. Arrangements of r objects selected from among n distinguishable objects.
   1. The order is significant.
   2. The order is irrelevant.
   For each of these, we consider two additional alternative conditions.
   1. No element may be selected more than once.
   2. Repitition is allowed.
2. Occupancy of n distinct cells by r objects. These objects are
   1. Distinguishable.
   2. Indistinguishable.
   The occupancy may be
   1. Exclusive.
   2. Nonexclusive (i.e., more than one object per cell)

   The results in the four cases may be summarized as follows:

   1. 1. Ordered arrangements, without repetition ( permutations ). Distinguishable objects, exclusive occupancy.
         $$P(n,r)=\frac{n!}{(n-r)!}$$
      2. Ordered arrangements, with repitition allowed. Distinguishable objects, nonexclusive occupancy.
         $$U(n,r)=n^r$$
   2. 1. Arrangements without repetition, order irrelevant ( combinations ). Indistinguishable objects, exclusive occupancy.
         $$C(n,r)=\frac{n!}{r!(n-r)!}=\frac{P(n,r)}{r!}$$
      2. Unordered arrangements, with repetition. Indistinguishable objects, nonexclusive occupancy.
         $$S(n,r)=C(n+r-1,r)$$
3. Matching n distinguishable elements to a fixed order. Let $M(n,k)$ be the number of permutations which give k matches.

We reduce the problem to determining $m(n,0)$ , as follows:

1. Select k places for matches in $C(n,k)$ ways.
2. Order the $n-k$ remaining elements so that no matches in the other $n-k$ places.
   $$M(n,k)=C(n,k)M(n-k,0)$$
   Some algebraic trickery shows that $M(n,0)$ is the integer nearest $n!/e$ . These are easily calculated by the MATLAB command M = round(gamma(n+1)/exp(1)) For example >>M = round(gamma([3:10]+1)/exp(1));>>disp([3:6;M(1:4);7:10;M(5:8)]')3 2 7 1854 4 9 8 148335 44 9 133496 6 265 10 1334961

Extended binomial coefficients and the binomial series

• The ordinary binomial coefficient is ${\displaystyle C(n,k)=\frac{n!}{k!(n-k)!}}$ for integers $n>0,0\le k\le n$
  For any real x , any integer k , we extend the definition by
  $$C(x,0)=1,C(x,k)=0\text{for}k<0,\text{and}C(n,k)=0\text{for}\text{a}\text{positive}\text{integer}k>n$$
  and
  $$C(x,k)=\frac{x(x-1)(x-2)\cdots (x-k+1)}{k!}\text{otherwise}$$
  Then Pascal's relation holds: $C(x,k)=C(x-1,k-1)+C(x-1,k)$
  The power series expansion about $t=0$ shows
  $${(1+t)}^x=1+C(x,1)t+C(x,2)t^2+\cdots \forall x,-1<t<1$$
  For $x=n$ , a positive integer, the series becomes a polynomial of degree n .

Cauchy's equation

1. Let f be a real-valued function defined on $(0,\infty )$ , such that
   1. $f(t+u)=f\left(t\right)+f\left(u\right)\text{for}t,u>0$ , and
   2. There is an open interval I on which f is bounded above (or is bounded below).
   Then $f\left(t\right)=f\left(1\right)t\forall t>0$
2. Let f be a real-valued function defined on $(0,\infty )$ such that
   1. $f(t+u)=f\left(t\right)f\left(u\right)\forall t,u>0$ , and
   2. There is an interval on which f is bounded above.
   Then, either $f\left(t\right)=0\text{for}t>0$ , or there is a constant a such that $f\left(t\right)=e^{at}\text{for}t>0$

[For a proof, see Billingsley, Probability and Measure , second edition, appendix A20]

Countable and uncountable sets

A set (or class) is countable iff either it is finite or its members can be put into a one-to-one correspondence with the natural numbers.

Examples

• The set of odd integers is countable.
• The finite set $\{n:1\le n\le 1000\}$ is countable.
• The set of all rational numbers is countable. (This is established by an argument known as diagonalization).
• The set of pairs of elements from two countable sets is countable.
• The union of a countable class of countable sets is countable.

A set is uncountable iff it is neither finite nor can be put into a one-to-one correspondence with the natural numbers.

Examples

• The class of positive real numbers is uncountable. A well known operation shows that the assumption of countability leads to a contradiction.
• The set of real numbers in any finite interval is uncountable, since these can be put into a one-to-one correspondence of the class of all positive reals.