Martingale sequences: the concept, examples, and basic patterns (Page 2/2)

Page 2 / 2

Notation . When we write $(X_{N}, Z_{N})$ is a martingale (submartingale, supermartingale), we are asserting X _N is integrable, Z _N is a decision sequence, $X_{N} \sim Z_{N}$ , and X _N is a MG (SMG, SRMG) relative to Z _N .

Definition . If Y _N is integrable and Z _N is a decision sequence, then

Y _N is absolutely fair relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] = 0 a . s . \forall n \in N$
Y _N is favorable relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] \geq 0 a . s . \forall n \in N$
Y _N is unfavorable relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] \leq 0 a . s . \forall n \in N$

Notation . When we write $(Y_{N}, Z_{N})$ is absolutely fair (favorable, unfavorable), we are asserting Y _N is integrable, Z _N is a decision sequence, $Y_{N} \sim Z_{N}$ , and Y _N is absolutely fair (favorable, unfavorable) relative to Z _N . IXA2-2

Ixa2-1

If X _N is a basic sequence and Y _N is the corresponding incremental sequence, then

$(X_{N}, Z_{N})$ is a martingale iff $(Y_{N}, Z_{N})$ is absolutely fair.
$(X_{N}, Z_{N})$ is a submartingale iff $(Y_{N}, Z_{N})$ is favorable.
$(X_{N}, Z_{N})$ is a supermartingale iff $(Y_{N}, Z_{N})$ is unfavorable.

Let * be any one of the symbols $=, \geq$ , or $\leq$ . Then by linearity and (CE7)

E [X_{n + 1} | W_{n}] = E [Y_{n + 1} | W_{n}] + E [X_{n} | W_{n}] = E [Y_{n + 1} | W_{n}] + X_{n} * X_{n} a . s . iff E [Y_{n + 1} | W_{n}] * 0 a . s .

Remarks

$(X_{N}, Z_{N})$ is a SMG iff $(- X_{N}, Z_{N})$ is a SRMG
We write (S)MG to indicate the same statement can be made for a MG or a SMG with the appropriate choice of = or $\geq$
We write ( ≥ ) to indicate simultaneously two cases:
- $(\geq)$ read as = in all places (for a MG)
- $(\geq)$ read as $\geq$ in all places (for a SMG)

Some basic patterns

Ixa3-1

If $(X_{N}, Z_{N})$ is a (S)MG and $X_{N} \sim H_{N}$ , with $H_{N} \sim Z_{N}$ , then $(X_{N}, H_{N})$ is a (S)MG.

Let $K_{n} = (H_{0}, H_{1}, \dots, H_{n})$ . By (CE9) , the (S)MG definition, monotonicity, and (CE7)

E [X_{n + 1} | K_{n}] = E {E [X_{n + 1} | W_{n}] | K_{n}} (\geq) E [X_{n} | K_{n}] = X_{n} a . s .

Ixa3-2

For integrable $X_{N} \sim Z_{N}$ , the following are equivalent

$\begin{matrix} a & (X_{N}, Z_{N}) & is a (S)MG \\ b & E [X_{n + k} | W_{n}] (\geq) X_{n} a . s . & \forall & n, k \in N \\ c & E [I_{C} X_{n + 1}] (\geq) E [I_{C} X_{n}] & \forall & C \in σ (W_{n}) & \forall & n \in N \\ d & E [I_{C} X_{n + k}] (\geq) E [I_{C} X_{n}] & \forall & C \in σ (W_{n}) & \forall & n, k \in N \end{matrix}$

b $\Rightarrow$ a as a special case
a $\Rightarrow$ b By (CE9) , (a), and monotonicity
$E [X_{n + k} | W_{n}] = E {E [X_{n + k} | W_{n + k - 1}] | W_{n}} (\geq) E [X_{n + k - 1} | W_{n}] a . s .$
$k - 1$ iterations yield $E [X_{n + k} | W_{n}] (\geq) X_{n} a . s .$
d $\Rightarrow$ c as a special case
c $\Rightarrow$ a By (CE1) and (c), $E [I_{C} X_{n + 1}] = E {I_{C} E [X_{n + 1} | W_{n}]} (\geq) E [I_{C} X_{n}] a . s .$ . Since $X_{n} \sim W_{n} a . s .$ and $E [X_{n + 1} | W_{n}] \sim W_{n} a . s .$ , the result follows from the uniqueness property (E5)
b $\Rightarrow$ d By (CE1) , (b), and monotonicity $E [I_{C} X_{n + k}] = E {I_{C} E [X_{n + k} | W_{n}]} (\geq) E [I_{C} X_{n}]$

We thus have $d \Rightarrow c \Rightarrow a \Leftrightarrow b \Rightarrow d$

Ixa3-3

If $(X_{N}, Z_{N})$ is a (S)MG, then $E [X_{n + k}] (\geq) E [X_{n}] (\geq) E [X_{0}]$

Ixa3-4

$(X_{N}, Z_{N})$ is a (S)MG iff $E [X_{q} - X_{p} | W_{n}] (\geq) 0 a . s . \forall n \leq p < q \in N$

EXERCISE. Note $X_{q} - X_{p} = Y_{p + 1} + \dots + Y_{q}$

IXA3-2

Ixa3-5

If $(X_{N}, Z_{N})$ is an $L^{2}$ MG, then

$\begin{matrix} E [X_{q} - X_{p}] = 0 & \forall & p < q \in N \\ E [X_{n} (X_{q} - X_{p})] = 0 & \forall & n \leq p < q \in N \\ E [(X_{n} - X_{m}) (X_{q} - X_{p})] = 0 & \forall & m < n \leq p < q \in N \\ E [X_{p} X_{q}] = E [X_{p \land q}^{2}] & \forall & p, q \in N \\ E [{(X_{q} - X_{p})}^{2}] = E [X_{q}^{2}] - E [X_{p}^{2}] \geq 0 & \forall & p < q \in N \\ E [X_{p}^{2}] = \sum_{k = 0}^{p} E [Y_{k}^{2}] & \forall & p \in N \end{matrix}$

$E [X_{q} - X_{p}] = E {E [X_{q} - X_{p} | W_{n}]} = 0$ by (CE1b) and Thm IXA3-4
$E [X_{n} (X_{q} - X_{p})] = E {X_{n} E [X_{q} - X_{p} | W_{n}]} = 0$ by (CE1b) , (CE8) , and Thm IXA3-4
As in b, since $X_{n} - X_{m} \sim W_{n}$
Suppose $p < q$ . Then, since $X_{p} \sim W_{p}$ , $E [X_{p} X_{q}] = E {X_{p} E [X_{q} | W_{p}]} = E [X_{p}^{2}]$ by definition of MG. For $q < p$ , interchange $p, q$ in the argument above.
$E [{(X_{q} - X_{p})}^{2}] = E [X_{q}^{2}] - 2 E [X_{p} X_{q}] + E [X_{p}^{2}] = E [X_{q}^{2}] - 2 E [X_{p}^{2}] + E [X_{p}^{2}]$ by d, above
By c, $E [Y_{j} Y_{k}] = 0$ for $j \neq k$ . Hence, $E [X_{p}^{2}] = E [{(\sum_{k = 0}^{p} Y_{k})}^{2}] = \sum_{j} \sum_{k} E [Y_{j} Y_{k}] = \sum_{k = 0}^{p} E [Y_{k}^{2}]$

A variety of weighted sums of increments are useful.

Ixa3-6

Suppose $(X_{N}, Z_{N})$ is a (S)MG and Y _N is the incremental sequence. Let H ₀ be a (nonnegative) constant and let $H_{n} \sim W_{n - 1}, n \geq 1$ , be bounded (nonnegative). Set

X_{n}^{*} = \sum_{k = 0}^{n} H_{k} Y_{k} = \sum_{k = 0}^{n} Y_{k}^{*} \forall n \in N

Then $(X_{N}^{*}, Z_{N})$ is a (S)MG.

$E [Y_{n + 1}^{*} | W_{n}] = E [H_{n + 1} Y_{n + 1} | W_{n}] = H_{n + 1} E [Y_{n + 1} | W_{n}] a . s .$ by (CE8)

For MG case: $E [Y_{n + 1}^{*} | W_{n}] = 0 a . s .$ for arbitrary bounded H _n

For SMG case: $E [Y_{n + 1}^{*} | W_{n}] \geq 0 a . s .$ for $H_{n} \geq 0$ , bounded

The conclusion follows from [link]

Remark . This result extends the pattern in the introductory gambling example. [link] IXA3-3

Ixa3-7

In Theorem IXA3-6 , if $E [X_{0}] \geq 0$ and $0 \leq H_{n} \leq 1 a . s . \forall n \in N$ , then $0 \leq E [X_{n}^{*}] \leq E [X_{n}] \forall n \in N$

$E [Y_{n + 1} | W_{n}] \geq H_{n + 1} E [Y_{n + 1} | W_{n}] (\geq) 0 a . s .$ , by hypothesis, and $H_{n + 1} E [Y_{n + 1} | W_{n}] = E [Y_{n + 1}^{*} | W_{n}] a . s .$ , by (CE8) . Thus, by monotonicity and (CE1b)

E [Y_{n + 1}] (\geq) E [Y_{n + 1}^{*}] (\geq) 0 \forall n \in N and E [Y_{0}] = E [X_{0}] \geq H_{0} E [Y_{0}] = E [Y_{0}^{*}]

Hence

E [X_{n}] = \sum_{k = 0}^{n} E [Y_{k}] \geq \sum_{k = 0}^{n} E [Y_{k}^{*}] = E [X_{n}^{*}] \geq 0

Some important special cases

Ixa3-8

Suppose integrable $X_{N} \sim Z_{N}$ . If $X_{n + 1} - X_{n} (\geq) 0 a . s . \forall n \in N$ , then $(X_{N}, Z_{N})$ is a (S)MG.

Apply monotonicity and Theorem IXA3-4

Ixa3-9

Suppose X _N has independent increments.

If $E [X_{n}] = c$ , invariant with n , then X _N is a MG.
If $E [X_{n + 1} - X_{n}] (\geq) 0, \forall n \in N$ , then $(X_{N}$ is a (S)MG.

For any n , consider any $C \in σ (U_{n})$ . By independent increments, ${I_{C}, (X_{n + 1} - X_{n})}$ is independent. Hence, $E [I_{C} X_{n + 1}] - E [I_{C} X_{n}] = E [I_{C} (X_{n + 1} - X_{n})] = E [I_{C}] E [(X_{n + 1} - X_{n})] (\geq) 0$ . The desired result follows from Theorem IXA3-2(c) .

Ixa3-10

Suppose g is a convex Borel function on an interval I which contains the range of all X _n and

$E [| g (X_{n}) |] < \infty \forall n \in N$ , Let $H_{n} = g (X_{n}) \forall n \in N$ ,

If $(X_{N}, Z_{N})$ is a MG, then $(H_{N}, Z_{N})$ is a SMG.
If g is nondecreasing and $(X_{N}, Z_{N})$ is a SMG, then so is $(H_{N}, Z_{N})$

a. By Jensen's inequality and the definition of a MG
$E [g (X_{n + 1}) | W_{n}] \geq g (E [X_{n + 1} | W_{n}]) = g (X_{n}) a . s .$
b. By Jensen's inequality
$E [g (X_{n + 1}) | W_{n}] \geq g (E [X_{n + 1} | W_{n}]) a . s .$
Since $E [X_{n + 1} | W_{n}] \geq X_{n} a . s .$ and g is nondecreasing, we have
$g (E [X_{n + 1} | W_{n}]) \geq g (X_{n}) a . s .$

Some commonly utilized convex functions

$g (t) = | t |$
$g (t) = t^{2}$ g is increasing for $t \geq 0$
$g (t) = u (t) t$ $g (X_{n}) = X_{n}^{+}$ g nondecreasing for all t
$g (t) = - u (- t) t$ $g (X_{n}) = X_{n}^{-}$ g nonincreasing for all t
$g (t) = e^{a t}, a > 0$ g is increasing for all t

Ixa3-11

Consider integrable $X_{N} \sim Z_{N}$ .

If $E [X_{n + 1} | W_{n}] = a X_{n} a . s . \forall n$ and $X_{n}^{*} = \frac{1}{a^{n}} X_{n} \forall n$ , then $(X_{N}^{*}, Z_{N})$ is a MG
If $E [X_{n + 1} | W_{n}] \geq a X_{n} a . s ., a > 0, \forall n$ and $X_{n}^{*} = \frac{1}{a^{n}} X_{n} \forall n$ , then $(X_{N}^{*}, Z_{N})$ is a SMG

E [X_{n + 1}^{*} | W_{n}] = \frac{1}{a^{n + 1}} E [X_{n + 1} | W_{n}] (\geq) \frac{1}{a^{n + 1}} a X_{n} = X_{n}^{*} a . s .

The restrictionl $a > 0$ is needed in the $\geq$ case.

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Topics in applied probability. OpenStax CNX. Sep 04, 2009 Download for free at http://cnx.org/content/col10964/1.2

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Topics in applied probability' conversation and receive update notifications?

Ask

	1 Human A&P 2- Test 1 By Madison Christian Start Flashcards
	3 Psychology MCQ 2009 2 Exam By John Gabrieli Start Exam
	American Politics MCQ By Nicole Bartels Start Quiz
	Vocabulary for "A Rose for Emily" By Bonnie Hurst Start Quiz
	27 AP 27 Reproductive System MCQ By OpenStax Start Quiz
	8 Psychology MCQ 2011 1 Exam By John Gabrieli Start Exam
	8 Sociology 08 Media and Technology MCQ By OpenStax Start Quiz
	1 AP 01 Human Body Anatomy Physiology Essay By OpenStax Start Flashcards
	2 SCEA Java Architect By JavaChamp Team Start Exam
	28 Biology 28 Invertebrates MCQ By OpenStax Start Quiz