0.16 Markov chains  (Page 3/4)

1. The matrix $A$ is not a regular Markov chain because every power of it has an entry 0 in the first row, second column position. In fact, we will show that all 2 by 2 matrices that have a zero in the first row, second column position are not regular. Consider the following matrix $M$ .

   $M=\left[\begin{array}{cc}a& 0\\ b& c\end{array}\right]$
   $M^2=\left[\begin{array}{cc}a& 0\\ b& c\end{array}\right]\left[\begin{array}{cc}a& 0\\ b& c\end{array}\right]=\left[\begin{array}{cc}a\cdot a+0\cdot b& a\cdot 0+0\cdot c\\ b\cdot a+c\cdot b& b\cdot 0+c\cdot c\end{array}\right]$

   Observe that the first row, second column entry, $a\cdot 0+0\cdot c$ , will always be zero, regardless of what power we raise the matrix to.

2. The transition matrix $B$ is a regular Markov chain because

   $B^2=\left[\begin{array}{cc}\text{.}\text{40}& \text{.}\text{60}\\ \text{.}\text{24}& \text{.}\text{76}\end{array}\right]$

   has only positive entries.

At this point, the reader may have already guessed that the answer is yes if the transition matrix is a regular Markov chain. We try to illustrate with the following example from [link] .

A small town is served by two telephone companies, Mama Bell and Papa Bell. Due to their aggressive sales tactics, each month 40% of Mama Bell customers switch to Papa Bell, that is, the other 60% stay with Mama Bell. On the other hand, 30% of the Papa Bell customers switch to Mama Bell. The transition matrix is given below.

If the initial market share for Mama Bell is 20% and for Papa Bell 80%, we'd like to know the long term market share for each company.

Let matrix $T$ denote the transition matrix for this Markov chain, and $M$ denote the matrix that represents the initial market share. Then $T$ and $M$ are as follows:

$T=\left[\begin{array}{cc}\text{.}\text{60}& \text{.}\text{40}\\ \text{.}\text{30}& \text{.}\text{70}\end{array}\right]$ and $M=\left[\begin{array}{cc}\text{.}\text{20}& \text{.}\text{80}\end{array}\right]$

Since each month the towns people switch according to the transition matrix $T$ , after one month the distribution for each company is as follows:

After two months, the market share for each company is

After three months the distribution is

After four months the market share is

After 30 months the market share is $\left[\begin{array}{cc}3/7& 4/7\end{array}\right]$ .

The market share after 30 months has stabilized to $\left[\begin{array}{cc}3/7& 4/7\end{array}\right]$ .

This means that

Once the market share reaches an equilibrium state, it stays the same, that is, $\text{ET}=E$ .

This helps us answer the next question.

The answer to the second question provides us with a way to find the equilibrium vector $E$ .

The answer lies in the fact that $\text{ET}=E$ .

Since we have the matrix $T$ , we can determine $E$ from the statement $\text{ET}=E$ .

Suppose $E=\left[\begin{array}{cc}e& 1-e\end{array}\right]$ , then $\text{ET}=E$ gives us

Therefore, $E=\left[\begin{array}{cc}3/7& 4/7\end{array}\right]$

We will show that the final market share distribution for a Markov chain does not depend upon the initial market share. In fact, one does not even need to know the initial market share distribution to find the long term distribution. Furthermore, the final market share distribution can be found by simply raising the transition matrix to higher powers.

Consider the initial market share $\left[\begin{array}{cc}\text{.}\text{20}& \text{.}\text{80}\end{array}\right]$ , and the transition matrix $T=\left[\begin{array}{cc}\text{.}\text{60}& \text{.}\text{40}\\ \text{.}\text{30}& \text{.}\text{70}\end{array}\right]$ for Mama Bell and Papa Bell in the above example. Recall we found $T^n$ , for very large $n$ , to be $\left[\begin{array}{cc}3/7& 4/7\\ 3/7& 4/7\end{array}\right]$ .

Clearly, $\left[\begin{array}{cc}\text{.}\text{20}& \text{.}\text{80}\end{array}\right]\left[\begin{array}{cc}3/7& 4/7\\ 3/7& 4/7\end{array}\right]=\left[\begin{array}{cc}3/7& 4/7\end{array}\right]$

No matter what the initial market share, the product is $\left[\begin{array}{cc}3/7& 4/7\end{array}\right]$ .

If the initial share is $\left[\begin{array}{cc}\text{.}\text{10}& \text{.}\text{90}\end{array}\right]$ , then

For any distribution $A=\left[\begin{array}{cc}a& 1-a\end{array}\right]$ , for example,

It makes sense, because the entry $3/7\left(a\right)+3/7\left(1-a\right)$ , for example, will always equal $3/7$ .

Just as the sum of the parts equals the whole, the sum of the parts of $3/7$ equals $3/7$ .