16.1 Elements of markov sequences  (Page 7/13)

Remarks

• A similar treatment shows that for the nonhomogeneous case the distribution at any stage is determined by the initial distribution andthe class of one-step transition matrices. In the nonhomogeneous case, transition probabilities $p_{n,n+1}(i,j)$ depend on the stage n .
• A discrete-parameter Markov process, or Markov sequence, is characterized by the fact that each member $X_{n+1}$ of the sequence is conditioned by the value of the previous member of thesequence. This one-step stochastic linkage has made it customary to refer to a Markov sequence as a Markov chain . In the discrete-parameter Markov case, we use the terms process, sequence, or chain interchangeably.

The transition diagram and the transition matrix

The previous examples suggest that a Markov chain is a dynamic system, evolving in time. On the other hand, the stochastic behavior of a homogeneous chain is determined completelyby the probability distribution for the initial state and the one-step transition probabilities $p(i,j)$ as presented in the transition matrix P . The time-invariant transition matrix may convey a static impression of the system.However, a simple geometric representation, known as the transition diagram , makes it possible to link the unchanging structure, represented by thetransition matrix, with the dynamics of the evolving system behavior.

Definition . A transition diagram for a homogeneous Markov chain is a linear graph with one node for each state and one directededge for each possible one-step transition between states (nodes).

We ignore, as essentially impossible, any transition which has zero transition probability. Thus, the edges on the diagram correspond topositive one-step transition probabilities between the nodes connected. Since for some pair $(i,j)$ of states, we may have $p(i,j)>0$ but $p(j,i)=0$ , we may have a connecting edge between two nodes in one direction, but none in the other. The system can be viewed as an object jumping from state to state (node tonode) at the successive transition times . As we follow the trajectory of this object, we achieve a sense of the evolution of the system.