A reorder problem-- electronic store

A reorder problem

An electronic store stocks a certain type of DVD player. At the end of each week, an order is placed for early delivery the following Monday. A maximum of fourunits is stocked. Let the states be the number of units on hand at the end of the sales week: $\mathbf{E}=\{0,1,2,3,4\}$ . Two possible actions:

• Order two units, at a cost of $150 each
• Order four units, at a cost of $120 each

Units sell for $200. If demand exceeds the stock in hand, the retailer assumes a penalty of $40 per unit (in losses due to customer dissatisfaction, etc.).Because of turnover, return on sales is considered two percent per week, so that discount is $\alpha =1/1.02$ on a weekly basis.

In state 0, there are three possible actions: order 0, 2, or 4. In states 1 and 2 there are two possible actions: order 0 or 2. In states 3 and 4, the only actionis to order 0. Customer demand in week $n+1$ is represented by a random variable $D_{n+1}$ . The class is iid, uniformly distributed on the values 0, 1, 2, 3, 4. If X _n is the state at the end of week n , then $\{X_n,D_{n+1}\}$ is independent for each n .

Analyze the system as a Markov decision process with type 3 gains, depending upon current state, action, and demand. Determine the transition probability matrixPA (properly padded) and the gain matrix (also padded). Sample calculations are as follows:

$\begin{array}{cc}\text{State 0, action 0:}& p_{00}\left(0\right)=1\text{(all other}{p}_{0k}\left(0\right)=0\\ \text{State 0, action 2:}& p_{00}\left(2\right)=P(D\ge 2)=3/5,p_{01}\left(2\right)=P(D=1)=1/5\text{, etc.}\\ \text{State 2, action 2:}& p_{2j}\left(k\right)=1/5,k=0,1,2,3,4\end{array}$

For state = i , action = a , and demand = k , we seek $g(i,a,k)$

$\begin{array}{cc}g(0,0,k)=-40k& \text{0 -40 -80 -120 -160}\\ g(0,2,k)=-300+200min\{k,2\}-40max\{k-2,0\}& \text{-300 -100 100 60 20}\\ g(0,4,k)=-480+200k& \text{-480 -280 -80 120 320}\end{array}$

1. Complete the transition probability table and the gain table.
2. Determine an optimum infinite-horizon strategy with no discounting.
3. Determine an optimum infinite-horizon strateby with discounting (alpha = 1/1.02).
4. The manager decides to set up a six-week strategy, after which new sales conditions may be established. Determine an optimum strategy for the six-week period.

Transition probabilities and gains