Markov Chain

In many applications, some variables evolve over time in a random (stochastic) way in that even if you know everything up to time t, it is still impossible to know for sure which values the variables take in the future.
Stochastic processes are mathematical models that describe these phenomena when the randomness is driven by something out of the control of the relevant decision makers. For example, the stock prices can be (and usually are) modeled as stochastic processes, since it is difficult for an investor to affect them. However, in a chess game, the uncertainty in your opponents’ future moves are not modeled as stochastic processes as they are made by a decision maker with the goal to defeat you and they may be adjusted based on your moves.
One simple and widely used class of stochastic processes are Markov chains. In this question, we study
Markov chains on a finite state space. There is a sequence of random variables x0, x1, …, xt, …, each taking value in a finite set S called the state space. The subscripts have the interpretation of time. Therefore given an integer time t, x0, …, xt are assumed to be known at the time, while xt+1, xt+2, … remain random. For convenience, we label states with positive integers: S = {1, …, n}, where n is the number of possible states.
(a) Is S a good choice of sample space for describing this stochastic process? No matter what the answer is, for the rest of the question we assume that a good sample space Ω (which might be S if your answer is
“yes”) has been chosen to carry the random variables.
(b) The fundamental assumption of Markov chains is that given x0, …, xt, the probability that xt+1 = j is pji where i is the value of xt. This holds for every t = 0, 1, … and the numbers pji are independent of t. More precisely, for any fixed t and the event A that x0 = a0, …, xt = at (where a0, a1, …, at are integers in S), P(x −1
t+1(j) ∩ A) = P(A)pj,at
; this holds for all a0, a1, …, at. (The so-called Markov property means that
the value of xt+1 only depends on xt and not on history further back.)
Let P be the n × n matrix whose (j, i) entry is pji for all i, j ∈ S; it is called the transition matrix of the
Markov chain. Show that for the probabilities to be well-defined for all laws of x0, each column of P sums
up to one.
(c) A law on the state space S can be represented by an n × 1 matrix (column vector), whose (i, 1) entry is
the probability of {i}. A function f : S → R can be represented by a 1 × n matrix whose (1, i) entry is
the number f(i). (Notice that f is not a random variable unless your answer to Part (a) is “yes”.) If the
law of x0 is v0, then what is the interpretation of fv0 (matrix product)?
(d) In what follows, fix a law v0 of x0 and a function f : S → R. What is the law of x1?
(e) Notice that fP is a 1×n matrix, so it represents a function on S. What does that function mean intuitively?
(As we can see, the transformation from f to fP is another way to describe the transition matrix P, and
this transformation plays an important role when studying more complicated Markov processes.)
(f) What is the law of x

Don't use plagiarized sources. Get Your Custom Essay on
Markov Chain
Just from $13/Page
Order Essay
Academic Heros
Calculate your paper price
Pages (550 words)
Approximate price: -

Why Work with Us

Top Quality and Well-Researched Papers

Our writers are encouraged to read and research widely to have rich information before writing clients’ papers. Therefore, be it high school or PhD level paper, it will always be a well-researched work handled by experts.

Professional and Experienced Academic Writers

For one to become part of our team, thorough interview and vetting is undertaken to make sure their academic level and experience are beyond reproach, hence enabling us give our clients top quality work.

Free Unlimited Revisions

Once you have received your paper and feel that some issues have been missed, just request for revision and it will be done. In addition, you can present your work to the tutor and he/she asks for improvement/changes, we are always ready to assist.

Prompt Delivery and 100% Money-Back-Guarantee

All our papers are sent to the clients before the deadline to allow them time to review the work before presenting to the tutor. If for some reason we feel our writers cannot meet the deadline, we will contact you to ask for more time. If this is not possible, then the paid amount will be refunded.

Original & Confidential

Our writers have been trained to ensure work produced is free of plagiarism. Software to check originality are also applied. Our clients’ information is highly guarded from third parties to ensure confidentiality is maintained.

24/7 Customer Support

Our support team is available 24 hours, 7 days a week. You can reach the team via live chat, email or phone call. You can always get in touch whenever you need any assistance.

Try it now!

Calculate the price of your order

Total price:

How it works?

Follow these simple steps to get your paper done

Place your order

Fill in the order form and provide all details of your assignment.

Proceed with the payment

Choose the payment system that suits you most.

Receive the final file

Once your paper is ready, we will email it to you.

Our Services

You have had a hectic day, and still need to complete your assignment, yet it is late at night. No need to panic. Place your order with us, retire to bed, and once you wake up, the paper will be ready.


Essay Writing Service

It does not matter the urgency of your paper, or the academic level, our team is ready to help you 24/7. Just contact us and all your academic needs will be sorted.


Admission Essays & Business Writing Help

A student is often required to write an admission letter requesting to be admitted in a certain institution. For you to be gain that admission in your dream institution, you must write a convincing letter. You can depend on our team for the best admission letters.


Editing Support

Academic writing is not just about getting information and throwing it all over. Our team will ensure you have a polished paper that is coherent and has a good flow of information. We also ensure the paper follows the correct formatting styles like APA, Harvard, MLA, Chicago/Turabian.


Revision Support

If our writers write a paper but you are not satisfied in one way or another, you can always ask for revision. This is totally free. Our writers and editors will revise your paper untill you are be totally satisfied. We as well offer revision for papers not done by our writers at a small fee.