Category : markov-decision-process

I’m working on a time-series problem and I want to estimate the transition probability matrix of a 2D time-series data (n_samples=997, state height= 5, state width= 3) and there are only 5 possible numbers {-100,0,100,200,300} for each element of the states. I’m wondering if anyone can suggest an efficient way to estimate the transition probabilities ..

Read more