⁹In the simplistic case of a finite number of discrete states, computing the value function is not a problem at all, only learning is. But in realistic scenarios, the value function would be computed by something like a neural network based on sensory input, and these computations can be challenging.